System, program, and the like

Abstract

To use information captured by an imaging apparatus capable of acquiring image data having characteristics different from a visible light image.SOLUTION: A system comprises control means 200 which receives input information obtained by capturing an image of an imaging target by a special imaging apparatus capable of acquiring image data having characteristics different from a visible light image and executes a predetermined process by using the input information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to, for example, a system and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, systems have been developed that perform monitoring and the like using visible light images captured by a monitoring camera or the like (for example, Patent Document 1). However, analyzing visible light images has not been easy. On the other hand, thermal imaging inspection devices that use far-infrared rays can acquire image data with characteristics different from visible light images, and although these images contain useful information, they are not being fully utilized. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-135436 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to provide something superior to conventional things, for example, by utilizing information captured by an imaging device that can acquire image data having characteristics different from those of visible light images.

[0005] The purpose of the present invention is not limited to this, and the applicant intends to obtain rights for configurations that aim to achieve the effects achieved by the components disclosed in the specification and drawings, etc., through divisional applications, amendments, etc. For example, this specification discloses problems in which the phrase "can be" is read as "the problem is...." Each problem is described as an independent problem, and the applicant intends to obtain rights for configurations that solve each problem separately through divisional applications, amendments, etc. Even if the problem is implicitly understood from the description in the specification, the applicant intends to include part of the configuration described in this specification in the scope of the patent claim through amendments or divisional applications. Furthermore, the applicant has disclosed configurations that solve problems that combine these independent problems, and the applicant intends to obtain rights for these configurations. [Means for solving the problem]

[0006] (1) The system may include a control means for receiving input information obtained by capturing an image of an object using a special imaging device capable of acquiring image data having characteristics different from those of visible light images, and for carrying out predetermined processing using the input information.

[0007] In this way, it is possible to obtain information about the imaging target using characteristics different from those of the visible light image.

[0008] The "visible light image" may be, for example, an image that can be seen by humans. The "visible light image" may be, for example, an image captured by an imaging device that captures visible light, such as a general digital camera (hereinafter, also referred to as a "visible light imaging device," "visible light imaging device," or "visible light camera," as appropriate). The "visible light image" may be, for example, image data having two or more picture elements (also referred to as "pixels"). A pixel may be, for example, the smallest unit of point having color information that constitutes a digital image. The number of pixels may be the number of solid-state imaging elements, which will be described later.

[0009] The "visible light image" may be, for example, two-dimensional image data obtained by splitting the visible light region into red, green, and blue using a color filter or the like. The "visible light image" may be, for example, image data obtained by superimposing three images of red, green, and blue (RGB) obtained by imaging the subject on a solid-state image sensor through a lens. The solid-state image sensor may be, for example, a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor).

[0010] The term "visible light" refers to, for example, electromagnetic waves with wavelengths that can be seen by the human eye. Electromagnetic waves other than visible light may include, for example, infrared rays and ultraviolet rays.

[0011] The "special imaging device" may be, for example, an imaging device that captures electromagnetic waves other than visible light, a multispectral camera, a hyperspectral camera, a distance imaging camera, or a combination of two or more of these.

[0012] The "imaging device that captures electromagnetic waves other than visible light" may be, for example, a thermal imaging inspection device (also referred to as a "thermocamera" as appropriate) that measures infrared rays emitted from the object to be imaged, converts this into temperature, and displays the temperature distribution as an image.

[0013] A "multispectral camera" and a "hyperspectral camera" may be, for example, a camera that has two-dimensional information in the X and Y directions, as well as multiple spectral bands in the Z direction as wavelength information. A "multispectral camera" may be, for example, a camera capable of acquiring image data in multiple bands, with the number of bands being up to several tens. A "hyperspectral camera" may be, for example, a camera that can acquire image data in multiple bands, and that can acquire a number of bands that exceeds that of a multispectral camera.

[0014] The "distance imaging camera" may be, for example, a device that measures an object in an imaging space in real time using multiple pixels. The "distance imaging camera" may be, for example, a device that can obtain three-dimensional distance information to an object even in the dark without being affected by sunlight or shadows.

[0015] The "image capture target" may be, for example, a desired object. The "image capture target" may be, for example, a predetermined area in which the desired object is present. The "image capture target" may be, for example, one or more objects present in the predetermined area. The "predetermined area" may be determined according to the imaging range of the special imaging device. As an example, the following may be taken as the subject of imaging: The area to be monitored or objects present in the area to be monitored - Items that are located indoors, such as in a factory, a room, or a car - Using the characteristics that can be photographed by special imaging equipment, the object to be inspected -Items that can utilize characteristics derived from images captured by special imaging devices

[0016] The "image data having characteristics different from those of a visible light image" may be, for example, an image of an object captured by the special imaging device described above. The "image of an object captured" may have, for example, two or more pixels. The "image data having characteristics different from those of a visible light image" may be, for example, data that represents, for each pixel, characteristics different from those of a visible light image for two or more pixels that make up an image of an object captured using the special imaging device described above.

[0017] The "input information" may be, for example, information based on image data having characteristics different from those of a visible light image. The "information based on image data having characteristics different from those of a visible light image" may be, for example, "image data having characteristics different from those of a visible light image" obtained by capturing an image of an object using the special imaging device described above. As an example, when the special imaging device is a thermal imaging inspection device, the information may be an image showing the temperature distribution described above. The "information based on image data having characteristics different from those of visible light images" may be, for example, information obtained by processing "image data having characteristics different from those of visible light images" captured by the special imaging device described above of an imaging target. As an example, when the special imaging device is a thermal imaging inspection device, the information may be information obtained by measuring infrared rays emitted from an imaging target, converting the measured infrared rays into temperature values, and displaying the converted values ​​for each pixel.

[0018] The "input information" may be, for example, an image of the entire object captured at once by a special imaging device. The "input information" may be, for example, a plurality of images captured by a special imaging device after dividing the imaging target into a plurality of parts. As an example, if the viewing angle of the special imaging device is narrow compared to the entire imaging target, the "input information" may be images captured by the special imaging device after dividing the imaging target into parts and capturing them in order.

[0019] The "input information" may be input to the control means periodically or at a predetermined timing. For example, when monitoring an object to be imaged, the input information may be input to the control means at preset intervals. The "input information" may be input to the control means, for example, when requested by a user. For example, when a user wants to inspect an object to be imaged using the characteristics that the special imaging device can capture, or when a user wants to use an image of an object captured by the special imaging device, the input information may be input to the control means based on a user's instruction.

[0020] The "predetermined processing" may be, for example, acquiring characteristics of the imaging target that differ from those of the visible light image from the input information. The "predetermined processing" may be, for example, at least one of the following processes using characteristics different from those of a visible light image. - Visualizing characteristics different from visible light images - Making at least a part of the image more visible (e.g., coloring, adding graphics or text) Combining a visible light image of an object with at least a portion of the characteristics that differ from the visible light image. -Detecting abnormalities in the imaging subject Detecting abnormalities in the imaged object and displaying the detected abnormalities in a visible light image of the imaged object - Providing information specific to the imaged subject using characteristics different from those of visible light images (e.g., the code described below) Comparing the information specific to the imaged object with other information (e.g., matching codes) and associating the information specific to the imaged object with other information.

[0021] (2) The input information is information based on image data having characteristics different from those of the visible light image, and the control means receives the visible light image of the object to be imaged, and the predetermined processing generates an image that visualizes the input information and generates processing information that combines at least a portion of the visualized image with the visible light image by matching the position of the object to be imaged.

[0022] In this way, it is possible to provide information that combines an image that visualizes characteristics different from the visible light image with the visible light image.

[0023] "An image visualizing the input information is generated, and the visualized image" (hereinafter also referred to as "an image visualizing the input information") may be generated by using the input information. As an example, characteristics different from those of the visible light image may be expressed by an image using color coding, figures, numbers, characters, etc.

[0024] "At least a portion of the image in which the input information is visualized" may be, for example, the entire image in which the input information is visualized. The "at least a portion of the image in which the input information is visualized" may be, for example, a portion of the image in which the input information is visualized that satisfies a predetermined condition. The "at least a portion of the image in which the input information is visualized" may be, for example, an image in which the input information is visualized that corresponds to a predetermined object. The "image in which the input information is visualized" may be, for example, an image in which a predetermined area is extracted from the image in which the input information is visualized. Here, the above-mentioned predetermined condition, predetermined object, or predetermined area may be stored in advance in a storage area that can be referenced by the control means, for example.

[0025] "Matching the position of the imaged object" may, for example, be achieved by matching the position of the imaged object between the visible light image and the image visualized from the input information using any object present in the imaged object. "Matching the position of the imaged object" may, for example, be achieved by matching the positions of the imaged object between the visible light image and the image visualized from the input information, by associating each pixel constituting the visible light image of the imaged object with the input information. The correspondence between each pixel constituting the visible light image and the input information may, for example, be acquired in advance, and the acquired information may be stored in a storage area accessible by the control means.

[0026] The "visible light image of the imaging target" may be, for example, an image of the imaging target captured by a visible light imaging device.

[0027] "The control means receives the visible light image of the imaging target" may, for example, mean inputting an image of the imaging target captured by a visible light imaging device to the control means. As one example, the control means may receive the input of the visible light image periodically or at a predetermined timing. As another example, the control means may acquire the visible light image at a desired timing. The phrase "the control means receives the visible light image of the imaging target" may refer to, for example, inputting the above-mentioned visible light image to the control means in advance, and the control means storing the visible light image. As an example, when the imaging target does not change, the control means may store the visible light image captured in advance.

[0028] The processing information that combines at least a portion of an image that visualizes input information with a visible light image may be, for example, processing information created by one of the following processes or by combining multiple processes: -Uniformly superimposing the image that visualizes the input information and the visible light image - Semi-transparently overlaying an image that visualizes input information with a visible light image - Representing the parts of the input information that meet certain conditions, which have characteristics different from the visible light image, in the visible light image. - Representing characteristics of the imaged object that differ from the visible light image of a preset object in the visible light image. - Representing at least a part of the characteristics obtained from the input information that are different from the visible light image by color coding. - Expressing at least a part of the characteristics obtained from the input information that are different from the visible light image using characters, figures, coloring, or a combination of these.

[0029] For example, the input information may be temperature information obtained by measuring infrared rays emitted from the image capture target and converting them into temperature, the control means may receive a visible light image of the image capture target, and the predetermined processing may generate processed information that combines at least a portion of the temperature information with the visible light image by aligning the position of the image capture target. In this way, it is possible to provide information that combines the visible light image and the temperature information.

[0030] The "temperature information" may be, for example, the image of the temperature distribution described above (hereinafter also referred to as "thermography" or "thermoimage" as appropriate). The temperature distribution may be, for example, an image in which the temperature of the imaged object is represented using multiple colors. The "temperature information" may be, for example, a numerical value representing the temperature distribution of the image capture target. The "temperature information" may be, for example, information that enables acquisition of an image of temperature distribution that represents the temperature of the imaging target using multiple colors.

[0031] (3) The control means may use the input information to obtain a characteristic image of the object to be imaged, which represents characteristics different from those of the visible light image using multiple colors, and the processing information may be an image in which at least a portion of the characteristic image is superimposed on the visible light image.

[0032] In this way, it is possible to superimpose a characteristic different from that of the visible light image of the captured object on the visible light image and display it.

[0033] The control means may, for example, receive as input information an image in which a characteristic of the imaged object that is different from the visible light image is expressed in multiple colors. The control means may, for example, receive as input information a numerical value representing a characteristic of the imaged object that is different from the visible light image, and use the numerical value representing the characteristic that is different from the visible light image to create at least a portion of the image in which the characteristic that is different from the visible light image of the imaged object is expressed in multiple colors. The numerical value representing the characteristic that is different from the visible light image may, for example, be associated with information indicating the position of the imaged object. The numerical value representing the characteristic that is different from the visible light image may, for example, be associated with two or more pixels constituting the imaged object.

[0034] For example, when the input information is the aforementioned temperature information, the control means may use the temperature information to acquire a temperature distribution that represents the temperature of the imaged object using multiple colors, and the processing information may be an image in which at least a portion of the temperature distribution is superimposed on the visible light image. In this way, the temperature characteristics of the imaged object can be superimposed on the visible light image. The "processing information" may, for example, superimpose a portion of the temperature distribution that satisfies a predetermined condition on the visible light image of the imaged object. The predetermined condition may, for example, be a portion where the temperature exceeds a threshold value, a portion where the temperature is below a threshold value, a portion where the temperature exceeds a predetermined range, a portion where the temperature difference per unit time exceeds a threshold value, a pre-specified object or area, or a combination thereof. The "processing information" may, for example, add information indicating a warning to a portion of the temperature distribution that satisfies a predetermined condition. The warning information may, for example, be represented by a specific color, a graphic, a character, or a combination thereof.

[0035] (4) The characteristic image may have a lower resolution than the visible light image, and the processing information may be an image in which one pixel of the characteristic image corresponds to multiple pixels of the visible light image.

[0036] This makes it easier to associate the characteristic image with the visible light image, and also makes it easier to visually recognize the shape of an object that is unclear in the characteristic image.

[0037] The visible light image may have, for example, 1080×1920 pixels, and the characteristic image may have, for example, 60×80 pixels.

[0038] The processing information may, for example, indicate in a different color at least a portion of the pixels forming the outer edge of a region of pixels in the characteristic image that satisfies a predetermined condition. In this way, the outer edge of the temperature distribution can be clearly defined, and the outline of the temperature range of interest can be clearly defined.

[0039] The "predetermined condition" may be, for example, a range of characteristics different from those of the visible light image that are displayed in the same color. The "predetermined condition" may be, for example, a range (either an upper limit and a lower limit, or an upper limit and a lower limit) of a characteristic that is different from the visible light image and that you want to focus on. The "predetermined condition" may be, for example, a threshold value set for a characteristic where an abnormality is expected to occur. As an example, the "predetermined condition" may be set to, for example, a temperature range of interest or a temperature at which an abnormality is expected to occur.

[0040] "At least a portion of a pixel" may be, for example, the entire pixel. The "at least a portion of the pixel" may be, for example, the frame of the pixel. The "at least a portion of the pixel" may be, for example, a side that contacts a pixel of another temperature.

[0041] For example, when the input information is the temperature information described above, the characteristic image may be the temperature distribution described above. This makes it easier to visually recognize the shape of an object that is unclear from the temperature distribution. Also, the visible light image and the temperature distribution can be easily associated. This allows for a wider range of applications for low-resolution temperature distributions.

[0042] (5) The control means may hold display information from the input information that identifies an object to be incorporated into the visible light image, obtain characteristics of the object that are different from the visible light image from the input information, and the processing information may be an image in which the characteristics that are different from the visible light image of the object are added to the visible light image.

[0043] In this way, it is possible to display a specific object among the captured images in a way that makes it easy to focus attention on it.

[0044] The "display information" may be set to, for example, an object that is present in the image capture target and whose characteristics can be acquired by the special image capture device and that is to be monitored.

[0045] As an example, when the input information is the temperature information described above, the control means may hold display information from the temperature information that sets the object to be incorporated into the visible light image, obtain the temperature of the object from the temperature information, and the processing information may be an image in which the temperature of the object is added to the visible light image.

[0046] The processing information may include, for example, the temperature of the object in at least one of the following ways: - Associating multiple temperature ranges with multiple colors and expressing temperatures with colors Adding temperature values · Expressing multiple temperature ranges in letters The target object may be, for example, an object that emits heat, such as a factory device or an electrical appliance in a room, or an object that passes through a predetermined location.

[0047] (6) The control means may hold a first condition relating to a characteristic different from that of the visible light image, and the predetermined processing may use the input information to extract a portion of the imaged subject that satisfies the first condition, and add information representing the extracted portion to the visible light image.

[0048] In this way, it is possible to provide a display that makes it easy to focus on the part of the image capture target that satisfies the first condition.

[0049] The "first condition" may be, for example, that an upper limit value, a lower limit value, or both an upper limit value and a lower limit value are satisfied with respect to a characteristic different from that of the visible light image. The "first condition" may be stored in a storage area that can be referenced by the control means, for example.

[0050] For example, when the input information is the temperature information described above, the control means may hold the temperature condition as a first condition, and the predetermined process may use the temperature information to extract temperature portions of the imaged object that satisfy the first condition, and add information representing the extracted temperature portions to the visible light image. In this way, a specific temperature distribution of the imaged object can be detected, and a display that makes it easy to focus on the detected temperature distribution can be provided. The "information representing the temperature portions" may be represented, for example, by color, text, graphics, or a blinking display.

[0051] (7) The first condition may be a plurality of conditions set in association with a plurality of regions of the imaging target, and the predetermined processing may use the input information to extract, for each of the plurality of regions, a portion that satisfies the condition associated with each region.

[0052] In this way, it is possible to provide a display that makes it easy to focus on characteristics different from those of a visible light image, using conditions according to each region.

[0053] For example, when the input information is the temperature information described above, the first condition may be a plurality of temperature conditions set in association with a plurality of regions of the image capture target, and the predetermined process may use the temperature information to extract, for each of the plurality of regions, temperature portions that satisfy the temperature conditions associated with each region, and add information representing the extracted temperature portions to the visible light image. In this way, a temperature rise can be detected using the temperature conditions associated with each region.

[0054] (8) The control means may hold a second condition relating to a characteristic different from that of the visible light image, and the predetermined processing may extract portions of the input information that satisfy the second condition and count the number of the extracted portions.

[0055] In this way, when the object has a different characteristic from the other areas in terms of the characteristics different from the visible light image, the number of objects can be easily counted.

[0056] The "second condition" may be, for example, one of satisfying an upper limit, satisfying a lower limit, or satisfying both an upper limit and a lower limit for a characteristic different from the visible light image. The second condition may be the same as the first condition. The "second condition" may be stored, for example, in a memory area accessible by the control means.

[0057] For example, when the input information is the aforementioned temperature information, the control means may store a temperature threshold as the second condition, and the predetermined process may extract over-temperature portions exceeding the temperature threshold from the temperature information and count the number of over-temperature portions. In this way, the number of objects can be easily counted when the objects have a temperature different from that of other areas. For example, the number of people present in a closed space (e.g., a factory, a room, or a vehicle such as a bus) can be counted. This makes it possible to detect, for example, the number of people present in a room or a new intruder. The "temperature threshold" may be set, for example, to the minimum temperature of the object to be detected among the objects present in the imaging area.

[0058] (9) The control means may hold a third condition relating to characteristics different from the visible light image and a display method for representing the portion that satisfies the third condition, and the predetermined processing may extract the portion that satisfies the third condition from the input information and generate processing information that represents the extracted portion using the display method.

[0059] In this way, it is possible to provide a display that makes it easy to focus on the part that satisfies the third condition among the characteristics that differ from the visible light image.

[0060] The "third condition" may be, for example, any one of exceeding an upper limit, being less than a lower limit, and satisfying both an upper limit and a lower limit with respect to a characteristic different from the visible light image, or a combination thereof. As an example, a temperature threshold for monitoring the imaged object, a temperature threshold for detecting an abnormality in the imaged object, or the like may be set as the third condition. The third condition may be the same as the first or second condition described above. The "third condition" may be stored in a storage area that can be referenced by the control means, for example.

[0061] The "display method" may be, for example, color coding, adding characters, adding graphics, or a combination of these. The "processing information" may be, for example, an image that warns the user when an event that satisfies the third condition occurs. (10) The display method is a method for generating an image that visualizes the portion that satisfies the third condition, and it is preferable that at least a portion of the pixels that form the outer edge of the extracted portion be displayed in a different color.

[0062] In this way, it is possible to clearly identify the outline of an object that has characteristics different from those of the visible light image that one is interested in.

[0063] The "at least a portion of the pixels" may be the same as (4) above.

[0064] (11) The object to be imaged may have areas where the measured values ​​of infrared rays are different, the input information may be the amount of infrared rays measured from the object to be imaged, and the predetermined processing may generate processing information related to the amount of infrared rays.

[0065] In this way, it becomes possible to acquire and use the amount of infrared light for an object to be imaged that has regions where the measured values ​​of infrared light are different.

[0066] The "areas where the measured values ​​of infrared rays differ" may be areas where the measured amount of infrared rays differs when infrared rays of an object to be imaged are measured using a special imaging device capable of measuring infrared rays, for example. The image capturing target may have areas where the measured infrared light values ​​are different so as to form a predefined code. The "predefined code" may be, for example, a one-dimensional code. The "predefined code" may be, for example, a two-dimensional code. The "predefined code" may be, for example, a QR code (registered trademark).

[0067] The imaging target may have regions with different thermal transmittances or different thermal reflectances, which can result in regions with different amounts of infrared light when irradiated with far-infrared rays. The special imaging device may include, for example, an imaging unit that measures the amount of infrared light from an object to be imaged. The imaging unit may include, for example, a plurality of imaging elements.

[0068] The "processing information" may be, for example, information representing areas where the measured amount of infrared light differs. The "processing information" may be, for example, the previously described predefined code using areas where the amount of infrared light differs. The "processing information" may be, for example, a pattern representing areas where the amount of infrared light differs. The "processing information" may be, for example, the result of comparing information representing areas where the amount of infrared light differs with other information. The control means may, for example, generate the processing information using an error-corrected value for the measured amount of infrared light.

[0069] (12) The control means may hold verification information for verifying the processing information, and the predetermined process may further verify the processing information using the verification information.

[0070] In this way, the result of matching the matching information with the processing information can be obtained.

[0071] The "verification information" may be, for example, a drawing, a mathematical formula, a character string, or a combination of these. The mathematical formula may be, for example, a hash function or a one-way hash function. The control means may, for example, output a result of matching the matching information with the processing information. The control means may, for example, output a warning sound based on the result of matching the matching information with the processing information. The control means may, for example, display the result of matching the matching information with the processing information on a display screen. The control means may, for example, output a comparison result of comparing the matching information with the processing information.

[0072] (13) The area where the measured values ​​of infrared rays are different is an area where at least one of the thermal transmittance and the thermal reflectance is different, and the input information may be the amount of infrared rays measured when far infrared rays are irradiated onto the object to be imaged.

[0073] In this way, it is possible to create a difference in the amount of infrared rays using at least one of the thermal transmittance and the thermal reflectance.

[0074] The "regions with different thermal transmittances or different thermal reflectances" may be formed, for example, by combining materials with different thermal transmittances or different thermal reflectances. The "regions with different thermal transmittances or different thermal reflectances" may be formed, for example, by combining regions with different thermal transmittances or different thermal reflectances so as to form a predefined code.

[0075] (14) The special imaging device may include an irradiation means for irradiating the imaging target with far-infrared rays.

[0076] In this way, it is possible to measure the amount of infrared rays in an area where at least one of the thermal transmittance and the thermal reflectance is different.

[0077] When the imaging target is a combination of materials with different thermal reflectivities, the "irradiation means" is preferably arranged on the same side as the imaging element of the special imaging device. When the imaging target is a combination of materials with different thermal transmittances, the "irradiation means" is preferably arranged opposite the imaging element of the special imaging device, with the imaging target sandwiched between them.

[0078] (15) The special imaging device may be capable of capturing an image of the entire imaging target.

[0079] In this way, it becomes possible to capture an image of the object in one go. The special imaging device may be, for example, capable of capturing an image of the entire plane of the imaging target. The special imaging device may be, for example, equipped with an imaging means capable of capturing an image of the entire imaging target. The special imaging device may be, for example, equipped with an imaging means large enough to capture the one-dimensional code described above.

[0080] (16) The special imaging device may be capable of capturing an image of a partial area of ​​the imaging target, and may divide the imaging target into multiple parts and capture the entire image, and the input information may be information based on image data having characteristics different from the visible light image captured by the special imaging device in multiple parts.

[0081] In this way, even if the imaging target is large compared to the size that can be imaged by the special imaging device, the entire imaging target can be imaged.

[0082] The imaging means may be sized to capture, for example, the length of one side of the two-dimensional code described above. The imaging means may be sized to capture, for example, the length of the diameter of the circumscribing circle of the imaging target. The imaging means may be sized to capture, for example, a portion of the imaging target.

[0083] The special-purpose imaging device may include, for example, an imaging control means for controlling the imaging means. The imaging control means may, for example, control the imaging means to rotate around an axis at the center of the imaging means to capture an image of the entire imaging target. For example, when the imaging target is rectangular, the imaging control means may control the imaging means to move from one side of the imaging target to the opposite side to capture an image of the imaging target.

[0084] (17) In the systems (1) to (16) described above, the special imaging device may be at least one of a thermal imaging inspection device, a multispectral camera, a hyperspectral camera, or a distance imaging camera.

[0085] In this way, a system can be configured that performs predetermined processing according to the special imaging device.

[0086] (18) The systems described above in (1) to (17) may further include the special imaging device.

[0087] In this way, a system can be configured in which the control means acquires input information of the imaging target from the special imaging device, which facilitates consistency of information input and output between the control means and the special imaging device.

[0088] (19) Another aspect of the present invention may be a program that causes a computer to execute the functions of the control means included in the system described in any one of (1) to (18) above.

[0089] The inventions described in (1) to (19) above can be combined in any way. For example, a configuration may be created by adding at least a portion of the configuration of at least one of the inventions described in (2) and subsequent items to all or a portion of the configuration of the invention described in (1). In particular, an invention may be created by adding at least a portion of the configuration of at least one of the inventions described in (2) and subsequent items to the invention described in (1). Furthermore, any configuration may be extracted from the inventions described in (1) to (19) and combined. The applicant of this application intends to obtain rights to inventions including these configurations. Furthermore, even if a description is made of "in the case of..." or "when...," it is not intended to describe a configuration that is limited to that case or time. These are merely examples of better configurations, and the applicant intends to obtain rights to configurations that are not in these cases or times. Furthermore, any descriptions that specify an order are not limited to this order. Configurations in which some parts are deleted or the order is changed are also disclosed, and the applicant intends to obtain rights to them. [Effects of the Invention]

[0090] According to the present invention, it is possible to achieve something better than the prior art, such as utilizing information captured by an imaging device capable of acquiring image data having characteristics different from those of visible light images.

[0091] The effects of the present invention are not limited to these, and the effects achieved by the components disclosed in the specification and drawings, etc. are also disclosed, and the applicant intends to obtain rights to the components that achieve these effects through divisional applications, amendments, etc. For example, in this specification, a phrase such as "can..." clearly states the effect achieved, and there are parts that demonstrate the effect even without the phrase "can...". Furthermore, there are effects that can be understood from the configuration even without such a phrase. [Brief explanation of the drawings]

[0092] [Figure 1] FIG. 2 is a diagram illustrating an example of the configuration of a control means according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of a system according to a first embodiment. [Figure 3] 1 is a schematic diagram illustrating an example of the appearance of an imaging device. [Figure 4] FIG. 1 is a block diagram illustrating an example of the configuration of an imaging device. [Figure 5] 10A and 10B are diagrams illustrating an example of processing information when a high temperature abnormality occurs in an imaging target. [Figure 6] FIG. 2 is a diagram illustrating another example of the system according to the first embodiment. [Figure 7] 1A and 1B are diagrams illustrating pixels that form a visible light image and a temperature distribution of an image capture target. [Figure 8] FIG. 1 is a diagram illustrating an example in which the system of the first embodiment is applied to illegal intrusion monitoring. [Figure 9] 1A and 1B are diagrams illustrating an example of applying the system of embodiment 1 to monitoring inside a room, in which (A) is a diagram illustrating an example of a display screen when selecting a target to monitor, and (B) is a diagram illustrating an example of a display screen during monitoring. [Figure 10] FIG. 10 is a diagram illustrating a display example in which the temperature of a room being monitored is displayed in real time. [Figure 11] FIG. 10 is a diagram illustrating an example of a display in which the temperature of a room being monitored is displayed as a numerical table. [Figure 12] FIG. 10 is a diagram illustrating a display example in which the temperature of a room being monitored is displayed in a graph. [Figure 13] FIG. 10 is a diagram illustrating an example of a thermal code according to a second embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of using a thermal code for card authentication. [Figure 15] 1A and 1B are diagrams illustrating an example of using a thermal code as a key, where (A) is a plan view and (B) is a front view. [Figure 16] FIG. 1 is a diagram illustrating an example of a thermal code reader that reads a heat-reflective thermal code. [Figure 17] FIG. 1 is a diagram illustrating an example of a thermal code reader that reads a heat-transmitting thermal code. [Figure 18] FIG. 10 is a diagram illustrating an example of reading a circular thermal code with a one-dimensional thermal camera. [Figure 19] FIG. 10 is a diagram illustrating an example of reading a rectangular thermal code with a one-dimensional thermal camera. [Figure 20] 1A and 1B are diagrams showing an example of the configuration of a power generation means for supplying power to a thermal authentication system, in which (A) is the initial state, (B) is forward power generation, and (C) is a diagram explaining power generation by a power-generating mainspring. [Figure 21] FIG. 2 is a diagram illustrating an example of the configuration of a system mainspring. [Figure 22] FIG. 2 is a diagram illustrating an example of the configuration of a power generation circuit. [Figure 23] 1 is a flowchart illustrating an example of the operation of a power-supply type thermal authentication system. [Figure 24] 10 is a flowchart illustrating an example of the operation of a power-generating thermal authentication system. DETAILED DESCRIPTION OF THE INVENTION

[0093] Hereinafter, one embodiment of the present invention will be described with reference to the drawings. For clarity of description, the following description and drawings have been omitted or simplified as appropriate. In each drawing, components and corresponding parts having the same configuration or function are designated by the same reference numerals, and their description will be omitted.

[0094] One embodiment of the system of the present invention includes a control unit that receives input information obtained by capturing an image of an object using a special imaging device capable of acquiring image data having characteristics different from those of visible light images, and performs predetermined processing using the input information, thereby making it possible to utilize information captured by the imaging device capable of acquiring image data having characteristics different from those of visible light images.

[0095] FIG. 1 is a diagram illustrating an example of the configuration of a control unit included in a system according to an embodiment. The control means 200 includes an input unit 201 , a processing unit 202 , an output unit 203 , and a holding unit 204 .

[0096] The input unit 201 receives input information. The input information may be, for example, the information described in the Summary of the Invention or the information described later in each embodiment. The input unit 201 may receive the input information from an output destination that outputs the input information via a signal line, or may receive the input information via a communication means. The communication means may be either wired or wireless. The input unit 201 may include, for example, an interface or communication means for receiving information. The input unit 201 may receive, for example, input of information other than the input information. For example, the input unit 201 may receive, in addition to the input information, an input of a visible light image of an imaging target.

[0097] The processing unit 202 performs the predetermined processing described above using the input information to generate processing information. The predetermined processing may be, for example, the processing described in the Summary of the Invention or the processing described later in each embodiment.

[0098] The output unit 203 outputs the processing information generated by the processing unit 202. The output unit 203 outputs the processing information to, for example, a display means that displays the processing information. The output unit 203 outputs, for example, a warning sound as the processing information. The output unit 203 instructs other devices to perform processing based on, for example, the processing information. The processing instructed to other devices may be, for example, displaying the processing information, outputting a warning sound or the like based on the processing information, or a combination of these.

[0099] The storage unit 204 stores preset information. The stored information may be the above-mentioned first condition, second condition, third condition, predetermined condition, predetermined object, predetermined area, etc. The storage unit 204 may be a storage area that can be referenced by the control means.

[0100] The control means 200 may be realized, for example, in part or in whole, by a program, which causes a computer to execute the functions of the control means. The computer includes, for example, a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), and a hard disk drive (HDD).

[0101] The CPU executes various programs stored in the ROM etc., and controls the entire system based on arithmetic processing and control programs. In one embodiment, the CPU executes a group of program instructions that realize the functions of the control means. RAM is a volatile storage medium for reading and writing information at high speed, and functions as a work area when the CPU executes programs. The ROM is a read-only non-volatile storage medium in which programs and the like are stored. The HDD is a storage medium that stores programs executed by the CPU. The ROM or HDD stores threshold values ​​used in one embodiment, predetermined information that is set in advance, other fixed data, etc. The ROM or HDD may be provided with a storage area for the storage unit 204.

[0102] The control means may be realized, for example, in part or in whole, by hardware, or by a combination of hardware and software, such as an embedded system, or by any one of hardware, firmware, and software, or a combination of two or more of these.

[0103] Embodiment 1. In the first embodiment, a thermal imaging inspection device is used as the special imaging device, and temperature information obtained by measuring infrared rays emitted from an object to be imaged and converting the measured temperature into temperature information is input to a control unit. In this embodiment, an example is described in which the control unit generates processing information using the temperature information of the object to be imaged.

[0104] Current surveillance cameras, for example, can capture the shape of objects by capturing visible light, but cannot measure temperature. Therefore, even if one or more parts of the object being imaged are abnormally hot, they cannot detect the abnormality until a fire breaks out or significant deformation occurs. Furthermore, these cameras require lighting when shooting in the dark, making it difficult to capture a wide area. Furthermore, even when using high-intensity lights to illuminate distant objects, various problems arise, such as the fact that areas close to the light source are bright, making distant areas dark due to exposure issues.

[0105] Therefore, in one aspect of this embodiment, the control means generates processing information that combines a visible light image of the image capture target with temperature information. As a predetermined process, the control means receives, for example, input information and the visible light image of the image capture target, and generates processing information that combines at least a portion of the temperature information with the visible light image by matching the position of the image capture target. The temperature information is as described above.

[0106] The processed information that combines at least a portion of the temperature information with the visible light image may be, for example, processed information created by one of the following processes or a combination of multiple processes. - Uniformly superimposing temperature distribution images and visible light images - Semi-transparent overlay of temperature distribution image and visible light image - Displaying the parts of the temperature distribution image that meet certain conditions in a visible light image - Displaying the temperature of pre-set objects in a visible light image - At least a part of the temperature distribution image is displayed in color. - At least a part of the temperature distribution image is displayed using text, graphics, coloring, or a combination of these.

[0107] In another aspect of this embodiment, the control unit generates processing information using temperature information as the predetermined processing, without using a visible light image, for example.

[0108] Next, an example in which one aspect of the system of this embodiment is applied will be described. <System example 1> FIG. 2 is a schematic diagram illustrating an example of a factory monitoring system. The monitoring system 1 in FIG. 2 includes a machine tool 2, a terminal device 3, and an imaging device 100.

[0109] <Imaging device> The imaging device 100 of this embodiment is an example of a monitoring device that monitors the work content of a machine tool 2 operating on a factory production line, and records images and temperatures of the machine tool 2. The imaging device 100 is temporarily installed beside the machine tool 2 on a tripod.

[0110] The machine tool 2 to be monitored performs screw tightening work on a production line. The machine tool 2 repeatedly performs this screw tightening work at a predetermined work cycle (for example, every 30 seconds).

[0111] The imaging device 100 captures images of the machine tool 2, for example, periodically. Video recorded by the imaging device 100 is played back using a dedicated browser, which is display / viewing software that runs on the terminal device 3. The dedicated browser is a program executed by the CPU of the terminal device 3, and is installed on the terminal device 3 in an administrative department that is separate from the factory where the imaging device 100 is installed. In addition to playing back videos recorded by the imaging device 100, the dedicated browser can also be used to set various settings such as frame rate, image quality, recording method, and whether or not overwriting is possible, and to format the memory card 190 for the imaging device 100.

[0112] Fig. 3 is a schematic diagram illustrating an example of the appearance of an imaging device, and Fig. 4 is a block diagram illustrating an example of the configuration of an imaging device, shown together with a memory card. The imaging device 100 includes a controller 110, a first imaging unit 120, a second imaging unit 130, an event input terminal 140, a memory card slot 160, and an image signal output terminal 180 inside a substantially cylindrical housing 105. Screw holes (not shown) for attaching the imaging device to a tripod head or a shooting location are provided in part of the cylindrical side surface of housing 105. First imaging unit 120 and second imaging unit 130 are arranged inside housing 105 with the screw holes facing downward.

[0113] The control controller 110 is configured to include a microcomputer equipped with a CPU, main memory such as ROM and RAM, a timer, etc. (not shown). The ROM of the control controller 110 stores various programs such as an image processing program for saving images captured by the first imaging unit 120 and the second imaging unit 130 in the main memory or a memory card 190 inserted in the memory card slot 160, and an OS (Operating System). The CPU of the control controller 110 performs various controls in accordance with these control programs. The control controller 110 also detects information indicating the occurrence of an event that is input to an event input terminal 140, and executes control in accordance with the detected information.

[0114] The first imaging unit 120 functions as a visible light imaging device that captures a visible light image of the imaging target. The second imaging unit 130 functions as a thermal imaging device that measures infrared rays of the imaging target and converts them into temperature to obtain an image of the temperature distribution. The first imaging unit 120 and the second imaging unit 130 are provided at one end of the longitudinal direction of a substantially cylindrical housing 105, with the longitudinal direction of the housing 105 as the optical axis.

[0115] The first imaging unit 120 is composed of a lens unit 122 and an imaging element 124. The lens unit 122 has a zoom lens and can set the angle of view depending on the size of the subject and the distance to the subject. The imaging element 124 is a CCD, CMOS image sensor, or the like, and acquires the image of the subject formed by the lens unit 122 as a captured image. The second imaging unit 130 is composed of a lens unit 132 and an imaging element 134. The lens unit 132 is equipped with a zoom lens, and is capable of setting the angle of view according to the size of the subject to be photographed and the distance to the subject. The imaging element 134 is an infrared imaging element having an infrared sensitivity range, and acquires the image of the subject to be photographed formed by the lens unit 132 as a captured image.

[0116] The event input terminal 140 is a signal input terminal provided on the side surface of the housing 105. The event input terminal 140 receives a trigger signal, which notifies the occurrence of an event, from outside the imaging device 100. For example, the imaging device 100 may be configured to start imaging based on a trigger signal output from the terminal device 3. The event input terminal 140 may receive the trigger signal via, for example, a wired or wireless communication means. The trigger signal is transmitted to the controller 110 and detected as information indicating the occurrence of an event. Note that the imaging device 100 may not be configured to include the event input terminal 140.

[0117] The image signal output terminal 180 is a terminal that outputs an image signal for outputting images captured by the first imaging unit 120 and the second imaging unit 130 to the external terminal device 3.

[0118] The memory card slot 160 is provided at the end of the housing 105 opposite to the first imaging unit 120 and the second imaging unit 130. A memory card 190 serving as a storage medium is inserted into the memory card slot 160. Under the control of the controller 110, files storing images captured by the first imaging unit 120 and the second imaging unit 130 are recorded on the memory card 190.

[0119] <Terminal Device> A personal computer, for example, may be used as the terminal device 3. The terminal device 3 operates a control means. Here, the control means and the dedicated browser may be, for example, separate programs. The control means may be configured to control the functions of the dedicated browser. Furthermore, the control means may be configured to have the functions of the dedicated browser.

[0120] The terminal device 3 performs, for example, the following processing in order to monitor a factory (an example of an imaging target) using the system. (1) A dedicated browser displays a visible light image and a temperature distribution image on the same screen. (2) Set the range to be monitored among the imaging targets. Multiple ranges to be monitored can be set.

[0121] (3) Set the conditions for determining an abnormal state. For example, set the temperature at which an abnormal state is determined (upper or lower threshold, temperature range, etc.), how to handle multiple monitoring ranges, and what to do when an abnormal state occurs. When multiple monitoring ranges are to be handled, it is advisable to decide whether (a) an abnormal state processing should be performed when even one abnormality occurs, or (b) whether an abnormal state is determined to have occurred using the average value of multiple monitoring ranges. When an abnormal state occurs, for example, a warning may be displayed on the display screen of the terminal device 3, an alarm may be output, or a trigger signal may be output to a function that starts processing in response to the occurrence of an abnormal state.

[0122] (4) When an abnormality occurs, video of the abnormal state is recorded as an abnormal area. For example, a warning light may be controlled. It is desirable that the video of the abnormal area be recorded in a memory card or the like so that it can be easily read. (5) Provide a function to play back video of the abnormal range. In addition, by using the mouse to associate the locations to be monitored (such as installation locations) in the visible light image and temperature distribution image, and then highlighting the locations to be monitored in the visible light image when playing back the abnormal area, it becomes easy to check which equipment is abnormal in the visible light image. Association can be done by specifying a point or a range (such as a square, circle, or any shape).

[0123] Fig. 5 is a diagram illustrating an example of processing information when a temperature abnormality (for example, an abnormality exceeding a temperature threshold) occurs in the machine tool 2 shown in Fig. 2. Fig. 5 shows an example in which a temperature abnormality detected by temperature information is added to a visible light image, and shows an example of processing information in which the temperature abnormality is represented by a dotted circle and a warning is also displayed. As shown in Figure 5, by using the visible light image and the temperature distribution image, the control means can generate processing information to notify the user of the occurrence of an abnormal condition caused by a temperature rise that cannot be detected by the visible light image. Also, even if it is difficult to grasp an abnormal condition from the temperature distribution image alone due to low resolution of the temperature distribution image, it becomes easier to grasp the abnormal condition by combining it with the visible light image.

[0124] As described above, the factory monitoring system shown as an example in Figures 2 to 5 has been described as an example of a system that detects the occurrence of abnormalities that are not normal conditions on a production line. Figure 5 shows an example of a case where an abnormality has occurred in machine tool 2. However, there are many different types of abnormalities that can occur on a production line, such as line abnormalities, machine abnormalities, and the presence (intrusion) of foreign objects. Therefore, in order to detect various abnormalities, it is advisable to have the system learn images of normal conditions, for example. When detecting machine abnormalities, an image of the temperature distribution is acquired while the machine is in operation. When an abnormality occurs in the machine, such as heat generation due to friction or excessive power supply, an abnormality can be detected by the appearance of a heat-generating area that is not in a normal state in the temperature distribution image. In addition, when detecting the presence of a foreign object, it is advisable to acquire an image of the temperature distribution in the factory under normal conditions and detect the intrusion by detecting a temperature distribution (a heat-generating area due to the presence of a living organism) that does not exist in the image of the temperature distribution under normal conditions.

[0125] <System example 2> In System Example 1, an example of a system for monitoring a factory using a combination of visible light images and temperature distribution images was described, but a factory may also be monitored using only temperature distribution images. For example, when the resolution of image data captured by a thermal imaging device is high, it is advisable to monitor the object using temperature distribution images. For example, when the temperature distribution images enable the shape of the object to be determined and useful information to be obtained, it is advisable to monitor the object using temperature distribution images without using visible light images.

[0126] The monitoring system performs monitoring in the following manner, for example. The control means first sets a temperature threshold value for determining an abnormal state. Next, when the control means detects a portion whose temperature exceeds the temperature threshold value, it uses the temperature distribution image captured by the thermal imaging device to change the color of the temperature distribution of the portion or area determined to be in an abnormal state to a color that notifies the user of the abnormal state. The control means also outputs a trigger signal or generates an alarm sound to notify the user of the occurrence of an abnormal state. At this time, it is preferable to generate a trigger signal or the like to warn the user if any portion of the imaged object exceeds the temperature threshold value.

[0127] <System example 3> In this system example, we consider a case where the thermal imaging device has the ability to capture images at a wide viewing angle. A thermal imaging device with a wide viewing angle can capture images of a wide range of objects. However, the size of the objects present in the image capture area becomes smaller. Therefore, unless the thermal imaging device has high resolution and high image quality, it may be difficult to observe the objects. Therefore, when high image quality is desired for observing the object to be imaged, it is advisable to use, for example, a thermal imaging device that has a narrow viewing angle but high image quality, and make the device itself movable so that a wide range of the object can be imaged. Furthermore, an imaging device with a wide viewing angle is recommended for surveillance where wide-area observation is prioritized over high-resolution observation, such as when it is sufficient to detect areas that exceed a temperature threshold and no discrimination is required. Furthermore, depending on the surveillance application, a high quality thermal imaging device and a wide viewing angle thermal imaging device may be used in combination.

[0128] <System Example 4> Next, another example using a surveillance camera will be described. Fig. 6 is a diagram illustrating another example of the system of embodiment 1 (human / animal determination, temperature gradation). In this system example, an example is described in which the control means generates processing information for combining a visible light image and a thermographic image. In Fig. 6, the upper row shows a "visible light image" and a "thermal image," and the lower row shows a "visible light image + thermographic image" that is a combination of the visible light image and the thermographic image.

[0129] Figure 6 shows an example of a temperature distribution exceeding a predetermined temperature threshold (e.g., 30°C) superimposed on a visible light image. In the thermal image shown in Figure 6, the temperature range between 25°C and 43°C is displayed using multiple color gradations. The temperature gradation can be set by the temperature range, maximum value, minimum value, center temperature, span, etc. Referring to Figure 6, an example of human-animal discrimination between a small animal 31 and a human 42 in an image of "visible light image + thermal image" will be described. The small animal 31 is not wearing any clothing, so the entire temperature is captured. This results in a single temperature distribution. On the other hand, the human 32 has exposed skin. This results in a temperature distribution divided into multiple regions. Due to these differences, the small animal 31 and the human 32 can be easily distinguished from each other using their temperature distributions.

[0130] Here, the resolution of the visible light image and the temperature distribution image of the captured image will be described. The temperature distribution image is composed of multiple pixels, but generally has a lower resolution than the visible light image. Therefore, the control means generates, as processing information, an image in which one pixel of the temperature distribution image corresponds to multiple pixels of the visible light image. This makes it easier to associate the temperature distribution with the visible light image. Furthermore, it makes it easier to visually recognize the shape of an object that is unclear in the temperature distribution image.

[0131] Figure 7 illustrates the pixels that make up the visible light image and temperature distribution of the imaged object, where (A) is the visible light image and (B) is the temperature distribution image. (A) shows a schematic representation of a group of pixels corresponding to one pixel of the temperature distribution in the upper left corner, and (B) shows a schematic representation of one pixel. 7 schematically shows an example in which the visible light image has, for example, 1080 x 1920 pixels, and the temperature distribution has, for example, 60 x 80 pixels. Therefore, one pixel in the temperature distribution corresponds to 432 (18 x 24) pixels in the visible light image.

[0132] Since the image of the temperature distribution has low resolution, the temperature distribution can be made easier to understand by expressing the processing information as follows, for example. The processing information may, for example, indicate in a different color at least a portion of the pixels forming the outer edge of a region of pixels in the temperature distribution that meets a predetermined condition. At least a portion of a pixel may be, for example, the entire pixel, the pixel frame, the edge adjacent to a pixel of another temperature, or a combination of these. This makes it possible to clarify the outer edge of the temperature distribution. It also makes it easier to visually recognize the shape of an object that is unclear in a temperature distribution image. Furthermore, it makes it easy to associate a visible light image with a temperature distribution. These features can expand the range of applications for low-resolution temperature distribution images.

[0133] <System Example 5> FIG. 8 is a diagram illustrating an example in which the system of the first embodiment is applied to monitoring illegal intrusions. Figure 8 shows an example of monitoring for unauthorized entry and the like using an image of the temperature distribution in a room. The top row of Figure 8 is a visible light image of the room captured by a visible light imaging device, and the bottom row is a thermal image showing the temperature distribution captured by a thermal imaging device. The left side of Figure 8 is an example of an image under normal conditions, and the right side is an example of an image when an abnormality such as unauthorized entry occurs.

[0134] When it is bright, the visible light image makes it possible to visually identify an intruder 44 or a small animal 45 that has entered the vehicle, but when it gets dark, the intruder or small animal 45 disappears from the image. In the thermographic image, under normal circumstances, the temperatures of a fluorescent light 41, a fax machine 42, and a personal computer (server) 43 are higher than those of other areas. If an intruder 44 or a small animal 45 enters such a room, they will appear in the temperature distribution as high-temperature objects that are not normally present. Therefore, the thermal imaging device can be used as a surveillance camera to monitor the inside of a room at night.

[0135] For example, if there is a heat-generating object separated into multiple regions, it will be recognized as a human, as described above. Therefore, if a heat-generating object recognized as a human appears during security hours, it can be determined to be an intruder 44. Also, if there is a heat-generating object moving alone in one region, it can be determined to be a small animal 45. In the example of the room shown in FIG. 8, when an area where a temperature exceeds a preset value is detected, it can be determined that an abnormality has occurred.

[0136] In this way, the thermal imaging device can be used as a surveillance camera. For example, if the object being monitored is one that exceeds a certain temperature, it is possible to monitor it using only the thermal image without using visible light images. In addition, when a visible light imaging device and a thermal imaging device are used in combination as a surveillance camera, it becomes possible to monitor by combining visible light images and thermal images. Furthermore, if the object to be monitored does not change, it is also possible to monitor by combining visible light images captured in advance with thermal images captured at predetermined intervals.

[0137] <System Example 6> In this system example, one mode will be described in which one or more monitoring targets are set in advance among the imaging targets, and the set monitoring targets are monitored using a temperature threshold value. 9 to 12 are diagrams illustrating an example in which the system of the first embodiment is applied to monitoring a room. Here, the monitoring target may be, for example, a part of the area of ​​the imaging target, or an object present in the imaging target. In FIGS. 9 to 12, an example in which an electrical appliance present in the imaging target is the monitoring target is described.

[0138] The control unit stores display information that identifies an object to be incorporated into a visible light image from among the temperature information. For example, in the configuration example of FIG. 1, the control unit 200 may store the display information in advance in the storage unit 204. The control means, for example, acquires the temperature of the object from the temperature information and generates an image in which the temperature of the object is added to the visible light image as processed information. In this way, a display that makes it easy to focus on a pre-specified object can be achieved.

[0139] The display information may be, for example, set to indicate an object whose temperature is to be monitored among objects present in the image capture target. For example, the control means selects an object to be monitored (hereinafter also referred to as "target") from among the electrical appliances present in the image capture target. At this time, the control means may display an operation screen on the terminal device or the like that allows the user to select the object. In Fig. 9, (A) is a diagram illustrating an example of a display screen when a target to be monitored is selected, and (B) is a diagram illustrating an example of a display screen during monitoring. In Fig. 9, TGn (n is a positive integer) represents a target selected by the user. The user selects a target to be monitored and sets the display method, etc., using the following procedure, for example.

[0140] (1) Select the target whose temperature you want to measure (TG1~6). Here, the following selected examples are described: Working state or standby state represents the state of the appliance. TG1: Refrigerator (working) TG2: Microwave (standby) TG3: Electric kettle (standby) TG4: Rice cooker (standby) TG5: Ventilation fan (operating) TG6: TV (standby) In addition, even if there is a temperature change in an object other than the target, that point will automatically be set as the target and the temperature will be displayed. For example, Figure 9(B) shows an example of a state in which a temperature rise near a window due to sunlight has been detected in the imaged object.

[0141] (2) Temperature and time settings Sampling time Measurement period Measurement temperature range Temperature Limit Alarm (display, sound, target stop command, etc.) The temperature limit sets the temperature that separates a normal state from an abnormal state, such as a temperature threshold that is the upper or lower limit of the temperature, or an appropriate temperature range.

[0142] (3) Select the display method for the temperature measurement data -Display of temperature distribution added to visible light image Real-time display (current value of each target, temperature of any specified location) -Table-based numerical display Graph display In the display method, the temperature and time range can be set arbitrarily.

[0143] The control means may, for example, display a screen prompting the user to perform the above-mentioned operations (1) to (3) and receive the process selected by the user. The control means may also, for example, determine the monitoring conditions, display method, etc. based on the received information. For example, in the configuration example of FIG. 1, the control means 200 may store information related to the monitoring conditions, display method, etc. in the storage unit 204.

[0144] Figure 9(B) and Figures 10 to 12 are diagrams illustrating an example of a method for displaying temperature measurement data. Figure 9(B) shows a display in which temperature distribution is added to a visible light image, Figure 10 shows a real-time display, Figure 11 shows a numerical display in a table, and Figure 12 shows an example of a graph display. Fig. 9(B) is an example showing a temperature distribution above a temperature threshold. Fig. 9(B) may show temperatures within a predetermined temperature range using multiple color gradations, as shown in Fig. 6, for example.

[0145] In Figure 10, the thermographic image shows the current temperature of each target as well as the temperature of any specified location as a numerical value. In Figure 10, (A) is an example of the display before an abnormal condition occurred, and (B) is an example of the display when an abnormal condition occurred. Fig. 11 is an example of a table that numerically represents the temperature distribution within a predetermined time. In Fig. 11, for example, temperatures that are determined to be in an abnormal state may be displayed in a warning color (for example, red). Figure 12 shows an example of a display when an abnormal condition occurs due to a temperature rise in the ventilation fan (TG5) selected as the target. Figure 12 shows an example in which the temperature threshold for detecting an abnormality in TG5 is set to 80°C or higher.

[0146] <System Example 7> In this system example, an example of a system for monitoring using a temperature threshold will be described. The control means extracts portions of the temperature information that exceed the temperature threshold, and generates processing information using the extracted information. For example, the number of extracted portions is counted. In this way, the number of objects can be easily counted when the object has a temperature different from other areas. 1, the control means 200 may, for example, store the temperature threshold value in the storage unit 204. Furthermore, the processing unit 202 may extract over-temperature portions that exceed the temperature threshold value from the temperature information and count the number of over-temperature portions.

[0147] For example, it can be used to count the number of people in a closed space (e.g., a factory, a room, or a vehicle such as a bus), enabling detection of the number of people in a room and the detection of new intruders. It can also measure the number of people per unit area. For example, it can measure the number of people in a partitioned room such as a cram school classroom, and if the number of attendees differs from a preset capacity, it can be deemed an abnormality. It can also measure the number of people entering and leaving a room (for example, entering and leaving a toilet or a library). It can also be used to monitor children in daycare centers.

[0148] The temperature threshold may be set to, for example, the lowest temperature of an object to be detected among objects present in the imaging area. Furthermore, the control means may be configured to measure an area exceeding the minimum temperature as an over-temperature area when the area is a group of pixels (a group of consecutive pixels) with a number of pixels greater than a preset value. The preset value may be determined according to the size of the object to be measured. This makes it possible to eliminate measurements of unexpected objects and improve the accuracy of the object being measured.

[0149] The ideal imaging direction for the object is directly above, but the direction does not matter as long as multiple detected heat sources are locked and tracked. Even if heat sources overlap and are behind other heat sources, management is possible by continuing to track them the next time they appear. However, if a heat source overlaps in a direction that allows people to leave the room (such as a door), it becomes impossible to determine whether the heat source has left the room or not. Therefore, when measuring the entrance and exit of a heat source, it is preferable to determine the imaging direction so that the entrance and exit of the heat source can be captured.

[0150] <Other system examples> An example of the system of this embodiment has been described with reference to FIGS. 2 to 12. For example, by implementing the processing described below, advantageous effects can be achieved. For example, by overlaying images from a thermograph and a visible light imaging device, it is possible to measure the temperature of each object. The temperature of the subject can be displayed in full color gradation, so the temperature distribution can also be displayed using color. In a surveillance camera, for example, it is possible to use a visible light image taken in bright light as the background.

[0151] Temperature distribution makes it easy to determine whether a subject is human or animal, even if it cannot be captured by a visible light imaging device. Since animals (especially wild animals that are pests) generally do not wear clothing, there is a crucial difference in how they appear compared to humans wearing clothes, making it easy to determine whether they are human or animal. In addition, the fact that animals generally have higher body temperatures than humans can also be used to determine whether they are human or animal.

[0152] Consider a situation in a kitchen or factory where there is an object that has become so hot that touching it will cause burns. Since the temperature cannot be determined through visible light images (images seen by the human eye), touching the object will result in burns or other accidents. In addition, while fires caused by far-infrared rays or electrical leakage on the backside of a wall are difficult to detect with the naked eye, thermal imaging inspection equipment can capture the heat and detect fires or electrical leakage behind the wall. In addition, by using a visible light image as the background, it is possible to see the temperature of each subject even in the dark.

[0153] Thermal imaging inspection devices also have the function of measuring temperature change per unit time (ΔT). Furthermore, thermal imaging inspection devices also have the function of automatically detecting abnormalities based on temperature change per unit time. These functions can be combined with visible light images to monitor the object being imaged. For example, if the use of an electrical appliance such as an old electric fan could cause a fire, or if using a power strip beyond its rated capacity could cause the cable to melt and start a fire, the temperature change per unit time can be detected and the power cut off to prevent the fire.

[0154] In detecting moving objects, even if the subject is small and cannot be detected using visible light, because objects with temperature move, using a thermal imaging inspection device can improve detection accuracy compared to detection using visible light images. Using a thermal imaging inspection device, you can select multiple pinpoint temperatures, display them numerically, and also log them. If the obstacle is made of a material that transmits heat, it is possible to pass through it and photograph subjects behind the obstacle. For home appliances that are connected via IoT (Internet of Things), using them as the subject can detect abnormal heat and prevent fires by turning off the power.

[0155] If installed at an entrance or other location, it can measure the body temperature of people passing by and can also determine whether they have a fever-causing illness. When a thermography camera that does not emit visible light is used as a camera to monitor young children, privacy is protected to the greatest extent possible, so it can be used with peace of mind. It is also possible to automatically count the number of people per unit area and manage the increase or decrease in the number of people. By learning the thermal conditions of the subject, it is also possible to identify registered people.

[0156] In palm authentication, individuals can be identified by learning temperature distribution, making it possible to provide security that cannot be easily faked. In the case of palm authentication based on temperature, it is possible to perform authentication using only the palm of the hand, avoiding areas where temperature changes drastically, such as the fingertips.

[0157] It can be used for indoor and outdoor surveillance, and can also detect suspicious people hiding in the dark or in the shadows. By using a heat-impermeable material such as glass as an obstacle, it is easy to hide areas that you do not want to measure. Since it is not possible to measure the heat of materials in water, it is also possible to measure the cooling rate when a heated metal or other object is partially immersed in water and cooled.

[0158] In the above-described system examples 2 to 7, each system may be configured to include at least the imaging device 100 and the terminal device 3, similar to the system example 1 described in FIG. Furthermore, the imaging device 100 in FIG. 3 is just one example, and the first imaging unit that captures a visible light image and the second imaging unit that measures the amount of infrared light to obtain an image of the temperature distribution may be realized as multiple imaging devices provided in separate housings.

[0159] Embodiment 2. In this embodiment, a thermal image inspection device is used as the special imaging device, and an imaging target having regions with different measured infrared values ​​is used as a code.

[0160] Currently, barcodes and QR codes (registered trademarks) are the mainstream methods for identifying products, and no next-generation codes have been considered. Furthermore, credit cards and cash cards are also primarily magnetic or IC (Integrated Circuit) chips, which have a long history and are becoming increasingly vulnerable in terms of security. Therefore, in this embodiment, an aspect will be described in which areas with different measured values ​​of infrared rays are used as a code. The object to be imaged may have a code made up of a combination of materials with different thermal transmittances or thermal reflectances, and the amount of infrared rays may be measured by irradiating it with far infrared rays. This makes it extremely difficult to counterfeit the code. In the following description, a code that combines materials with different thermal transmittances or reflectances will be referred to as a "thermal code."

[0161] <Thermal code> Fig. 13 is a diagram illustrating an example of a thermal code. The thermal code 300 is composed of, for example, a predetermined number of pixels. Fig. 13 shows an example of a thermal code 300 with 10 x 10 pixels. Fig. 13 shows an example in which the thermal code 300 is formed from a combination of black pixels 301 and white pixels 302. In the following description, it is assumed that the imaging target and the thermal code are the same. However, the imaging target may have a plurality of pixels that form the thermal code in a portion thereof.

[0162] The thermal code can also change its transmittance depending on the heat it is irradiated with. In addition to determining whether a heat source is coming from the back, thermal codes can also determine whether a heat source is coming from the front. More specifically, thermal codes can be of two types: heat-reflective and heat-transmitting. The heat-reflective type irradiates far-infrared rays and measures the amount of infrared rays reflected by the thermal code. The heat-transmitting type irradiates far-infrared rays and measures the amount of infrared rays that penetrate the thermal code.

[0163] Thermal codes can be made as thin as magnetic cards. Thermal codes cannot be seen by the naked eye or by optical instruments using visible light, making them impossible to visually confirm or optically copy. Furthermore, like the Davunchi code, it is impossible to detect the presence or absence of a thermal code without a thermal camera. Even if a thermal camera were available, the thermal code itself is so small that it would be impossible to determine its presence without close proximity at current resolutions.

[0164] Even with a low-resolution thermal camera (imaging means), the number of combinations is infinite. For example, with 10 x 10 pixels and two levels of transmittance, the number of combinations is astronomically high, or 2 to the power of 100, or approximately 1.27 x 1030. The size of the thermal code can be made the same as the size of the thermal camera sensor (image sensor), making it possible to create thermal codes on the order of micrometers. Because the thermal code can be made smaller, it can be used not only for cards, but also for car and house keys, driver's licenses, and My Number cards. Thermal codes can also be used as an alternative to bar codes and QR codes (registered trademark).

[0165] 14 shows an example of using a thermal code for card authentication, in which a thermal code 300 is placed on a card 310. Because the thermal code 300 has a small area, it is easy to place the thermal code in addition to other codes (for example, an IC chip).

[0166] Figure 15 shows an example of using a thermal code in a key, in which a heat-transmitting thermal code is formed on a key 320. In Figure 15, (A) is a plan view, and (B) is a front view. In key 320, thermal code 300 is placed in thermal code placement area 322 of key board 321. If the thermal code is a heat-transmitting type, a hole for placing the thermal code is formed in placement area 322. Thermal code 300 is covered on both the flat (top) and bottom sides by slide cap 323, and is exposed when slide cap 323 moves along slide rail 324.

[0167] 15 shows an example of a heat-transmitting thermal code, so the slide caps and slide rails 324 are placed on the flat and bottom sides of the key board 321. In a heat-reflective thermal code 300, the slide caps 323 and slide rails 324 are placed on the side of the key board 321 where the thermal code 300 is placed. Note that the slide caps 323 are provided to protect the thermal code 300, and the slide caps 323 and slide rails 324 may not be provided.

[0168] Silicon is an example of a material with high thermal transmittance, and thermal cords can be easily produced by applying wafer manufacturing techniques. Furthermore, by making the thermal cord from silicon, it has the advantage of being resistant to deterioration caused by temperature, humidity, ultraviolet rays, etc., and is hardly deformed by heat.

[0169] <Control means using thermal code> In the system of this embodiment, the control means receives as input information information based on the amount of infrared light or the infrared ridge of the thermal code measured by a thermal code reader (described later), generates processing information related to the amount of infrared light, and outputs the generated processing information. For example, the control means may generate the processing information using a value obtained by correcting the amount of infrared light received.

[0170] The input information may be, for example, information indicating the amount of infrared light emitted by each pixel constituting the thermal code. For example, the input information may be a numerical value indicating the amount of infrared light, or image data that indicates the amount of infrared light by color. The input information may be, for example, image data that can be acquired by a thermal code reader (described later), or information based on the image data.

[0171] The processing information may be, for example, information representing a thermal code. As an example, the processing information may be information indicating the amount of infrared light for each pixel constituting the code, or an image representing the code. The image representing the code may be, for example, a pattern representing areas with different amounts of infrared light. As an example, a pattern representing the amount of infrared light divided into two levels (for example, color-coded using two different colors such as white and black) as shown in FIG. 13 may be used.

[0172] The processing information may be the result of comparing the information indicating the amount of infrared light with the verification information. In this case, in the configuration example of Fig. 1, the control means 200 may store the verification information in the storage unit 204, for example.

[0173] <Thermal code reader> Next, a device for reading the above-mentioned thermal code will be described. Currently, there is no equipment that can read the thermal code described above. Therefore, we propose a thermal code reader as an example of a special imaging device. Here, we will explain an example of equipment that can read a thermal code that is a combination of materials with different thermal transmittances or reflectances. The thermal code reader includes, for example, a thermal camera as an imaging means and an irradiation means.

[0174] The thermal camera measures the amount of infrared rays when far-infrared rays are irradiated onto the thermal code by an irradiation means. The thermal camera has multiple sensors (image pickup elements). The thermal camera may be, for example, a 2D thermal camera or a 1D thermal camera. The 2D thermal camera may have, for example, multiple sensors arranged on a 2D plane. The 1D thermal camera may have, for example, multiple sensors arranged on a line (e.g., a straight line).

[0175] The irradiation means irradiates the thermal code with far-infrared rays. The irradiation means may be, for example, a far-infrared generator. The thermal code reader of this embodiment can be configured to read the thermal code by applying heat to the surface of the thermal code and reading the reflected heat (thermal reflection type), or by applying heat to the back of the thermal code and reading the transmitted heat (thermal transmission type). If the thermal code is a thermal reflection type that combines materials with different thermal reflectivities, the irradiation means is located on the same side as the thermal camera. If the thermal code is a thermal transmission type that combines materials with different thermal transmittances, the irradiation means is located on the opposite side of the thermal code from the thermal camera, sandwiching the thermal code between them.

[0176] 16 is a diagram illustrating an example of a thermal code reader that reads a heat-reflective thermal code 300A, where the thermal code 300A is a two-dimensional code. In the thermal code reader 410, a far-infrared generator 412 irradiates heat from the front of the thermal code 300A. A two-dimensional thermal camera 411 captures the reflected heat. In this case, the thermal code 300A is made of a heat-reflecting material.

[0177] 17 is a diagram illustrating an example of a thermal code reader that reads a heat-transmitting thermal code 300B, where the thermal code 300B is a two-dimensional code. In the thermal code reader 420, a far-infrared generator 422 is placed behind the thermal code 300B as a heat source. A two-dimensional thermal camera 421 captures the heat that passes through the thermal code 300B. In this case, the distance between the two-dimensional thermal camera and the code can be made quite short, and the structure is simple and inexpensive. Note that the heat source behind the thermal code 300B may not be necessary in some cases.

[0178] It is preferable that the 2D thermal cameras 411, 421 be capable of capturing an image of the entire thermal code at once. Therefore, the 2D thermal cameras 411, 421 should have sensors corresponding to the pixels that make up the thermal code. This allows the entire thermal code to be captured. Furthermore, the control means can obtain input information based on image data captured at once by, for example, a thermal code reader.

[0179] If the thermal camera is equipped with a sensor smaller than the thermal code, it will capture the entire thermal code by dividing the area of ​​the thermal code into multiple parts. This allows the entire thermal code to be obtained even if the thermal code is larger than the thermal code reader can capture in one shot. In this way, the control unit can obtain input information based on image data captured by the thermal code reader in multiple separate images, for example.

[0180] 18 and 19, a case where a two-dimensional thermal code is read using a one-dimensional thermal camera will be described. 18 and 19 are diagrams illustrating an example of a thermal code reader when the thermal camera has a sensor smaller than the thermal code. A one-dimensional thermal camera may have a sensor that can capture the length of one side of a two-dimensional code. A one-dimensional thermal camera may be large enough to capture the diameter of the circumscribing circle of the object being imaged. The thermal code reader may also include an imaging control means (not shown) for controlling the thermal camera.

[0181] Figure 18 is a diagram illustrating an example of reading a circular thermal code 300C with a one-dimensional thermal camera 431. In Figure 18, the imaging control means controls the one-dimensional thermal camera 431 to rotate around its center as an axis and capture an image of the entire thermal code 300C. In the thermal code reader 430, the one-dimensional thermal camera 431 rotates half a turn around the center of the thermal camera as an axis to read the circular thermal code 300C.

[0182] In order for the thermal code reader 430 to accurately read the thermal code 300C, it is preferable to keep the distance between the thermal code 300C and the one-dimensional thermal camera 431 as small as possible. It is also preferable that the thermal code 300C and the thermal code reader 430 be heat-transmitting rather than heat-reflective. Furthermore, a positioning hole is provided in the center of the back of the thermal code 300C, and a boss (protrusion) on the thermal code reader 430 fits into this hole, making it easy to align the center of the one-dimensional thermal camera 431 with the center of the thermal code 300C.

[0183] Fig. 19 is a diagram illustrating an example in which a rectangular thermal code 300D is read by a one-dimensional thermal camera 441. In Fig. 19, for example, when the thermal code 300D is rectangular, the imaging control means controls the one-dimensional thermal camera 441 to move from one side of the thermal code 300D toward the opposite side to capture an image of the thermal code 300D. In the thermal code reader 440, the one-dimensional thermal camera 441 moves from one side of the thermal code 300D to the opposite side to read the rectangular code. In order for the thermal code reader 440 to accurately read the thermal code 300D, it is preferable to make the distance between the thermal code 300D and the one-dimensional thermal camera 441 as small as possible. Furthermore, it is preferable that the thermal code 300D and the thermal code reader 440 be of a transmission type rather than a reflection type.

[0184] In this way, the thermal code reader can read two-dimensional codes by rotating the one-dimensional thermal camera.The thermal code reader can also read two-dimensional thermal codes by tracing them with the one-dimensional thermal camera.

[0185] <Thermal authentication system> Next, an example of a system using a thermal code will be described. Here, a thermal authentication system will be described that outputs an authentication result by comparing preset matching information (also called authentication information) with a thermal code. The thermal authentication system may include, for example, at least a thermal code reader and an authentication circuit. The authentication circuit functions as a control means that uses the thermal code.

[0186] First, the power supply for the thermal authentication system will be described. The thermal authentication system can be configured as a power supply type that includes a power supply means for supplying power from an external source. The power supply type thermal authentication system is preferably configured so that when a card or key having a thermal code is inserted into the cylinder, the switch of the power supply means is turned on and power is supplied to the thermal authentication system. The thermal authentication system may also be configured as a power generation type having a power generation means. For example, power can be generated when a card or key having a thermal code is inserted into or removed from a thermal code reader. In this way, it becomes possible to eliminate the need for a power supply. An example of a power generation means is described below.

[0187] 20 is a diagram illustrating an example of the configuration of a power generation means for supplying power to a thermal authentication system. In FIG. 20, the power generation means is explained using an example of using the thermal code of the key 320 shown in FIG. 20 shows an example in which the power generating means is realized using a cylinder 610, a slider 620, a pulley 630, a wire 640, and a system mainspring 650. In FIG. 20, (A) is a diagram illustrating the initial state, (B) is forward power generation, and (C) is a diagram illustrating power generation by the power generating mainspring. 21 is a diagram illustrating an example of the configuration of a system spring 650. The system spring 650 has a wire winding spring 651, a power generating spring 652, a gear box 653, and a power generating motor 654.

[0188] When the key 320 is inserted into the cylinder 610 (FIG. 20(A)) of the thermal authentication system, the slider 620 is pushed and the wire 640 is pulled. At this time, forward power generation is performed by the system mainspring 650 (FIG. 20(B)). Furthermore, when the slider 620 is inserted further than a certain position, the discharge switch of the power generation circuit is turned on and the generated power is supplied to the thermal authentication system.

[0189] When the key 320 is pressed into the cylinder 610 and the slider 620 exceeds the limit of the power generating spring, the power generating motor 654 generates electricity using the wound power generating spring 652 (FIG. 20(C)). At the same time, when the power generating spring 652 starts generating power, power supply to the authentication circuit begins, and when authentication is complete, the cylinder lock is released, allowing the key 320 to be turned. After unlocking, when the key is removed, the wire winding spring 651 returns the slider 620 to its initial position. At this time, the gear of the gear box 653 winds the power generating spring 652. At the same time, the power supply to the authentication circuit stops.

[0190] 22 is a diagram illustrating an example of the configuration of a power generation circuit. The power generation circuit 800 is an example of a circuit that supplies power to the thermal authentication system 700. The power generation circuit 800 includes a motor 810 (corresponding to the power generation motor 654), a step-up transformer 820, a rectifier 830, a smoothing capacitor 840, a battery 850, a discharge switch 860, and a DC-DC converter 870. However, the configuration is not limited to this, and power may be generated using, for example, a known energy harvesting technique.

[0191] Next, an example of the authentication operation of the thermal authentication system will be described. Below, we will explain the case where a locked door is unlocked using a key that uses a thermal code. We will explain the power supply type (Fig. 23) and the power generation type (Fig. 24) thermal authentication systems.

[0192] FIG. 23 is a flowchart illustrating an example of the operation of the power-supply type thermal authentication system. When a key is inserted into the cylinder (S11), the power supply means is switched on, power is supplied to the thermal camera's irradiation means, and far-infrared rays are emitted (S12). The thermal camera reads the thermal code (S13). If the thermal code is successfully read (YES in S13), the thermal code is sent to the authentication circuit.

[0193] The authentication circuit compares the thermal code with the verification information (S14). If the thermal code matches the verification information (YES in S14), the authentication circuit unlocks the cylinder (S15) and opens the door (S16). On the other hand, if the thermal code is not read (NO in S13) or if the thermal code does not match the verification information (NO in S14), the authentication circuit notifies the verification failure (S17) and keeps the door locked.

[0194] Fig. 24 is a flowchart illustrating an example of the operation of a power-generating thermal authentication system, which will be described using the power generation means of Figs. 20 to 22 as an example. When a key is inserted into the cylinder (S21), forward power generation begins (S22), the discharge switch of the power generation circuit is turned ON (S23), and power generation by the power generation spring (power generation by the system spring, which rotates the power generation motor by releasing the power generation spring) begins (S24). When power is supplied from the system spring, power is supplied to the irradiation means of the thermal camera, far-infrared rays are emitted, and the thermal camera reads the thermal code (S25). When the thermal code is read (YES in S25), the thermal code is sent to the authentication circuit.

[0195] The authentication circuit compares the thermal code with the verification information (S26). If the thermal code matches the verification information (YES in S26), the authentication circuit unlocks the cylinder (S27) and opens the door (S28). On the other hand, if the thermal code is not read (NO in S25) or if the thermal code does not match the verification information (NO in S26), the authentication circuit notifies the system that verification has failed (S29), and the door remains locked.

[0196] The above-described thermal code reader can also be modified as follows. The one-dimensional or two-dimensional thermal camera mentioned above can also measure the temperature ratio of each pixel (each spot) of the thermal code, making it possible to make the code three-dimensional. The thermal code reader described above can read the thermal code whether the thermal code and the thermal camera are far apart or close together.

[0197] By using a thermal code reader equipped with a thermal camera with a high number of pixels, even if the thermal code position is misaligned, correction can be made by pattern matching. Thermal code readers can be used, for example, in card readers that read thermal codes on cards, key cylinders for homes and cars, and security authentication devices. A filter is attached to the lens of the thermal camera, allowing it to pass or block specific wavelengths of far infrared rays.

[0198] The thermal code reader of this embodiment can be configured as a heat reflective or heat transmissive type as described above, or can utilize external heat to read codes without applying heat. For example, the thermal code reader may be provided with an external irradiation means. The thermal code reader of this embodiment can capture infrared light of wavelengths that cannot be captured by a visible light imaging device such as a CCD. The thermal code reader of this embodiment may use a thermal infrared sensor such as a thermopile or a bolometer.

[0199] Embodiment 3. Other special imaging devices Using the special imaging device, for example, the following processing can be further executed. It detects whether there is water inside the car and warns if the temperature inside the car is likely to rise when the car is parked. It is recommended that this be detected using a near-infrared camera. It detects the temperature of the water inside the car and issues a warning if the water temperature rises. A thermal camera or similar device is recommended.

[0200] It would be good to detect sugar and issue a warning or notify the area. In the case of a car, it would be good to detect stains on the seats. It would be good to use a device that can detect sugar.

[0201] It is advisable to use a hyperspectral camera to photograph road structures, buildings, etc. from a moving vehicle and create an assessment map of salt damage to concrete, etc. It would be a good idea to capture the meteorite fall with a thermal camera and record it as a trigger. It is expected that the video type will sell for a high price.

[0202] It is recommended to use a biomedical multispectral camera to photograph the driver. This camera should be detachable so that it can photograph passengers who become ill and transmit information to emergency services. The first band (735 nm) represents deoxyhemoglobin (deoxygenated or enzyme), the second band (800 nm) represents total hemoglobin (whole blood), and the third band (930 nm) represents lipids, so these information can be provided.

[0203] It would be possible to measure the moisture content of the skin and hair of drivers and passengers, and praise them when their skin or hair is in good condition. This would be possible using a hyperspectral camera. It is recommended to use a hyperspectral camera to distinguish the colors of road signs, electronic signs, taillights, white lines, and vegetation. It is also recommended to use images from a visible light camera to distinguish between objects with similar spectra, such as white lines and the sky.

[0204] It is advisable to combine a TOF camera (distance imaging camera) and a hyperspectral camera, obtain correspondence between the positions of each image, and obtain correspondence between spectrum and distance. It is best to process the data at the edge (terminal) and send only the results to the cloud. It is a good idea to attach a thermal camera to guided vehicles (AGVs), robots, forklifts, etc. to warn of approaching hot or cold places. It is also a good idea to combine it with a distance imaging camera.

[0205] It is possible to record the image from the special camera together with the image from the visible light camera in a time-synchronized manner, and check the image from the visible light imaging device at the time when a specific phenomenon was detected by the special camera, and combine them to display on a single screen or print on paper.

[0206] It is advisable to use information from a hyperspectral camera, thermal camera, TOF, etc. to tag each pixel of the visible light camera and automatically create training data for machine learning.

[0207] The overlapping sensitivity bands of the hyperspectral camera and (multiple) light sources with specific bands are used for processing (for example, by taking the difference, switching the light source, or changing to a light source (LED) with a different spectrum). Two types of beauty light source LEDs are turned on alternately to obtain the conditions of each skin depth region.

[0208] Other embodiments In each of the above embodiments, the system may be configured to include at least a control unit, which may be implemented by a computer such as the terminal device 3 described above. The system may be configured to include, in addition to a control means, a special imaging device, a display means for displaying processing information, an operation means for accepting instructions from a user, an output means for outputting warning sounds, etc., or two or more of these. In this specification, an image (such as a visible light image or a temperature distribution image) may be either a still image or a moving image. The system may be configured as follows: Body temperature management for livestock and pets After measuring and recording a healthy profile, the system may be equipped with a function to detect abnormalities in the body temperature of livestock and pets by comparing the current state with the healthy profile and issuing an alarm. · Measurement of ambient temperature and food temperature. Food temperature can also affect digestion and health, so it is advisable to have a function that detects abnormalities in the food temperature and issues an alarm. - Accumulation of livestock movement: It is difficult to measure movement using visible light, but with thermography, it is relatively easy to measure because all you need to do is accumulate the amount of movement of the heat source. For example, in the case of dairy cows, by comparing the above data with the components of the milk collected daily, it is possible to obtain information on the temperature environment and movement amount that resulted in good milk being collected. It is also possible to calculate the causal relationship with milk volume. The above information has the advantage that if a standard is set and any deviation from the standard is detected as an abnormality, there is no need for constant monitoring by a person. ·Measures against harmful animals such as pigeons, bears, wild boars, and rats The system should be set up to monitor areas that are affected by harmful animals such as pigeons, bears, wild boars, and rats, or areas where damage by harmful animals is expected. The vehicle may be configured to measure the driver's body temperature with a thermal camera and output a warning if the temperature rises or falls to a predetermined temperature within a predetermined time. A warning warning to be cautious of drowsiness is particularly desirable. For example, a warning may be output if the temperature drops by 0.5 degrees within five minutes. Since the temperature inside the vehicle affects the driver immediately after getting into the vehicle, the warning may be suppressed for a predetermined time after getting into the vehicle. For example, the warning may not be output for 10 minutes after the engine start is detected or after the device is turned on, and then the warning may be enabled.

[0209] The scope of the present invention is not limited to the structures expressly described in the specification, but also includes combinations of various aspects of the invention disclosed herein. While the structures of the invention sought to be patented are specified in the accompanying claims, it is the intention to claim structures disclosed herein in the future, even if they are not currently specified in the claims.

[0210] The present invention is not limited to the configurations described in the above-described embodiments. The components of the above-described embodiments, system examples, and variations may be arbitrarily selected and combined. Furthermore, any component of each embodiment, system example, or variation may be arbitrarily combined with any component described in the Summary for Solving the Invention or any component embodying any component described in the Summary for Solving the Invention. The present invention also intends to obtain rights to these configurations through amendments or divisional applications of this application. Furthermore, even if a description is made of "in the case of..." or "when...," this does not mean that the configuration is limited to that case or time. Configurations that are not limited to those cases or times are also disclosed, and the present invention intends to obtain rights to them. Furthermore, even if a description is made in an order, the order is not limited to this order. Configurations in which some parts are deleted or the order is changed are also disclosed, and the present invention intends to obtain rights to them.

[0211] Furthermore, by filing a conversion application to a design application, the applicant intends to obtain rights to the overall design or partial design. The drawings depict the entire device in solid lines, but they also include partial designs claimed for parts of the device. For example, it is possible to include some components of the device as partial designs, and it is also possible to include some parts of the device as partial designs regardless of the components. A part of the device may be a part of the device, or it may be part of that component. The applicant intends to obtain rights not only to the overall design, but also to partial designs in which any part of the solid line portion of the drawings is shown as a broken line. [Explanation of symbols]

[0212] 1. Imaging system 2 Machine tools 3 Terminal Devices 100 Imaging device 105 Case 110 Controller 120 First Imaging Unit 130 Second imaging unit 140 Event input terminal 160 memory card slot 180 image signal output terminal 200 Control Means 201 Input section 202 Processing section 203 Output section 204 Holding part 300, 300A~300D thermal code 310 Cards 320 keys 410, 420, 430, 440 Thermal Code Reader 411, 421 2D thermal camera 412, 422 Far infrared generator 431, 441 1D thermal camera 610 cylinder 620 Slider 630 Pulley 640 Wire 650 system mainspring

Claims

1. A system for monitoring an object using images of temperature distribution captured by a thermal imaging device, The system includes a control means for distinguishing between small animals and humans using the aforementioned temperature distribution image. The control means is If a heat-generating object is found to be divided into multiple regions, it will be recognized as a human. If a single, independently moving heat source is present in a given area, it will be identified as a small animal. A system characterized by the following features.

2. The control means is Using a temperature distribution that exceeds a predetermined temperature threshold, small animals and humans are distinguished. The system according to feature 1.

3. The imaging target is a living room. The control means is Using the temperature distribution image of the aforementioned room, we monitor for illegal intrusions, etc. An abnormal occurrence occurs when an intruder or small animal enters the aforementioned room, and the intruder or small animal appears in the temperature distribution as an object with a high temperature that does not normally exist. The system according to claim 1 or 2, characterized in that it is the same as described in claim 1 or 2.

4. A program for causing a computer to perform the functions of the control means provided in any one of claims 1 to 3.

Images