Object recognition device based on thermal image and object recognition method thereof

Abstract

The invention discloses an object recognition device based on a thermal image and an object recognition method thereof. The object recognition device comprises a thermal image sensor, an image sensor and an operation host. An image sensor senses an imaging region to generate a thermal image, divides the thermal image into a plurality of thermal image sub-regions, obtains first temperature information and second temperature information from each of the thermal image sub-regions, and uses the first temperature information and the second temperature information as converted thermal images. The image sensor senses the imaging area to generate an optical image. The computing host performs object recognition based on the optical image and the converted thermal image.

Description

Technical Field

[0001] The present invention relates to an object recognition technology, and in particular to an object recognition device based on thermal images and an object recognition method thereof. Background Art

[0002] Object recognition technology is a major research direction of automation technology. Object recognition technology can be achieved by combining color images (such as RGB images) with artificial intelligence (AI). In recent years, in addition to using color images, thermal images can also be combined to improve the recognition efficiency of artificial intelligence (AI) through the temperature data in thermal images.

[0003] Each pixel in a thermal image has a corresponding temperature value. When the image resolution is higher, the amount of thermal image data and the amount of computation will increase accordingly, so it is necessary to expand the corresponding hardware equipment to process this data in order to maintain the original performance. If the original equipment is used to process thermal images with high image resolution, the recognition performance of AI will be reduced, becoming a performance bottleneck. Summary of the invention

[0004] The present invention provides an object recognition device based on thermal images and an object recognition method thereof, which can improve the transmission and use efficiency of thermal image data in object recognition technology based on artificial intelligence (AI) algorithms.

[0005] The object recognition device based on thermal image of the present invention comprises a thermal image sensor, an image sensor and a computing host. The thermal image sensor senses an imaging area to generate a thermal image, divides the thermal image into a plurality of thermal image sub-areas, obtains first temperature information and second temperature information from each thermal image sub-area, and uses the first temperature information and the second temperature information in each thermal image sub-area as a converted thermal image. The image sensor senses the imaging area to generate an optical image. The computing host couples the thermal image sensor and the image sensor. The computing host performs object recognition based on the optical image and the converted thermal image.

[0006] The object recognition method based on thermal images of the present invention includes the following steps: sensing an imaging area through a thermal image sensor to generate a thermal image; dividing the thermal image into a plurality of thermal image sub-areas through the thermal image sensor, obtaining first temperature information and second temperature information from each thermal image sub-area, and using the first temperature information and the second temperature information in each thermal image sub-area as a converted thermal image; sensing the imaging area through an image sensor to generate an optical image; and performing object recognition based on the optical image and the converted thermal image.

[0007] Based on the above, the thermal image-based object recognition device and object recognition method described in the embodiment of the present invention divides the thermal image into several thermal image sub-areas, determines and obtains the first temperature information (e.g., the one with the maximum temperature in this thermal image sub-area) and the second temperature information (e.g., the one with the minimum temperature in this thermal image sub-area), and provides the first temperature information and the second temperature information of each thermal image sub-area to the back-end computing host. In this way, the computing host does not need to receive the thermal image with a huge amount of information, but can only perform object recognition based on the maximum temperature and the minimum temperature of each thermal image sub-area based on the artificial intelligence algorithm, thereby greatly shortening the data transmission time and computing time of the thermal image, and effectively improving the overall efficiency of object recognition. In addition, the computing host of this embodiment will also control information to the thermal image sensor to notify or adjust the thermal image sensor's way of distinguishing the thermal image sub-areas, so as to adjust the accuracy of object recognition according to needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a functional block diagram of an object recognition device according to an embodiment of the present invention.

[0009] Figure 2 is a schematic diagram of an exemplary object recognition device according to an embodiment of the present invention.

[0010] Figure 3 is a flow chart of an object recognition method according to an embodiment of the present invention.

[0011]

Explanation of symbols

[0012] 100: Object recognition device

[0013] 105: Imaging area

[0014] 110: Thermal image sensor

[0015] 115:Converted thermal image

[0016] 120: Image sensor

[0017] 130:Computing host

[0018] 132: Processor

[0019] 135: Memory

[0020] 212: Thermal image receiver

[0021] 214: Data conversion device

[0022] 216: Data transmission device

[0023] 237: Artificial Intelligence Algorithms

[0024] S310~S340: Steps

[0025] CS: Control signal

[0026] IM: Optical Image

[0027] T1A~T9A、T1B~T9B、T1C~T9C、T1D~T9D、T1E~T9E: Temperature information DETAILED DESCRIPTION

[0028] Figure 1 is a functional block diagram of an object recognition device 100 according to an embodiment of the present invention. The object recognition device 100 is mainly used to recognize whether a specific object (e.g., a flame, a specific gas that may leak) appears in the imaging area. The object recognition device 100 runs an artificial intelligence (AI) algorithm to recognize the aforementioned specific object, and can be applied to various AI recognition application scenarios, such as industrial safety protection, security monitoring, automatic driving, industrial inspection, etc. The object recognition device 100 can also be set on a charging station or a charging battery exchange station for an electric vehicle or electric locomotive to sense whether a flame appears and a combustion occurs.

[0029] The object recognition device 100 mainly includes a thermal image sensor 110, an image sensor 120 and a computing host 130. In addition to sensing the imaging area to generate a thermal image, the thermal image sensor 110 of the present embodiment also divides the thermal image into a plurality of thermal image sub-areas and obtains first temperature information and second temperature information from each thermal image sub-area. The first temperature information of the present embodiment is the highest of the plurality of temperature information corresponding to the plurality of pixels in each thermal image sub-area, and the second temperature information is the lowest of the plurality of temperature information corresponding to the plurality of pixels in the aforementioned thermal image sub-area. The thermal image sensor 110 uses the first temperature information and the second temperature information in the aforementioned thermal image sub-areas as a converted thermal image and transmits the converted thermal image to the computing host 130. In this way, the amount of data of the thermal image is changed from each pixel having corresponding temperature information to the highest temperature and the lowest temperature in each thermal image sub-area in the converted thermal image, thereby greatly reducing the data transmission time.

[0030] The image sensor 120 senses the aforementioned imaging area to generate an optical image. The image sensor 120 of the present embodiment may be an RGB image sensor or a corresponding sensing device. The computing host 130 is coupled to the thermal image sensor 110 and the image sensor 120. The computing host 130 obtains a converted thermal image from the thermal image sensor 110 and obtains an optical image from the image sensor 120. The computing host performs object recognition based on the optical image and the converted thermal image. The computing host 130 includes a processor 132 and a memory 135. The computing host 130 runs an artificial intelligence algorithm through the processor 132 and the memory 135 to perform object recognition. The artificial intelligence algorithm determines whether there is a specific object in the imaging area based on the optical image and the converted thermal image. The artificial intelligence algorithm of the present embodiment only analyzes the first temperature information (e.g., the highest temperature) and the second temperature information (e.g., the lowest temperature) of each thermal image sub-area contained in the converted thermal image for object recognition. Since there is no need to process the temperature value of each pixel, the artificial intelligence algorithm of the present embodiment can shorten the data processing and operation time for the thermal image, effectively improving the sorting efficiency.

[0031] The computing host 130 may also provide control information CS to the thermal image sensor 110 to notify or adjust the thermal image sensor 110 of the method of distinguishing the thermal image sub-regions. For example, when the object recognition device 100 is turned on, the computing host 130 of the present embodiment will notify the thermal image sensor 110 through the control information CS that the method of distinguishing the thermal image sub-regions is the first method of distinguishing (e.g., dividing the thermal image into 3x3 thermal image sub-regions). When the object recognition device 100 is in operation and the computing host 130 believes that more accurate temperature information is needed, the control information CS may be used to notify the thermal image sensor 110 that the method of distinguishing the thermal image sub-regions is the second method of distinguishing (e.g., dividing the thermal image into 4*4 or 5*5 thermal image sub-regions), thereby obtaining more accurate temperature information, and then adjusting the accuracy of object recognition according to needs.

[0032] Figure 2 is a schematic diagram of an object recognition device 100 according to an embodiment of the present invention. Figure 2 The imaging region 105 is presented for illustration. The thermal image sensor 110 includes a thermal image receiver 212, a data conversion device 214, and a data transmission device 216. The image sensor 120 senses the imaging region 105 to generate an optical image IM.

[0033] When the object recognition device 100 is turned on, the data conversion device 214 in the thermal image sensor 110 can obtain the control information CS from the computing host 130 to know the method of distinguishing the thermal image sub-regions, or distinguish the thermal impact by a preset distinction method. The distinction method of this embodiment is to divide the thermal image into 3x3 thermal image sub-regions as an example.

[0034] The thermal image receiver 212 in the thermal image sensor 110 senses the imaging area 105 to generate a thermal image. Each pixel in the thermal image of this embodiment has corresponding temperature information. The data conversion device 214 is coupled to the thermal image receiver 212. The data conversion device 214 divides the thermal image into a plurality of thermal image sub-areas according to the aforementioned division method. For example, this embodiment divides the thermal image 112 into 9 thermal image sub-areas, and the number of thermal image sub-areas depends on the use requirements. In addition, each thermal image sub-area can have a large number of diversified temperature information obtained by the corresponding temperature information of each pixel in this thermal image sub-area. For example, in this embodiment, the nine thermal image sub-areas in the thermal image 112 respectively have corresponding information: first temperature information T1A~T9A (e.g., maximum temperature), second temperature information T1B~T9B (lowest temperature), third temperature information T1C~T9C (the first quartile of the thermal image sub-area sorted by temperature), fourth temperature information T1D~T9D (the median of the thermal image sub-area sorted by temperature), and fifth temperature information T1E~T9E (the third quartile of the thermal image sub-area sorted by temperature). The amount of corresponding information depends on usage requirements.

[0035] Then, the data conversion device 214 extracts the first temperature information T1A-T9A (e.g., the highest temperature) and the second temperature information T1B-T9B (e.g., the lowest temperature) in each thermal image sub-region, and uses the first temperature information T1A-T9A and the second temperature information T1B-T9B in each thermal image sub-region as the converted thermal image 115, and transmits the converted thermal image 115 to the data transmission device 216. The data transmission device 216 transmits the converted thermal image 115 to the computing host 130. The data transmission device 216 can be a universal serial bus (USB) data transmission hardware module, a wired local area network data transmission hardware module, or a wireless network data transmission hardware module. The converted thermal image 115 of this embodiment only includes the first temperature information T1A-T9A and the second temperature information T1B-T9B in each thermal image sub-region.

[0036] In other words, the thermal image sensor 110 of the present embodiment divides the thermal image into a plurality of thermal image sub-areas through the data conversion device 214, finds the highest temperature and the lowest temperature of each thermal image sub-area, and transmits only the highest temperature and the lowest temperature of each thermal image sub-area to the computing host 130. In this way, since only the highest temperature and the lowest temperature of each thermal image sub-area are provided, the amount of data of the converted thermal image 115 will be greatly reduced. For example, the present embodiment divides the thermal image into 9 (3×3) thermal image sub-areas, so the amount of data of the converted thermal image 115 mainly includes 18 data (9 first temperature information T1A to T9A and 9 second temperature information T1B to T9B), while the original temperature information has 45 data (9 first temperature information T1A to T9A, 9 second temperature information T1B to T9B, 9 third temperature information T1C to T9C, 9 fourth temperature information T1D to T9D, and 9 fifth temperature information T1E to T9E). Therefore, the artificial intelligence algorithm 237 running on the computing host 130 does not need to process a large amount of temperature data, which can greatly reduce the amount of computation required for AI recognition applications, thereby improving the overall efficiency of object recognition.

[0037] Figure 3 is a flow chart of an object recognition method according to an embodiment of the present invention. Figure 3 The object recognition method is applicable to Figure 1 and Figure 2 Please refer to the object recognition device 100. Figure 1 and Figure 3 In step S310 , the imaging area is sensed by the thermal image sensor 110 to generate a thermal image.

[0038] In step S320, the thermal image is divided into a plurality of thermal image sub-regions by the thermal image sensor 110, first temperature information and second temperature information are obtained from each thermal image sub-region, and the first temperature information and second temperature information in each thermal image sub-region are used as the converted thermal image.

[0039] In step S330, the imaging area is sensed by the image sensor 120 to generate an optical image. In step S340, the computing host 130 performs object recognition based on the optical image and the converted thermal image. Figure 3 For details of each step S310-S340, please refer to the aforementioned embodiments.

[0040] In summary, the thermal image-based object recognition device and object recognition method described in the embodiment of the present invention divides the thermal image into several thermal image sub-areas, determines and obtains the first temperature information (e.g., the one with the maximum temperature in this thermal image sub-area) and the second temperature information (e.g., the one with the minimum temperature in this thermal image sub-area), and provides the first temperature information and the second temperature information of each thermal image sub-area to the back-end computing host. In this way, the computing host does not need to receive the thermal image with a huge amount of information, but can only perform object recognition based on the maximum temperature and the minimum temperature of each thermal image sub-area based on the artificial intelligence algorithm, thereby greatly shortening the data transmission time and computing time of the thermal image, and effectively improving the overall efficiency of object recognition. In addition, the computing host of this embodiment will also control information to the thermal image sensor to notify or adjust the thermal image sensor's way of distinguishing the thermal image sub-areas, so as to adjust the accuracy of object recognition according to needs.

[0041] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Those skilled in the art may make slight changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the scope defined in the attached claims.

Claims

1. An object recognition device based on thermal images, comprising: A thermal image sensor senses an imaging area to generate a thermal image, divides the thermal image into a plurality of thermal image sub-areas, obtains first temperature information and second temperature information from each thermal image sub-area, and uses the first temperature information and the second temperature information in each thermal image sub-area as a converted thermal image; An image sensor that senses the imaging area to generate an optical image; as well as A computing host is coupled to the thermal image sensor and the image sensor, wherein the computing host performs object recognition based on the optical image and the converted thermal image. 2 . The object recognition device as claimed in claim 1 , wherein the converted thermal image only includes the first temperature information and the second temperature information in each thermal image sub-region.

3. An object identification device as described in claim 2, wherein the first temperature information is the highest of multiple temperature information corresponding to multiple pixels in each thermal image sub-region, and the second temperature information is the lowest of the temperature information corresponding to the pixels in each thermal image sub-region. 4 . The object recognition device as claimed in claim 1 , wherein the computing host further provides control information to the thermal image sensor to notify or adjust the thermal image sensor's method of distinguishing the thermal image sub-areas.

5. The object recognition device as claimed in claim 1, wherein the computing host runs an artificial intelligence algorithm to perform the object recognition, and the artificial intelligence algorithm determines whether there is a specific object in the imaging area based on the optical image and the converted thermal image.

6. The object recognition device according to claim 1, wherein the thermal image sensor comprises: a thermal image receiver, sensing the imaging area to generate the thermal image; a data conversion device, coupled to the thermal image receiver, dividing the thermal image into the thermal image sub-regions, extracting the first temperature information and the second temperature information in each thermal image sub-region, and using the first temperature information and the second temperature information in each thermal image sub-region as a converted thermal image; as well as The data transmission device is coupled to the data conversion device to transmit the converted thermal image to the computing host.

7. A method for object recognition based on thermal images, comprising: sensing an imaging area by a thermal image sensor to generate a thermal image; Dividing the thermal image into a plurality of thermal image sub-regions by the thermal image sensor, obtaining first temperature information and second temperature information from each thermal image sub-region, and using the first temperature information and the second temperature information in each thermal image sub-region as a converted thermal image; sensing the imaging area by an image sensor to generate an optical image; as well as Object recognition is performed based on the optical image and the converted thermal image. 8 . The object recognition method according to claim 7 , wherein the converted thermal image only includes the first temperature information and the second temperature information in each thermal image sub-region.

9. The object recognition method as described in claim 8, wherein the first temperature information is the highest of multiple temperature information corresponding to multiple pixels in each thermal image sub-region, and the second temperature information is the lowest of the temperature information corresponding to the pixels in each thermal image sub-region.

10. The object recognition method according to claim 8, further comprising: Control information is provided to the thermal image sensor to notify or adjust the thermal image sensor's method of distinguishing the thermal image sub-areas.

Images