Distance estimation device and distance estimation method

Abstract

The present invention appropriately estimates a distance to a person even if a low-resolution image captured by a thermo-camera is used.　A distance estimation device 10 comprises: an acquisition unit 11 that acquires an image obtained by capturing with a thermo-camera 100; a person estimation unit 12 that detects a portion in the image of a person captured in the image acquired by the acquisition unit 11 and estimates a type of the person from the detected portion; and a distance estimation unit 13 that estimates a distance from the thermo-camera 100 to the person from a temperature of the portion of the person indicated by the image acquired by the acquisition unit 11 on the basis of a reference corresponding to the type of the person estimated by the person estimation unit 12.

Description

Distance Estimation Device and Distance Estimation Method

[0001] The present invention relates to a distance estimation device and a distance estimation method for estimating the distance to a person.

[0002] Patent Document 1 shows that an image obtained by an infrared camera is processed to measure the distance to a target.

[0003] Japanese Unexamined Patent Application Publication No. 10-281759

[0004] As shown in Patent Document 1, it is conceivable to estimate the distance from a thermal camera to a person shown in an image using the image obtained by imaging with the thermal camera. However, when a person is imaged with high resolution, there is a risk of privacy problems such as the ability to identify an individual from the captured image. Therefore, it is conceivable to use a low-resolution image captured by a thermal camera, which is difficult to identify an individual from the image, for distance estimation. However, when the resolution of the image is low, it becomes difficult to estimate an appropriate distance.

[0005] One embodiment of the present invention has been made in view of the above, and an object thereof is to provide a distance estimation device and a distance estimation method capable of appropriately estimating the distance to a person even when using a low-resolution image captured by a thermal camera.

[0006] To achieve the above object, a distance estimation device according to an embodiment of the present invention includes an acquisition unit that acquires an image obtained by imaging with a thermal camera, a person estimation unit that detects a portion in the image of a person shown in the image acquired by the acquisition unit and estimates the type of the person from the detected portion, and a distance estimation unit that estimates the distance from the thermal camera to the person from the temperature of the portion of the person shown by the image acquired by the acquisition unit based on a criterion corresponding to the type of the person estimated by the person estimation unit.

[0007] In a distance estimation device according to one embodiment of the present invention, the distance from the thermal camera to the person is estimated from the temperature of the part of the person shown in the acquired image, based on criteria corresponding to the type of person. As a result, the distance estimation device according to one embodiment of the present invention can appropriately estimate the distance to the person.

[0008] Incidentally, one embodiment of the present invention can be described as an invention of a distance estimation device as described above, or as an invention of a distance estimation method as described below. These are substantially the same invention, differing only in category, and produce similar functions and effects.

[0009] That is, a distance estimation method according to one embodiment of the present invention includes: an acquisition step in which a distance estimation device acquires an image obtained by imaging with a thermal camera; a person estimation step in which the distance estimation device detects a portion of a person in the image acquired in the acquisition step and estimates the type of person from the detected portion; and a distance estimation step in which the distance estimation device estimates the distance from the thermal camera to the person based on the temperature of the portion of the person shown in the image acquired in the acquisition step, according to a criterion corresponding to the type of person estimated in the person estimation step.

[0010] According to one embodiment of the present invention, even when using low-resolution images captured by a thermal camera, the distance to a person can be appropriately estimated.

[0011] This diagram shows the configuration of a distance estimation device according to an embodiment of the present invention. This is an example of an image used for distance estimation by the distance estimation device. This is an example of a histogram used for setting a threshold used for estimating the type of person. This flowchart shows the distance estimation method, which is a process performed by the distance estimation device according to an embodiment of the present invention. This diagram shows the hardware configuration of the distance estimation device according to an embodiment of the present invention.

[0012] Embodiments of the distance estimation device and distance estimation method according to the present invention will be described in detail below with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant explanations are omitted.

[0013] Figure 1 shows the distance estimation device 10 according to this embodiment. The distance estimation device 10 is a device (system) that analyzes images obtained by imaging with a thermal camera 100 and estimates the distance from the thermal camera 100 to a person shown in the image.

[0014] The image used for distance estimation by the distance estimation device 10 is a low-resolution image. Figure 2 shows an example of an image used for distance estimation by the distance estimation device 10. Note that the image in Figure 2 also shows parts P1 and P2 of a person (the parts indicated by rectangles in the image).

[0015] The image in Figure 2 has a resolution of 24 pixels vertically and 32 pixels horizontally. This image resolution is less than 1 / 10th the resolution of a typical commercially available infrared thermal camera. The distance estimation device 10 may perform downsampling to reduce the image resolution. For example, a 24-pixel vertical x 32-pixel horizontal image may be downsampled to an 18-pixel vertical x 24-pixel horizontal image, or a 12-pixel vertical x 16-pixel horizontal image.

[0016] As shown in Figure 2, the above images are of such low quality that it is difficult to smoothly identify the same person even when comparing two images of the same person. Furthermore, the above images are of such low quality that individual identification is impossible by visual inspection.

[0017] Thus, because the images used by the distance estimation device 10 are low-resolution, it becomes possible to estimate distances using images from locations where identifying individuals is inappropriate. In other words, the distance estimation device 10 can realize sensing technology that takes privacy into consideration. For example, the distance estimation device 10 can estimate distances in areas where privacy is a major concern. The distance estimated by the distance estimation device 10 can be used for any service. For example, the estimated distance may be used in medical and nursing care facilities to detect and alert about the risk of collisions between patients and medical personnel at night.

[0018] In this embodiment, the distance estimation device 10 can be a conventional server device or a computer such as a PC (personal computer). The distance estimation device 10 may also be a computer system including multiple computers. Furthermore, the distance estimation device 10 may have a communication function, enabling it to send and receive information with other devices.

[0019] The thermal camera 100 is a device that takes images and acquires images to be used by the distance estimation device 10. The thermal camera 100 takes images of the space to be imaged and acquires an image of the heat distribution in the space to be imaged. The thermal camera 100 may take continuous images of the space to be imaged and acquire moving images. As described above, the images taken by the thermal camera 100 are low-resolution images.

[0020] The thermal camera 100 is pre-positioned and fixedly installed at a location where it can capture images of a person whose distance is the target of the distance estimation device 10. Alternatively, the thermal camera 100 does not have to be fixedly installed. The thermal camera 100 has a communication function (for example, a wireless LAN (Local Area Network) communication function) and transmits the images obtained by imaging to the distance estimation device 10 using this communication function. A conventional thermal camera module (for example, an MLX90640 thermal camera unit) can be used as the thermal camera 100 (the unit that performs imaging).

[0021] Next, the functions of the distance estimation device 10 according to this embodiment will be described. As shown in Figure 1, the distance estimation device 10 according to this embodiment comprises an acquisition unit 11, a person estimation unit 12, and a distance estimation unit 13.

[0022] The acquisition unit 11 is a functional unit that acquires images obtained by imaging with the thermal camera 100. The acquisition unit 11 may acquire an image in which the vertical and horizontal pixels each have 32 or fewer pixels.

[0023] The acquisition unit 11 receives and acquires images transmitted from the thermal camera 100. As shown in the example in Figure 2, the images may be low-resolution images with 32 or fewer pixels in each direction (vertical and horizontal). The images may also be other than those described above, as long as they are images obtained by imaging with the thermal camera 100 and can be used for distance estimation. Furthermore, the acquisition unit 11 may acquire images obtained by imaging with the thermal camera 100 using methods other than those described above. The acquisition unit 11 outputs the acquired images to the person estimation unit 12 and the distance estimation unit 13.

[0024] The person estimation unit 12 is a functional unit that detects parts of a person in an image acquired by the acquisition unit 11 and estimates the type of person from the detected parts. The person estimation unit 12 may estimate whether the person is an adult or a child as the type of person. The person estimation unit 12 may detect multiple parts of a person and estimate the type of person from the sizes of those multiple parts. The person estimation unit 12 may detect the whole body and head as multiple parts of a person. The person estimation unit 12 detects parts of a person in an image and estimates the type of person, for example, as follows.

[0025] The person estimation unit 12 receives an image from the acquisition unit 11. The person estimation unit 12 detects parts of the image that depict a person. The parts of the person to be detected from the image are predetermined. There may be multiple parts of the person to be detected from the image. For example, the parts of the person to be detected from the image may be two parts: the whole body and the head. In addition, the parts of the person to be detected from the image may be other than those mentioned above, or there may be three or more parts.

[0026] The person estimation unit 12 detects parts of a person from an image according to a pre-stored method. For example, the person estimation unit 12 detects parts of a person from an image using a pre-stored model. The model used for detecting parts of a person is, for example, the following: The model is a trained model generated by machine learning. The model takes an image as input and outputs information indicating the location of parts of a person in the image. There may be separate models for each type of person part to be detected from the image. In the above example, two models may be used: one for detecting the whole body of a person and another for detecting the head of a person. Alternatively, one model may be used to detect multiple types of parts.

[0027] For example, the image shown in Figure 2 is the image input to the model. The positions of the rectangular parts P1 and P2, which are parts of a person in the image shown in Figure 2, are the information output from the model. In the image shown in Figure 2, rectangular part P1 is the whole body of the person, and rectangular part P2 is the head of the person. The model may be capable of detecting parts for each person if there are multiple people in the image. The model may also output an identifier (ID) that identifies the person in the image.

[0028] The model is composed of a neural network. The input layer of the model is provided with neurons for receiving an image. For example, the input layer of the model is provided with neurons that receive the pixel values ​​of a certain number of pixels in the image. The output layer of the model is provided with neurons for outputting information indicating the location of parts of a person in the image. For example, the output layer of the model is provided with neurons that output the location of parts of a person (for example, rectangular parts P1 and P2 in the image shown in Figure 2).

[0029] If the image acquired by the acquisition unit 11 is a moving image, the person estimation unit 12 may detect parts of a person in each image that makes up the moving image. That is, the person estimation unit 12 may detect the trajectory of parts of a person. In this case, the model may take a time-series of images as input and output information indicating the trajectory of parts of people that appear in the time-series of images. For example, the model may take a time-series of images as input in chronological order from oldest to newest and output an identifier (ID) that identifies the person in the image and information indicating the position of the part of that person for each input image.

[0030] The positional information of the parts of a person output from the model is arranged chronologically for each identifier that identifies that person, creating a trajectory of that person's parts. The person identifier is assigned by the model so that the same person is assigned the same identifier across images, and different people are assigned different identifiers.

[0031] In this case, the model may include a recurrent neural network and recursively use the input time-series images to calculate the position of the part of the person to be output. Specifically, each time an image is input, the model uses the information of that image to calculate the position of the part of the person depicted in that image, but this calculation uses the results of calculations performed within the model on previously input images.

[0032] The above model may be a conventional pre-trained model, such as DeepSORT, which tracks people. However, any model generated by machine learning that can be used to detect parts of people in time-series images is also acceptable.

[0033] The trained model used in the person estimation unit 12 is intended to be used as a program module that is part of artificial intelligence software. The model is used, for example, in a computer equipped with hardware such as a CPU (Central Processing Unit) and memory, and the computer's CPU operates according to instructions from the model stored in memory. For example, the computer's CPU operates in accordance with the instructions to input information to the model, perform calculations according to the model, and output a result from the model. Specifically, the computer's CPU operates in accordance with the instructions to input information to the input layer of the neural network, perform calculations based on the trained weighting coefficients etc. in the neural network, and output a result from the output layer of the neural network.

[0034] The person estimation unit 12 inputs the image received from the acquisition unit 11 into a pre-stored model, performs calculations according to the model, and obtains information indicating the position of a part of the person in the image as output from the model.

[0035] This document describes a method for generating a model used to detect parts of a person in an image, i.e., a model training method. Training images are obtained by imaging with a thermal camera (training thermal camera), and training position information indicating the location of parts of a person in the training images is acquired. Alternatively, the training position information may be obtained by acquiring a position estimation image, which is of a different type from the training image and corresponds to the training image, and estimating the location of the parts of a person in the acquired position estimation image. A model is then generated by performing machine learning based on the acquired training images and training position information.

[0036] The generation of the above model may be carried out specifically as follows. A training thermal camera and a training RGB camera are prepared for model generation. The training thermal camera is a device that takes images and acquires training images, which are images used for model generation. The training thermal camera has the same functions as the thermal camera 100 and acquires images similar to the images obtained by the thermal camera 100. Images similar to the images obtained by the thermal camera 100 are, for example, images with the same frame rate as the images obtained by the thermal camera 100. Furthermore, when obtaining moving images by the thermal camera 100, the training thermal camera acquires moving images similar to the moving images obtained by the thermal camera 100. Moving images similar to the moving images obtained by the thermal camera 100 are, for example, moving images with the same resolution and frame rate as the moving images obtained by the thermal camera 100.

[0037] Furthermore, images similar to those obtained by the thermal camera 100 do not necessarily need to have the same resolution and frame rate; any image that can be used to appropriately generate a model for estimating the position of a person's body part from the image obtained by the thermal camera 100 is sufficient.

[0038] The thermal camera used for learning may be the same as thermal camera 100, or of the same type as thermal camera 100. The thermal camera used for learning is pre-positioned and fixedly installed in a location where it can capture images of a person for creating a model.

[0039] A training RGB camera is a device that captures images to acquire images used for model generation. The images obtained by the training RGB camera are used to identify the location of parts of a person. The training RGB camera captures the space being captured by the training thermal camera at the same timing as when the training thermal camera is capturing images, and acquires a visible light image of the target area.

[0040] The image captured by the learning RGB camera is an image with a resolution that enables identification of the position of the human part. Unlike the images captured by the thermal camera 100 and the learning thermal camera, the image captured by the learning RGB camera does not necessarily need to be of low resolution. The learning RGB camera is pre-positioned and fixedly installed at a position corresponding to the position where the learning thermal camera is installed as described above. For example, the learning thermal camera and the learning RGB camera are installed side by side. As the functional part for performing the imaging of the learning RGB camera, a module of a conventional RGB camera can be used.

[0041] The learning thermal camera and the learning RGB camera provided at corresponding positions are used in combination.

[0042] For generating the model, a learning image obtained by imaging with the learning thermal camera is acquired. As described above, the learning image is a low-resolution image similar to the image acquired by the acquisition unit 11.

[0043] When imaging is performed with the learning thermal camera, the camera may be installed in the space of the imaging target of the learning thermal camera, etc., so that the learning image is an image in which a light that can be a heat source is reflected. That is, the learning image may be an image in which a person and a light are reflected. By using the learning image as such an image, the estimation accuracy of the model can be improved.

[0044] Note that the learning image may be other than the above as long as it corresponds to the image acquired by the acquisition unit 11 and is appropriately used for generating the model.

[0045] For generating the model, in addition to acquiring the learning image, learning position information indicating the position of the human part shown in the learning image is acquired. The learning position information is a label for model learning. The learning position information is, for example, information indicating the position of the human part shown in the learning image at the timing related to the learning image. The learning position information is information of the same type as the information output from the model.

[0046] For acquiring learning position information, an image obtained by imaging with a learning RGB camera is acquired. The image obtained by imaging with the learning RGB camera is an image using visible light (RGB image), and is different in type from the image obtained by imaging with the learning thermal camera.

[0047] The image for acquiring learning position information does not necessarily have to be an image using visible light (RGB image), as long as it can identify the position of the part of the person shown in the image for model generation. For example, the image for acquiring learning position information may be an image of the thermal distribution with a higher resolution than the image of the thermal distribution obtained by imaging with the thermal camera 100, and may be an image with high resolution that can appropriately identify the position of the part of the person.

[0048] The image obtained by imaging with the learning RGB camera is, for example, an image captured at the same timing as the learning image obtained by imaging with the learning thermal camera. This is for associating the learning image with the position of the part of the person indicated by the learning position information, as will be described later. However, if this association is possible, the image transmitted from the learning RGB camera does not have to be the above.

[0049] From the image obtained by imaging with the learning RGB camera, a part in the image of the person shown in the image is detected. The detection of the part in the image of the person may be performed by a conventional method, for example, a method using a conventional person detection model (YOLO (You Only Look Once)). As described above, the image obtained by imaging with the learning RGB camera is different from the images obtained by imaging with the above-described thermal camera 100 and the learning thermal camera, and does not have to be low resolution, so the position of the person can be detected by a conventional method.

[0050] Based on the information indicating the position of a part of the person's image obtained as described above, training position information is generated. Since the training thermal camera and the training RGB camera are positioned at different locations, the relative position of the person with respect to the training thermal camera is different from the relative position of the person with respect to the training RGB camera. Therefore, based on the positional relationship (relative position) between the training thermal camera and the training RGB camera, the position of the part of the person's image obtained from the image captured by the training RGB camera is corrected so that it becomes the position detected by the training thermal camera, thereby generating training position information. The above correction based on the positional relationship is, for example, a linear shift correction, which can be performed by conventional methods.

[0051] Machine learning is performed to generate a model using training images as input and training location information as the model's output (ground truth). The training of the machine learning process itself, i.e., updating the model parameters, can be done in the same way as conventional machine learning training.

[0052] When the model takes time-series images as input, it uses time-series training images as input to the model, and training location information corresponding to each training image as the model's output (ground truth) to generate the model through machine learning. When time-series training images are input to the model, the time-series training images are input in chronological order, starting with the oldest. Furthermore, when training location information is output from the model, the training location information at the time corresponding to the training image input to the model is made to correspond to the model's output.

[0053] In the model, the number of past images used recursively (the number of past images traced back from the input image) is set in advance. Low-resolution images make it difficult to obtain the physical characteristics of a person. Therefore, if too many past images are used, the posture and orientation of the person will differ too much, making it difficult to associate images with the same person. For this reason, the number of past images used recursively may be kept to a minimum. The number of past images used recursively may be set to 10, for example, through parameter tuning. The above is a method for generating a model used to detect parts of a person in an image.

[0054] The person estimation unit 12 may use a model other than the one described above to detect the person's portion from the image input from the acquisition unit 11. Alternatively, the person estimation unit 12 may use a pre-stored method other than the model to detect the person's portion from the image input from the acquisition unit 11.

[0055] The person estimation unit 12 estimates the type of person from the detected portion. The estimated type of person is used to determine the criterion for distance estimation, as will be described later. The temperature of a person shown in the image obtained by the thermal camera may vary depending on the type of person. This type is a type of person that can distinguish the temperature. For example, this type could be an adult or a child. Adults and children may be distinguished by age. Generally, a child's body temperature is higher than an adult's, and the temperature of a child shown in the image will be higher than that of an adult.

[0056] The person estimation unit 12 identifies the size of each detected part. For example, the person estimation unit 12 identifies the vertical length of the detected part as its size. For example, if the imaging direction of the thermal camera 100 is horizontal, the vertical direction of the image corresponds to the vertical direction. Therefore, in this case, the vertical length of the part is the vertical length of the part in the image. The person estimation unit 12 may also identify a value other than the above as the size of the detected part.

[0057] If the parts to be detected are the whole body and the head of a person, the person estimation unit 12 calculates the ratio of the whole body size to the head size. For example, the person estimation unit 12 calculates (whole body size) / (head size). This value becomes the head-to-body ratio of the person detected from the image.

[0058] The person estimation unit 12 estimates the type of person from the size-related values ​​using pre-stored criteria. For example, the person estimation unit 12 compares the calculated ratio value with a pre-set and stored threshold value to estimate whether the person is an adult or a child. If the ratio value is greater than or equal to the threshold value, the person estimation unit 12 classifies (estimates) the person as an adult. If the ratio value is less than the threshold value, the person estimation unit 12 classifies (estimates) the person as a child.

[0059] The thresholds mentioned above are set in advance, for example, by the following method: A person whose status as an adult or child is known in advance is imaged using a thermal camera. From the images obtained from the imaging, the ratio of the whole body size to the head size is calculated for that person in the same manner as above. This sampling is performed on a large number of adults and children. Figure 3 shows a histogram of the sampling results. In the histogram, the horizontal axis is the head-to-body ratio value (the ratio value mentioned above), and the vertical axis is the frequency (number of people). In Figure 3, the data for adults and children are shown with different hatching. The data distributed towards the larger head-to-body ratio is for adults, and the data distributed towards the smaller head-to-body ratio is for children.

[0060] The threshold value is the head-to-body ratio at the boundary between adults and children in the histogram. More specifically, for example, the threshold value is the head-to-body ratio at the boundary corresponding to the position of the deepest valley between two peaks in the histogram (modal method).

[0061] Furthermore, the person estimation unit 12 may estimate the type of person by methods other than those described above, as long as it estimates the type of person from a portion of the image of the detected person. Also, the estimated type does not have to be adult or child; it may be any other type that can be used to determine the criterion for distance estimation. In addition, there may be three or more estimated types.

[0062] The person estimation unit 12 outputs information indicating the part of the image of the detected person, and information indicating the type of person the estimated person is, to the distance estimation unit 13.

[0063] The distance estimation unit 13 is a functional unit that estimates the distance from the thermal camera 100 to the person based on criteria corresponding to the type of person estimated by the person estimation unit 12, using the temperature of the part of the person shown in the image acquired by the acquisition unit 11. The distance estimation unit 13 may also estimate the distance from the thermal camera 100 to the person from statistical values ​​of the temperature of the part of the person shown in the image. The distance estimation unit 13 estimates the above distance, for example, as follows.

[0064] The distance estimation unit 13 receives an image from the acquisition unit 11. The distance estimation unit 13 receives information from the person estimation unit 12 indicating the part of the image in which a person has been detected, and information indicating the estimated type of the person.

[0065] The distance estimation unit 13 generates (calculates) information used for distance estimation from the temperature of the part of the person indicated by the input image. The information used for distance estimation is, for example, the following values. The part used for calculation may be one of several parts detected by the person estimation unit 12. The part used for calculation should be a part that appropriately reflects the temperature of the person. For this reason, the part used for calculation may be the head of the person. Alternatively, the person estimation unit 12 may detect a part to be used for calculation by the distance estimation unit 13 (for example, a specific part such as the face), and the distance estimation unit 13 may calculate a value used for distance estimation from the temperature of that part.

[0066] The distance estimation unit 13 calculates a statistical value of the temperature of the relevant portion as a value to be used for estimating the distance. Specifically, for example, the distance estimation unit 13 calculates the average temperature of the relevant portion.

[0067] The distance estimation unit 13 stores in advance a criterion used to estimate the distance from the temperature of a part of a person. This criterion is pre-set. This criterion shows the correspondence (correlation) between the temperature of a part of a person (for example, the average temperature of that part) and the distance. This correspondence utilizes the property that far-infrared energy from an object is absorbed into the atmosphere and attenuates inversely proportional to the square of the distance. Therefore, the correspondence increases as the temperature decreases. This correspondence can be generated (calculated) by prior measurement.

[0068] The distance estimation unit 13 stores the criteria for each type of person. For example, if the type of person is either an adult or a child, as described above, the distance estimation unit 13 stores the criteria for adults and the criteria for children. As mentioned above, a child's body temperature is usually higher than an adult's, so for the same temperature, the distance will be greater for a child than for an adult.

[0069] The distance estimation unit 13 uses a criterion corresponding to the type of person indicated by the information input from the person estimation unit 12 as the criterion for estimating the distance. Based on this criterion, the distance estimation unit 13 estimates the above distance from the average temperature of the part of the person calculated.

[0070] Furthermore, the criteria used for estimating distance may be other than those described above, as long as the distance can be estimated from the temperature of the part of the person shown in the image. The distance estimation unit 13 may also estimate the distance using methods other than those described above, as long as the estimation is based on the temperature of the part of the person according to the type of person. Distance estimation in the distance estimation device 10 may be performed in real time.

[0071] The distance estimation unit 13 outputs information indicating the distance estimation result. For example, the distance estimation unit 13 transmits information indicating the estimation result to the device in which the distance is used. Alternatively, the distance estimation unit 13 may output information indicating the estimation result in a format that is recognizable to the user. For example, the distance estimation unit 13 may display information indicating the estimation result on a display device provided by the distance estimation device 10. The output of information indicating the estimation result from the distance estimation unit 13 may be performed by methods other than those described above, and to output destinations other than those described above.

[0072] Furthermore, if the image acquired by the acquisition unit 11 is a moving image, the distance estimation device 10 may estimate the time-series distance from each image of the moving image. This time-series distance may be used to estimate whether the person is approaching or moving away from the thermal camera 100. The speed of these movements may also be estimated. This determination may be made by the distance estimation device 10. The above describes the functions of the distance estimation device 10 according to this embodiment.

[0073] Next, using the flowchart in Figure 4, we will explain the distance estimation method, which is the process (operation method performed by the distance estimation device 10) executed by the distance estimation device 10 according to this embodiment.

[0074] In this process, first, the acquisition unit 11 acquires an image obtained by imaging with the thermal camera 100 (S01, acquisition step). Next, the person estimation unit 12 detects a portion of the image of a person that is captured in the image acquired by the acquisition unit 11 (S02, person estimation step). Subsequently, the person estimation unit 12 estimates the type of person from the detected portion (S03, person estimation step).

[0075] Next, the distance estimation unit 13 estimates the distance from the thermal camera 100 to the person based on the temperature of the part of the person shown in the image acquired by the acquisition unit 11, using a criterion corresponding to the type of person estimated by the person estimation unit 12 (S04, distance estimation step). Subsequently, the distance estimation unit 13 outputs information indicating the estimation result (S05). The above is the distance estimation method according to this embodiment.

[0076] In this embodiment, the distance from the thermal camera 100 to the person is estimated from the temperature of the part of the person shown in the acquired image, based on criteria corresponding to the type of person. As a result, according to this embodiment, the distance to the person can be appropriately estimated.

[0077] Furthermore, based on the above estimation, this embodiment allows for accurate estimation of the distance to a person even when using low-resolution images captured by the thermal camera 100. Using low-resolution images also enables distance estimation that respects privacy and does not identify individuals. Additionally, distance estimation can be performed using only a single thermal camera, making it a cost-effective method.

[0078] Furthermore, as in this embodiment, the person estimation unit 12 may also estimate whether the person is an adult or a child. With this configuration, the person's type can be appropriately estimated from the image, and the distance can be estimated based on appropriate criteria. However, the person's type does not necessarily have to be an adult or a child; it is sufficient if it can be estimated from the image and appropriate criteria can be selected for estimating the distance.

[0079] The person estimation unit 12 may detect multiple parts of a person and estimate the person's species from the sizes of those parts. In this case, the person estimation unit 12 may also detect the person's whole body and head as multiple parts of the person. With this configuration, the person's species can be estimated appropriately and reliably, and as a result, the distance to the person can be estimated appropriately and reliably. However, the parts used to estimate the person's species do not necessarily have to be the whole body and head. Alternatively, the person's species may be estimated from just one part of the person.

[0080] The distance estimation unit 13 may estimate the distance from the thermal camera 100 to the person based on statistical values ​​of the temperature of the part of the person shown in the image. This configuration allows for accurate and reliable estimation of the distance to the person. However, it is not always necessary to use statistical values ​​of the temperature of the part of the person to estimate the distance to the person.

[0081] Furthermore, as mentioned above, the image used for distance estimation may be one with 32 or fewer pixels in each direction (vertical and horizontal). However, the image used for distance estimation may have a resolution other than that specified above.

[0082] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more of the above devices.

[0083] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0084] For example, the distance estimation device 10 in one embodiment of the present disclosure may function as a computer that performs information processing according to the present disclosure. Figure 5 is a diagram showing an example of the hardware configuration of the distance estimation device 10 according to one embodiment of the present disclosure. The distance estimation device 10 described above may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0085] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the distance estimation device 10 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0086] Each function in the distance estimation device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.

[0087] The processor 1001 controls the entire computer, for example, by running the operating system. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. For example, each function in the distance estimation device 10 described above may be implemented by the processor 1001.

[0088] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each function of the distance estimation device 10 may be implemented by a control program stored in the memory 1002 and operated on the processor 1001. Although the above-described processes have been explained as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from the network via a telecommunications line.

[0089] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for carrying out information processing according to one embodiment of the present disclosure.

[0090] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The storage medium provided by the distance estimation device 10 may be, for example, a database, server, or other suitable medium including at least one of the memory 1002 and the storage 1003.

[0091] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0092] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0093] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0094] Furthermore, the distance estimation device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0095] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements using exemplary order and are not limited to the specific order presented.

[0096] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0097] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0098] Each aspect / embodiment described in this disclosure may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0099] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0100] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0101] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0102] The terms “system” and “network” as used in this disclosure are interchangeable.

[0103] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0104] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0105] The terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables, and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain, and optical (both visible and invisible) domain.

[0106] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0107] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0108] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0109] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0110] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0111] The distance estimation device and distance estimation method disclosed herein have the following configurations: [1] A distance estimation device comprising: an acquisition unit that acquires an image obtained by imaging with a thermal camera; a person estimation unit that detects a portion of a person in the image acquired by the acquisition unit and estimates the type of person from the detected portion; and a distance estimation unit that estimates the distance from the thermal camera to the person based on the temperature of the portion of the person shown in the image acquired by the acquisition unit, according to a criterion corresponding to the type of person estimated by the person estimation unit. [2] The distance estimation device according to [1], wherein the person estimation unit estimates whether the person is an adult or a child as the type of person. [3] The distance estimation device according to [1] or [2], wherein the person estimation unit detects multiple parts of the person and estimates the type of person from the size of the multiple parts. [4] The distance estimation device according to [3], wherein the person estimation unit detects the whole body and head of the person as multiple parts of the person. [5] A distance estimation device according to any one of [1] to [4], wherein the distance estimation unit estimates the distance from the thermal camera to the person from statistical values ​​of the temperature of the part of the person shown in the image. [6] A distance estimation device according to any one of [1] to [5], wherein the acquisition unit acquires an image as the image, with each vertical and horizontal pixel count being 32 or less. [7] A distance estimation method comprising: an acquisition step in which the distance estimation device acquires an image obtained by imaging with a thermal camera; a person estimation step in which the distance estimation device detects a part of the image of a person shown in the image acquired in the acquisition step and estimates the type of person from the detected part; and a distance estimation step in which the distance estimation device estimates the distance from the thermal camera to the person from the temperature of the part of the person shown in the image acquired in the acquisition step, based on a criterion corresponding to the type of person estimated in the person estimation step.

[0112] 10... Distance estimation device, 11... Acquisition unit, 12... Person estimation unit, 13... Distance estimation unit, 100... Thermal camera, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. A distance estimation device comprising: an acquisition unit that acquires an image obtained by imaging with a thermal camera; a person estimation unit that detects a portion of a person in the image acquired by the acquisition unit and estimates the type of person from the detected portion; and a distance estimation unit that estimates the distance from the thermal camera to the person based on the temperature of the portion of the person shown in the image acquired by the acquisition unit, according to a criterion corresponding to the type of person estimated by the person estimation unit.

2. The distance estimation device according to claim 1, wherein the person estimation unit estimates whether the person is an adult or a child.

3. The distance estimation device according to claim 1, wherein the person estimation unit detects multiple parts of the person and estimates the type of the person from the sizes of the multiple parts.

4. The distance estimation device according to claim 3, wherein the person estimation unit detects the whole body and head of the person as multiple parts of the person.

5. The distance estimation device according to claim 1, wherein the distance estimation unit estimates the distance from the thermal camera to the person from statistical values ​​of the temperature of the part of the person shown in the image.

6. The distance estimation device according to claim 1, wherein the acquisition unit acquires an image in which the vertical and horizontal pixels each number 32 or less.

7. A distance estimation method comprising: an acquisition step in which a distance estimation device acquires an image obtained by imaging with a thermal camera; a person estimation step in which the distance estimation device detects a portion of a person in the image acquired in the acquisition step and estimates the type of person from the detected portion; and a distance estimation step in which the distance estimation device estimates the distance from the thermal camera to the person based on the temperature of the portion of the person shown in the image acquired in the acquisition step, according to a criterion corresponding to the type of person estimated in the person estimation step.

Images