Worn article estimation device and worn article estimation method

Abstract

To appropriately estimate a worn article of a person in consideration of privacy issues.　A worn article estimation device 10 comprises: an acquisition unit 11 that acquires an image obtained by imaging of a thermal camera 100; a person detection unit 12 that detects a portion in the image of a person in the image acquired by the acquisition unit 11; and a worn article estimation unit 13 that estimates a worn article of the person from a temperature of the portion of the person detected by the person detection unit 12, the temperature being indicated by the image acquired by the acquisition unit 11.

Description

Wearing item estimation device and wearing item estimation method

[0001] The present invention relates to a wearing item estimation device and a wearing item estimation method for estimating a person's wearing item.

[0002] Patent Document 1 shows that it is possible to determine the type of clothing worn by a person depicted in an image from the image.

[0003] International Publication No. 2018 / 163238

[0004] As shown in Patent Document 1, when using an RGB image for estimating a person's wearing item, there is a risk of privacy problems such as enabling the identification of an individual from the captured image.

[0005] One embodiment of the present invention has been made in view of the above, and an object thereof is to provide a wearing item estimation device and a wearing item estimation method that can appropriately estimate a person's wearing item in consideration of privacy problems.

[0006] In order to achieve the above object, a wearing item estimation device according to an embodiment of the present invention includes an acquisition unit that acquires an image obtained by imaging with a thermal camera, a person detection unit that detects a portion in the image of a person depicted in the image acquired by the acquisition unit, and a wearing item estimation unit that estimates the wearing item of the person from the temperature of the portion of the person detected by the person detection unit, indicated by the image acquired by the acquisition unit.

[0007] In the wearing item estimation device according to an embodiment of the present invention, the wearing item of the person is estimated from the temperature of the portion in the image of the person depicted in the acquired image. As the image used for the estimation, a low-resolution image in which it is difficult to identify an individual from the image can be used. Therefore, according to the wearing item estimation device according to an embodiment of the present invention, it is possible to appropriately estimate a person's wearing item in consideration of privacy problems.

[0008] By the way, one embodiment of the present invention can be described as an invention of a wearing item estimation device as described above, and can also be described as an invention of a wearing item estimation method as follows. These are inventions with different categories, but are substantially the same invention and exhibit the same operations and effects.

[0009] That is, a clothing estimation method according to one embodiment of the present invention includes an acquisition step in which a clothing estimation device acquires an image obtained by imaging with a thermal camera; a person detection step in which the clothing estimation device detects a portion of a person in the image acquired in the acquisition step; and a clothing estimation step in which the clothing estimation device estimates the clothing of a person from the temperature of the portion of the person detected in the person detection step, which is shown by the image acquired in the acquisition step.

[0010] According to one embodiment of the present invention, it is possible to appropriately estimate a person's clothing while taking privacy issues into consideration.

[0011] This figure shows the configuration of the clothing estimation device according to an embodiment of the present invention. This is an example of an image used by the clothing estimation device to estimate a person's clothing. This is an example of a histogram, which is frequency information for each temperature, used to estimate a person's clothing. This is a flowchart of the clothing estimation method, which is a process performed by the clothing estimation device according to an embodiment of the present invention. This figure shows the hardware configuration of the clothing estimation device according to an embodiment of the present invention.

[0012] Embodiments of the garment estimation device and garment estimation method according to the present invention will be described in detail below with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant explanations are omitted.

[0013] Figure 1 shows the clothing estimation device 10 according to this embodiment. The clothing estimation device 10 is a device (system) that analyzes images obtained by imaging with a thermal camera 100 to estimate the clothing worn by a person in the image.

[0014] The clothing estimated by the clothing estimation device 10 may be, for example, the upper body clothing, such as whether it is a long-sleeved or short-sleeved garment. Alternatively, the clothing estimated by the clothing estimation device 10 may be, for example, the lower body clothing, such as whether it is long trousers or shorts. Thus, the clothing estimated by the clothing estimation device 10 may be for each part of the person (for example, each part of the upper body or each part of the lower body).

[0015] The images used by the clothing estimation device 10 to estimate a person's clothing are low-resolution images. Figure 2 shows an example of an image used by the clothing estimation device 10 to estimate a person's clothing. Note that the image in Figure 2 also shows a portion of the person P1 (the portion indicated by the rectangle in the image).

[0016] The image in Figure 2 has a resolution of 24 pixels vertically and 32 pixels horizontally. This image resolution is less than 1 / 10th the resolution of a typical commercially available infrared thermal camera. The garment estimation device 10 may perform downsampling to reduce the image resolution. For example, a 24-pixel vertical x 32-pixel horizontal image may be downsampled to an 18-pixel vertical x 24-pixel horizontal image, or a 12-pixel vertical x 16-pixel horizontal image.

[0017] As shown in Figure 2, the above images are of such low quality that it is difficult to smoothly identify the same person even when comparing two images of the same person. Furthermore, the above images are of such low quality that individual identification is impossible by visual inspection.

[0018] Thus, because the images used by the clothing estimation device 10 are low resolution, it is possible to estimate the clothing worn by a person using images from locations where it is not appropriate to identify an individual. In other words, the clothing estimation device 10 can realize sensing technology that takes privacy into consideration. For example, the clothing estimation device 10 estimates the clothing worn by a person in an area where privacy is a major concern. The clothing worn by a person estimated by the clothing estimation device 10 can be used for various services. For example, it may be used to enhance security services in private spaces. Specifically, it may be used as an alternative to surveillance cameras in private spaces such as restrooms to identify the characteristics of a suspicious person who has fled by identifying their clothing. Alternatively, it may be used for comprehensive searching of lost persons within a facility, including areas where privacy needs to be considered. Alternatively, it may be used for automating the control of air conditioning equipment in private spaces such as hotels.

[0019] In this embodiment, the clothing estimation device 10 can be a conventional server device or a computer such as a PC (personal computer). Alternatively, the clothing estimation device 10 may be a computer system including multiple computers. Furthermore, the clothing estimation device 10 may have a communication function, enabling it to send and receive information with other devices.

[0020] The thermal camera 100 is a device that takes images to acquire images used by the clothing estimation device 10. The thermal camera 100 takes images of the space to be imaged and acquires an image of the heat distribution in the space to be imaged. The thermal camera 100 may take continuous images of the space to be imaged and acquire moving images. As described above, the images taken by the thermal camera 100 are low-resolution images.

[0021] The thermal camera 100 is pre-positioned and fixedly installed in a location where it can capture images of a person wearing clothing that is the target of the clothing estimation device 10. Alternatively, the thermal camera 100 does not have to be fixedly installed. The thermal camera 100 has a communication function (for example, a wireless LAN (Local Area Network) communication function) and transmits the images obtained by imaging to the clothing estimation device 10 using this communication function. A conventional thermal camera module (for example, an MLX90640 thermal camera unit) can be used as the imaging function of the thermal camera 100.

[0022] Next, the functions of the clothing estimation device 10 according to this embodiment will be described. As shown in Figure 1, the clothing estimation device 10 according to this embodiment comprises an acquisition unit 11, a person detection unit 12, and a clothing estimation unit 13.

[0023] The acquisition unit 11 is a functional unit that acquires images obtained by imaging with the thermal camera 100. The acquisition unit 11 may acquire an image in which the vertical and horizontal pixels each have 32 or fewer pixels.

[0024] The acquisition unit 11 receives and acquires images transmitted from the thermal camera 100. As shown in the example in Figure 2, the images may be low-resolution images with 32 or fewer pixels in each direction (vertical and horizontal). However, the images may be other than those described above, as long as they are images obtained by imaging with the thermal camera 100 and can be used to estimate the clothing worn by a person. The acquisition unit 11 may also acquire images obtained by imaging with the thermal camera 100 by methods other than those described above. The acquisition unit 11 outputs the acquired images to the person detection unit 12 and the clothing estimation unit 13.

[0025] The person detection unit 12 is a functional unit that detects parts of a person in an image acquired by the acquisition unit 11. The person detection unit 12 detects parts of a person in an image as follows, for example.

[0026] The person detection unit 12 receives an image from the acquisition unit 11. The person detection unit 12 detects a part of the image containing a person. The part of the person to be detected from the image is predetermined. The part of the person to be detected from the image may be the entire person, or it may be a part of the person's clothing that is subject to estimation (for example, the upper body or the lower body). If there are multiple parts of the person's clothing that are subject to estimation, multiple parts of the person to be detected from the image may be detected. If there are multiple parts of the person's clothing that are subject to estimation, the person's clothing may be estimated for each part as follows.

[0027] The person detection unit 12 detects parts of a person from an image according to a pre-stored method. For example, the person detection unit 12 detects parts of a person from an image using a pre-stored model. The model used for detecting parts of a person is, for example, the following: The model is a trained model generated by machine learning. The model takes an image as input and outputs information indicating the position of the part of the person in the image that is depicted in the image. There may be separate models for each type of part of a person to be detected from the image. For example, two models may be used: one for detecting the upper body of a person and another for detecting the lower body of a person. Alternatively, one model may be used to detect multiple types of parts.

[0028] For example, the image shown in Figure 2 is the image input to the model. The position of the rectangular portion P1, which represents the part of the person in the image shown in Figure 2, is the information output from the model. In the image shown in Figure 2, the upper rectangular portion P1 represents the upper body of the person, and the lower rectangular portion P1 represents the lower body of the person. The model may also be capable of detecting parts for each person if there are multiple people in the image. The model may also output an identifier (ID) that identifies the person in the image.

[0029] The model is composed of a neural network. The input layer of the model is provided with neurons for receiving an image. For example, the input layer of the model is provided with neurons that receive the pixel values ​​of a certain number of pixels in the image. The output layer of the model is provided with neurons for outputting information indicating the location of a part of a person in the image. For example, the output layer of the model is provided with neurons that output the location of a part of a person (for example, the rectangular part P1 in the image shown in Figure 2).

[0030] If the image acquired by the acquisition unit 11 is a moving image, the person detection unit 12 may detect a person's part in each image that makes up the moving image. That is, the person detection unit 12 may detect the trajectory of a person's part. In this case, the model may take a time-series of images as input and output information indicating the trajectory of a person's part in the time-series of images. For example, the model may take a time-series of images as input in chronological order from oldest to newest and output an identifier (ID) that identifies the person in the image and information indicating the position of a part of that person for each input image.

[0031] The positional information of the parts of a person output from the model is arranged chronologically for each identifier that identifies that person, creating a trajectory of that person's parts. The person identifier is assigned by the model so that the same person is assigned the same identifier across images, and different people are assigned different identifiers.

[0032] In this case, the model may include a recurrent neural network and recursively use the input time-series images to calculate the position of the part of the person to be output. Specifically, each time an image is input, the model uses the information of that image to calculate the position of the part of the person depicted in that image, but this calculation uses the results of calculations performed within the model on previously input images.

[0033] The above model may be a conventional pre-trained model, such as DeepSORT, which tracks people. However, any model generated by machine learning that can be used to detect parts of people in time-series images is also acceptable.

[0034] The trained model used in the person detection unit 12 is intended to be used as a program module that is part of artificial intelligence software. The model is used, for example, in a computer equipped with hardware such as a CPU (Central Processing Unit) and memory, and the computer's CPU operates according to instructions from the model stored in memory. For example, the computer's CPU operates in accordance with the instructions to input information to the model, perform calculations according to the model, and output a result from the model. Specifically, the computer's CPU operates in accordance with the instructions to input information to the input layer of the neural network, perform calculations based on the trained weighting coefficients in the neural network, and output a result from the output layer of the neural network.

[0035] The person detection unit 12 inputs the image received from the acquisition unit 11 into a pre-stored model, performs calculations according to the model, and obtains information indicating the position of a part of the person in the image as output from the model.

[0036] This document describes a method for generating a model used to detect parts of a person in an image, i.e., a model training method. Training images are obtained by imaging with a thermal camera (training thermal camera), and training position information indicating the location of parts of a person in the training images is acquired. Alternatively, the training position information may be obtained by acquiring a position estimation image, which is of a different type from the training image and corresponds to the training image, and estimating the location of the parts of a person in the acquired position estimation image. A model is then generated by performing machine learning based on the acquired training images and training position information.

[0037] The generation of the above model may be carried out specifically as follows. A training thermal camera and a training RGB camera are prepared for model generation. The training thermal camera is a device that takes images and acquires training images, which are images used for model generation. The training thermal camera has the same functions as the thermal camera 100 and acquires images similar to the images obtained by the thermal camera 100. Images similar to the images obtained by the thermal camera 100 are, for example, images with the same frame rate as the images obtained by the thermal camera 100. Furthermore, when obtaining moving images by the thermal camera 100, the training thermal camera acquires moving images similar to the moving images obtained by the thermal camera 100. Moving images similar to the moving images obtained by the thermal camera 100 are, for example, moving images with the same resolution and frame rate as the moving images obtained by the thermal camera 100.

[0038] Furthermore, images similar to those obtained by the thermal camera 100 do not necessarily need to have the same resolution and frame rate; any image that can be used to appropriately generate a model for estimating the position of a person's body part from the image obtained by the thermal camera 100 is sufficient.

[0039] The learning thermo camera may be the same as or of the same type as the thermo camera 100. The learning thermo camera is pre-positioned and fixedly installed at a position where it can image a person for creating a model.

[0040] The learning RGB camera is a device that captures images and acquires images used for generating a model. The images obtained by the learning RGB camera are used to identify the positions of parts of a person. The learning RGB camera images the space of the imaging target of the learning thermo camera at the same timing as when imaging is performed by the learning thermo camera, and acquires an image of visible light in the imaging target.

[0041] The image captured by the learning RGB camera is an image with a resolution that enables identification of the positions of parts of a person. Unlike the images captured by the thermo camera 100 and the learning thermo camera, the image captured by the learning RGB camera does not necessarily have to be a low-resolution image. The learning RGB camera is pre-positioned and fixedly installed at a position corresponding to the position where the learning thermo camera is installed as described above. For example, the learning thermo camera and the learning RGB camera are installed side by side. As the functional part for performing imaging of the learning RGB camera, a module of a conventional RGB camera can be used.

[0042] The learning thermo camera and the learning RGB camera provided at corresponding positions to each other are used in combination.

[0043] For generating a model, a learning image obtained by imaging with the learning thermo camera is acquired. As described above, the learning image is a low-resolution image similar to the image acquired by the acquisition unit 11.

[0044] When imaging is performed by the learning thermo camera, the learning image may be an image in which a light that can be a heat source is reflected by installing a camera in the space of the imaging target of the learning thermo camera. That is, the learning image may be an image in which a person and a light are reflected. By making the learning image such an image, the estimation accuracy of the model can be improved.

[0045] Note that the learning image may be other than the above as long as it corresponds to the image acquired by the acquisition unit 11 and can be appropriately used for generating the model.

[0046] For generating the model, in addition to acquiring the learning image, learning position information indicating the position of the person's part shown in the learning image is acquired. The learning position information is a label for model learning. The learning position information is, for example, information indicating the position of the person's part shown in the learning image at the timing related to the learning image. The learning position information is the same type of information as the information output from the model.

[0047] To acquire the learning position information, an image obtained by imaging with the learning RGB camera is acquired. The image obtained by imaging with the learning RGB camera is an image by visible light (RGB image), and is different in type from the image obtained by imaging with the learning thermal camera.

[0048] The image for acquiring the learning position information does not necessarily have to be an image by visible light (RGB image), as long as it can identify the position of the person's part shown for generating the model. For example, the image for acquiring the learning position information may be an image of the thermal distribution with a higher resolution than the image of the thermal distribution obtained by imaging with the thermal camera 100, and may be an image with high resolution that can appropriately identify the position of the person's part.

[0049] The image obtained by imaging with the learning RGB camera is, for example, an image imaged at the same timing as the learning image obtained by imaging with the learning thermal camera. This is for associating the learning image with the position of the person's part indicated by the learning position information as described later. However, as long as this association is possible, the image transmitted from the learning RGB camera does not have to be the above.

[0050] The system detects the portion of a person in an image from an image obtained by a training RGB camera. The detection of the portion of a person can be performed using conventional methods, such as conventional person detection models (YOLO (You Only Look Once)). As mentioned above, the image obtained by the training RGB camera does not need to be low resolution, unlike the image obtained by the thermal camera 100 and the training thermal camera, so the position of the person can be detected using conventional methods.

[0051] Based on the information indicating the position of a part of the person's image obtained as described above, training position information is generated. Since the training thermal camera and the training RGB camera are positioned at different locations, the relative position of the person with respect to the training thermal camera is different from the relative position of the person with respect to the training RGB camera. Therefore, based on the positional relationship (relative position) between the training thermal camera and the training RGB camera, the position of the part of the person's image obtained from the image captured by the training RGB camera is corrected so that it becomes the position detected by the training thermal camera, thereby generating training position information. The above correction based on the positional relationship is, for example, a linear shift correction, which can be performed by conventional methods.

[0052] Machine learning is performed to generate a model using training images as input and training location information as the model's output (ground truth). The training of the machine learning process itself, i.e., updating the model parameters, can be done in the same way as conventional machine learning training.

[0053] When the model takes time-series images as input, it uses time-series training images as input to the model, and training location information corresponding to each training image as the model's output (ground truth) to generate the model through machine learning. When time-series training images are input to the model, the time-series training images are input in chronological order, starting with the oldest. Furthermore, when training location information is output from the model, the training location information at the time corresponding to the training image input to the model is made to correspond to the model's output.

[0054] In the model, the number of past images used recursively (the number of past images traced back from the input image) is set in advance. Low-resolution images make it difficult to obtain the physical characteristics of a person. Therefore, if too many past images are used, the posture and orientation of the person will differ too much, making it difficult to associate images with the same person. For this reason, the number of past images used recursively may be kept to a minimum. The number of past images used recursively may be set to 10, for example, through parameter tuning. The above is a method for generating a model used to detect parts of a person in an image.

[0055] The person detection unit 12 may use a model other than the one described above to detect the person's portion from the image input from the acquisition unit 11. Alternatively, the person detection unit 12 may use a pre-stored method other than the model to detect the person's portion from the image input from the acquisition unit 11. The person detection unit 12 outputs information indicating the portion of the detected person's image to the clothing estimation unit 13.

[0056] The clothing estimation unit 13 is a functional unit that estimates the clothing worn by a person based on the temperature of parts of the person detected by the person detection unit 12, as shown by the image acquired by the acquisition unit 11. The clothing estimation unit 13 may also estimate the clothing worn by the person based on the frequency of each temperature of the person's body part. The clothing estimation unit 13 may also estimate the length of the clothing worn by the person. The clothing estimation unit 13 estimates the clothing worn by a person as follows, for example.

[0057] The clothing estimation unit 13 estimates, for example, the length of a person's clothing. Specifically, the clothing estimation unit 13 estimates whether the person is wearing long-sleeved or short-sleeved clothing. That is, the clothing estimation unit 13 estimates the length of the sleeves the person is wearing. Alternatively, the clothing estimation unit 13 estimates whether the person is wearing long trousers or short trousers. That is, the clothing estimation unit 13 estimates the length of the trousers the person is wearing. The clothing estimation unit 13 may also estimate the length of other clothing items (for example, skirts, coats, or jackets). Furthermore, the clothing estimation unit 13 may perform estimations of clothing items other than the length of a person's clothing.

[0058] The clothing estimation unit 13 receives an image from the acquisition unit 11. The clothing estimation unit 13 receives information from the person detection unit 12 indicating the part of the image of the person detected from the image.

[0059] The clothing estimation unit 13 generates (calculates) information used to estimate the clothing of a person from the temperature of the part of the person shown in the input image. As information used to estimate clothing, the clothing estimation unit 13 generates, for example, frequency information for each temperature of the part of the person. The clothing estimation unit 13 counts the pixels in the part of the person in the image for each temperature (temperature range) and generates information on the frequency distribution for each temperature. Figure 3 shows an example of a histogram of the frequency distribution information for each temperature. In the histogram, the horizontal axis is temperature (°C) and the vertical axis is frequency (number of pixels). The histogram in Figure 3(a) corresponds to the upper rectangular part (upper body of the person) P1 of the image shown in Figure 2. The histogram in Figure 3(b) corresponds to the lower rectangular part (lower body of the person) P1 of the image shown in Figure 2.

[0060] The histogram above reflects the degree of skin exposure in different parts of a person's body. Generally, clothing worn by a person corresponds to the amount of skin exposed. For example, if the upper body of a person shows a uniform distribution of low-temperature pixels, the person is likely wearing long-sleeved clothing. Conversely, if there are pixels with exceptionally high temperatures in the upper body of a person, the person is likely wearing short-sleeved clothing. Therefore, the clothing worn by a person can be estimated from the histogram above.

[0061] The clothing estimation unit 13 may calculate statistical values ​​of the temperature of the parts of the person shown in the input image as information used to estimate the clothing of the person. For example, the clothing estimation unit 13 may calculate feature quantities obtained from the temperature distribution, such as the maximum, minimum, mean, median, variance, mode, quartiles, skewness, and kurtosis of the temperature, and use these as information used to estimate the clothing of the person.

[0062] The clothing estimation unit 13 stores in advance criteria used to estimate a person's clothing from the temperature of parts of the person. These criteria are pre-set. For example, the clothing estimation unit 13 estimates a person's clothing using a pre-stored model. The model used to estimate a person's clothing is, for example, the following: The model is a trained model generated by machine learning. The model takes the above-mentioned information used to estimate a person's clothing (for example, information on the frequency of each temperature of a person's body part) as input and outputs information indicating the estimated clothing of the person.

[0063] For example, the histogram shown in Figure 3 is the information input to the model. Information indicating the clothing worn by a person is output from the model. The information output from the model regarding a person's clothing may also indicate the likelihood (degree of likelihood) that a person is wearing a pre-defined type of clothing (e.g., long-sleeved clothing and short-sleeved clothing). In this case, the clothing with the highest likelihood of being worn may be used as the estimated clothing worn by the person.

[0064] The model is constructed using a neural network. The input layer of the model contains neurons for receiving images. For example, the input layer of the model contains neurons that receive the frequency for each temperature (or temperature range). The output layer of the model contains neurons for outputting information about the clothing worn by a person. For example, the output layer of the model contains neurons that output the identifier (ID) of the clothing worn by the person, or information indicating the possibility of the above for each type of clothing.

[0065] The model used in the wear estimation unit 13 is intended to be used as a program module that is part of artificial intelligence software. The model is used, for example, in a computer equipped with hardware such as a CPU (Central Processing Unit) and memory, and the computer's CPU operates according to instructions from the model stored in memory. For example, the computer's CPU operates in accordance with the instructions to input information to the model, perform calculations according to the model, and output a result from the model. Specifically, the computer's CPU operates in accordance with the instructions to input information to the input layer of the neural network, perform calculations based on the learned weighting coefficients etc. in the neural network, and output a result from the output layer of the neural network.

[0066] The garment estimation unit 13 inputs the information used to estimate the garment generated from the image into a pre-stored model, performs calculations according to the model, and obtains information indicating the garment estimation result as output from the model.

[0067] The above model generation may be carried out specifically as follows: Images are obtained by imaging a person whose clothing is known in advance using a training thermal camera. The training thermal camera may be the same as or different from the one used to generate the model used by the person detection unit 12. The images may also be the same as or different from the training images used to generate the model used by the person detection unit 12.

[0068] From the acquired image, information used to estimate the clothing worn by a person, which will serve as input to the model, is generated using the same method as described above. In addition, information indicating the clothing worn by a person, corresponding to the model's output, is generated from the clothing worn by the person in the image. These are used as training data, and machine learning is performed to generate the model. That is, the information used to estimate the clothing worn by a person is used as input to the model, and the information indicating the clothing worn by a person is used as the model's output (ground truth), and machine learning is performed to generate the model. The training of the above machine learning, i.e., updating the model parameters, can be done in the same way as conventional machine learning training.

[0069] The model may be a supervised machine learning classification model such as SVM (Support Vector Machine).

[0070] The clothing estimation unit 13 may estimate the clothing worn by a person from the temperature of a part of the person using a model other than the one described above. Alternatively, the clothing estimation unit 13 may estimate the clothing worn by a person from the temperature of a part of the person using a pre-stored method other than a model. Furthermore, it is sufficient to use the temperature of a part of the person to estimate the clothing worn by a person, and it is not necessarily required to generate and use the frequency for each temperature of a part of the person as described above. The estimation of a person's clothing in the clothing estimation device 10 may be performed in real time.

[0071] The clothing estimation unit 13 outputs information indicating the estimation result of the person's clothing. For example, the clothing estimation unit 13 transmits the information indicating the estimation result to a device that uses information indicating the person's clothing. Alternatively, the clothing estimation unit 13 may output the information indicating the estimation result in a format that is recognizable to the user. For example, the clothing estimation unit 13 may display the information indicating the estimation result on a display device provided by the clothing estimation device 10. The output of the information indicating the estimation result from the clothing estimation unit 13 may be performed by a method other than those described above, and to an output destination other than those described above. The above describes the functions of the clothing estimation device 10 according to this embodiment.

[0072] Next, using the flowchart in Figure 4, we will explain the clothing estimation method, which is a process (a method of operation performed by the clothing estimation device 10) executed by the clothing estimation device 10 according to this embodiment.

[0073] In this process, first, the acquisition unit 11 acquires an image obtained by imaging with the thermal camera 100 (S01, acquisition step). Next, the person detection unit 12 detects a portion of the image of a person that is captured in the image acquired by the acquisition unit 11 (S02, person detection step). Next, the clothing estimation unit 13 estimates the clothing worn by the person based on the temperature of the portion of the person detected by the person detection unit 12, which is shown in the image acquired by the acquisition unit 11 (S03, clothing estimation step). Next, the clothing estimation unit 13 outputs information indicating the estimation result (S04). The above is the clothing estimation method according to this embodiment.

[0074] In this embodiment, the clothing worn by a person is estimated from the temperature of a portion of the image of the person captured in the image. Low-resolution images that make it difficult to identify individuals can be used as the image for this estimation. For example, unlike high-resolution images, low-resolution images make it difficult to distinguish the boundaries of clothing, making it difficult to detect clothing features and identify the clothing from the image. In contrast, this embodiment uses temperature, making it possible to estimate the clothing worn by a person. Even if the image used for estimation in this embodiment has blurred boundaries or is noisy and difficult to draw lines, the temperature of pixels near the boundaries will correspond to the clothing, making it possible to estimate the clothing worn by a person. For example, if a person is wearing long-sleeved clothing, the temperature will be lower, and if a person is wearing short-sleeved clothing, the temperature will be higher, making it possible to distinguish between them. Therefore, according to this embodiment, it is possible to appropriately estimate the clothing worn by a person while taking privacy issues into consideration.

[0075] Furthermore, as in this embodiment, the clothing estimation unit 13 may estimate the clothing worn by a person based on the frequency of temperature changes for each part of the person's body. This configuration allows for the appropriate and reliable estimation of a person's clothing. However, it is not always necessary to use the frequency of temperature changes for each part of the person's body to estimate their clothing.

[0076] Furthermore, as mentioned above, the image used to estimate the clothing worn by a person may be an image with 32 or fewer pixels in both the vertical and horizontal directions. However, the image used to estimate the clothing worn by a person may have a resolution other than that mentioned above.

[0077] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more of the above devices.

[0078] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0079] For example, the clothing estimation device 10 in one embodiment of the present disclosure may function as a computer that performs information processing according to the present disclosure. Figure 5 is a diagram showing an example of the hardware configuration of the clothing estimation device 10 according to one embodiment of the present disclosure. The clothing estimation device 10 described above may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0080] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the wearable item estimation device 10 may include one or more of the devices shown in the figure, or it may be configured without some of the devices.

[0081] Each function in the clothing estimation device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which causes the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

[0082] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. For example, each function in the above-described clothing estimation device 10 may be implemented by the processor 1001.

[0083] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each function of the clothing estimation device 10 may be implemented by a control program stored in the memory 1002 and operated on the processor 1001. Although the above-described processes have been explained as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0084] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for carrying out information processing according to one embodiment of the present disclosure.

[0085] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The storage medium provided by the wearer estimation device 10 may be, for example, a database, server, or other suitable medium including at least one of the memory 1002 and the storage 1003.

[0086] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0087] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0088] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0089] Furthermore, the wearable garment estimation device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0090] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements using exemplary order and are not limited to the specific order presented.

[0091] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0092] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0093] Each aspect / embodiment described in this disclosure may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0094] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0095] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0096] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0097] The terms “system” and “network” as used in this disclosure are interchangeable.

[0098] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0099] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0100] The terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables, and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain, and optical (both visible and invisible) domain.

[0101] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0102] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0103] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0104] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0105] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0106] The clothing estimation device and clothing estimation method of the present disclosure have the following configurations: [1] A clothing estimation device comprising: an acquisition unit that acquires an image obtained by imaging with a thermal camera; a person detection unit that detects a part of a person in the image acquired by the acquisition unit; and a clothing estimation unit that estimates the clothing of a person from the temperature of the part of the person detected by the person detection unit, as shown by the image acquired by the acquisition unit. [2] The clothing estimation device according to [1], wherein the clothing estimation unit estimates the clothing of a person from the frequency of each temperature of the part of the person. [3] The clothing estimation device according to [1] or [2], wherein the clothing estimation unit estimates the length of the clothing of the person. [4] The clothing estimation device according to any one of [1] to [3], wherein the acquisition unit acquires an image in which the vertical and horizontal pixels each number 32 or less. [5] Clothing estimation method comprising: an acquisition step in which the clothing estimation device acquires an image obtained by imaging with a thermal camera; a person detection step in which the clothing estimation device detects a portion of a person in the image acquired in the acquisition step; and a clothing estimation step in which the clothing estimation device estimates the clothing of a person from the temperature of the portion of the person detected in the person detection step, which is shown by the image acquired in the acquisition step.

[0107] 10... Clothing estimation device, 11... Acquisition unit, 12... Person detection unit, 13... Clothing estimation unit, 100... Thermal camera, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. Clothing estimation device comprising: an acquisition unit that acquires an image obtained by imaging with a thermal camera; a person detection unit that detects a portion of a person in the image acquired by the acquisition unit; and a clothing estimation unit that estimates the clothing worn by a person from the temperature of the portion of the person detected by the person detection unit, as shown in the image acquired by the acquisition unit.

2. The clothing estimation device according to claim 1, wherein the clothing estimation unit estimates the clothing worn by a person based on the frequency of temperature changes for each part of the person.

3. The garment estimation device according to claim 1, wherein the garment estimation unit estimates the length of the garment worn by the person.

4. The garment estimation device according to claim 1, wherein the acquisition unit acquires an image in which the vertical and horizontal pixels are 32 or less.

5. A clothing estimation method comprising: an acquisition step in which a clothing estimation device acquires an image obtained by imaging with a thermal camera; a person detection step in which the clothing estimation device detects a portion of a person in the image acquired in the acquisition step; and a clothing estimation step in which the clothing estimation device estimates the clothing of a person from the temperature of the portion of the person detected in the person detection step, which is shown by the image acquired in the acquisition step.

Images