Image processing device capable of controlling temperature of image, warm / cold sensation presenting device, method for controlling image processing device, and program

The image processing device addresses the challenge of selective temperature control for specific objects in image processing by integrating object detection and type identification, allowing for precise thermal sensation presentation tailored to object types and scenes, enhancing user experience with flexible temperature adjustments.

WO2024241907A1PCT designated stage expired Publication Date: 2024-11-28CANON KK
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/017370
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-19
Filing Date
2024-05-10
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Conventional image processing devices that control temperature for thermal sensation presentation lack the ability to selectively apply temperature control to specific objects within an image, leading to unwanted temperature sensations and limited adjustability based on the type of object.

Method used

An image processing device equipped with an object detection and type identification system, which determines whether to perform temperature control on a specific object and adjusts the temperature range, using a temperature data table to set precise thermal sensations for different objects and scenes.

Benefits of technology

Enables precise control of temperature for desired objects within an image, allowing users to experience intended thermal sensations while avoiding unwanted ones, with flexibility in temperature settings based on object type and scene.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024017370_28112024_PF_FP_ABST
    Figure JP2024017370_28112024_PF_FP_ABST
Patent Text Reader

Abstract

[Problem] The objective of the present invention is to provide an image processing device capable of determining whether or not to control the temperature of a desired object in an image and having a degree of freedom in a control range of the temperature to be presented, a warm / cold sensation presenting device, a method for controlling the image processing device, and a program. [Solution] A warm / cold sensation presenting device 100 comprises: an image data input unit 101 into which image data of an image are input; an object detecting unit 104 which detects an object in the image on the basis of the image data and identifies the type of the object; a temperature control determining unit 111 which determines whether or not to execute temperature control with respect to an object occupation region that is occupied by the object within the image, on the basis of the identification result obtained by the object detecting unit 104; and a temperature determining unit 112 which determines the temperature of the object occupation region if it is determined, as the determination result obtained by the temperature control determining unit 111, that temperature control is to be executed.
Need to check novelty before this filing date? Find Prior Art

Description

Image processing device capable of controlling the temperature of an image, thermal sensation presentation device, and control method and program for the image processing device

[0001] The present invention relates to an image processing device capable of controlling the temperature of an image, a thermal sensation presentation device, a control method for an image processing device, and a program.

[0002] Conventionally, displays capable of displaying images using Peltier elements that can be switched between cooling and heating, arranged in a matrix, have been known (see, for example, Non-Patent Document 1). The display described in Non-Patent Document 1 can cool or heat the display by controlling the Peltier elements while an image is displayed. This allows a user using the display to feel cold or warm by touching the display, i.e., a thermal sensation is presented.

[0003] Kumiko Kushiyama, Yukiteru Doi, Shinji Sasada, and Tetsuaki Baba, "Thermo Drawing: Development of a temperature drawing system using a small tactile display with cold and hot presentation," Information Processing Society of Japan Interaction 2012, March 16, 2012, pp. 723-728

[0004] However, when the display described in Non-Patent Document 1 performs temperature control (temperature adjustment) of the entire image, even if the user desires to have the warmth and cold sensation of a specific object in the image presented, the display will also present the warmth and cold sensation of the part where the warmth and cold sensation is not desired. Also, depending on various conditions such as the type of specific object, the temperature control range cannot be freely adjusted or set.

[0005] The present invention provides an image processing device, a thermal sensation presentation device, a control method for an image processing device, and a program that can determine whether or not to control the temperature of a desired object in an image and have flexibility in the control range of the temperature to be presented.

[0006] The image processing device of the present invention is an image processing device capable of processing images including objects, and is characterized by comprising: an image data input means for inputting image data of the image; an object detection means for detecting the object in the image based on the image data; a type identification means for identifying the type of the object detected by the object detection means; a control judgment means for determining whether or not to perform temperature control on an object-occupied area occupied by the object in the image based on the identification result of the type identification means; and a temperature determination means for determining the temperature of the object-occupied area when the control judgment means determines that the temperature control should be performed.

[0007] According to the present invention, it is possible to determine whether or not to control the temperature of a desired object in an image, and there is a degree of freedom in the control range of the temperature to be presented.

[0008] 1 is a block diagram showing the hardware configuration of a warm / cold sensation presentation device according to a first embodiment. FIG. 2 is a schematic diagram for explaining the relationship between image data input to an image data input unit and warm / cold sensation presentation in a warm / cold sensation presentation unit. FIG. 3 is a diagram for explaining the creation of warm / cold sensation presentation area data by linking a warm / cold sensation presentation area with temperature data. FIG. 4 is a flowchart showing the cold sensation presentation process executed in a warm / cold sensation presentation device. FIG. 5 is a schematic diagram showing the hardware configuration of a warm / cold sensation presentation device according to a third embodiment. FIG. 7 is a schematic diagram showing the use state of the warm / cold sensation presentation device shown in FIG.

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each component constituting the present invention can be replaced with any configuration that can perform the same function. Furthermore, any component may be added. Furthermore, any two or more configurations (features) of each embodiment can be combined.

[0010] A first embodiment will be described below with reference to Figures 1 to 4. Figure 1 is a block diagram showing the hardware configuration of a warm / cold sensation presentation device 100 according to the first embodiment. In Figure 1, the warm / cold sensation presentation device (warm / cold sensation presentation device) 100 is a device that applies an image processing device capable of processing images that include objects. The warm / cold sensation presentation device 100 has an image data input unit (image data input means) 101, an image memory 102, a preprocessing unit 103, an object detection unit 104, a scene determination unit (scene determination means) 105, an area extraction unit 106, and an authentication unit (authentication means) 107.

[0011] The warm-cold sensation presentation device 100 also has an object data memory 108, a temperature data table memory (storage means) 109, a temperature control unit 110, a temperature control memory 113, and a warm-cold sensation presentation unit 114. The warm-cold sensation presentation device 100 also has a memory (not shown) for storing various programs. These programs are not particularly limited and include, for example, programs for causing a computer (CPU) to execute each unit and each means (control method of an image processing device) of the warm-cold sensation presentation device 100.

[0012] Image data of an image is input to the image data input unit 101 (image data input step). The image data input unit 101 has an I / F (interface) for inputting a digital image data signal. This I / F is not particularly limited, and for example, an HDMI (registered trademark) terminal can be used. In this case, the image data input unit 101 receives a video signal that complies with the HDMI standard and extracts image data from the video signal.

[0013] The image data input unit 101 writes this extracted image data into the image memory 102. The image data input unit 101 may be configured to have an interface into which a memory card capable of reading and writing high-resolution image data at high speed, such as an SD card or a CFexpress card, can be inserted or removed. In this case, the image data input unit 101 reads image data recorded on the memory card and writes the read image data into the image memory 102. The image memory 102 is, for example, a DRAM or an SRAM, and records image data. The pre-processing unit 103 reads the image data recorded in the image memory 102 and performs image processing on the read image data.

[0014] The image processing is suitable for the processing performed by, for example, the object detection unit 104, the scene determination unit 105, the region extraction unit 106, and the authentication unit 107. Note that, for example, when the image data is composed of YUV signals, the image processing also includes gamma correction, shading correction, white balance correction, and the like, which are performed on the YUV signals. Furthermore, when the image data is composed of a Bayer pixel pattern, the image processing may also include demosaic processing, which performs color interpolation processing so that one pixel has the values ​​of all RGB components, geometric deformation processing, noise reduction processing, and resizing processing. Furthermore, the preprocessing unit 103 writes the image data that has undergone image processing to the image memory 102.

[0015] The object detection unit 104 reads image data recorded in the image memory 102 and detects at least one predetermined object from the image based on the read image data (object detection process). The object detection unit 104 can also detect the position and size of the object within the image. The object detection unit 104 then identifies the type of object detected in the object detection process (type identification process).

[0016] The object type is not particularly limited, and examples include animals such as humans, dogs, cats, birds, and insects, moving objects such as automobiles and airplanes, and still objects such as balls. When an animal, including a human, is identified as the object type, the animal's body parts are also identified. For example, when a human is identified, the head, chest, abdomen, arms, legs, etc. are further identified. Furthermore, the object detection unit 104 writes information associating the object type identification result with a reliability as object data into the object data memory 108. It should be noted that known methods can be used for the object detection method for detecting objects and the type identification method for identifying object types.

[0017] The object detection unit 104 may also use machine learning to detect objects or identify object types. In this case, the object detection unit 104 preferably includes a GPU, a TPU, or the like. The object detection unit 104 can be configured as a class classifier that has been trained using training data. The machine learning algorithm may be, but is not limited to, deep learning. For example, a support vector machine, a logistic regression, a decision tree, a random forest, or the like may also be used. Such machine learning allows for more accurate object detection and object type identification.

[0018] In this embodiment, the object detection unit 104 functions as an object detection means for detecting an object in an image and as a type identification means for identifying the type of object detected by the object detection means. That is, the object detection means and the type identification means are integrated into a single object detection unit 104 (unit). This allows the hardware configuration of the thermal sensation presentation device 100 to be simplified. The object detection means and the type identification means may be configured as independent units. In this case, hardware (devices) suitable for the object detection means and the type identification means can be used.

[0019] The scene determination unit 105 reads image data recorded in the image memory 102 and determines what kind of scene the image is based on the read image data. The scene determination unit 105 also writes this determination result into the object data memory 108. The scene of the image is not particularly limited, and examples include "indoors," "night scene," "evening scene," "landscape (spring)," "landscape (summer)," "landscape (autumn)," "landscape (winter)," and "backlit." The scene determination unit 105 is configured, for example, with a scene classifier that has been trained using training data. The machine learning algorithm used in the scene classifier is not particularly limited.

[0020] The region extraction unit 106 reads image data recorded in the image memory 102 and detects at least one region of a predetermined type based on the read image data. For each detected region, the region extraction unit 106 writes map information that maps the region occupied within the image and the type of region (e.g., sky, human skin, grass, etc.) as region data into the object data memory 108. Note that the region extraction unit 106 is configured, for example, with a region map generator that has been trained using training data. Furthermore, the machine learning algorithm used in the region map generator is not particularly limited.

[0021] The authentication unit 107 can authenticate the type of object identified by the object detection unit 104. Specifically, the authentication unit 107 writes authentication data necessary for authentication in the object data memory 108 in advance, and reads the object data written by the object detection unit 104 from the object data memory 108. Then, if the object data contains the type of the object that is the authentication target, the authentication unit 107 reads the authentication data from the object data memory 108. In addition to reading this authentication data, the authentication unit 107 also cuts out the area of ​​the object that is the authentication target from the image data recorded in the image memory 102, and reads it as authentication target data.

[0022] The authentication unit 107 performs object authentication by calculating the degree of match between the authentication data and the authentication target data, and writes the result of the object authentication to the object data memory 108. The method for calculating the degree of match between the authentication data and the authentication target data is not particularly limited. For example, the degree of match is calculated as the total absolute difference between the pixel values ​​of the authentication data and the authentication target data. If this total is less than a predetermined threshold, it is determined that the authentication data and the authentication target data match. For example, assume that a human face is identified as the type of object. If a human face is identified as the type of object, the positions of the human's eyes, nose, mouth, etc. are extracted by a body part extractor, and the degree of match between the positions of these extracted body parts and the positions of these body parts in the authentication data is calculated.

[0023] Furthermore, when an object is photographed from multiple viewpoints and multiple sets of image data are acquired, the three-dimensional shape of the object can be calculated based on the acquired multiple sets of image data, and the degree of match can be calculated from the difference between the structure of the authentication data and the structure of the authentication target data. Note that the machine learning algorithm used in the authentication unit 107 is not particularly limited. The object data memory 108 is, for example, a DRAM or an SRAM, and stores, for example, information (data) about the object, information about the scene, information about the area, and the result of object authentication.

[0024] The temperature data table memory 109 is, for example, a DRAM or an SRAM, and stores temperature information related to the temperature for each type of object in the form of a table. For example, the temperature information is 36 degrees for the type of object "human," 38 degrees for the type of object "dog," and 40 degrees for the type of object "bird." Furthermore, if the type of object is a high-temperature object such as "fireworks," the temperature information is, for example, 50 degrees. The temperature data table memory 109 also stores, in the form of a table, temperature information related to the temperature for each scene determined by the scene determination unit 105.

[0025] The temperature information for a scene is, for example, 15 degrees when the scene is a "night scene," 20 degrees when it is a "sunset scene," and 25 degrees when it is an "indoor." Temperature data table memory 109 also stores in advance, as a table, temperature information relating to the temperature for each type of area detected by area extraction unit 106. The temperature information for each type of area is, for example, 15 degrees when the type of area is "sky," 36 degrees when it is "human skin," and 24 degrees when it is "grass."

[0026] The temperature control unit 110 has a temperature control determination unit (control determination means) 111 and a temperature determination unit (temperature determination means) 112. The temperature control determination unit 111 reads out from the object data memory 108 the object data that the object detection unit 104 has written to the object data memory 108. Based on the object type identification result included in the read object data, the temperature control determination unit 111 determines whether or not to execute temperature control on an "object-occupied area," which is an area occupied by an object in an image (control determination step).

[0027] When the temperature control determination unit 111 determines that temperature control should be performed, the temperature determination unit 112 determines the temperature of the object-occupied area based on the temperature information stored in the temperature data table memory 109 (temperature determination process). The temperature determination unit 112 also writes information related to the determined temperature into the temperature control memory 113. The warm / cold sensation presentation unit 114 presents a sensation of warmth and cold to the user using the warm / cold sensation presentation device 100, i.e., conveys warmth and coldness to the user. The warm / cold sensation presentation unit 114 can read the determined temperature written in the temperature control memory 113 from the temperature control memory 113 and reflect the read determined temperature in the warm / cold sensation presentation.

[0028] FIG. 2 is a schematic diagram illustrating the relationship between image data input to the image data input unit 101 and the warm / cold sensation presentation by the warm / cold sensation presentation unit 114. The warm / cold sensation presentation unit 114 has a warm / cold sensation presentation area 304 that presents a warm / cold sensation. In the warm / cold sensation presentation area 304, a plurality of warm / cold sensation presentation elements 305, each of which is formed, for example, by Peltier elements, are arranged in a matrix (seven rows vertically and eight columns horizontally in the configuration shown in FIG. 2). The input image data 301 input to the image data input unit 101 is image data including a person 303 and a tree 300. The input image data 301 includes information about a rectangular person area 302 inscribed by the person 303. The warm / cold sensation presentation unit 114 can also scale the entire field of view of the input image data 301 to fit the warm / cold sensation presentation area 304.

[0029] At this time, a person-corresponding region 306 having the same aspect ratio as the person region 302 is formed in the thermal sensation presentation region 304 at a position corresponding to the person region 302. A person-corresponding region 307 is then set, which is configured by one or more thermal sensation presentation elements 305, at least a portion of each of which overlaps at least a portion of the person-corresponding region 306. The person-corresponding region 307 corresponds to an object-occupied region in the image occupied by an object. The temperature control determination unit 111 determines whether to perform temperature control on the person-corresponding region 307, i.e., whether to set the person-corresponding region 307 as a target region for temperature control. If the person-corresponding region 307 is set as a target region for temperature control, the temperature to be presented in the person-corresponding region 307 is determined by the temperature determination unit 112. In the example shown in FIG. 2 , the person-corresponding region 307 is an region that presents the thermal sensation of a person 303 (person), and therefore the presented temperature is determined to be 36 degrees. The user can feel the temperature (body temperature) of the person 303 by touching the human hot / cold sensation presentation area 307 with a finger 308 .

[0030] FIG. 3 is a diagram for explaining the creation of warm / cold sensation presentation area data by linking a warm / cold sensation presentation area 405 with temperature data. The warm / cold sensation presentation unit 401 shown in FIG. 3 is the warm / cold sensation presentation unit 114. The warm / cold sensation presentation unit 401 has a plurality of warm / cold sensation presentation elements 402 arranged in a matrix (seven rows and eight columns in the configuration shown in FIG. 3 ) on the back side of the warm / cold sensation presentation unit 401. In this embodiment, each warm / cold sensation presentation element 402 is assigned coordinates ranging from (0,0) to (7,6) from the upper left to the lower right in FIG. 3 . Each warm / cold sensation presentation element 402 is composed of, for example, a Peltier element. Cooling and heating can be achieved by electrically controlling each warm / cold sensation presentation element 402. This allows temperature adjustment for each warm / cold sensation presentation element 402, thereby allowing a warm / cold sensation to be presented for each warm / cold sensation presentation element 402.

[0031] In this way, the multiple warm / cold sensation presentation elements 402 constitute a warm / cold sensation presentation region 405 that presents a warm / cold sensation. The warm / cold sensation presentation unit 401 can scale the entire angle of view of input image data (e.g., input image data 301) to the entire area of ​​the warm / cold sensation presentation region 405. In this case, the temperature control unit 110 calculates which coordinates of the warm / cold sensation presentation elements 402 in the warm / cold sensation presentation region 405 correspond to an "object region," which is the region occupied by an object read from the input image data. The temperature control unit 110 assigns the temperature data of the object to the warm / cold sensation presentation element 402 with the calculated corresponding coordinates.

[0032] For example, if the person-corresponding region 403 as an object region is a rectangular region surrounded by a dashed line in Figure 3, the warm / cold sensation providing elements 402 at least partially overlapping with at least partially the person-corresponding region 403 are the warm / cold sensation providing elements 402 at coordinates (4,3) to (4,6) and (5,3) to (5,6). The warm / cold sensation providing elements 402 at coordinates (4,3) to (4,6) and (5,3) to (5,6) constitute the person warm / cold sensation providing region 404. The temperature information "36 degrees" of the object type "person" read from the temperature data table memory 109 and determined by the temperature determination unit 112 is assigned to each of the warm / cold sensation providing elements 402 constituting the person warm / cold sensation providing region 404.

[0033] In this embodiment, the warm / cold sensation presentation unit 401 includes a display 406 disposed on the front side of the warm / cold sensation presentation unit 401. The display 406 is not particularly limited, and examples thereof include an organic electroluminescence (EL) display and a liquid crystal display (LCD). This allows the warm / cold sensation presentation unit 401 to function as a display unit that displays input image data as an image, and can display an image including a person 407, as shown in FIG. 3 . The person-corresponding area 403 is an area inscribed by the person 407. Furthermore, each warm / cold sensation presentation element 402 functions as a temperature adjustment unit that adjusts the temperature of the person 407 displayed on the display 406 to the temperature determined by the temperature determination unit 112. This allows the user to feel the temperature of the person 407 by touching the warm / cold sensation presentation elements 402 constituting the person warm / cold sensation presentation area 404 with their fingers (body).

[0034] Furthermore, the display 406 (warm / cold sensation presentation unit 401) may visualize and display the temperature of the human warm / cold sensation presentation area 404 on an image. This "visualization and display" method is not particularly limited. However, it is preferable that the display 406 (warm / cold sensation presentation unit 401) display the temperature of the human warm / cold sensation presentation area 404 with a higher density of red as the temperature is higher, or the display 406 (warm / cold sensation presentation unit 401) display the temperature of the human warm / cold sensation presentation area 404 with a higher density of blue as the temperature is lower. This allows the user to visually grasp the body temperature of an object (e.g., a human 407) by viewing the display 406 without touching the human warm / cold sensation presentation area 404.

[0035] The display 406 may also visualize and display on an image the degree of deviation between the temperature of the human warm-cold sensation presentation area 404 (object-occupied area) and the average body temperature of the human (animal). The method of "visualizing and displaying" here is not particularly limited. However, for example, when the human warm-cold sensation presentation area 404 deviates from the average body temperature in a direction of increasing temperature, the display 406 preferably displays the image in a higher density of red. Conversely, for example, when the human warm-cold sensation presentation area 404 deviates from the average body temperature in a direction of decreasing temperature, the display 406 preferably displays the image in a higher density of blue. This allows the user to understand that the body temperature is higher by visually checking the display 406, even without touching the human warm-cold sensation presentation area 404, if the temperature of the human 407 is higher than the average body temperature, for example, due to heatstroke or the like.

[0036] Furthermore, for example, if an image contains multiple objects and the object regions overlap in the thermal sensation presentation area 405, the object region may be assigned the temperature information of the object with a higher temperature or the temperature information of the object with a lower temperature. Furthermore, the temperature information of the objects may be assigned based on the priority of the object type. The priority of the object type can be determined arbitrarily in advance.

[0037] The warm / cold sensation presentation device 100 has a setting means (not shown) for setting in advance the type of object that can be the temperature control target in the warm / cold sensation presentation unit 114. As a result, for example, in the case of an image that includes a person 303 and a tree 300 as shown in Fig. 2, the type of object that can be the temperature control target can be set to the person 303 rather than the tree 300. The setting means is not particularly limited, and examples thereof include an operation unit through which a user inputs various information, commands, etc. to the warm / cold sensation presentation device 100.

[0038] As described above, the temperature control determination unit 111 of the temperature control unit 110 reads from the object data memory 108 the object data that the object detection unit 104 has written to the object data memory 108. Then, the temperature control determination unit 111 determines whether or not to execute temperature control on the object-occupied area in the image based on the identification result of the object type included in the read object data, i.e., the identification result by the object detection unit 104.

[0039] When making this determination, the temperature control determination unit 111 determines to execute temperature control on the object-occupied region if the type of object identified by the object detection unit 104 matches the type of object preset by the setting means. Furthermore, when executing temperature control, the temperature control determination unit 111 calculates, based on the position and size of the object in the image that may be a temperature control target, which region of the warm / cold sensation presentation region 304 (405) of the warm / cold sensation presentation unit 114 the object-occupied region occupied by the object corresponds to. Based on this calculation result, the temperature control determination unit 111 determines the warm / cold sensation presentation region 304 (405) to be the temperature control target. On the other hand, when the type of object identified by the object detection unit 104 does not match the type of object preset by the setting means, the temperature control determination unit 111 determines not to execute temperature control on the object-occupied region.

[0040] Furthermore, the temperature control determination unit 111 outputs information regarding the type of object to the temperature determination unit 112. Based on this information, the temperature determination unit 112 outputs temperature data for the type of object to the temperature control determination unit 111. As a result, the temperature control determination unit 111 receives the temperature data for the type of object from the temperature determination unit 112, associates the received temperature data with the thermal sensation presentation area 304 (405), and writes them into the temperature control memory 113 as temperature control data.

[0041] In this embodiment, the determination of whether or not to perform temperature control on the object-occupied area is made based on whether or not the type of object identified by the object detection unit 104 matches the type of object previously set by the setting means. However, the determination of whether or not to perform temperature control on the object-occupied area is not limited to this, and may be made based on, for example, whether or not the type of object identified by the object detection unit 104 is authenticated by the authentication unit 107.

[0042] Specifically, the authentication unit 107 reads the result of object authentication written in the object data memory 108. If the authentication unit 107 passes the object authentication, i.e., if the type of object identified by the object detection unit 104 is authenticated by the authentication unit 107, the temperature control determination unit 111 determines to execute temperature control. Following this determination, the temperature determination unit 112 also makes a determination. On the other hand, if the authentication unit 107 fails to pass the object authentication, i.e., if the type of object identified by the object detection unit 104 is not authenticated by the authentication unit 107, the temperature control determination unit 111 may determine not to execute temperature control. Note that the determination of whether or not to execute temperature control based on the authentication result by the authentication unit 107 may be arbitrarily settable in the warmth and cold sensation presentation device 100.

[0043] The temperature data table memory 109 stores temperature data listed for each type of object. The temperature determination unit 112 reads out the temperature data for each type of object from the temperature data table memory 109 based on the type of object input from the temperature control determination unit 111. The temperature determination unit 112 outputs the read temperature data to the temperature control determination unit 111. Note that the thermal sensation presentation device 100 may further include, in the image data input unit 101, a temperature data input unit that inputs temperature data corresponding to the entire angle of view of image data from outside the thermal sensation presentation device 100, for example.

[0044] The temperature determination unit 112 may extract temperature data of the object region from the temperature data input via the temperature data input unit and output the extracted temperature data to the temperature control determination unit 111. Furthermore, when the temperature data is composed of multiple pieces of temperature data for multiple regions within the object region, the temperature determination unit 112 may perform a smoothing process on the multiple pieces of temperature data to obtain one piece of temperature data and output the obtained one piece of temperature data to the temperature control determination unit 111. Furthermore, different temperature data may be output to the temperature control determination unit 111 for each thermal sensation providing element 402 (see FIG. 3 ), i.e., for each coordinate.

[0045] The thermal sensation presentation device 100 may also have a temperature inference device that has learned using learning data and infers the temperature of a specific type of object. In this case, input image data may be input to the temperature inference device, and temperature data that is the inference result obtained by the temperature inference device may be output to the temperature control determination unit 111 as temperature data of the object. If there are multiple methods (acquisition means) for acquiring temperature data, such as "extraction from input temperature data," "reading temperature data from a temperature data table (list)," or "inference by a temperature inference device," the acquisition means may be switched appropriately depending on the type of object input from the temperature control determination unit 111.

[0046] For example, if the object type is "person," temperature data is acquired through inference by the temperature inference unit. If the object type is "dog" or "cat," temperature data is extracted from the input temperature data. If the object type is "fireworks," temperature data is read from the temperature data table. By switching the temperature data acquisition means based on the object type input from the temperature control determination unit 111 in this way, it becomes possible to acquire more accurate temperature data, for example, even if accurate temperature data cannot be acquired depending on the object type. In this way, when determining the temperature of the object-occupied area, the temperature determination unit 112 can change the temperature determination method depending on the type of object and determine an appropriate temperature.

[0047] Furthermore, when determining the temperature of the object-occupied area, the temperature determination unit 112 can set at least one of an upper temperature limit and a lower temperature limit depending on the type of object. For example, in the case of a high-temperature object such as a "firework," the actual temperature of the object is 370 degrees for a sparkler and 1100 degrees or more for a balloon firework. Such high-temperature output is practically impossible if the thermal sensation providing element 305 is configured with a Peltier element. Furthermore, even if high-temperature output were possible, the user would be unable to touch the object-occupied area of ​​the hot object.

[0048] Therefore, if the object is a high-temperature object, it is preferable to set an upper limit on the temperature of the object and determine the temperature. This allows the temperature of the warm-cold sensation providing element 305 to be adjusted, allowing the user to touch the object-occupied area of ​​the high-temperature object. The upper limit depends on the type of object; for example, if the object is a "firework," it may be set to 50 degrees. The upper limit is pre-stored in the temperature data table memory 109 for each type of object. The temperature determination unit 112 can set a lower limit, similar to the upper limit of the temperature. Furthermore, when determining the temperature, a temperature between the upper and lower limits of the temperature that can be output by the warm-cold sensation providing element 305 may be used. For example, if the upper limit of the temperature that can be output by the warm-cold sensation providing element 305 is 40 degrees, the temperature may be uniformly set to 40 degrees for all objects whose temperature exceeds 40 degrees. Furthermore, if the lower limit of the temperature that can be output by the warm-cold sensation providing element 305 is 5 degrees, the temperature may be uniformly set to 5 degrees for all objects whose temperature is below 5 degrees.

[0049] As described above, the scene determination unit 105 determines what type of scene the image represents and writes this determination result to the object data memory 108. The temperature control determination unit 111 reads the determination result of the scene determination unit 105, i.e., scene information indicating what type of scene the image represents, from the object data memory 108 and outputs it to the temperature determination unit 112. The temperature determination unit 112 outputs temperature information of the scene based on the scene information read from the object data memory 108 to the temperature control determination unit 111.

[0050] This allows the temperature control determination unit 111 to receive scene temperature information from the temperature determination unit 112. In this case, the temperature control determination unit 111 may have a scene temperature generation function that acquires an offset temperature of the entire warm / cold sensation presentation area in the warm / cold sensation presentation unit 114 based on the scene temperature information and writes the offset temperature to the temperature control memory 113. As a result, for example, when the entire warm / cold sensation presentation area 304 (405) is the object area, an offset process is performed in which the temperature of the scene is added to or subtracted from the temperature data of the object. Furthermore, both the temperature data of the object and the temperature information of the scene may be reflected in the object area (the entire warm / cold sensation presentation area 304 (405)). Furthermore, for example, when the entire warm / cold sensation presentation area 304 (405) is the object area, the temperature of the scene may be ignored.

[0051] Furthermore, the temperature determination unit 112 may determine the temperature of the object-occupied area based on the determination result of the scene determination unit 105. Specifically, when the scene temperature generation function is enabled, the temperature determination unit 112 reads out temperature data prepared for each scene from the temperature data table memory 109 based on the scene information from the temperature control determination unit 111. The temperature determination unit 112 reflects this read-out temperature data in the temperature of the object-occupied area.

[0052] The temperature control determination unit 111 outputs the region data read from the object data memory 108 to the temperature determination unit 112. The temperature control determination unit 111 receives temperature information for a specific type region from the temperature determination unit 112. In this case, the temperature control determination unit 111 may have a region temperature generation function that writes the temperature information for the specific type region as an offset temperature into the temperature control memory 113. As a result, for example, when a specific type region overlaps with an object region, an offset process is performed in which the temperature of the specific type region is added to or subtracted from the temperature data for the object.

[0053] Furthermore, both the temperature data of the object and the temperature data of the specific type area may be reflected in the object area. For example, if a specific type area overlaps with an object area, the temperature of the specific type area may be ignored. When the area temperature generation function is enabled, the temperature determination unit 112 reads out temperature data prepared for each type area from the temperature data table memory 109 based on the area data received from the temperature control determination unit 111. The temperature determination unit 112 reflects this read temperature data in the temperature of the object-occupied area.

[0054] 4 is a flowchart showing the cold sensation presentation process executed by the warm / cold sensation presentation device 100. First, in step S200, the image data input unit 101 writes input image data into the image memory .

[0055] In step S201, the preprocessing unit 103 reads the image data written in step S200 from the image memory 102. The preprocessing unit 103 then performs image processing on the read image data suitable for "object detection processing" in which the object detection unit 104 detects at least one predetermined object from the image (preprocessing). The preprocessing unit 103 writes the image data after image processing into the image memory 102.

[0056] In step S202, the object detection unit 104 reads the image data written in step S201 from the image memory 102 and performs object detection processing.

[0057] In step S203, the object detection unit 104 determines whether or not at least one object has been detected. If it is determined in step S203 that an object has been detected, the object detection unit 104 writes the detection result (the position, size, etc. of the detected object) into the object data memory 108, and the process proceeds to step S204. On the other hand, if it is determined in step S203 that an object has not been detected, the process ends.

[0058] In step S204, the object detection unit 104 identifies the type of the object detected in step S203, and writes the identification result in the object data memory 108.

[0059] In step S205, the temperature control determination unit 111 reads the identification result written in step S204 from the object data memory 108. Then, based on the read identification result, the temperature control determination unit 111 determines whether the type of object identified as the identification result matches the type of object that will be the target of the predetermined warm / cold sensation presentation control. If it is determined in step S205 that the type of object identified as the identification result matches the type of object that will be the target of the predetermined warm / cold sensation presentation control, the process proceeds to step S206. On the other hand, if it is determined in step S205 that the type of object identified as the identification result does not match the type of object that will be the target of the predetermined warm / cold sensation presentation control, the process proceeds to step S207.

[0060] In step S206, the temperature determination unit 112 receives information regarding the type of object from the temperature control determination unit 111. The temperature determination unit 112 also reads the temperature data for the corresponding object type from the temperature data list in the temperature data table memory 109 (determines the temperature of the object-occupied area), and outputs the read temperature data to the temperature control determination unit 111. The temperature control determination unit 111 associates the received temperature data with the thermal sensation presentation area 304 (405) of the corresponding object, and writes the associated data into the temperature control memory 113 as temperature control data.

[0061] In step S207, the temperature control determination unit 111 determines whether the processes of steps S204 to S206 have been completed for all objects detected by the object detection unit 104. If it is determined in step S207 that the processes have been completed for all detected objects, the process proceeds to step S208. On the other hand, if it is determined in step S207 that the processes have not been completed for all detected objects, the process returns to step S204, and the subsequent steps are executed in order.

[0062] In step S208, the temperature control unit 110 reads out temperature data associated with each warm / cold sensation presentation area 304 (405) from the temperature control memory 113, and controls the temperature of each warm / cold sensation presentation area 304 (405) of the warm / cold sensation presentation unit 114 to the corresponding read-out temperature. This allows the user of the warm / cold sensation presentation device 100 to receive a warm / cold sensation by touching the warm / cold sensation presentation unit 401 with a finger, a palm, or the like.

[0063] The above-described warm / cold sensation presentation device 100 can determine whether to perform temperature control only on the object-occupied area, which is the area in the image occupied by the desired object to be temperature-controlled. If the determination results in a determination to perform temperature control, the temperature of only the object-occupied area can be determined. This allows the user to grasp the temperature of the desired object in the image by touching the object-occupied area on the warm / cold sensation presentation unit 401.

[0064] Furthermore, regardless of the type of object, the degree of freedom can be set for the temperature control range in the object-occupied area by, for example, restricting the upper and lower limits of the temperature. Furthermore, in the thermal sensation presentation device 100, the temperature control determination unit 111 can set multiple object-occupied areas. In this case, the temperature control determination unit 111 determines whether or not to execute temperature control for each object-occupied area. Then, the temperature determination unit 112 can determine the temperature for each object-occupied area.

[0065] With this configuration, for example, when an animal, including a human, is identified as the type of object, the temperature control determination unit 111 can set an object-occupied area for each part of the animal's body (e.g., head, chest, etc.) and determine whether to execute temperature control for each object-occupied area. The temperature determination unit 112 can then determine the temperature of each object-occupied area set for each part of the body. This allows the user to grasp the temperature of each part of the body. In other words, the user can grasp which part of the body is hot or cold.

[0066] The second embodiment will be described below with reference to Figures 5 and 6, focusing on the differences from the first embodiment described above, and omitting a description of the same aspects as the first embodiment. The second embodiment differs from the first embodiment mainly in the configurations of the temperature control unit and the warm / cold sensation presentation unit, but is otherwise similar to the first embodiment. Figure 5 is a block diagram showing the hardware configuration of a warm / cold sensation presentation device according to the second embodiment.

[0067] The warm / cold sensation presentation device 500 includes an image data input unit 501, an image memory 502, a preprocessing unit 503, an object detection unit 504, and an object data memory 508. The warm / cold sensation presentation device 500 also includes a temperature data table memory 509, a temperature control unit 510, a temperature control memory 513, and a warm / cold sensation presentation unit 514. The image data input unit 501 is similar to the image data input unit 101 in the first embodiment, the image memory 502 is similar to the image memory 102 in the first embodiment, and the preprocessing unit 503 is similar to the preprocessing unit 103 in the first embodiment, so their description will be omitted. The object detection unit 504 is similar to the object detection unit 104 in the first embodiment, and the object data memory 508 is similar to the object data memory 108 in the first embodiment, so their description will be omitted. Furthermore, the temperature data table memory 509 is similar to the temperature data table memory 109 in the first embodiment, and the temperature control memory 513 is similar to the temperature control memory 113 in the first embodiment, so a description thereof will be omitted.

[0068] Fig. 6 is a schematic diagram showing the state of use of the warm / cold sensation presentation device shown in Fig. 5. As shown in Fig. 6, when using the warm / cold sensation presentation device 500, the user wears a head-mounted display (display means) 706. The head-mounted display 706 displays a virtual reality space 701 as an image. This allows the user to visually recognize a human being 702, which is an object, in the virtual reality space 701.

[0069] The virtual reality space 701 is an image obtained from image data input to the image data input unit 501. The warm / cold sensation presentation unit 514 is a glove-shaped warm / cold sensation presentation unit 703 that can be worn on the hand of a user who views the person 702 in the virtual reality space 701. The warm / cold sensation presentation unit 703 may have a ring shape that can be worn on a fingertip, for example, and is not limited to the shape described above. In other words, the warm / cold sensation presentation unit 703 may have any shape that can present a warm / cold sensation and can be worn on the user's body (for example, the user's hand or finger).

[0070] Furthermore, a Peltier element serving as temperature adjustment means for adjusting the temperature to the temperature determined by temperature control unit 510 is disposed in glove-shaped thermal sensation presentation unit 703 and comes into contact with the user's fingers and / or the palm or back of the hand. Temperature control unit 510 has a function of calculating the position and size of virtual reality space 701 and determining whether the user is touching human 702.

[0071] While wearing the head-mounted display 706 and the thermal sensation presentation unit 703, the user can touch a person 702 in a virtual reality space 701 displayed on the head-mounted display 706 via the thermal sensation presentation unit 703. At this time, if temperature control for the object type "person" is enabled in advance, the temperature control unit 510 determines whether the user's finger 704 is touching the person 702. If it is determined that the finger 704 is touching the person 702, the temperature control unit 510 reads temperature data for the object type "person" from the temperature data table memory 509. The temperature control unit 510 controls the temperature of the Peltier element disposed in the thermal sensation presentation unit 703 so that the user's finger 704 can feel the temperature of the object type "person." This allows the user to grasp the temperature of the person 702 by touching the person 702 with the finger 704. On the other hand, if it is determined that the finger 704 is not touching the person 702, the temperature control unit 510 does not control the temperature of the Peltier element.

[0072] The method for determining whether the finger 704 is touching the person 702 is not particularly limited. For example, the head-mounted display 706 emits invisible laser light, and a sensor in the head-mounted display 706 receives the light reflected from the finger 704. This allows the temperature control unit 510 to measure the distance from the head-mounted display 706 to the finger 704. Then, based on the difference between the distance and angle from the user's pupil position 705 to the finger 704, which are converted into arbitrary coordinates in the virtual reality space 701, and the distance and angle from the pupil position 705 to the person 702, it becomes possible to determine whether the finger 704 is touching the person 702. For example, if the difference falls within a certain numerical value, the temperature control unit 510 determines that the finger 704 is touching the person 702. On the other hand, if the difference does not fall within the certain numerical value, the temperature control unit 510 determines that the finger 704 is not touching the person 702.

[0073] The temperature control unit 510 may control the thermal sensation presentation by the thermal sensation presentation unit 703, taking into account the attenuation of temperature according to the distance between the finger 704 and the person 702. For example, a state in which the finger 704 (thermal sensation presentation unit 703) is touching the person 702 is defined as a 100% distance. Also, a state in which the finger 704 is not touching the person 702 but is relatively close is defined as a 0% distance. The temperature may then be changed linearly according to the distance between 0% and 100%. For example, when the distance is 0%, no temperature control is performed, and when the distance has changed to 100%, temperature control is performed to linearly increase the temperature.

[0074] 7 and 8, the third embodiment will be described, focusing on the differences from the previously described embodiment, and the same matters as the previously described embodiment will not be described. The third embodiment is different from the previously described embodiment mainly in that the thermal sensation presentation device has an object generation unit 615 and a display unit, but is otherwise similar to the second embodiment.

[0075] 7 is a block diagram showing the hardware configuration of a warm / cold sensation presentation device 600 according to the third embodiment. The warm / cold sensation presentation device 600 includes an object generation data input unit 601, an object data memory 608, a temperature data table memory 609, a temperature control unit 610, a temperature control memory 613, a warm / cold sensation presentation unit 614, an object generation unit 615, and a display unit 616.

[0076] Fig. 8 is a schematic diagram showing a state in which the warm / cold sensation presentation device shown in Fig. 7 is being used. As shown in Fig. 8 , in the third embodiment, similarly to the second embodiment, when using the warm / cold sensation presentation device 600, the user wears a head-mounted display (display means) 806. The head-mounted display 806 displays a virtual reality space 800 including a polygon object (three-dimensional image) 801 as an image. This allows the user to visually recognize the polygon object 801 in the virtual reality space 800.

[0077] In the third embodiment, the polygon object 801 is a human being. Similarly to the warm / cold sensation presentation unit 703 of the second embodiment, the warm / cold sensation presentation unit 614 of the third embodiment is a glove-shaped warm / cold sensation presentation unit 802 that can be worn on the user's hand. The warm / cold sensation presentation unit 802 may have a ring shape that can be worn on a fingertip, for example, and is not limited to the shape described above. In other words, the warm / cold sensation presentation unit 802 may have any shape that can present a warm / cold sensation and can be worn on the user's hand or finger.

[0078] The object generation data input unit (image data input means) 601 has an I / F for inputting three-dimensional object generation data (image data) from an external device. This I / F is not particularly limited, and examples thereof include an I / F that can insert and remove a memory card that can read and write data at high speed, such as an SD card or a CFexpress card. The object generation data input unit 601 reads three-dimensional object generation data recorded on the memory card and writes it to the object data memory 608.

[0079] "Three-dimensional object generation data" refers to data including, for example, the local three-dimensional coordinates of each polygon vertex for generating multiple polygons that make up an object (e.g., polygon object 801). This data also includes texture mapping coordinates for applying texture to each polygon surface, texture data, material data for each polygon surface, and object type information for determining the object's temperature.

[0080] The object generation unit 615 generates a polygon object 801 to be displayed in the virtual reality space 800 based on the three-dimensional object generation data. The display unit 616 performs a rendering process on the polygon object 801 generated by the object generation unit 615 and displays the processing result (rendering result) as image data. As a result, the polygon object 801 is displayed in the virtual reality space 800 on the head-mounted display 806, allowing the user to visually recognize the polygon object 801.

[0081] In this embodiment, the temperature control unit 610 functions as an object detection unit that detects the polygon object 801 and as a type identification unit that identifies the type of the polygon object 801. In the thermal sensation presentation device 600, the object generation unit 615 may also function as an object detection unit and a type identification unit. The temperature control unit 610 also has a function that calculates the position and size of the virtual reality space 800 and can determine whether or not the user is touching the polygon object 801.

[0082] While wearing the head-mounted display 806 and the thermal sensation presentation unit 802, the user can touch a polygon object 801 in a virtual reality space 800 displayed on the head-mounted display 806 via the thermal sensation presentation unit 802. At this time, if temperature control for the object type "human" is enabled in advance, the temperature control unit 610 determines whether the user's finger 803 is touching the polygon object 801. If it is determined that the finger 803 is touching the polygon object 801, the temperature control unit 610 reads temperature data for the object type "human" from the temperature data table memory 609. The temperature control unit 610 controls the temperature of the Peltier element disposed in the thermal sensation presentation unit 802 so that the user's finger 803 can feel the temperature of the object type "human." This allows the user to grasp the temperature of the human by touching the polygon object 801 with the finger 803. On the other hand, if it is determined that the finger 803 is not touching the polygon object 801, the temperature control unit 610 does not control the temperature of the Peltier element.

[0083] The method for determining whether the finger 803 is touching the polygon object 801 is not particularly limited. For example, the head-mounted display 806 emits invisible laser light, and a sensor in the head-mounted display 806 receives the light reflected from the finger 803. This allows the temperature control unit 610 to measure the distance from the head-mounted display 806 to the finger 803. The temperature control unit 610 then acquires the positional relationship between the finger position coordinates generated from the distance and angle from the user's pupil position 804 to the finger 803, which are converted into arbitrary coordinates in the virtual reality space 800, and the inner coordinates of the polygon object 801 in the virtual reality space 800. Based on this positional relationship, it is possible to determine whether the polygon object 801 is being touched. For example, if the finger position coordinates are located at the inner coordinates, it is determined that the polygon object 801 is being touched. On the other hand, if the finger position coordinates are not located at the inner coordinates, it is determined that the polygon object 801 is not being touched.

[0084] Furthermore, various types of notification may be provided. For example, if the temperature data associated with the polygon object 801 indicates a temperature lower than, for example, 20 degrees, the display unit 616 may notify the user that the polygon object 801 is cold when displaying the polygon object 801 after rendering. This notification method is not particularly limited, and examples include a method of drawing an effect and a method of displaying a notification frame around the polygon object 801. Other notification methods include a method of drawing a notification icon near the polygon object 801, a method of displaying a notification message, a method of emitting a notification sound, and the like. Such notification allows the user to know that the polygon object 801 is cold before touching the polygon object 801.

[0085] Furthermore, when the temperature data associated with the polygon object 801 indicates a temperature higher than, for example, 40 degrees, the display unit 616 may notify the user that the polygon object 801 is hot when displaying the rendered polygon object 801. This notification method is not particularly limited, and examples include a method of drawing an effect and a method of displaying a notification frame around the polygon object 801. Other notification methods include a method of drawing a notification icon near the polygon object 801, a method of displaying a notification message, a method of emitting a notification sound, and the like. Such notification allows the user to know that the polygon object 801 is hot before touching it.

[0086] The fourth embodiment will be described below with reference to Fig. 9. The differences from the previously described embodiments will be mainly described, and the same points as those in the previously described embodiments will not be described. The fourth embodiment is different from the previously described embodiments mainly in the configuration of the thermal sensation presentation unit, and is otherwise similar to the first embodiment.

[0087] 9 is a schematic diagram showing the configuration of a warm / cold sensation presentation device according to a fourth embodiment. The warm / cold sensation presentation device 90 includes a screen (display means) 901 and a projector (image projection device) 907 disposed at a distance from the screen 901. The screen 901 may be omitted from the configuration of the warm / cold sensation presentation device 90. In this case, the screen 901 or a screen equivalent thereto is prepared separately from the warm / cold sensation presentation device 90. The projector 907 projects an image 904 obtained from input image data onto the screen 901. This causes the image 904 to be displayed on the screen 901. The position of the screen 901 is then adjusted relative to the projection angle of the projector 907.

[0088] The image 904 includes a person 903 and a tree 900. The screen 901 has a thermal sensation presentation unit 114 on the back side thereof, which has a plurality of Peltier elements (thermal sensation presentation elements 305) arranged in a matrix. The area of ​​the screen 901 that overlaps with the person 903 (the area on the screen 901 where the person 903 is projected) is subject to temperature control and is adjusted. While viewing the person 903 projected on the screen 901, the user can feel the temperature corresponding to the person 903 by touching the area of ​​the screen 901 that corresponds to the person 903.

[0089] The fifth embodiment will be described below with reference to Fig. 10. The differences from the previously described embodiments will be mainly described, and the same points as those in the previously described embodiments will not be described. The fifth embodiment differs from the previously described embodiments mainly in the configuration of the thermal sensation presentation unit, and is otherwise similar to the first embodiment.

[0090] FIG. 10 is a schematic diagram showing the configuration of a warm / cold sensation presentation device 10 according to a fifth embodiment. The warm / cold sensation presentation device 10 includes a warm / cold sensation presentation unit 1005 and a printing unit 1007. The printing unit 1007 may be omitted from the configuration of the warm / cold sensation presentation device 10. In this case, the printing unit 1007 or an equivalent printing unit is provided separately from the warm / cold sensation presentation device 10. The printing unit 1007 is preferably communicatively connected to the warm / cold sensation presentation device 10. The printing unit 1007 forms and prints an image 1004 obtained from input image data on a sheet 1001, and outputs (prints out) the resulting printout 1008. As a result, the image 1004 is displayed on the sheet 1001. The image 1004 includes a person 1003 and a tree 1000.

[0091] The warm / cold sensation presentation unit 1005 is a warm / cold sensation presentation unit 114 having a plurality of Peltier elements (warm / cold sensation presentation elements 305) arranged in a matrix on its front side. Each Peltier element of the warm / cold sensation presentation unit 1005 may be convex. The warm / cold sensation presentation device 10 is used by overlaying a printed material 1008 on the warm / cold sensation presentation unit 1005. The warm / cold sensation presentation unit 1005 preferably includes a detection unit that detects that the printed material 1008 has been overlaid and an alignment unit that aligns the printed material 1008 with the printed material 1008. The portion of the warm / cold sensation presentation unit 1005 that overlaps with the person 1003 is subject to temperature control and is adjusted to a desired temperature. A user can feel the temperature corresponding to the person 1003 by touching the area of ​​the person 1003 on the printed material 1008 while visually recognizing the person 1003 depicted on the printed material 1008.

[0092] In the above-described embodiments, the warm / cold sensation presentation device is, for example, the warm / cold sensation presentation device 100 applied to an image processing device, but is not limited to this. For example, the warm / cold sensation presentation device may have an image processing device built in. Other examples of applications of the warm / cold sensation presentation device include a system including an image processing device and a warm / cold sensation presentation device configured separately from the image processing device, which receives temperature information generated by the image processing device and presents a warm / cold sensation.

[0093] The present invention can also be realized by a process in which a program that realizes one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

[0094] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.

[0095] This application claims priority based on Japanese Patent Application No. 2023-083047, filed on May 19, 2023, the entire contents of which are incorporated herein by reference.

[0096] REFERENCE SIGNS LIST 100 Warm / cold sensation presentation device 101 Image data input unit (image data input means) 104 Object detection unit 105 Scene determination unit (scene determination means) 107 Authentication unit (authentication means) 109 Temperature data table memory 111 Temperature control determination unit (control determination means) 112 Temperature determination unit (temperature determination means) 114 Warm / cold sensation presentation unit

Claims

1. An image processing device capable of processing images including objects, comprising: image data input means for inputting image data of the image; object detection means for detecting the object in the image based on the image data; type identification means for identifying the type of the object detected by the object detection means; control decision means for deciding whether or not to execute temperature control for an object occupied area occupied by the object in the image based on the identification result by the type identification means; and temperature decision means for deciding the temperature of the object occupied area when the control decision means decides to execute the temperature control as a result of the decision.

2. An image processing device as described in claim 1, further comprising a setting means for setting the type of the object in advance, wherein the control decision means decides to execute the temperature control when the type of the object identified by the type identification means matches the type of the object preset by the setting means, and decides not to execute the temperature control when the type of the object identified by the type identification means does not match the type of the object preset by the setting means.

3. The image processing device according to claim 1, characterized in that the control judgment means is capable of setting a plurality of object occupancy areas and judges whether or not to execute the temperature control for each object occupancy area, and the temperature determination means is capable of determining a temperature for each object occupancy area.

4. An image processing device as described in claim 1, further comprising a storage means for storing in advance temperature information relating to the temperature for each type of object, and the temperature determination means determines the temperature of the object-occupied area based on the temperature information.

5. The image processing device according to claim 1, wherein said temperature determining means is capable of changing a method of determining said temperature depending on the type of said object when determining the temperature of said object-occupied area.

6. An image processing device as described in claim 1, characterized in that the temperature determination means, when determining the temperature of the object-occupied area, is capable of setting at least one of an upper limit of the temperature and a lower limit of the temperature depending on the type of the object.

7. An image processing device according to claim 1, further comprising a scene determination means for determining a scene of the image, and the temperature determination means determines the temperature of the object-occupied area based on a result of the determination by the scene determination means.

8. The image processing device described in claim 1, characterized in that when the type identification means identifies the type of object as an animal including a human, the control judgment means sets the object occupancy area for each part of the animal's body and judges whether or not to execute the temperature control, and the temperature determination means determines the temperature of each of the object occupancy areas set for each part of the body.

9. An image processing device as described in claim 8, further comprising a display means for displaying the image, the display means being capable of displaying on the image the degree of deviation between the temperature of each object-occupied area determined by the temperature determination means and the average body temperature of the animal.

10. An image processing device as described in claim 1, further comprising a display means for displaying the image, the display means being capable of displaying on the image a display corresponding to the temperature of the object occupying area determined by the temperature determination means.

11. The image processing device according to claim 1, further comprising: a display means for displaying the image; and a temperature adjustment means for adjusting the temperature of the image displayed on the display means to the temperature of the object-occupied area determined by the temperature determination means.

12. The image processing device according to claim 11, wherein said temperature adjustment means is configured to be wearable on the body of a user who uses said display means.

13. An image processing apparatus according to claim 11, wherein said display means displays a virtual reality space as said image.

14. The image processing apparatus according to claim 11, wherein said display means displays a three-dimensional image as said image.

15. An image processing apparatus according to claim 11, wherein said display means projects said image onto a screen and displays said image on said screen.

16. An image processing apparatus according to claim 11, wherein said display means displays said image by printing said image on a sheet.

17. An image processing device as described in claim 1, further comprising an authentication means for authenticating the type of the object identified by the type identification means, and when the type of the object identified by the type identification means is authenticated by the authentication means, a judgment is made by the control judgment means and a decision is made by the temperature determination means.

18. The image processing device according to claim 1, characterized in that the object detection means detects the object using machine learning.

19. The image processing device according to claim 1, wherein the type identification means identifies the type of the object using machine learning.

20. The image processing device according to claim 1, wherein said object detection means and said type identification means are integrated into one unit, or are independent units.

21. A thermal sensation presentation device comprising the image processing device according to claim 1.

22. A control method for controlling an image processing device capable of processing an image including an object, comprising: an image data input step for inputting image data of the image; an object detection step for detecting the object in the image based on the image data; a type identification step for identifying the type of the object detected in the detection step; a control judgment step for determining whether or not to perform temperature control on an object occupied area occupied by the object in the image based on the identification result in the type identification step; and a temperature determination step for determining the temperature of the object occupied area when it is determined as a result of the control judgment step that the temperature control is to be performed.

23. A program for causing a computer to execute each of the means of the image processing device according to claim 1.

Citation Information

Patent Citations

  • Coldness and warmth feeling display device and method for driving the same

    JP2011180678A

  • Electronic apparatus and control method for electronic apparatus

    JP2019194833A

  • User stimulus system

    JP2020135191A