Image processing apparatus, method for controlling the same, image information presentation device, image information presentation system, and program

Abstract

To provide an image information presentation device, a method for controlling the same, and a program for presenting an input image such as a picture so that a visually impaired person can grasp the details of the input image by touching the image.SOLUTION: An image processing apparatus comprises: an image input unit 111 for inputting image data; a detection unit 114 that detects the type of an object and an area of the object on the basis of the image data inputted by the image input unit 111; and a presentation mode determination unit 117 that determines a presentation mode for a tactile sense presentation area corresponding to the area of the object detected by the detection unit 114 on the basis of the type of the object detected by the detection unit 114.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device that presents the content of an input image in a manner that allows it to be grasped tactilely, a control method thereof, an image information presentation device, an image information presentation system, and a program. [Background technology]

[0002] A presentation device has been proposed that presents the tactile sensation of text, figures, and unevenness of a picture displayed on a computer screen by independently raising and lowering multiple pins arranged in a two-dimensional shape, in response to the unevenness of the text, figures, and unevenness of a picture, and is used by visually impaired people, etc. This type of presentation device is called a "pin display."

[0003] Also proposed is a presentation device that electrically controls two-dimensionally arranged semiconductor thermoelectric elements called "Peltier elements" to generate heat in response to the unevenness of text, figures, pictures, etc. displayed on a computer screen, to present the sensation of warmth and cold in response to the unevenness of text, figures, pictures, etc. Such presentation devices are also used by visually impaired people and are called "thermal sensation displays" (see Non-Patent Document 1).

[0004] Additionally, when presenting image data such as photographs to visually impaired people instead of the displays on PC monitors, smartphones, TVs, etc. used by people with normal vision ("sighted people"), the following displays may be used. For example, instead of devices such as PC monitors, smartphones, and TVs, a "tactile display" may be used to recognize image data through the sense of touch. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Kumiko Kushiyama, Yuki Doi, Shinji Sasada, and Tetsuaki Baba; Thermo Drawing: "Development of a temperature drawing system using a small tactile display with hot and cold display; Information Processing Society of Japan, Interaction" pp. 723-728 (2012) Summary of the Invention [Problem to be solved by the invention]

[0006] However, the above-mentioned "tactile display" is far from providing a tactile presentation suitable for, for example, viewing photographs to visually impaired people. In this regard, there is a method of creating a textured photograph or the like to provide a tactile presentation, but providing textured images to image data is extremely time-consuming and expensive. In addition, since photographs and the like contain a large amount of information, if they are directly expressed as textured images, the textured images become noise and may interfere with viewing the photograph. In the case of color textured photographs, it was also time-consuming to consult with visually impaired people each time to decide where to place the textured images.

[0007] An object of the present invention is to provide an image processing device, a control method thereof, an image information presentation device, an image information presentation system, and a program that present the content of an input image such as a photograph to a visually impaired person in a manner that allows the visually impaired person to grasp the content of the input image by tactile means. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the present invention is characterized in comprising an image input unit for inputting image data, a detection unit for detecting the type of object and the area of ​​the object based on the image data input by the image input unit, and a presentation mode determination unit for determining the presentation mode of a tactile presentation area corresponding to the area of ​​the object detected by the detection unit based on the type of object detected by the detection unit. Effect of the Invention

[0009] According to the present invention, it is possible to provide an image processing device, a control method thereof, an image information presentation device, an image information presentation system, and a program that present the contents of an input image such as a photograph to a visually impaired person in a manner that allows them to grasp the contents of the input image by tactile means. [Brief description of the drawings]

[0010] [Figure 1] 1 is a configuration diagram of an image information presentation device according to an embodiment of the present invention. [Diagram 2] 5 is a flowchart showing an object and area detection process of the image information presentation device according to the first embodiment of the present invention. [Diagram 3] 5 is a flowchart showing a tactile presentation process of the image information presentation device according to the first embodiment of the present invention. [Figure 4] FIG. 2 is an explanatory diagram of an input / output image in the first embodiment of the present invention. [Diagram 5] 13 is a flowchart showing an object and area detection process of an image information presentation device according to a second embodiment of the present invention. [Figure 6] 10 is a flowchart showing a tactile presentation process of the image information presentation device according to the second embodiment of the present invention. [Figure 7] 13 is a flowchart showing an object and area detection process of the image information presentation device according to the third embodiment of the present invention. [Figure 8] 13 is a flowchart showing a tactile presentation process of the image information presentation device according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in the embodiments. In this specification, "touch" is used to mean various sensations that occur when the hand touches an object. Furthermore, "temperature and cold" is used to mean the sensation of hotness or cold felt by the hand, and "irregularity" is used to mean the sensation of unevenness felt by the hand. Therefore, "touch" includes "temperature and cold" and "irregularity".

[0012] <Processing Overview of Image Information Presentation Device 100> The detection unit 114 of the image information presentation device 100 according to the embodiment of the present invention detects the "type of object, the area of ​​the object" and the like based on the image data input by the image input unit 111. The distance acquisition unit 115 acquires the distance of the object based on the image data. The tactile presentation unit 130 presents the "area of ​​the object" and the like for "each type of object". The tactile presentation unit 130 also presents the "area of ​​the object" and the like for "each type of object". Furthermore, the tactile presentation unit 130 presents the "area of ​​the object" and the like for "each distance of the object" and "each type of object". More specifically, in response to the operation control of the presentation control unit 119, the tactile presentation unit 130 controls the displacement amount of each pin of the "pin display" according to the presentation mode determined by the presentation mode determination unit 117, and presents information so that it can be grasped by the sense of touch. Similar information presentation is performed by controlling the temperature of each Peltier element of the "thermal display" according to the presentation mode determined by the presentation mode determination unit 117. The detection information presentation unit 131 presents the same information in the same manner as a Braille display and audio.

[0013] <Figure 1: Configuration of image information presentation device 100> 1 is a configuration diagram of an image information presentation device 100 according to an embodiment of the present invention. The image information presentation device 100 has an image processing unit 110, a tactile presentation unit 130, a detection information presentation unit 131, a system control unit 140, an operation unit 141, a non-volatile memory 142, and a system memory 143. The image processing unit 110 also has an image input unit 111, an image memory 112, a preprocessing unit 113, a detection unit 114, a distance acquisition unit 115, an object information memory 116, a presentation mode determination unit 117, a presentation control memory 118, and a presentation control unit 119.

[0014] <Image Input Unit 111> The image input unit 111 has an "I / F (interface)" for inputting digital image data. The image input unit 111 has, for example, an HDMI (registered trademark) terminal as the "I / F", receives a video signal conforming to the HDMI standard, extracts image data from the received video signal, and executes a process of writing the extracted image data to the image memory 112. The image input unit 111 may also be configured to have a "physical I / F" into which a memory card capable of reading and writing high-resolution image data at high speed, such as an SD card or a CFexpress card, can be inserted and removed. In this case, the image input unit 111 reads image data recorded on the inserted memory card, and writes the read image data to the image memory 112.

[0015] <Image memory 112: Preprocessing unit 113> Image memory 112 is composed of semiconductor memory such as DRAM, SRAM, etc., and stores image data input by image input unit 111. Pre-processing unit 113 reads out image data stored in image memory 112. Pre-processing unit 113 performs optimal image processing on the read out image data for processing in detection unit 114 and distance acquisition unit 115, and writes the image data that has been subjected to optimal image processing into image memory 112. Here, optimal image processing refers to gamma correction processing, shading correction processing, white balance correction processing, etc., performed on the "YUV signal" when the image data is composed of, for example, a "YUV signal." In addition to these correction processes, noise reduction processing, resizing processing, etc. may also be performed.

[0016] <Detection unit 114> The detection unit 114 reads out image data stored in the image memory 112, and detects one or more predetermined object regions from the read out image data. The detection unit 114 detects the position of the object in the image data, the size of the object, the type of object (car, airplane, bird, insect, human body, head, eye, cat, dog, ball, etc.), the object region, etc. The detection unit 114 writes "object data" in which the position of the object in the image data, the size of the object, the type of object, the reliability of each type, etc. are associated with each detected object region in the image data, into the object information memory 116.

[0017] The detection unit 114 detects "candidate regions" using known techniques for detecting "characteristic regions" such as the facial regions of people and animals. For example, the detection unit 114 may be configured as a trained class classifier using training data. There are no particular limitations on the classification algorithm. The detection unit 114 can be realized by "training" a classifier equipped with a multi-class logistic regression, a support vector machine, a random forest, a neural network, or the like. In addition, the detection unit 114 writes "map information" that maps the region of the object that occupies the image data and "type of region (sky, human skin, grass, etc.)" as "region data" in the object information memory 116 for each region of the detected object.

[0018] <Distance acquisition unit 115> The distance acquisition unit 115 reads out image data stored in the image memory 112, and estimates the distance to each object based on the read image data. As "distance information", a "distance map" in which the "distance information" for each pixel is converted into map data is written into the object information memory 116. For distance estimation, a known technique using, for example, a trained neural network can be used. In addition, various methods for estimating distance from an image have been proposed. The subject distance estimation method in the distance acquisition unit 115 may be any method that allows distance estimation. Note that, when the "distance information" is input in advance together with the image data, the "distance map" may be generated based on the input "distance information".

[0019] <Object Information Memory 116> The object information memory 116 is realized by a storage device such as a RAM, and stores the detection information detected by the detection unit 114, the distance information acquired by the distance acquisition unit 115, and the like.

[0020] <Presentation mode determination unit 117> Based on instructions from the system control unit 140, which will be described later, the presentation mode determination unit 117 determines a "presentation mode" in a "tactile presentation area" that corresponds to the area occupied by each type of object detected by the detection unit 114, and writes the determined mode in the presentation control memory 118, which will be described later. At this time, the presentation mode determination unit 117 also writes various types of detection information in the "tactile presentation area" (e.g., type of object, number of detected objects, detection coordinates of the objects, detection area of ​​the objects, etc.) in the presentation control memory 118, which will be described later.

[0021] The "presentation mode" will be described later, but includes "a mode of presentation for each type of object," "a mode of presentation for each distance and each type of object," "a mode of presentation for each distance," etc. The presentation mode determination unit 117 further determines the presentation mode depending on the type of "tactile presentation device (pin display, thermal display, etc.)" to be used.

[0022] When the "tactile presentation device" is a "pin display", the pin displacement amount is determined according to the area of ​​each object to be presented. The pin displacement amount for the area of ​​each object can be arbitrarily determined according to the resolution of the pin displacement amount. The pin displacement amount corresponding to the area of ​​each object may be, for example, the maximum displacement amount of the pin or the minimum displacement amount, or may be a displacement amount of "1 / N (N is an integer of 2 or more)" of the "maximum displacement amount". When the "tactile presentation device" is a "thermal display", the temperature of the area of ​​each object to be presented is determined. For example, in the case of a "thermal display" capable of presenting temperatures from "10 degrees" to "40 degrees", any presented temperature within this temperature range can be determined.

[0023] In conclusion, when a pin display is used, the presentation mode determination unit 117 determines the displacement amount of the pin corresponding to the object area, making it possible to distinguish between the object area and the non-object area by tactile sensation. Also, when a thermal display is used, the presentation mode determination unit 117 determines the temperature of the Peltier element corresponding to the object area, making it possible to distinguish between the object area and the non-object area by thermal sensation.

[0024] <Presentation control memory 118> The presentation control memory 118 is a storage device for storing the detection information by the detection unit 114, the distance information acquired by the distance acquisition unit 1115, etc. The presentation control memory 118 stores necessary information such as "type of object, number of detected objects, coordinates of the object, area of ​​the object, distance of the object," etc.

[0025] <Operations of Presentation Control Unit 119 and Tactile Presentation Unit 130> The presentation control unit 119 controls the operation of the tactile presentation unit 130, which will be described later, by referring to the contents stored in the presentation control memory 118. As a result, the tactile presentation unit 130 executes the following process using at least one of the information presentation means of "thermal display" and "pin display" in accordance with the presentation mode determined by the presentation mode determination unit 117. That is, the tactile presentation unit 130 presents images of the area of ​​the object detected by the detection unit 114 on multiple sheets individually so that the area can be grasped by the sense of touch for each type of object.

[0026] In addition, the tactile presentation unit 130 is controlled by the presentation control unit 119 to perform the following process using at least one of the information presentation means of the "thermal display" and the "pin display" according to the presentation mode determined by the presentation mode determination unit 117. That is, the tactile presentation unit 130 presents a plurality of images of the area of ​​an object for each type of object detected by the detection unit 114 in a manner that allows the image to be grasped by the sense of touch, based on the distance information acquired by the distance acquisition unit 115. Similarly, the tactile presentation unit 130 is controlled by the presentation control unit 119 to perform the following process using at least one of the information presentation means of the "thermal display" and the "pin display" according to the presentation mode determined by the presentation mode determination unit 117. That is, the tactile presentation unit 130 presents a plurality of images of the area of ​​an object for each distance of the object detected by the detection unit 114 in a manner that allows the image to be grasped by the sense of touch, based on the distance information acquired by the distance acquisition unit 115. Note that presenting for each distance includes not only presenting for exactly the same distance, but also presenting for each range of a predetermined distance. For example, distances A and less than distance B and distances B and less than distance C may be presented separately.

[0027] <Configuration of the tactile sense providing unit 130, etc.> The tactile presentation unit 130 can be realized by, for example, a "pin display" or a "thermal display". Any type of tactile presentation device may be used as long as it is a device capable of presenting tactile sensations and can achieve the object of the present invention. A "pin display" is a "means for presenting information using a sense of unevenness" and a "thermal display" is a "means for presenting information using a sense of warmth and cold". The "means for presenting information using a sense of warmth and cold" and the "means for presenting information using a sense of unevenness" are also referred to simply as "means for presenting information using a sense of warmth and cold" and "means for presenting information using a sense of unevenness", respectively.

[0028] A "pin display" has pins corresponding to multiple pixels arranged two-dimensionally, and by controlling the amount of displacement of each pin, it is possible to grasp the "area of ​​the object" and "other" through the sense of unevenness (touch) of the pins. On the other hand, a "thermal display" has Peltier elements corresponding to multiple pixels arranged two-dimensionally, and by controlling the temperature of each Peltier element, it is possible to grasp the "area of ​​the object" and "other" through the sense of hot and cold (touch).

[0029] The tactile presentation unit 130 presents information in a tactile manner using either the "thermal display" or the "pin display" information presentation means according to the presentation mode determined by the presentation mode determination unit 117. Alternatively, the tactile presentation unit 130 presents information in a tactile manner using both the "thermal display" and the "pin display". There are various modes in which both are used in combination. For example, the types of objects are divided into "people" and "other things", and information is presented for these areas in a tactile manner. As an example, information on the "people area" is presented using the "pin display", while information on the "non-people area" is presented using the "thermal display". This allows visually impaired people, whose main purpose is to grasp "people", to instantly grasp the size, number, etc. of the "people area" by touching the pins on the "pin display".

[0030] Another example is to present information near the center of an image using a "pin display" and to present information for the remaining image using a "thermal display." This allows visually impaired people to instantly grasp the "area of ​​objects, the number of objects, the size of objects, etc." that exist near the center of the image by tactile sensation to the pins of the "pin display." On the other hand, the "area of ​​objects, the number of objects, the size of objects, etc." in the image parts other than the center of the image can be grasped by the thermal sensation to the Peltier element of the "thermal display." Basically, interest in "objects" that exist near the center of the image is higher than interest in other parts. Therefore, by presenting information using different displays for "objects" in the "near the center of the image" and "other image parts," visually impaired people can instantly grasp information presented about "objects" that exist near the center of the image. However, the manner in which information is presented using both displays is not limited to these.

[0031] <Detection information presentation unit 131> The detection information presentation unit 131 is controlled in operation by the presentation control unit 119 to present information on objects that can be presented by the tactile presentation unit 130 or information on presented objects in a manner that allows the information to be understood in Braille or audio. Specifically, the detection information presentation unit 131 presents a plurality of object area images for each type of object, or presents a plurality of object area images for each type of object based on distance information. The detection information presentation unit 131 further presents a plurality of object area images for each distance of the object based on distance information. These presentations are performed in a manner that allows the information to be understood in Braille or audio.

[0032] The detection information presenting unit 131 is controlled by the presentation control unit 119 to present at least one of the detection information of the object detected by the detection unit 114 and the distance information acquired by the distance acquisition unit 115 by "Braille display and / or voice". Specifically, the information presented by the detection information presenting unit 131 is "object type, number of detected objects, object coordinates, object area, object distance", etc. This information is presented to the user (visually impaired person) by "Braille display and / or voice". As an example of information presented by "voice", information such as "objects A to N exist", "the distances of objects A to N are close, far, or intermediate", and "the types of objects A to N are 'person', 'dog', 'car', etc." are explained by voice.

[0033] <System control unit 140> The system control unit 140 controls the entire image information presentation device 100. A processor such as a CPU or DSP (not shown) executes a program recorded in a non-volatile memory 142 (described later), thereby implementing each process of this embodiment (described later) and each unit in FIG.

[0034] <Operation unit 141> The operation unit 141 is an operation means for giving various operation instructions to the system control unit 140. The operation unit 141 can utilize known techniques such as input operation using a keyboard or voice input operation using voice recognition. In addition, the operation unit 141 may be configured to include a switching unit that switches between switching and presenting the "type of object" detected by the detection unit 114 or switching and presenting the "distance information" acquired by the distance acquisition unit 115, depending on the operation of the operation unit 141 by the user.

[0035] <Non-volatile memory 142> The non-volatile memory 142 is an electrically erasable and recordable memory, and can be realized by, for example, an EEPROM. The non-volatile memory 142 stores constants and programs for operating the system control unit 140 in a non-volatile manner. As described above, the programs recorded in the non-volatile memory 142 are for executing various flowcharts in this embodiment.

[0036] <System Memory 143> System memory 143 is realized by, for example, a RAM. System memory 143 stores constants, variables, etc. for operation of system control unit 140. When system control unit 140 executes a program, it reads the program from non-volatile memory 142, expands it in system memory 143, and executes the expanded program. This realizes the processes in each embodiment and each unit shown in FIG. 1.

[0037] <<First embodiment>> Next, the processing in the first embodiment will be described with reference to the flowcharts in Figures 2 and 3. Note that when describing the processing in the first embodiment, the explanatory diagram of input and output images shown in Figure 4 will also be referenced. The first embodiment is characterized in that the detection unit 114 detects "type of object, position of object, size of object, area of ​​object" and the like, and divides the object into multiple parts (multiple sheets) for each type of object and presents them on a tactile presentation device such as a pin display.

[0038] <FIG. 2: Flowchart showing "object / area detection processing" in the first embodiment> The "object / area detection process" will be described with reference to Fig. 2. First, in S200, image input unit 111 writes input image data to image memory 112. Next, in S201, pre-processing unit 113 reads image data from image memory 112, performs image processing suitable for detection unit 114 to execute the "object / area detection process", and writes the image data after image processing to image memory 112.

[0039] Next, in S202, the detection unit 114 reads image data from the image memory 112 and performs "object / area detection." That is, the detection unit 114 detects one or more object areas set in advance based on the read image data. For each detected object area, the detection unit 114 detects the object position, object size, object type (car, airplane, bird, insect, human body, head, eye, cat, dog, ball, etc.) in the image data, etc.

[0040] Then, in S203, detection unit 114 writes the detection information (object position, object size, object type, etc.) to object information memory 116. More specifically, as described above, detection unit 114 writes object data in object information memory 116 that associates "object position, object size, object type, and reliability for each type" in the image data for each detected region.

[0041] <FIG. 3: Flowchart showing tactile presentation processing in the first embodiment> Next, the tactile presentation process will be described with reference to Fig. 3. First, in S300, the detection information presenting unit 131 presents the detection information written in the object information memory 116 in S203 of Fig. 2. The presented detection information is information such as "type of object, number of detected objects of each type, detected position (coordinates) of object, area of ​​object", etc. The detection information presenting unit 131 presents information by Braille display and audio, so that visually impaired people can understand the presented information.

[0042] Next, in S301, the user designates an object that the user desires to present from the detection information presented in S300 using the operation unit 141. For example, the designation is made by keyboard input, voice input using voice recognition, or the like.

[0043] Next, in S302, the tactile presentation unit 130 presents tactile information. Then, in S303, if the system control unit 140 determines that the user has not operated the operation unit 141 to instruct the presentation of the next object (No), the process ends. On the other hand, in S303, if the system control unit 140 determines that the user has operated the operation unit 141 to instruct the presentation of the next object (Yes), the process returns to S301 and continues.

[0044] In the presentation process in the flowchart shown in Fig. 3, the user is configured to indicate the type of object to be presented, but the detected object types may be presented tactilely in a preset order and the detection information may be presented automatically. Examples of the "preset order" include "in alphabetical order of object names" and "in order of object size" for object types. In this case, since there are individual differences in the time it takes to recognize tactile information to switch from the presented information to the presentation of a different object type, it is preferable that the user himself be able to give the instruction.

[0045] <Summary of the First Embodiment> In the first embodiment, the detection unit 114 detects the type of object and the area of ​​the object based on the image data input by the image input unit 111. The presentation mode determination unit 117 determines the presentation mode of the tactile presentation area corresponding to the area of ​​the detected object based on the type of the detected object. The tactile presentation unit 130 uses at least one of the information presentation means of a "thermal display" and a "pin display" in accordance with the determined presentation mode to present images of the area of ​​the object across multiple sheets for each type of detected object in a manner that allows the image to be grasped by the sense of touch.

[0046] The presentation mode determination unit 117 may also be configured to further include a change unit that changes the degree to which at least one of the information presentation means of the "thermal display" and the "pin display" is presented so as to be graspable by touch, depending on the type of object detected by the detection unit 114. When either the "thermal display" or the "pin display" is used, this change unit changes the degree to which the one of the displays is presented so as to be graspable by touch, depending on the type of object detected by the detection unit 114. When both displays are used in combination, this change unit changes the degree to which both displays are presented so as to be graspable by touch, depending on the type of object detected by the detection unit 114. In other words, this change unit changes the pin displacement amount of the pin display and the temperature of the Peltier element in the thermal display depending on the type of object detected.

[0047] <Figure 4: Illustration of input / output image> Next, an overview of the input / output image will be described with reference to Fig. 4. Fig. 4(a) is an "input image". The "input image" in Fig. 4(a) shows a "person" and "dog A" facing each other near the center of the image in a natural setting with "grass, grass" and "sky". Also, "dog B" at the bottom right of the image is running from the front toward the "person" located near the center in the background. For this example "input image", the detection unit 114 detects the "type of object" and "area of ​​object" of "person", "dog A, dog B", "sky", and "grass".

[0048] In accordance with the first embodiment, the "person" and "dog A" in Fig. 4(a) are objects at approximately the same distance. However, even if the distance information is the same, the "person" and "dog A" are presented separately because the type of object is taken into consideration. This allows the visually impaired person to distinguish between them.

[0049] Figure 4(b) is an explanatory diagram showing the correspondence between the "input image" and each pixel of the "thermal display." Specifically, it shows the correspondence between the "input image" and the "thermal display" with a resolution of "10 pixels" horizontally and "8 pixels" vertically. Figure 4(c) shows the case where a "human area" is presented in the "input image." In Figure 4(c), the "shaded area" is the "human area" in the "input image." By changing the presented temperature between the "shaded area" and the rest of the image, the user can recognize the "human area (position)."

[0050] FIG. 4(d) shows the case where the "areas" of "Dog A and Dog B" are presented in the "input image". The "shaded area" in FIG. 4(d) is the "area of ​​Dog A and Dog B" in the "input image". By changing the presentation temperature between the "shaded area" and the rest of the image, the user can recognize the "area (position) of Dog A and Dog B". Similarly, FIG. 4(e) shows the case where the "sky area" is presented in the "input image", and FIG. 4(f) shows the case where the "grass area" is presented in the "input image".

[0051] As described above, the image information presentation device 100 according to the first embodiment can present "object position, object size, etc." for each type of object based on type information of the object in the input image. This enables visually impaired people to identify "object position, object size, etc." for each type of object in the input image even in a photograph with a large amount of information.

[0052] In the first embodiment, the image input unit 111 inputs a photographic image or the like, and the detection unit 114 detects "subject area" and "type information" (metadata) for each type of object based on the detected detection information. The tactile presentation unit 130 presents multiple sheets of image data stored for each type of object in the presentation control memory 118 in a time-division manner. In this way, by presenting "object position, object size" and the like for each type of object, visually impaired people can identify "object position, object size" and the like for each type of object appearing in a photograph, even if the photograph contains a large amount of information, and can therefore enjoy viewing the photograph.

[0053] <<Second embodiment>> Next, the processing of the second embodiment will be described with reference to Fig. 5 and Fig. 6. In the second embodiment, the "type of object", "position of object", "size of object", "area", etc. are detected by the "object / area detection processing" executed by the detection unit 114. Furthermore, the distance acquisition unit 115 acquires distance information, divides it into "for each object distance" and "for each object type" multiple times, and presents it on a tactile presentation device such as a pin display.

[0054] <FIG. 5: Flowchart showing "object / area detection processing" in the second embodiment> The "object / area detection process" will be described with reference to Fig. 5. Note that the processes in S500, S501, and S502 are the same as those in S200, S201, and S202 in Fig. 2 in the first embodiment, so duplicated descriptions will be omitted.

[0055] In S503, the distance acquisition unit 115 reads out the image data stored in the image memory 112 and estimates the "distance information" based on the read out image data. If the "distance information" is directly input together with the image data, the "distance information" is not estimated and the input "distance information" is used as is.

[0056] Then, in S504, system control unit 140 associates the "detection information (type of object, position of object, size of object, etc.)" detected by detection unit 114 with the "distance information" acquired by distance acquisition unit 115, and writes the information in object information memory 116. Specifically, the system control unit 140 writes the detection information of detection unit 114 in object information memory 116 for "each type" of object detected by detection unit 114 and for "each distance information (each distance)" acquired by distance acquisition unit 115.

[0057] <FIG. 6: Flowchart showing tactile presentation processing according to the second embodiment> Next, the tactile presentation process will be described with reference to Fig. 6. S601, S602, and S603 are the same processes as S301, S302, and S303 in Fig. 3 described in the first embodiment, so duplicated explanations will be omitted. Only S600 is different from the first embodiment.

[0058] In S600, the detection information presenting unit 131 presents to the user the "detection information" linked to the "distance information" written in the object information memory 116. The "detection information" presented to the user includes "type of object, number of detected objects for each type of object, coordinates of the object, area information of the object, distance of the object," etc. At this time, the information is presented to the user by a Braille display, audio, etc.

[0059] In the second embodiment, the "type of object" linked to the "distance information" is presented. Therefore, even if the "type of object" is the same, the "distance information" can be presented in a way that makes it possible to understand that the "distance information" is different. In this way, the image information presentation device 100 according to the second embodiment presents the "position of the object", the "size of the object", etc. for each "distance information" and "type of object" based on the "information on the type of object" and the "distance information" in the input image. Therefore, even if the input image is a photograph or the like with a large amount of information, the visually impaired person can understand the "position of the object", the "size of the object", etc. by taking into account the depth for each type of object in the input image. Since the information in the depth direction is taken into account, the visually impaired person can grasp and identify the object in three dimensions.

[0060] An example of the second embodiment will be described with reference to FIG. 4 as follows. For example, if the distances of "sky", "person", and "dog B" in FIG. 4(a) are "KA", "KB", and "KC", respectively, then "KA>KB>KC". Even if the distances of each object are different in this way, according to the second embodiment, the object area is presented for "each object type" according to "distance information". Therefore, since the object area is presented for each distance information and each detection type, if the image data of each object is presented individually across multiple pages, visually impaired people can more easily grasp "objects, object positions (areas), object sizes" and the like in the input image.

[0061] In the second embodiment, the distance information acquired by the distance acquisition unit 115 and the distance information detected by the detection unit 114 are thing Based on the type of body, the "subject area (image)" and "type information / distance information" (metadata) are detected. The tactile presentation unit 130 presents multiple pieces of image data stored in the presentation control memory 118 in a time-division manner for "each type of object" and "each distance information." This allows visually impaired people to identify the "position and size of an object" for each type of object in a photograph according to the distance, allowing them to easily enjoy viewing photographs.

[0062] <<Third embodiment>> Next, the processing executed in the third embodiment will be described with reference to Fig. 7 and Fig. 8. In the third embodiment, the detection unit 114 detects the "type of object," the "position of the object," the "size of the object," the "area of ​​the object," and the like. Furthermore, the distance acquisition unit 115 acquires "distance information." Furthermore, the tactile presentation unit 130 presents the detection information by the detection unit 114 in multiple parts for each acquired "distance information."

[0063] <FIG. 7: Flowchart showing object / area detection processing in the third embodiment> 7 are the same processes as S500, S501, S502, and S503 in Fig. 5 described in the second embodiment, and therefore will not be described again. Only S704 differs from the second embodiment.

[0064] In S704, the detection information (type of object, position of object, size of object, etc.) detected by the detection unit 114 for each "distance information" acquired by the distance acquisition unit 115 is written to the object information memory .

[0065] <FIG. 8: Flowchart showing tactile presentation processing according to the third embodiment> The tactile presentation process of the third embodiment will be described with reference to Fig. 8. S801, S802, and S803 in Fig. 8 are the same processes as S601, S602, and S603 in Fig. 6 described in the second embodiment, so duplicated explanations will be omitted. Only S800 is different from the second embodiment.

[0066] In S800, the detection information presenting unit 131 presents the detection information of the detection unit 114 for "each distance information" written in the object information memory 116 to the user in a manner that allows the user to grasp it by means of a Braille display, voice, etc. The detection information to be presented includes "contents of object type, number of detected objects for each object type, object coordinates, object area information, object distance information," etc. In the third embodiment, since objects are presented for "each distance information," even if the types of objects are different, they can be presented to a visually impaired person at the same time as long as the "distance information" is the same.

[0067] As described above, the image information presentation device 100 according to the third embodiment presents, for each piece of distance information, the "area of ​​the object, the position of the object, the size of the object," etc., which are the information detected by the detection unit 114, based on the type information of the object in the input image and the acquired distance information. At this time, the tactile presentation unit 130 presents the information detected by the detection unit 114 (area of ​​the object, etc.) multiple times for each piece of acquired distance information.

[0068] Therefore, even if the input image is a photograph or other image with a lot of information, visually impaired people can grasp the "position and size of an object" taking into account the depth of each object in the input image. As a result, each object can be grasped three-dimensionally by touch, since the depth direction of each object is taken into account.

[0069] In the third embodiment, the "area (image) of the object's subject" and "type information / distance information (metadata)" detected by the detection unit 114 are detected based on the distance information acquired by the distance acquisition unit 115. The tactile presentation unit 130 presents multiple pieces of image data stored in the presentation control memory 118 based on the distance information in a time-division manner. This allows the visually impaired person to easily enjoy viewing photographs, since it becomes possible to identify the "position and size of the object" for each distance shown in the photograph.

[0070] <<Fourth embodiment>> Next, the process in the fourth embodiment will be described. The fourth embodiment is characterized in that the tactile presentation mode is changed according to the acquired "distance information", instead of presenting the detected object for each "distance information" as in the third embodiment.

[0071] For example, in the case of presentation using a "thermal display," the closer the object is as indicated by the "distance information," the higher the temperature is, and the farther the object is as indicated by the "distance information," the lower the temperature is. Alternatively, the opposite may be true, where the closer the object is as indicated by the "distance information," the lower the temperature is, and the farther the object is as indicated by the "distance information," the higher the temperature is. In this way, the sense of temperature changes depending on the distance of the object, allowing visually impaired people to easily grasp the distance of an object through their sense of temperature.

[0072] Furthermore, the presentation mode of the "pin display" is such that the closer the object is as indicated by the "distance information", the greater the pin displacement, and the farther the object is as indicated by the "distance information", the smaller the pin displacement. Or, conversely, the closer the object is as indicated by the "distance information", the smaller the pin displacement, and the farther the object is as indicated by the "distance information", the greater the pin displacement. In this way, the unevenness of the pin changes depending on the distance of the object, allowing visually impaired people to easily grasp the distance of an object by the unevenness of the pin.

[0073] As described above, the image information presentation device 100 according to the fourth embodiment changes the presentation mode of an object for each distance information based on the "object type information" in the input image and the acquired "distance information". Then, the "object position", "object size", etc., which are information detected by the detection unit 114, are presented. This also enables visually impaired people to grasp the "object position, object size", etc., taking into account the depth for each type of object in the input image, even for input images such as photographs with a large amount of information.

[0074] <<Fifth embodiment>> Next, the process in the fifth embodiment 5 will be described. In the fifth embodiment, instead of presenting objects for each "type of object" as in the first embodiment, the degree to which the objects are presented so as to be graspable by tactile sense is changed according to the "type of object."

[0075] For example, in a "thermal display," "object type A" is high temperature, "object type C" is low temperature, and "object type B" is an intermediate temperature. Or, conversely, "object type A" is low temperature and "object type C" is high temperature. In this way, the sense of temperature changes depending on the type of object, so visually impaired people can identify the type of object by their sense of temperature.

[0076] In addition, in the presentation by the "pin display," the pin displacement amount is set to the maximum for "object type A," the minimum for "object type C," and an intermediate displacement amount for "object type B." Or, conversely, the pin displacement amount is set to the minimum for "object type A" and the minimum for "object type C." In this way, the unevenness of the pins changes depending on the "type of object," so visually impaired people can grasp the type of object by the unevenness of the pins.

[0077] As described above, the image information presentation device 100 according to the fifth embodiment presents the "position of the object", "size of the object", etc. by changing the degree of presentation of the object for "each type of object" based on the type information of the object in the input image. This enables visually impaired people to grasp the "position of the object", "size of the object", etc. for "each type of object" in the input image even in a photo with a large amount of information.

[0078] <Summary of the Second to Fifth Embodiments> According to the explanation of the second to fifth embodiments, the detection unit 114 detects the "type of object" and the "area of ​​the object" based on the image data input by the image input unit 111, and the distance acquisition unit 115 acquires "distance information" based on the input image data. The presentation mode determination unit 117 determines the presentation mode of the "tactile presentation area" corresponding to the detected "area of ​​the object" based on the "type of object" detected by the detection unit 114 and the "distance information" acquired by the distance acquisition unit 115.

[0079] Furthermore, the tactile presentation unit 130 performs the following process using at least one of the information presentation means of "thermal display" and "pin display" in accordance with the presentation mode determined by the presentation mode determination unit 117. That is, the tactile presentation unit 130 presents multiple images of the object area for each "type of object" detected by the detection unit 114 in a manner that allows the image to be grasped by the sense of touch, based on the distance information acquired by the distance acquisition unit 115.

[0080] Furthermore, the tactile presentation unit 130 performs the following process using at least one of the information presentation means of "thermal display" and "pin display" in accordance with the presentation mode determined by the presentation mode determination unit 117. That is, based on the distance information acquired by the distance acquisition unit 115, the tactile presentation unit 130 presents multiple images of the area of ​​the object detected by the detection unit 114 for each "distance" of the object in a manner that allows the image to be grasped by the sense of touch.

[0081] In another aspect, the distance acquisition unit 115 acquires "distance information" based on image data input by the image input unit 111, and the presentation mode determination unit 117 determines the presentation mode of the tactile presentation area based on the "distance information" acquired by the distance acquisition unit 115.

[0082] The presentation mode determination unit 117 further includes a change unit that changes the degree to which at least one of the information presentation means of the "warm display" and the "pin display" is presented so as to be graspable by the sense of touch based on the "distance information" acquired by the distance acquisition unit 115. When either the "warm display" or the "pin display" is used, this change unit changes the degree to which the one of the displays is presented so as to be graspable by the sense of touch based on the "distance information" acquired by the distance acquisition unit 115.

[0083] Furthermore, when both displays are used in combination, the change unit changes the degree to which both displays are presented so as to be graspable by the tactile sense based on the "distance information" acquired by the distance acquisition unit 115. In other words, the change unit changes the amount of pin displacement of the pin display and changes the temperature of the Peltier element in the thermal display based on the "distance information" acquired by the distance acquisition unit 115.

[0084] <<Other embodiments>> In the above embodiment, an example has been shown in which an object detected by the detection unit 114 is presented as a tactile sensation. When the size of the detected object is small, it may become noise, so a process such as "masking" may be performed on objects smaller than a "certain threshold". However, this does not apply when the type of object to be preferentially detected (priority detection object) is smaller than a "certain threshold". In this way, when the "object area" detected by the detection unit 114 is smaller than a preset "certain threshold" and the object is not a "priority detection object", noise reduction, etc. can be achieved by providing a configuration with a masking unit that masks the object.

[0085] In the above-described embodiment, an example of tactile presentation using a pin display or a thermal display has been shown, but the display that presents the tactile sensation is not limited to this. For example, the pin display may present the "area of ​​the object" by providing a sense of unevenness, and the thermal display may present the "distance information" by providing a sense of warmth and cold. Alternatively, the pin display may present the "distance information" by providing a sense of unevenness, and the thermal display may present the "area of ​​the object" by providing a sense of warmth and cold.

[0086] Also, the distance information acquired by the distance acquisition unit 115 can be presented so that it can be grasped by the sense of warmth and cold using a "thermal display", and the area of ​​the object detected by the detection unit 114 can be presented so that it can be grasped by the sense of unevenness using a "pin display". Alternatively, conversely, the distance information acquired by the distance acquisition unit 115 can be presented so that it can be grasped by the sense of unevenness using a "pin display", and the area of ​​the object detected by the detection unit 114 can be presented so that it can be grasped by the sense of warmth and cold using a "thermal display".

[0087] <Application to virtual space> The present invention can also be applied to images of three-dimensional virtual spaces such as the metaverse. The virtual space of the metaverse is a three-dimensional virtual space of "360 degrees" in the horizontal circumferential direction that is grasped using "dedicated goggles", "dedicated glasses", etc. For this reason, the image input unit 111 inputs three-dimensional image data obtained through "dedicated goggles", "dedicated glasses", etc., for the image data of the metaverse. At this time, "360 degrees" is divided into a plurality of regions, and a pin display or a thermal display is associated with each divided region, and the information of "360 degrees" is presented so that it can be grasped by touch using a plurality of pin displays. At this time, the type of display associated with each divided region may be changed. For example, a pin display may be associated with 180 degrees in the forward direction, and a thermal display may be associated with 180 degrees in the backward direction.

[0088] The present invention may also be applied to a virtual space that does not require the use of "dedicated goggles", "dedicated glasses", or the like. Image data of the virtual space may be input to the image input unit 111, and the processing described in the above embodiment may be performed to present information. In this case, since "dedicated goggles", "dedicated glasses", or the like are not used, a realistic three-dimensional image is not presented, but the same presentation as the information presentation described above can be performed. Even in this case, the horizontal circumferential direction "360 degrees" may be divided into a plurality of regions, and a pin display or a thermal display may be associated with each divided region, so that "360 degrees" information can be presented in a tactile manner using a plurality of pin displays.

[0089] <Variation 1> The image information presentation device 100, excluding the tactile presentation unit 130 realized by the thermal display or pin display, can be realized by, for example, a single PC equipped with a CD drive and a card reader. Image data is recorded on the CD or card, and the PC reads this image data. The PC and the "pin display" are connected so that required information can be transmitted via short-distance wireless communication. In this case, for example, a communication method based on "Bluetooth (registered trademark)" can be adopted. Furthermore, the image information presentation device 100, excluding the tactile presentation unit 130, is equipped with a transmission unit.

[0090] The transmitting unit transmits "presentation mode information" to the "pin display." The CPU of the "pin display" controls the amount of displacement of the two-dimensionally arranged pins based on the received "presentation requirement information." The presentation mode determination unit 117 determines the amount of displacement of each pin based on information detected from image data such as a photograph, and transmits the determined amount of displacement of each pin to the pin display as "presentation mode information."

[0091] In response, the CPU of the "pin display" controls the amount of displacement of each pin based on the received "presentation mode information." This allows multiple visually impaired people to share information such as "object type, object area" for the same image data. Similar processing can also be performed on the "thermal display."

[0092] That is, an image information presentation device 100 is formed by connecting an image processing device (corresponding to the image processing unit 110) and a display (pin display, thermal display, etc.) so that required information can be wirelessly transmitted. The image processing device includes an image input unit 111 and a detection unit 114 that detects the type of object and the area of ​​the object based on image data input by the image input unit 111. The image processing device further includes a presentation mode determination unit 117 that determines the presentation mode of a "tactile presentation area" corresponding to the area of ​​the object detected by the detection unit 114 based on the type of object detected by the detection unit 114. The image processing device further includes a transmission unit that transmits the presentation mode determined by the presentation mode determination unit 117 to the display.

[0093] On the other hand, the display is configured such that members including pins and Peltier elements are arranged in a two-dimensional manner, and information is presented so as to be grasped tactilely by controlling and changing the physical quantities that change in the members, and includes a receiving unit that receives the presentation mode determined by the presentation mode determination unit 117. The display further includes a presentation unit that presents an area of ​​an object so as to be grasped tactilely, based on the presentation mode determined by the presentation mode determination unit 117 received by the receiving unit.

[0094] Therefore, the display side changes and controls the physical quantities of each member corresponding to the "tactile presentation area" based on the presentation mode received in response to the transmission of the presentation mode determined by the image processing device side, thereby making it possible to grasp the area of ​​the object detected by the detection unit 114 by touch. Here, "changing and controlling the physical quantities" means control to change the amount of pin displacement in the case of a "pin display," and control to change the temperature of the Peltier element in the case of a "thermal display." In other words, "changing and controlling the physical quantities of the members" means, even in other displays, changing various quantities that can be grasped by touch by making some kind of change in each two-dimensionally arranged member.

[0095] <Variation 2> The image information presentation device 100, except for the tactile presentation unit 130 realized by a thermal display or pin display, can also be realized by a "mobile terminal" such as a smartphone or tablet terminal. Information (presentation control information) processed in each of the above-mentioned embodiments for a photo taken by the "mobile terminal" is transmitted to the "pin display". The "presentation control information" is transmitted to the "pin display" using the "Bluetooth (registered trademark)" function of the mobile terminal. The CPU of the "pin display" controls the displacement amount of the pins arranged in a two-dimensional shape based on the received "presentation control information". For this purpose, for example, a dedicated program for enabling the processing in each embodiment to be executed can be downloaded from a specific site and installed in the "mobile terminal". In this way, multiple visually impaired people can share the "type of object, area of ​​object" and the like for image data taken by one visually impaired person on a mobile terminal. Similar processing can also be performed on the "thermal display".

[0096] <Additional Note> The disclosure of the present embodiment includes the following configurations, methods, systems, and programs. (Configuration 1) An image input unit for inputting image data; a detection unit that detects a type of object and a region of the object based on the image data input by the image input unit; An image processing device comprising: a presentation mode determination unit that determines a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection unit for each type of object detected by the detection unit. (Configuration 2) The image processing device of claim 1, further comprising a first presentation unit that presents multiple images of object areas for each type of object detected by the detection unit in a tactile manner using at least one of an information presentation means based on a sense of temperature and cold and an information presentation means based on a sense of unevenness in accordance with the presentation mode determined by the presentation mode determination unit. (Configuration 3) The presentation mode determination unit is The image processing device according to configuration 2 further comprises a change unit that changes the degree to which at least one of the information presentation means based on the sense of warmth and cold and the information presentation means based on the sense of unevenness is presented so as to be graspable by the sense of touch for each type of object detected by the detection unit. (Configuration 4) An image input unit for inputting image data; a distance acquisition unit that acquires distance information based on the image data input by the image input unit; a presentation mode determination unit that determines a presentation mode of a tactile sense presentation area for each distance based on the distance information acquired by the distance acquisition unit. (Configuration 5) The presentation mode determination unit is The image processing device according to configuration 4, further comprising a change unit that changes the degree to which at least one of the information presentation means based on the sense of warm and cold and the information presentation means based on the sense of unevenness is presented so as to be graspable by the sense of touch for each distance in accordance with the distance information acquired by the distance acquisition unit. (Configuration 6) A distance acquisition unit that acquires distance information based on the image data input by the image input unit, The image processing device described in configuration 1, characterized in that the presentation mode determination unit includes a presentation mode determination unit that determines a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection unit based on the type of object detected by the detection unit and the distance information acquired by the distance acquisition unit. (Configuration 7) The image processing device described in Configuration 6, further comprising a second presentation unit that uses at least one of an information presentation means based on the sense of warmth and cold and an information presentation means based on the sense of unevenness in accordance with the presentation mode determined by the presentation mode determination unit, and presents multiple images of the area of ​​an object for each type of object detected by the detection unit in a tactile manner based on the distance information acquired by the distance acquisition unit. (Configuration 8) The image processing device described in Configuration 6, further comprising a third presentation unit that uses at least one of an information presentation means based on a sense of warmth and cold and an information presentation means based on a sense of unevenness in accordance with the presentation mode determined by the presentation mode determination unit, and presents multiple images of the area of ​​the object detected by the detection unit for each distance to the object based on the distance information acquired by the distance acquisition unit in a manner that allows the image to be grasped by the sense of touch. (Configuration 9) The image processing device described in Configuration 6, further comprising a distance area presentation unit that presents the distance information acquired by the distance acquisition unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense, and presents the area of ​​the object detected by the detection unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense, or presents the distance information acquired by the distance acquisition unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense, and presents the area of ​​the object detected by the detection unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense. (Configuration 10) The image processing device according to any one of configurations 1, 2, 3, 7 and 8, further comprising an information presenting unit that presents detection information of an object detected by the detection unit by a braille display and / or audio. (Configuration 11) The image processing device according to any one of configurations 4 to 9, further comprising an information presenting unit that presents the distance information acquired by the distance acquisition unit by a Braille display and / or by voice. (Configuration 12) The image processing device according to any one of claims 1, 2, 3, 6, 7, 8 and 9, further comprising a masking unit that masks an object detected by the detection unit when the area of ​​the object detected by the detection unit is smaller than a predetermined threshold value and is not a priority detection object that is a predetermined priority detection target object. (Configuration 13) The image processing device according to any one of claims 1, 2, 3, 6, 7, 8 and 9, further comprising a switching unit that switches and presents the type of object detected by the detection unit in response to a user's operation of the operation unit. (Configuration 14) The image processing device according to any one of claims 2, 3, 5, 7, 8 and 9, characterized in that the means for presenting information based on the thermal sensation is a thermal display, and the means for presenting information based on the unevenness sensation is a pin display. (Configuration 15) The image processing device according to any one of configurations 1 to 9, further comprising information presenting means capable of presenting image information by a tactile sense based on the presentation mode determined by the presentation mode determination unit. (Configuration 16) A device that can be connected to an image information presentation device that has an information presentation means that can present information by touch, 10. The image processing device according to any one of configurations 1 to 9, further comprising a transmission unit that transmits the presentation mode determined by the presentation mode determination unit to the information presentation means. (Configuration 17) A device that can be connected to an image processing device, an information presentation means in which members including at least one of pins and Peltier elements are arranged in a two-dimensional form, and which is capable of presenting information through the sense of touch by controlling and changing a physical quantity that changes in the members; a receiving unit that receives an instruction of a presentation mode by the information presenting means, The information presentation means presents images for each type of object based on the presentation format received by the receiving unit. (Configuration 18) An image input unit for inputting image data; a detection unit that detects a type of object and a region of the object based on the image data input by the image input unit; a presentation mode determination unit that determines a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection unit based on a type of the object detected by the detection unit; an information presenting means in which members including at least one of pins and Peltier elements are arranged in a two-dimensional form, and which is capable of presenting information by tactile sensation by changing and controlling a physical quantity that changes in the members; The information presentation means presents images for each object type based on the presentation format determined by the presentation format determination unit. (Method 1) A method for controlling an image processing device, comprising the steps of: an image input step for inputting image data; a detection step of detecting a type of object and a region of the object based on the image data inputted by the image input step; A control method for an image processing device, comprising: a presentation mode determination step of determining a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection step for each type of object detected by the detection step. (Method 2) A method for controlling an image processing device, comprising the steps of: an image input step for inputting image data; a distance acquisition step of acquiring distance information based on the image data inputted by the image input step; A control method for an image processing device, comprising: a presentation mode determination step of determining a presentation mode of a tactile sense presentation area for each distance based on the distance information acquired in the distance acquisition step. (Program 1) A program for causing a computer to execute a control method for an image processing device, The control method includes: an image input step for inputting image data; a detection step of detecting a type of object and a region of the object based on the image data inputted by the image input step; A program comprising: a presentation mode determination step of determining a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected in the detection step for each type of object detected in the detection step. (Program 2) A program for causing a computer to execute a control method for an image processing device, The control method includes: an image input step for inputting image data; a distance acquisition step of acquiring distance information based on the image data inputted by the image input step; a presentation mode determination step of determining a presentation mode of a tactile sense presentation area for each distance based on the distance information acquired in the distance acquisition step.

[0097] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-mentioned embodiment to a system or device via a network or a recording medium, and having a computer processor of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Industrial Applicability]

[0098] This is an image information presentation device that presents information in a manner that can be grasped through the sense of touch, enabling visually impaired people to grasp the area of ​​an object based on the type of object and / or the distance of the object in image data such as a photograph. [Explanation of symbols]

[0099] 100 Image information presentation device 110 Image processing section 113 Pretreatment section 114 Detection unit 115 Distance acquisition part 116 Object Information Memory 117 Presentation mode determination unit 118 Presentation control memory 130 Tactile display unit 131 Detection information presentation unit 140 System control section 141 Operation section

Claims

1. an image input unit for inputting image data; a detection unit that detects a type of object and a region of the object based on the image data input by the image input unit; An image processing device comprising: a presentation mode determination unit that determines a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection unit for each type of object detected by the detection unit.

2. The image processing device according to claim 1, further comprising a first presentation unit that presents a plurality of images of the area of ​​an object for each type of object detected by the detection unit in a tactile manner using at least one of an information presentation means based on a sense of temperature and cold and an information presentation means based on a sense of unevenness in accordance with the presentation mode determined by the presentation mode determination unit.

3. The presentation mode determination unit is The image processing device according to claim 2, further comprising a change unit that changes the degree to which at least one of the information presentation means based on the sense of warmth and cold and the information presentation means based on the sense of unevenness is presented so as to be graspable by the sense of touch for each type of object detected by the detection unit.

4. an image input unit for inputting image data; a distance acquisition unit that acquires distance information based on the image data input by the image input unit; a presentation mode determination unit that determines a presentation mode of a tactile sense presentation area for each distance based on the distance information acquired by the distance acquisition unit.

5. The presentation mode determination unit is The image processing device according to claim 4, further comprising a change unit that changes the degree to which information is presented in a tactile manner for each distance, for at least one of the information presentation means based on the sense of warm and cold and the information presentation means based on the sense of unevenness, in accordance with the distance information acquired by the distance acquisition unit.

6. a distance acquisition unit that acquires distance information based on the image data input by the image input unit, The image processing device according to claim 1, characterized in that the presentation mode determination unit is provided with a presentation mode determination unit that determines a presentation mode of a tactile presentation area corresponding to an area of ​​an object detected by the detection unit based on the type of object detected by the detection unit and the distance information acquired by the distance acquisition unit.

7. The image processing device described in claim 6, further comprising a second presentation unit that presents multiple images of the area of ​​an object for each type of object detected by the detection unit in a tactile manner based on the distance information acquired by the distance acquisition unit, using at least one of an information presentation means based on a thermal / cold sensation and an information presentation means based on a sense of unevenness, in accordance with the presentation mode determined by the presentation mode determination unit.

8. The image processing device described in claim 6, further comprising a third presentation unit that uses at least one of an information presentation means based on a sense of warmth and cold and an information presentation means based on a sense of unevenness in accordance with the presentation mode determined by the presentation mode determination unit, and presents multiple images of the area of ​​the object detected by the detection unit for each distance to the object based on the distance information acquired by the distance acquisition unit in a manner that can be grasped by the sense of touch.

9. The image processing device according to claim 6, further comprising a distance area presentation unit that presents the distance information acquired by the distance acquisition unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense, and presents the area of ​​the object detected by the detection unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense, or that presents the distance information acquired by the distance acquisition unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense, and presents the area of ​​the object detected by the detection unit so that it can be grasped by the sense of hot and cold using an information presentation means based on hot and cold sense.

10. 9. The image processing device according to claim 1, further comprising an information presenting unit that presents detection information of an object detected by the detection unit by a braille display and / or audio.

11. 10. The image processing apparatus according to claim 4, further comprising an information presenting unit that presents the distance information acquired by the distance acquisition unit by a Braille display and / or by voice.

12. The image processing device according to any one of claims 1, 2, 3, 6, 7, 8 and 9, further comprising a masking unit that masks an object detected by the detection unit when the area of ​​the object detected by the detection unit is smaller than a predetermined threshold value and is not a priority detection object that is a predetermined priority detection target object.

13. 10. The image processing device according to claim 1, further comprising a switching unit that switches and presents the type of object detected by the detection unit in response to an operation of an operation unit by a user.

14. 10. The image processing apparatus according to claim 2, wherein the information presentation means based on the thermal sensation is a thermal display, and the information presentation means based on the unevenness sensation is a pin display.

15. 10. The image processing device according to claim 1, further comprising an information presenting unit capable of presenting image information by a tactile sense based on the presentation mode determined by the presentation mode determination unit.

16. The device is connectable to an image information presentation device having an information presentation means capable of presenting information by touch, 10. The image processing device according to claim 1, further comprising a transmission unit that transmits the presentation mode determined by the presentation mode determination unit to the information presentation means.

17. It is connectable to an image processing device, an information presentation means in which members including at least one of pins and Peltier elements are arranged in a two-dimensional form, and which is capable of presenting information through the sense of touch by controlling and changing a physical quantity that changes in the members; a receiving unit that receives an instruction of a presentation mode by the information presenting means, The information presentation means presents images for each type of object based on the presentation format received by the receiving unit.

18. an image input unit for inputting image data; a detection unit that detects a type of object and a region of the object based on the image data input by the image input unit; a presentation mode determination unit that determines a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection unit based on a type of the object detected by the detection unit; an information presenting means in which members including at least one of pins and Peltier elements are arranged in a two-dimensional form, and which is capable of presenting information by tactile sensation by changing and controlling a physical quantity that changes in the members; The information presentation means presents images for each object type based on the presentation format determined by the presentation format determination unit.

19. A control method for an image processing device, comprising: an image input step for inputting image data; a detection step of detecting a type of object and a region of the object based on the image data inputted by the image input step; A control method for an image processing device, comprising: a presentation mode determination step of determining a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected by the detection step for each type of object detected by the detection step.

20. A control method for an image processing device, comprising: an image input step for inputting image data; a distance acquisition step of acquiring distance information based on the image data inputted by the image input step; A control method for an image processing device, comprising: a presentation mode determination step of determining a presentation mode of a tactile sense presentation area for each distance based on the distance information acquired in the distance acquisition step.

21. A program for causing a computer to execute a control method for an image processing device, The control method includes: an image input step for inputting image data; a detection step of detecting a type of object and a region of the object based on the image data inputted by the image input step; A program comprising: a presentation mode determination step of determining a presentation mode of a tactile presentation area corresponding to an area of ​​the object detected in the detection step for each type of object detected in the detection step.

22. A program for causing a computer to execute a control method for an image processing device, The control method includes: an image input step for inputting image data; a distance acquisition step of acquiring distance information based on the image data inputted by the image input step; a presentation mode determination step of determining a presentation mode of a tactile sense presentation area for each distance based on the distance information acquired in the distance acquisition step.

Images