Tactile display device

Abstract

To provide a tactile display device that is usable for users such as visually-impaired persons and allows easy recognition of the content of an entire image.SOLUTION: A tactile display device 10 comprises: a tactile display 12 that displays a specific region image TRG being an image of a specific region TR within an entire image ZTG1; a data storage unit 14 that stores entire image data ZTG1(D); a support information output unit 16 that outputs support information to the user; a command input unit 18 for the user to input a request command; and a device control unit 20 that controls the operation of the tactile display 12 and the support information output unit 16. The command input unit 18 includes position designation means 18a for designating the position of the specific region TR. When a request command that inquires about the position of the specific region image TRG within the entire image ZTG1 is input, corresponding positional information (support information) is created and output from the support information output unit 16.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a tactile display device that enables visually impaired people to obtain image information. [Background technology]

[0002] As a device that makes visually perceived images tactile, there is a tactile display device equipped with a tactile display in which a large number of tactile pins protrude and sink to display an image. Since the number of tactile pins in a tactile display is limited, if the entire image is displayed on the tactile display at once, the resolution drops significantly, and in many cases the details of the image cannot be correctly recognized. Therefore, it is common to display only a part of the entire image (a specific area) on the tactile display, and change the display (move the position of the specific area) according to the user's operation, so that the entire image can be recognized.

[0003] Conventionally, there have been active studies on technologies for improving the usability of this type of tactile display device and technologies for supporting easier recognition of the contents of the overall image. For example, the tactile pin display device disclosed in Patent Document 1 has a function of generating movement information clarifying the positional relationship of the specific area before and after the change when the display of the tactile pin display changes (when the position of the specific area moves) and transmitting this information to the user through voice or the like. In addition, a function that allows the user to change the display magnification of the tactile pin display (the ratio of the size of the specific area to the overall image) and a function that displays only a specific shape specified by the user when multiple shapes overlap are added. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2001-242781 A Summary of the Invention [Problem to be solved by the invention]

[0005] In interviews with several visually impaired people, one of the most difficult things about using this type of tactile display device was losing track of which side of the overall image the area displayed on the tactile display was located, and many of them said that it would be helpful if some measure could be taken to prevent this.In addition, when the overall image is a flow chart or graph, it is believed that visually impaired people would find it much easier to recognize the content of the overall image if they could obtain support information that explains the content of each image element contained in the overall image (for example, the meaning of shapes and line segments). However, the tactile pin display device of Patent Document 1 does not provide sufficient support in these respects, and improvements are required.

[0006] The present invention has been made in view of the above-mentioned background art, and has an object to provide a tactile display device that is easy for users such as visually impaired people to use and allows them to easily recognize the contents of the entire image. [Means for solving the problem]

[0007] The present invention provides a tactile display device that makes it possible to tactilely display an entire image showing a figure, a symbol, or the like, comprising: a tactile display that displays a specific area image, which is an image of a specific area within the overall image, by projecting and recessing a plurality of tactile pins; a data storage unit that stores overall image data, which is image data of the overall image; a support information output unit that outputs support information for supporting a user in grasping the overall image, to a user; a command input unit for a user to input a request command; and a device control unit that controls operations of the tactile display and the support information output unit based on a default set condition and the request command, the command input unit includes a position designation means for designating a position of the specific area in the entire image, When a request command specifying the position of the specific area is input through the position designation means, the device control unit creates image data of the specific area corresponding to the request command based on the overall image data and determines the protruding state of the tactile pins, and when a request command requesting information indicating where in the overall image the specific area image displayed on the tactile display is located, creates position information corresponding to the request command and outputs it from the support information output unit as the support information.This is a tactile display device.

[0008] In this case, the device control unit can be configured to cause the support information output unit to output the created position information in the form of an audio signal or in the form of Braille to be displayed on the Braille display.

[0009] The present invention also provides a tactile display device that makes it possible to tactilely display an entire image showing figures, characters, etc., comprising: a tactile display that displays a specific area image, which is an image of a specific area in the entire image, by projecting and recessing a plurality of tactile pins; a data storage unit that stores overall image data, which is image data of the entire image; a command input unit for a user to input a request command; and a device control unit that controls an operation of the tactile display based on a default set condition and the request command, the command input unit includes a touch operation surface as a position designation means for designating a position of the specific area in the entire image, the touch operation surface having an outer shape corresponding to an outer shape of the entire image, and capable of designating which position in the entire image is to be the specific area based on a relative positional relationship between the outer shape of the touch operation surface and a position touched or tapped by a user; The device control unit is a tactile display device that, when the request command specifying the position of the specific area is input through the touch operation surface, creates image data of the specific area corresponding to the request command based on the overall image data and determines the protruding state of the tactile pins.

[0010] In this case, it is preferable that the device further comprises an assistance information output unit that outputs assistance information to a user to assist in comprehension of the overall image, and the device control unit controls the operation of the assistance information output unit.

[0011] It is preferable that the position designation means of the command input unit is capable of continuously changing the designation of the position of the specific area, and that the device control unit is configured to change the protruding and retracting state of the tactile pin at a change rate corresponding to the request command when the request command specifying the change rate of the specific area image displayed on the tactile display is input.

[0012] Also, a layer divider is provided which recognizes a plurality of image elements included in the overall image by analyzing the overall image data, divides the overall image into layer images representing the respective image elements, creates layer image data which is image data of the layer images, and stores the layer image data in the data storage unit, It is preferable that when the request command specifying the image element to be displayed on the tactile display is input through the position designation means, the device control unit creates image data of the specific area corresponding to the request command based on the layer image data, and determines the protruding and retracting state of the tactile pin.

[0013] It is preferable that the device control unit is configured to create, when the position of the specific area specified in the request command is a position where all or a part of the specific area protrudes outside the whole image, attention-calling information indicating that fact and output the attention-calling information from the support information output unit as the support information. In this case, the device control unit can be configured to output the created attention-calling information from the support information output unit in the form of a sound signal, a voice signal, or a vibration signal.

[0014] It is preferable that the data storage unit stores text data related to the overall image or image elements contained in the overall image, and the device control unit is configured to extract the text data corresponding to the request command from the data storage unit and output it from the support information output unit as the support information.

[0015] For example, the system may include a text data generator that analyzes the overall image data and generates text data related to the overall image or image elements included in the overall image, and the text data generated by the text data generator is included in the text data stored in the data storage unit. In this case, it is preferable that the text data generator is generated by machine learning using a plurality of image data labeled with text data related to the image data as learning data.

[0016] The image processing device may also be configured to include a character information extractor that analyzes the overall image data and extracts characters or character strings represented in the overall image as text data, and the text data extracted by the character information extractor may be included in the text data stored in the data storage unit.

[0017] Furthermore, the device control unit can be configured to cause the support information output unit to output the extracted text data in the form of an audio signal or in the form of Braille to be displayed on the Braille display. Effect of the Invention

[0018] The tactile display device of the present invention is configured to easily and accurately know which side of the entire image the specific area image displayed on the tactile display is located on, so that users such as visually impaired people can appropriately select the position of the specific area they want to display next, and can efficiently recognize each part of the entire image. In addition, while general-purpose Braille displays (types that display Braille text in one line) are inexpensive and easy to obtain, they have a very small number of tactile pins, so they have traditionally been considered unsuitable for displaying images. However, by applying the technology of the present invention, they have the advantage of being fully usable as tactile displays for displaying images.

[0019] Furthermore, by providing the user with support information, such as information indicating the meaning of each image element in the overall image and information on character strings displayed in the overall image, the content of the overall image can be recognized much more easily. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1A is a system configuration diagram showing a first embodiment of a tactile display device of the present invention, and FIG. 1B is a schematic diagram showing the relationship between an overall image and the display on the tactile display. [Diagram 2] FIG. 2A is a system configuration diagram showing a modified example of the tactile display device of the first embodiment, and FIG. 2B is a schematic diagram showing the relationship between the touch operation surface, the overall image, and the display of the tactile display. [Diagram 3] FIG. 11 is a system configuration diagram showing a second embodiment of a tactile display device of the present invention. [Figure 4] FIG. 13 is a diagram showing an example of an entire image [flowchart]. [Diagram 5] 5 is a diagram showing an example of an operation performed by a layer image divider when the entire image data of the entire image (flowchart) of FIG. 4 is input. [Figure 6] 5 is a diagram showing an example of the operations performed by a text data generator and a character information extractor when the entire image data of the entire image (flowchart) of FIG. 4 is input. [Figure 7]13 is a schematic diagram showing the relationship between an entire image and the display of a tactile display in a tactile display device of a second embodiment. FIG. [Figure 8] 13 is a diagram showing an example of an operation performed by the layer divider when the overall image data of another overall image [line graph] is input. [Figure 9] 13 is a diagram showing an example of the operations performed by the text data generator and the character information extractor when the overall image data of another overall image [line graph] is input. [Figure 10] 13 is a diagram showing an example of an operation performed by the layer image divider when the whole image data of another whole image [picture of a T-shirt] is input. [Figure 11] 13 is a diagram showing an example of the operations performed by the text data generator and the character information extractor when the whole image data of another whole image [picture of a T-shirt] is input. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] <<Tactile display device of the first embodiment and its modified examples>> A first embodiment of a tactile display device of the present invention and its modified examples will be described below with reference to FIGS. <Tactile display device 10 according to the first embodiment> The tactile display device 10 of the first embodiment is a device that makes it possible to tactilely display an entire image ZTG that represents, for example, a simple illustration of a picture, etc. As shown in Fig. 1(a), the tactile display device 10 includes a tactile display 12, a data storage unit 14, a support information output unit 16, a command input unit 18, and a device control unit 20, and the data storage unit 14 and the device control unit 20 are provided inside a computer.

[0022] As shown in Fig. 1(b), the tactile display 12 displays a specific area image TRG, which is an image of a specific area TR in the whole image ZTG, by projecting and sinking multiple tactile pins. Here, a general-purpose Braille display (displaying Braille text in one line) is used as the tactile display 12.

[0023] The data storage unit 14 is a block in which the entire image data ZTG(D) which is the image data of the entire image ZTG is stored.

[0024] The support information output unit 16 is a device that outputs support information to the user to support comprehension of the whole image ZTG, and here is composed of a speaker 16a that outputs the support information in the form of an audio signal, and a vibration device 16b that outputs the support information in the form of a vibration signal. It is also possible to output the support information in the form of Braille, in which case the tactile display 12 is also used as the support information output unit 16.

[0025] The command input unit 18 is a device used when the user inputs a request command, and can be configured, for example, with a personal computer keyboard, keys on the tactile display 12, a game console controller, a voice recognition device that enables input by the user's voice, etc. The command input unit 18 is characterized by having a position designation means 18a (arrow keys on a personal computer keyboard, a joystick on a game console controller, etc.) for designating the position of a specific region TR in the entire image ZTG, and by operating the position designation means 18a, the designation of the position of the specific region TR can be continuously changed.

[0026] The device control unit 20 is a block that controls the operation of the tactile display 12 and the support information output unit 16 based on default settings and requests from the user.

[0027] Next, the operation of the tactile display device 10 will be described with reference to Fig. 1(b). Here, the whole image ZTG will be an illustration of Mt. Fuji (whole image ZTG1), and the image data of the whole image ZTG1 will be referred to as whole image data ZTG1(D).

[0028] When the user operates the position designation means 18a of the command input unit 18 to input a request command designating the position of a specific area TR within the overall image ZTG1, the device control unit 20 creates image data of the specific area TR corresponding to this request command based on the overall image data ZTG1(D) stored in the data storage unit 14, determines the protruding and retracting state of the tactile pins, and sends a command to the tactile display 12 to display the specific area image TRG on the tactile display 12.

[0029] Then, when a request command to "explain the current display position" is input from the user, the device control unit 20 creates position information (support information) corresponding to this request command and outputs it from the support information output unit 16. For example, a voice saying "The current position is the lower right area of ​​the entire image" is output from the speaker 16a. Alternatively, the display of the specific area image TRG by the tactile display 12 may be temporarily interrupted, and a sentence saying "The current position is the lower right area of ​​the entire image" may be displayed in Braille on the tactile display 12.

[0030] The content of the position information can be freely set by the system administrator. The example above, "The current display position is the lower right area," is an example of position information in which the entire image ZTG1 is divided into nine areas in a grid pattern and each area is labeled with a label such as "top left / top / top right / left / middle / ...." In addition to this, for example, a method of specifying the relative position of the specific area TR with respect to the center of the entire image and making the content "The current display position is a position slightly shifted from the center toward the 4 o'clock direction" or a method of calculating the coordinates of the center point of the specific area TR and making the content "The current display position is at X=0.75, Y=0.20" is also conceivable.

[0031] In addition, the device control unit 20 has a function of creating attention-calling information (support information) indicating that the position of the specific region TR specified by the request command is a position where all or a part of the specific region TR protrudes outside the entire image ZTG1, and outputting the information from the support information output unit 16. For example, the device control unit 20 may output a voice message saying "The specified position protrudes to the lower right of the entire image" from the speaker 16a, or may output a warning sound such as "beep, beep, beep...". It is preferable that the volume of the warning sound "beep, beep, beep..." is increased or the interval between sounds is shortened as the amount of protrusion of the specific region TR increases, thereby enabling the user to intuitively recognize in which direction the specified position of the specific region TR should be corrected.

[0032] Also, attention-calling information (assistance information) may be output as a vibration signal from the vibration device 16b. In this case, it is preferable to make the vibration stronger as the amount of protrusion of the specific region TR increases, so that the user can intuitively recognize in which direction the position of the specific region TR should be corrected.

[0033] In this way, since the position information and attention calling information are output from the support information output unit 16, the user can easily recognize which side of the entire image ZTG1 the current display position is on.

[0034] In addition, the tactile display device 12 has a function that allows the user to freely set the speed of change when changing the display of the specific area image TRG. Specifically, when the user inputs a request command specifying the speed of change of the specific area image TRG, the device control unit 20 creates image data of the specific area TR so that the protruding and retracting state of the tactile pins changes at a speed of change corresponding to the request command, and displays the specific area image TRG on the tactile display 12. This allows the user to comfortably change (scroll) the specific area image TRG at a speed of their choice.

[0035] As described above, the tactile display device 10 is configured to easily and accurately recognize which side of the entire image ZTG1 the specific region image TRG displayed on the tactile display 12 is located on, so that a user such as a visually impaired person can appropriately select the position of the specific region TR that he or she wants to display next, and can efficiently recognize each part of the entire image ZTG1. In addition, while general-purpose Braille displays (types that display Braille text in one line) are inexpensive and easy to obtain, they have a very small number of tactile pins, and therefore have traditionally been considered unsuitable for displaying images. However, the tactile display device 10 has the advantage that they can be fully used as the tactile display 12.

[0036] <Modified Tactile Display Device 10x> A tactile display device 10x, which is a modified example of the above-mentioned tactile display device 10, will be described with reference to Fig. 2. In the tactile display device 10x, the command input unit 18 of the tactile display device 10 is replaced with a command input unit 18x, and the support information output unit 16 is deleted.

[0037] The command input unit 18x ​​is a device used when a user inputs a request command, similar to the command input unit 18. A characteristic feature of the command input unit 18x ​​is that it is provided with a touch operation surface 18b as a position designation means for designating the position of a specific region TR in the entire image ZTG1. The touch operation surface 18b is, for example, a touch panel of a smartphone or tablet terminal, and has an outer shape (similar outer shape or similar outer shape) corresponding to the outer shape of the entire image ZTG1, and can designate which position in the entire image ZTG1 is to be the specific region TR according to the relative positional relationship between the outer shape of the touch operation surface 18b and the position touched or tapped by the user.

[0038] 2(b), the user inputs a request command specifying the position of a specific region TR in the overall image ZTG1 by tapping or touching a specific position on the touch operation surface 18b. Then, the device control unit 20, similar to the case of the tactile display device 10, creates image data of the specific region TR corresponding to this request command based on the overall image data ZTG1(D) stored in the data storage unit 14, determines the protruding and retracting states of the tactile pins, and sends a command to the tactile display 12 to cause the tactile display 12 to display the specific region image TRG1.

[0039] In this case, the user specifies the position of the specific area TR after recognizing the positional relationship between the outline of the touch operation surface 18b and the position to tap, etc. In other words, since the position of the specific area TR is specified after recognizing which side of the entire image ZTG it is located on, there is little need to provide the user with the above-mentioned position information (support information) and attention-calling information (support information). Therefore, the support information output unit 16 is omitted from the tactile display device 10x.

[0040] In this way, although the tactile display device 10x does not include a support information output unit, it can obtain substantially the same effects as the tactile display device 10. Of course, it is also possible to provide a support information output unit 16 and add functions similar to those of the tactile display device 10.

[0041] <<Tactile display device of the second embodiment and its modified example>> A second embodiment of the tactile display device of the present invention and its modified example will be described below with reference to Figures 3 to 11. Here, the same components as those in the above-mentioned tactile display device 10 are given the same reference numerals and the description thereof will be omitted. <Tactile display device 22 according to the second embodiment> A tactile display device 22 of this embodiment is a more sophisticated and multifunctional version of the tactile display device 10 described above.

[0042] 3, the tactile display device 22 comprises the above-mentioned tactile display device 12, support information output unit 16, and command input means 18, and as new components, a data storage unit 24 and a device control unit 26 are provided instead of the data storage unit 14 and device control unit 20. In addition, a layer divider 28, a text data generator 30, and a character information extractor 32 are newly added. All of the new components are provided inside the computer.

[0043] First, an overview of the new configuration will be described. The data storage unit 24 is a block that stores overall image data ZTG(D), which is image data of the overall image ZTG, as in the data storage unit 14 described above, and further stores layer image data LYG(D) and text data, which will be described later.

[0044] The device control unit 26, like the above-mentioned device control unit 20, is a block that controls the operation of the tactile display 12 and the support information output unit 16 based on default settings and user requests. The device control unit 26 executes a significantly larger number of processes, and is therefore distinguished from the device control unit 20 by using different reference numerals.

[0045] The layer divider 28 is a block that recognizes a plurality of image elements contained in the overall image ZTG by analyzing the overall image data ZGD(D), divides the overall image ZTG into layer images LYG1, LYG2, ... representing each image element, and creates layer image data LYG1(D), LYG2(D), ... which is the image data of the layer images LYG1, LYG2, ..., and stores this in the data storage unit 24. The pixel elements will be explained later.

[0046] The text data generator 30 is a block that analyzes the overall image data ZTG(D), generates text data related to the overall image ZTG or image elements included in the overall image ZTG, and stores the generated text data in the data storage unit 24. For example, it is preferable to use a trained model created by machine learning using a plurality of image data labeled with text data related to the image data as training data, which is any image data. Alternatively, a generation AI may be used.

[0047] The character information extractor 32 is a block that analyzes the overall image data ZTG(D), extracts characters or character strings shown in the overall image ZTG, converts them into text data, and stores them in the data storage unit 24. For example, OCR (optical character recognition) technology can be used.

[0048] In addition to the text data generated by the text data generator 30 and the text data extracted by the character information extractor 32, the data storage unit 24 may also store text data manually input by a system administrator.

[0049] Next, an example of the operations of the layer divider 28, the text data generator 30, and the character information extractor 32 will be described with reference to Fig. 4 to Fig. 6. Here, the entire image ZTG will be described as the flowchart (entire image ZTG2) shown in Fig. 4, and the image data of the entire image ZTG2 will be described as entire image data ZTG2(D).

[0050] 5 shows a specific example of the operation of the layer divider 28. The layer divider 28 analyzes the overall image data ZGD2(D) and recognizes a plurality of image elements contained in the overall image ZTG2. Here, since the overall image ZTG2 is an image of a flowchart, the image elements may be represented as an "ellipse (terminal)" symbol, a "rectangle (processing)" symbol, a "diamond (branch)" symbol, an "upper trapezoid (repetition start)" symbol, a "lower trapezoid (repetition end)" symbol, an arrow symbol, or the like.

[0051] Then, the entire image ZTG2 is divided into layer images LYG1, LYG2, ... which represent each image element, and layer image data LYG1(D), LYG2(D), ... which is the image data of the layer images LYG1, LYG2, ... is created and stored in the data storage unit 24. Note that the system administrator can freely set what units the image elements will be divided into.

[0052] FIG. 6 shows a specific example of the operation of the text data generator 30 and the character information extractor 32. The text data generator 30 analyzes the entire image data ZTG2(D) and generates text data related to image elements for each layer image LYG1, LYG2, .... For example, for an "ellipse" symbol that is an image element of the layer image LYG1, text data "terminal" is generated, which is the meaning of this symbol. Also, for a "diamond" symbol that is an image element of the layer image LYG9, text data "branch" is generated, which is the meaning of this symbol. Then, each generated text data is linked to the corresponding layer image data LYG1(D), LYG2(D), ..., and the position coordinates (X1, Y1) and (X2, X2) of each image element, and stored in the data storage unit 24 with classification numbers 1, 2, .... Further, the whole image data ZTG2(D) is comprehensively analyzed, text data "flowchart" indicating that the whole image ZTG2 is an image of a flow chart is generated, and the classification number 22 is assigned to it and stored in the data storage unit 24.

[0053] The character information extractor 32 analyzes the whole image data ZTG2(D) and extracts characters or character strings as text data for each layer image LYG1, LYG2, .... For example, as shown in FIG. 6, for an "oval" symbol that is an image element of the layer image LYG1, text data of the character string "START" written within this symbol is extracted. Also, for a "diamond" symbol that is an image element of the layer image LYG9, text data of the character string "INPUT VALUE EVALUATION" written within this symbol is extracted. Then, each of the extracted text data is linked to the corresponding layer image data LYG1(D), LYG2(D), ... and the position coordinates (X1, Y1) and (X2, X2) of each image element, and stored in the data storage unit 24 with classification numbers 1, 2, .... In addition, the overall image data ZTG2(D) is comprehensively analyzed, and text data, the character string "Number Guessing Game" written in the upper right area of ​​the overall image ZTG2, is extracted, and the text data is linked to its position coordinates (X1, Y1) and (X2, X2), and is stored in the data storage unit 24 with the classification number 21 assigned.

[0054] The text data "Title" of classification number 21 is text data indicating that the character string "Guess the Number Game" is the title of whole image ZTG2, and is generated by text data generator 30. However, to automatically generate the "Titles" and "Flowcharts" of classification numbers 21 and 22, a relatively sophisticated text data generator 30 is required, so the system administrator may manually input them into data storage unit 24.

[0055] Next, the operation of the tactile display device 22 will be described with reference to Fig. 7. The tactile display device 22 has similar functions to the above-mentioned tactile display device 10, but the description of the similar functions will be omitted here and only new functions will be described.

[0056] For example, when the user operates the command input unit 18 to input a request command such as "Explain what image the overall image ZTG2 is (explain classification number 22)," the device control unit 26 browses the data storage unit 24, extracts the text data "flowchart" for classification number 22 corresponding to this request command, and outputs a voice saying "The overall image ZTG2 is a flowchart" from the speaker 16a.

[0057] In addition, when the user operates the command input unit 18 to input a request command such as "Explain classification number 9," the device control unit 26 extracts the text data "branch" and "input value evaluation" for classification number 9 that corresponds to this request command, and outputs a voice message stating "Classification number 9 is the branch symbol, and the content is input evaluation" from the speaker 16a.

[0058] In addition, when the user operates the command input unit 18 to input a request command such as "Explain classification number 21," the device control unit 26 accesses the data storage unit 24, extracts the text data "Title" and "Number Guessing Game" of classification number 21 corresponding to this request command, and outputs a voice saying "Classification number 21 is the title and the content is a number guessing game" from the speaker 16a.

[0059] The support information may be output in the form of Braille rather than as an audio signal, and the display of the specific area image TRG by the tactile display 12 may be temporarily interrupted and a Braille sentence such as "The overall image ZTG2 is a flowchart" may be displayed on the tactile display 12.

[0060] In addition, the user can operate the command input unit 18 to input a request command specifying the layer image LYG to be displayed on the tactile display 12. In other words, the user can select only one of all the layer images LYG to display, or select only multiple images to display simultaneously. This makes it possible to accurately recognize each image element by switching the designation of the layer image LYG, even when multiple image elements are crowded together or slightly overlapping.

[0061] As described above, the tactile display device 22 can achieve the same effects as the above-mentioned tactile display device 10, and further provides the user with support information such as information indicating the meaning of each image element in the overall image ZTG2 and information on character strings displayed in the overall image ZTG2, making it much easier to recognize the contents of the overall image ZTG2. <Other examples of operations for the whole image ZTG3> Here, we will explain an example of the operation of the layer divider 28, text data generator 30, and character information extractor 32 when the overall image ZTG3 [line graph] shown in the upper part of Figure 8 is displayed in a tactile manner on the tactile display device 22.

[0062] 8 shows a specific example of the operation of the layer divider 28. The layer divider 28 analyzes the overall image data ZGD3(D) and recognizes a plurality of image elements contained in the overall image ZTG3. Here, since the overall image ZTG3 is an image of a line graph, the image elements may be a "horizontal line (horizontal axis)", a "vertical line (vertical axis)", a "solid broken line (series α)", a "dashed broken line (series β)", etc.

[0063] Then, the entire image ZTG3 is divided into layer images LYG1, LYG2, ... which represent each image element, and layer image data LYG1(D), LYG2(D), ... which is the image data of the layer images LYG1, LYG2, ... is created and stored in the data storage unit 24. Note that the system administrator can freely set what units the image elements will be divided into.

[0064] FIG. 9 shows a specific example of the operation of the text data generator 30 and the character information extractor 32. The text data generator 30 analyzes the entire image data ZTG3(D) and generates text data related to image elements for each layer image LYG1, LYG2, .... For example, for a "horizontal line" which is an image element of the layer image LYG1, text data "horizontal axis" which is the meaning of this horizontal line is generated. Also, for a "solid broken line" which is an image element of the layer image LYG3, text data "series α" which is the meaning of this broken line is generated. Then, each generated text data is linked to the corresponding layer image data LYG1(D), LYG2(D), ... and the position coordinates (X1, Y1) and (X2, X2) of each image element, and stored in the data storage unit 24 with classification numbers 1, 2, .... Moreover, the overall image data ZTG3(D) is comprehensively analyzed, text data "Line graph" is generated indicating that the overall image ZTG2 is an image of a line graph, and the classification number 22 is assigned to it and stored in the data storage unit 24.

[0065] The character information extractor 32 analyzes the entire image data ZTG3(D) and extracts characters or character strings as text data for each layer image LYG1, LYG2,... For example, as shown in FIG. 9, for a "horizontal line" which is an image element of the layer image LYG1, text data of the character string "2019, 2020,..." written near the horizontal line is extracted. Also, for a "solid broken line" which is an image element of the layer image LYG3, text data of "Area A" which describes the legend of the solid broken line is extracted. Then, each extracted text data is linked to the corresponding layer image data LYG1(D), LYG2(D),... and the position coordinates (X1, Y1) and (X2, X2) of each image element, and stored in the data storage unit 24 with classification numbers 1, 2,.... In addition, the overall image data ZTG3(D) is comprehensively analyzed, and text data, the character string "Number of traffic accidents by year" written in the upper area of ​​the overall image ZTG3, is extracted. The text data is linked to its position coordinates (X1, Y1) and (X2, X2), and is stored in the data storage unit 24 with a classification number 21 assigned.

[0066] The text data "Title" of classification number 21 is text data indicating that the character string "Number of Traffic Accidents by Year" is the title of the whole image ZTG3, and is generated by the text data generator 30. Note that when the "Title" and "Line Graph" of classification numbers 21 and 22 are automatically generated, or when the burden on the text data generator 30 is large, the system administrator may manually input the data into the data storage unit 24.

[0067] The operation of the tactile display device 22 for the whole image ZTG3 is similar to that for the whole image ZTG2, and a description thereof will be omitted here. <Other concrete examples of operations for the whole image ZTG4> Next, an example of the operation of layer divider 28, text data generator 30, and character information extractor 32 when the entire image ZTG4 [picture of a T-shirt] shown in the upper part of Figure 10 is displayed in a tactile manner on tactile display device 22 will be described.

[0068] 10 shows a specific example of the operation of the layer divider 28. The layer divider 28 analyzes the overall image data ZGD4(D) and recognizes multiple image elements contained in the overall image ZTG4. In this case, since the overall image ZTG4 is a picture of two T-shirts, it is advisable to divide the image elements into a "left T-shirt" and a "right T-shirt".

[0069] Then, the entire image ZTG4 is divided into layer images LYG1 and LYG2 that represent each image element, and layer image data LYG1(D) and LYG2(D), which are the image data of the layer images LYG1 and LYG2, are created and stored in the data storage unit 24. Note that the system administrator can freely set how the image elements are divided, and here, the characters "Genki" and "LOVE" written large on the chest of the T-shirt are considered to be part of the T-shirt design (inseparable from the main fabric) and are treated as part of the image elements.

[0070] FIG. 11 shows a specific example of the operation of the text data generator 30 and the character information extractor 32. The text data generator 30 analyzes the entire image data ZTG4(D) and generates text data related to image elements for each of the layer images LYG1 and LYG2. For example, for the image element "T-shirt on the left" of the layer image LYG1, text data "yellow T-shirt with "cheer" written on it" indicating the characteristics of the T-shirt is generated. Also, for the image element "T-shirt on the right" of the layer image LYG2, text data "red T-shirt with "LOVE" written on it" indicating the characteristics of the T-shirt is generated. Then, each of the generated text data is linked to the corresponding layer image data LYG1(D), LYG2(D) and the position coordinates (X1, Y1) and (X2, X2) of each image element, and stored in the data storage unit 24 with classification numbers 1 and 2.

[0071] The character information extractor 32 analyzes the whole image data ZTG4(D) and extracts characters or character strings as text data for each of the layer images LYG1 and LYG2. For example, as shown in FIG. 11, text data of the character string "Genki" written at the chest position is extracted for the image element "T-shirt on the left side" of the layer image LYG1. Also, text data of "LOVE" written at the chest position is extracted for the image element "T-shirt on the right side" of the layer image LYG2. Then, each of the extracted text data is linked to the corresponding layer image data LYG1(D), LYG2(D),... and the position coordinates (X1, Y1) and (X2, X2) of each image element, and stored in the data storage unit 24 with classification numbers 1 and 2.

[0072] The text data relating to the writing of "genki" or "LOVE" may be generated only by the text data generator 30, or may be extracted only by the character information extractor 32. Alternatively, if the size of "genki" or "LOVE" is large enough that it can be felt even with the resolution of the tactile display 12, it may be possible to neither generate nor extract it, and this can be freely decided by the system administrator. Also, the text data may be manually input into the data storage unit 24 by the system administrator.

[0073] The operation of the tactile display device 22 for the whole image ZTG4 is similar to the operations for the whole images ZTG2 and ZTG3, and a description thereof will be omitted here.

[0074] <<Other embodiments and modifications>> The tactile display device of the present invention is not limited to the above-mentioned embodiment and modified examples. For example, the above-mentioned tactile display device 22 is a highly functional and multifunctional configuration equipped with a layer divider 28, a text data generator 30, and a character information extractor 32, but these blocks can be selectively deleted depending on the type and content of the entire image. For example, if the entire image ZTG1 [illustration of Mt. Fuji] shown in Figures 1 and 2 is a simple picture that many people can easily imagine, it is considered that almost no support information other than the position information indicating the position of the specific display area TR is necessary, so that the layer divider 28, the text data generator 30, and the character information extractor 32 can all be deleted, and the device can be configured very simply.

[0075] The above-mentioned tactile display 12 is a general-purpose Braille display (a type that displays Braille text on one line and has tactile pins arranged in 4 rows and n columns), but the present invention can also use a display with a larger number of tactile pins (for example, a type with 5 or more rows of tactile pins) and achieve the same effect.

[0076] In addition, the tactile display device of the present invention is not particularly limited in the type or content of the whole image to be displayed. For example, it may be an image with a simple pattern such as the whole body image ZTG1 [illustration of Mt. Fuji], or an image in which figures and text are mixed such as the whole images ZTG2 and ZTG3 [flowchart, line graph], or an intermediate image such as the whole image ZTG4 [picture of a T-shirt], and it is possible to provide a tactile display device that is easy for users to use. [Explanation of symbols]

[0077] 10,10x,22 Tactile display device 12 Tactile Display 14,24 Data storage section 16 Support information output section 16a Speaker 16b Vibration device 18,18x Command input section 18a Position designation means 18b Touch-sensitive surface as a means of positioning 20,26 Device control unit 28 Layer Divider 30 Text Data Generator 32 Character information extractor TR specific area TRG specific area image ZTG, ZTG1~ZTG4 Full image ZTG(D), ZTG1(D)~ZTG4(D) Full image data LYG,LYG1,LTG2,...Layer images LYG(D),LYG1(D),LTG2(D),...Layer image data

Claims

1. A tactile display device that makes it possible to tactilely display an entire image showing figures, symbols, etc. a tactile display that displays a specific area image, which is an image of a specific area within the overall image, by projecting and recessing a plurality of tactile pins; a data storage unit that stores overall image data, which is image data of the overall image; a support information output unit that outputs support information for supporting a user in grasping the overall image, to a user; a command input unit for a user to input a request command; and a device control unit that controls operations of the tactile display and the support information output unit based on a default set condition and the request command, the command input unit includes a position designation means for designating a position of the specific area in the entire image, The device control unit, when the request command specifying the position of the specific area is input through the position designation means, creates image data of the specific area corresponding to the request command based on the overall image data and determines the protruding state of the tactile pins, and when the request command requesting information indicating where in the overall image the specific area image displayed on the tactile display is located, creates position information corresponding to the request command and outputs it from the support information output unit as the support information, characterized in that

2. 2. The tactile display device according to claim 1, wherein the device control unit causes the support information output unit to output the created position information in the form of an audio signal or in the form of Braille displayed on the Braille display.

3. A tactile display device that makes it possible to tactilely display an entire image showing figures, characters, etc. a tactile display that displays a specific area image, which is an image of a specific area in the entire image, by projecting and recessing a plurality of tactile pins; a data storage unit that stores overall image data, which is image data of the entire image; a command input unit for a user to input a request command; and a device control unit that controls an operation of the tactile display based on a default set condition and the request command, the command input unit includes a touch operation surface as a position designation means for designating a position of the specific area in the entire image, the touch operation surface having an outer shape corresponding to an outer shape of the entire image, and capable of designating which position in the entire image is to be the specific area based on a relative positional relationship between the outer shape of the touch operation surface and a position touched or tapped by a user; The device control unit, when the request command specifying the position of the specific area is input through the touch operation surface, creates image data of the specific area corresponding to the request command based on the overall image data, and determines the protruding and retracting state of the tactile pins, characterized by the above.

4. The tactile display device according to claim 3 , further comprising an assistance information output unit that outputs assistance information to a user to assist in comprehension of the overall image, the device control unit controlling an operation of the assistance information output unit.

5. the position designation means of the command input unit is capable of continuously changing the designation of the position of the specific area, A tactile display device as described in claim 1 or 3, wherein when a request command specifying the rate of change of the specific area image displayed on the tactile display is input, the device control unit changes the protruding and retracting state of the tactile pins at a rate of change corresponding to the request command.

6. a layer divider that recognizes a plurality of image elements included in the overall image by analyzing the overall image data, divides the overall image into layer images representing the respective image elements, creates layer image data which is image data of the layer images, and stores the layer image data in the data storage unit; A tactile display device as described in claim 1 or 3, wherein when a request command specifying the image element to be displayed on the tactile display is input through the position designation means, the device control unit creates image data of the specific area corresponding to the request command based on the layer image data, and determines the protruding and retracting state of the tactile pins.

7. A tactile display device as described in claim 1 or 4, wherein when the position of the specific area specified in the request command is a position where all or part of the specific area extends outside the overall image, the device control unit creates warning information indicating this and outputs it as the support information from the support information output unit.

8. The tactile display device according to claim 7 , wherein the device control unit causes the support information output unit to output the created attention-calling information in the form of a sound signal, an audio signal, or a vibration signal.

9. the data storage unit stores text data related to the entire image or image elements included in the entire image; 5. The tactile display device according to claim 1, wherein the device control unit extracts the text data corresponding to the request command from the data storage unit, and causes the support information output unit to output the text data as the support information.

10. a text data generator that analyzes the overall image data and generates text data associated with the overall image or image elements contained in the overall image; 10. The tactile display device according to claim 9, wherein the text data stored in the data storage section includes the text data generated by the text data generator.

11. The tactile display device according to claim 10, wherein the text data generator is generated by machine learning using as learning data a plurality of image data, each of which is any image data and has text data associated with the image data labeled thereon.

12. a character information extractor that analyzes the overall image data and extracts characters or character strings represented in the overall image as text data; 10. The tactile display device according to claim 9, wherein the text data stored in the data storage section includes the text data extracted by the character information extractor.

13. 10. The tactile display device according to claim 9, wherein the device control unit causes the support information output unit to output the extracted text data in the form of an audio signal or in the form of Braille to be displayed on the Braille display.

Images