Color vision assist system
By generating and presenting color vision aid information based on object images and user color vision characteristics, the problem of users having difficulty recognizing actual colors is solved, improving the accuracy and convenience of color recognition.
Patent Information
- Application Number
- CN202280029851.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-04-27
- Filing Date
- 2022-02-21
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-02-21
AI Technical Summary
Users have difficulty accurately identifying the colors of objects they actually see, and existing technologies have failed to effectively solve the color recognition difficulties caused by the diversity of color vision.
Color vision aids are generated and presented using an information processing device, based on captured images of objects and user color vision characteristics, to help users identify actual colors.
It improves the accuracy and convenience of users in identifying object colors, reduces the operational burden of color vision calibration, and enhances the precision of color vision assistance.
Smart Images

Figure CN117203602B_ABST
Abstract
Description
Technical Field
[0001] This technology relates to information processing apparatus, information processing methods, and color vision assisting systems, and more particularly to technologies for assisting users in color vision based on their color vision characteristics. Background Technology
[0002] It is known that the diversity of color vision for red, blue, and green varies from person to person.
[0003] The patent document 1 below discloses a technique for improving visibility for users with color vision deficiencies by magnifying the content of the displayed information.
[0004] Furthermore, Patent Document 2 below discloses the following technology for using a color vision tester to determine the type and degree of color vision deficiency and the correction spectral characteristic curve related to color vision, and for designing glasses for correcting color vision based on this information.
[0005] In addition, the following patent document 3 discloses a technique for correcting the color of an image based on color vision characteristic information.
[0006] Citation List
[0007] Patent documents
[0008] Patent document 1: Japanese Patent Application Publication No. 2009-86612
[0009] Patent document 2: Japanese Patent No. 5110760
[0010] Patent document 3: Japanese Patent Application Publication No. 2009-71541 Summary of the Invention
[0011] The problem to be solved by the present invention
[0012] The purpose of this technology is to enable users to accurately identify the colors of objects they actually see.
[0013] Solution to the problem
[0014] The information processing apparatus according to the present technology includes a presentation processing unit configured to present color vision aid information generated based on a captured image of an object and color vision characteristic information of a user to a user.
[0015] As described above, color vision aids generated based on the captured image of the object and the user's color vision characteristics are presented to the user, thus helping the user to easily identify the color of the object they actually see.
[0016] The information processing method according to this technology is as follows: in this information processing method, the information processing device presents color vision auxiliary information generated based on the captured image of the object and the user's color vision characteristics information to the user.
[0017] Using such an information processing method, an information processing device based on the technology described above can be realized.
[0018] Furthermore, the color vision assist system according to this technology includes: an imaging unit configured to capture an image of an object; an assist information generation unit configured to generate color vision assist information based on the captured image of the object obtained by the imaging unit and the user's color vision characteristic information; and a presentation processing unit configured to present the color vision assist information generated by the assist information generation unit to the user.
[0019] As described above, color vision aids generated based on the captured image of the object and the user's color vision characteristics are presented to the user, thus helping the user to easily identify the color of the object they actually see. Attached Figure Description
[0020] Figure 1 This is a diagram showing an example of the external configuration of an information processing device 1 as an embodiment of the present technology.
[0021] Figure 2 This is a diagram showing an example of the electrical internal configuration of the information processing device 1.
[0022] Figure 3 This is a functional block diagram illustrating the functions of an information processing apparatus as an embodiment.
[0023] Figure 4 This is a diagram illustrating a specific example of calibration used to describe color vision characteristics.
[0024] Figure 5 This is an illustrative diagram illustrating an example of the first type of behavior.
[0025] Figure 6 This is an illustrative diagram illustrating an example of the second type of behavior.
[0026] Figure 7 This is a diagram illustrating an example of presenting color vision aids.
[0027] Figure 8 This is an illustrative diagram illustrating an example of color vision aids presented at the second-element level.
[0028] Figure 9 This is an explanatory diagram of a magnified image of the object.
[0029] Figure 10 This is another example of a diagram used to describe color vision aids at the second element level.
[0030] Figure 11 This is a flowchart illustrating the process related to calibration for obtaining information about a user's color vision characteristics.
[0031] Figure 12 This is a flowchart illustrating the process for detecting user behavior and presenting color vision assistive information in response to the behavior detection results.
[0032] Figure 13 This is an explanatory diagram of the first modification related to the device's construction.
[0033] Figure 14 This is an explanatory diagram of the second modification related to the device construction.
[0034] Figure 15 This is an explanatory diagram of the third modification related to the device's construction.
[0035] Figure 16 This is an explanatory diagram of the fourth modification related to the device structure.
[0036] Figure 17 This is an explanatory diagram of the fifth modification related to the device's construction. Detailed Implementation
[0037] The implementation methods will be described below in the following order.
[0038] <1. Configuration of the information processing device as an implementation method>
[0039] <2. Color Vision Assistance Method as an Implementation Method>
[0040] <3. Process>
[0041] <4. Modifications related to device construction>
[0042] <5. Modifications related to the presented information>
[0043] <6. Other modifications>
[0044] <7. Overview of Implementation Methods>
[0045] <8. This technology>
[0046] <1. Structure of the information processing device as an implementation method>
[0047] Figure 1 An example of the external structure of an information processing apparatus 1 as an embodiment of the present technology is shown.
[0048] The information processing device 1 includes a display screen 17a capable of displaying various types of information, and assists the user in terms of color perception by presenting information on the display screen 17a.
[0049] In this example, the information processing device 1 is configured as a wristwatch-type device and includes: a main body 1a on which a display screen 17a is formed; and a strap portion 1b connected to the main body 1a and attached to the user's wrist.
[0050] Furthermore, the information processing device 1 includes an imaging unit 21, which performs imaging via a solid-state imaging element and can sense images. In this example, the imaging unit 21 is disposed separately from the main body 1a, and specifically, is disposed at a predetermined position on the strap portion 1b. More specifically, in this example, the imaging unit 21 is disposed on the strap portion 1b so that it is located on the palm side when the information processing device 1 is worn, such that the main body 1a is located on the back side of the hand and on the palm side relative to the user's wrist.
[0051] In this example, a color image can be displayed on display screen 17a, and imaging unit 21 can generate a color image as a captured image.
[0052] As will be described later, the information processing device 1 presents to the user, via the display screen 17a, color vision assist information generated based on the captured image of the object obtained by the imaging unit 21 and the user's color vision characteristics information.
[0053] Note that in this example, the information processing device 1 is shaped to resemble a wristwatch, but does not necessarily have a clock function.
[0054] Figure 2 An example of the electrical internal configuration of the information processing device 1 is shown.
[0055] As shown, the information processing device 1 includes a central processing unit (CPU) 11, a read-only memory (ROM) 12, a random access memory (RAM) 13, and a non-volatile memory unit 14. The non-volatile memory unit 14 includes, for example, an electrically erasable programmable read-only memory (EEP-ROM).
[0056] The CPU 11 performs various types of processing based on programs stored in the ROM 12 or non-volatile memory unit 14, or programs loaded into the RAM 13 from memory unit 19 as described later. The RAM 13 also appropriately stores data, etc., required by the CPU 11 to perform various types of processing.
[0057] CPU 11, ROM 12, RAM 13 and non-volatile memory unit 14 are connected to each other via bus 23. In addition, input / output interface 15 is also connected to bus 23.
[0058] The input / output interface 15 can be connected to the following units: input unit 16, which is used by the user to perform input operations; display unit 17, which includes a liquid crystal panel, an organic electroluminescent (EL) panel, etc.; voice output unit 18, which includes a speaker, etc.; storage unit 19; and communication unit 20.
[0059] Here, the display screen 17a described above is the display screen included in the display unit 17.
[0060] Input unit 16 refers to an input device used by a user of information processing device 1. For example, it is assumed that various types of operating elements and devices such as keyboard, mouse, buttons, dial pad, touch panel, touchpad, and remote controller are used as input unit 16. The user's operation is sensed by input unit 16, and the signal corresponding to the input operation is interpreted by CPU 11.
[0061] Display unit 17 includes the display screen 17a described above, and can display various types of information on display screen 17a based on instructions from CPU 11. Furthermore, display unit 17 can display various types of operation menus, icons, messages, etc., based on instructions from CPU 11; that is, it displays as a graphical user interface (GUI).
[0062] Storage unit 19 includes, for example, a storage medium such as a hard disk drive (HDD) or a solid-state memory. Storage unit 19 can store various types of data, such as image data captured by imaging unit 21. In addition, storage unit 19 can also be used to store program data for causing CPU 11 to perform various types of processing.
[0063] The communication unit 20 performs communication processing via networks including the Internet and wired or wireless communication with peripheral devices (e.g., near-field communication).
[0064] In addition, in this example, the imaging unit 21 and the sensor unit 22 are connected to the input / output interface 15.
[0065] Imaging unit 21 includes, for example, a solid-state imaging element such as a complementary metal-oxide-semiconductor (CMOS) type or a charge-coupled device (CCD) type. For example, in a solid-state imaging element, multiple pixels are arranged in two dimensions, and each pixel has a photoelectric conversion element such as a photodiode. Imaging unit 21 performs, for example, correlated double sampling (CDS) processing, automatic gain control (AGC) processing, etc., on the electrical signals generated by the photoelectric conversion of each pixel, and further performs analog-to-digital (A / D) conversion processing to obtain captured image data as digital data.
[0066] Sensor unit 22 comprehensively indicates various types of sensors used to detect user behavior. In this example, sensor unit 22 is provided with motion sensors (e.g., accelerometers, angular velocity sensors, etc.) that detect the movement of body parts of the user wearing the information processing device 1.
[0067] Note that the sensors to be installed in the sensor unit 22 should be selected appropriately according to the form and application of the information processing device 1, and variations thereof will be described again later.
[0068] Furthermore, in this example, sensor unit 22 is equipped with a sensor that functions as a radio frequency identification (RFID) reader. Therefore, when the "object" described later is a product or the like with an attached RFID tag, the information stored in the RFID tag can be read.
[0069] <2. Color Vision Assistance Method as an Implementation Method>
[0070] Figure 3 This is a functional block diagram illustrating the function of the CPU 11 in the information processing apparatus 1 as an embodiment.
[0071] As shown, the CPU 11 includes a feature information acquisition unit F1, a behavior detection unit F2, an auxiliary information generation unit F3, and a presentation processing unit F4.
[0072] In this embodiment, each of these functional units implements the function of assisting the color vision of users whose color vision function differs from that of ordinary users. For convenience, in the following text, people whose color vision function differs from that of ordinary people will be referred to as "people with color vision diversity".
[0073] The characteristic information acquisition unit F1 acquires color vision characteristic information based on information input by the user. Color vision characteristic information is information indicating the characteristics of human color vision, and, for example, corresponds to information indicating functional differences when general color vision function is used as a reference for users with color vision diversity who have difficulty distinguishing between red and brown or between blue and purple.
[0074] By appropriately understanding the color vision characteristics of individuals with color vision diversity, one can provide appropriate assistance to them in terms of color vision based on this information.
[0075] In this embodiment, the user's color vision characteristics are obtained by performing a calibration of color vision characteristics. In this example, the calibration is performed by displaying a reference color and a sample color on the display screen 17a and allowing the user to select a sample color that looks the same as the reference color.
[0076] Reference Figure 4A specific example describing the calibration of color vision characteristics.
[0077] Calibration is essentially performed by using a predetermined color (such as red, green, or blue) as a reference color and allowing the user to select a sample color that looks the same as the reference color for each reference color.
[0078] Figure 4 A shows a screen for starting calibration for a certain reference color. As shown, the start screen displays a color image of the target reference color and a start button B1. Here, a color image refers to an image with a certain color. Examples of color images include a solid red image, etc.
[0079] When the Start button B1 is pressed on the Start screen, the color image of the sample color is displayed on the display screen 17a along with the color image of the reference color. In this example, the sample colors displayed side by side with the reference color are changed sequentially, and the user is allowed to select a sample color that looks the same as the reference color.
[0080] Specifically, Figure 4 B to Figure 4 D each shows the response to Figure 4 The state shown in Figure A, where the sample colors change sequentially due to the operation of the start button B1 on the start screen. Figure 4 In the diagram, the number of diagonal lines representing the sample color and the reference color is used.
[0081] In response to the operation of the start button B1, a color image of the sample color is displayed on the display screen 17a, and a stop button B2 is displayed in place of the start button B1. During the process of changing the sample color, the user operates the stop button B2 at a timed point when a sample color that appears to be the same as the reference color is displayed. The operation of the stop button B2 is for selecting the sample color, and the characteristic information acquisition unit F1 can specify the sample color displayed on the display screen 17a at the timed point when the stop button B2 is operated as a color that is visually recognized by the user as the same as the reference color.
[0082] Here, during the sequential changing of sample colors, it is desirable to temporarily hide the color image of the sample color when it is switched. This makes it easy for the user to recognize the switching of sample colors.
[0083] Furthermore, it is conceivable to change the sample color in different modes for each target reference color. For example, the hue of the sample color could be changed in different modes depending on whether the target reference color is red or blue. Here, the color change can be performed by altering at least one of the hue, saturation, or brightness of the color image to be displayed.
[0084] Figure 4 E shows in Figure 4 An example of the screen displayed in response to the operation of the stop button B2 when the timer of D is activated.
[0085] When the stop button B2 is operated, the color image of the reference color and the color image of the sample color when the stop button B2 is operated are displayed on the display screen 17a, and the reset button B3 and the next button B4 are displayed.
[0086] The user can instruct the information processing device 1 to reset the selection of the sample color by operating the reset button B3. Therefore, when the reset button B3 is operated, the characteristic information acquisition unit F1 displays on the display screen 17a... Figure 4 The start screen shown in A allows for the reselection of sample colors.
[0087] On the other hand, when the next button B4 is pressed, the characteristic information acquisition unit F1 acquires information about the sample color displayed on the display screen 17a at the timing when the stop button B2 is pressed, as information that the user visually recognizes as the same color as the reference color. Here, color information refers to color information specified by hue, saturation, and brightness.
[0088] Furthermore, when the next button B4 is pressed, the characteristic information acquisition unit F1 acquires the color vision characteristic information of the next reference color. Specifically, for the next reference color, the display and... Figure 4 The start screen shown in A is similar to the start screen in that it sequentially changes the sample color in response to the operation of the start button B1, and changes in response to the operation of the stop button B2. Figure 4 E similarly displays a screen with a reset button B3 and a next button B4.
[0089] When the reset button B3 is pressed, the characteristic information acquisition unit F1 re-displays the start screen for the next reference color and allows the user to reselect a sample color that appears to be the same as the reference color. Furthermore, when the next button B4 is pressed, the characteristic information acquisition unit F1 acquires information about the sample color displayed when the stop button B2 is pressed during the sample color change process of the next reference color, as information about a color that appears to be the same as the next reference color.
[0090] Here, when the target reference color is the latest reference color, the characteristic information acquisition unit F1 responds to the operation of the next button B4 and displays an end screen indicating the end of calibration on the display screen 17a.
[0091] The calibration for obtaining color vision information is performed using the method described above, thus eliminating the need for the user to change the sample color. Furthermore, since the sample and reference colors are displayed side-by-side, it is easy for the user to compare the reference and sample colors and obtain accurate color vision information.
[0092] Therefore, the accuracy of calibration can be improved, while reducing the burden of user operations related to calibration, and ultimately improving the precision of color vision assistance.
[0093] Here, it is known that human color vision characteristics include diurnal and daily variations. Considering these diurnal and daily variations, calibration should be performed relatively frequently. However, under heavy workloads, frequent calibration is difficult to perform, leading to a decrease in the accuracy of color vision assistance based on the user's color vision characteristics. Because the operational burden of calibration is reduced, it can be easily performed at a high frequency, thus improving the accuracy of color vision assistance.
[0094] Note that, as understood from the description above, it is expected that the calibration used to obtain information on color vision characteristics will be performed not just once, but multiple times. For example, taking into account the diurnal rhythm variations described above, the calibration should be performed multiple times a day. However, taking into account daily variations, the calibration may be performed, for example, once a day.
[0095] Description Return to Figure 3 .
[0096] The behavior detection unit F2 detects the user's behavior relative to an object. Specifically, in this example, the behavior detection unit F2 is based on... Figure 2 The sensor unit 22 shown detects the user's behavior relative to an object by using the detection signals of the motion sensor (in this example, the detection signals of acceleration or angular velocity). More specifically, the user's behavior is detected from the movement of the user's hand on one side of the wearable information processing device 1.
[0097] In this example, the first type of behavior and the second type of behavior are detected as user behavior relative to the object. Specifically, as Figure 5 As shown, the first type of behavior is the user extending his / her hand toward the object (the user moving his / her hand to get closer to the object). Figure 6 As shown, the second type of behavior is the behavior of the user initiating an object.
[0098] Notice, Figure 5 and Figure 6 The example shown is a shirt as a garment, but objects are not limited to this.
[0099] In this example, to detect the first type of behavior, the image captured by the imaging unit 21 is used together with the detection signal from the motion sensor. Specifically, for the detection signal from the motion sensor, the behavior detection unit F2 determines that the user has performed the first type of behavior if the detected signal waveform pattern matches or is similar to the signal waveform pattern when the hand extends forward, and obtains an image of the object gradually approaching based on image analysis of the captured image.
[0100] Furthermore, the behavior detection unit F2 uses at least the detection signal from the motion sensor to detect the second type of behavior. Specifically, within a predetermined time after the detection of the first type of behavior, the behavior detection unit F2 determines that the user has performed the second type of behavior if the motion sensor detection signal has identified that the user has moved his / her hand upward by a predetermined amount or more.
[0101] Note that the methods for detecting Type I and Type II behaviors are not limited to those described above, and various methods can be considered.
[0102] For example, the example of using the detection results of the first type of behavior in the detection of the second type of behavior has been described above, but it is also conceivable to perform the detection of the second type of behavior as an independent detection, not based on the detection results of the first type of behavior. In this case, it is conceivable to detect the second type of behavior, for example, under the following determining conditions: recognizing from the captured image that the user's hand and the object are moving together (relative to the background), and recognizing from the detection signal of the motion sensor that the user's hand has moved upward by a predetermined amount or more during the period when the user's hand and the object are moving together.
[0103] As will be described later, in this embodiment, the user's behavior results detected by the behavior detection unit F2 are used to present color vision assist information.
[0104] exist Figure 3 In this example, the auxiliary information generation unit F3 generates color vision auxiliary information based on the captured image of the object and its color vision characteristics. Color vision auxiliary information is information used to assist the user in color perception. Specifically, in this example, information used to help the user (such as someone with color vision diversity) easily identify the color of the object captured by the imaging unit 21 is generated as color vision auxiliary information.
[0105] The following will describe specific examples of color vision aids.
[0106] The presentation processing unit F4 presents color vision assistive information to the user. Specifically, in this example, the color vision assistive information is displayed on the display screen 17a of the display unit 17.
[0107] In this example, the presentation processing unit F4 presents information indicating the color as color vision aid information.
[0108] The "color-indicating information" mentioned here refers to information that enables users to recall a specific color when information is presented. Specific examples include information about the color image mentioned above (an image with a specific color), character information indicating the color name (such as "blue", "red", "purple", and "blue-purple"), etc.
[0109] Based on the captured image of the object and the user's color vision characteristics, information is generated to help the user (such as a person with color vision diversity) easily identify the color of the object captured by the imaging unit 21 as the color vision aid information described above.
[0110] Figure 7 This is a diagram illustrating an example of presenting color vision aids.
[0111] Figure 7 An example is shown where a color image corresponding to the color of an object captured by imaging unit 21 is displayed on display screen 17a as color vision aid information.
[0112] Here, the types of people with color vision diversity are classified according to which of the photoreceptors (red-sensitive photoreceptors, green-sensitive photoreceptors, and blue-sensitive photoreceptors) exhibits functional decline. Specifically, the type of photoreceptor is not limited to photoreceptors in which functional deterioration is observed only in any one of the red, green, and blue photoreceptors, but also includes some photoreceptors in which functional deterioration is observed in any two of the red, green, and blue photoreceptors.
[0113] For example, so-called dichroists, whose green-sensitive photoreceptors have deteriorated, find it difficult to distinguish between red and green, orange and yellow-green, green and brown, blue and purple. For instance, when an object is purple, the purple color appears more bluish than it actually is, making it difficult to differentiate that color from blue.
[0114] Therefore, in the case where the user has diverse color perception, and the shirt in question is purple, a reddish-purple image that intensifies the redness of purple is generated as color vision aid information, and this color image is displayed on the display screen 17a. Thus, the user (such as someone with diverse color perception) can easily identify the actual color of the object by referring to the color image displayed as color vision aid information. Specifically, in the case described above, it is easy for the user to identify that the color of an object perceived as blue is actually purple.
[0115] In color vision auxiliary information, the color to be presented to the user should be determined based on the type of color vision diversity (type of color vision characteristics) and the color of the imaged object. The type of color vision diversity can be obtained from the calibration results described above. Specifically, the characteristic information acquisition unit F1 specifies the type of color vision diversity based on the calibration results and obtains information about the type of color vision diversity as color vision characteristic information. Where the type of color vision diversity of the user can be specified, a color can be specified to make the actual color of the object easily perceptible to a person with color vision diversity (hereinafter referred to as the "auxiliary color").
[0116] In this example, as described above, the auxiliary information generation unit F3 obtains the auxiliary color to be presented as color vision auxiliary information based on the color vision characteristic information obtained by the characteristic information acquisition unit F1 (i.e., information on the type of color vision diversity specified by the captured image of the imaging unit 21 and information on the color of the object).
[0117] Note that auxiliary colors are obtained, for example, by preparing and referring to a table that stores information on auxiliary colors corresponding to each combination of types of color vision diversity and the colors of objects.
[0118] Here, in the case where the user is a person with color vision diversity, the auxiliary color indicated by the color vision aid information is a color different from the color of the object.
[0119] The color of an object perceived by a user (such as someone with color vision diversity) is different from the actual color of the object. Therefore, information about the auxiliary color, which is different from the object's color, is presented, and thus can help users with color vision diversity to identify the actual color of the object.
[0120] Furthermore, in this embodiment, the color vision characteristic information obtained through calibration can be reformulated as difference information between a reference color and a color that appears to be the same as the reference color. Then, in this embodiment, the auxiliary color of the object's color can be reformulated as a color determined based on the color vision characteristic information (such as this difference information).
[0121] Here, the difference information between the reference color and a color that appears to be the same as the reference color can be described as the difference information between the reference color and the color perceived by the user when perceiving the reference color. Therefore, in other words, as in this embodiment, the presentation processing unit F4, which obtains auxiliary colors and presents color vision auxiliary information indicating the auxiliary colors using the illustrated method described above, performs the following presentation processing. Therefore, the processing involves presenting information indicating a color determined based on the difference information between the object's color and the color perceived by the user when perceiving the object's color as color vision auxiliary information.
[0122] In this example, the presentation processing unit F4 presents color vision aiding information based on the user behavior detected by the behavior detection unit F2. Specifically, the presentation processing unit F4 switches the information element level of the information presented as color vision aiding information in response to the type of user behavior detected by the behavior detection unit F2.
[0123] The information element level mentioned here refers to the level of the elements that constitute information. Specifically, differences in information element levels include, for example, differences in the quantity or type of information to be presented, differences in the quality or specificity of the information to be presented, etc.
[0124] Here, the inventors of this application have, to date, gained knowledge through observation of the behavior of people with color vision diversity that differences in the types of behaviors, such as reaching out to an object, picking up an object, or changing the way an object is held, are related to differences in the level of information elements required by the user as information for decision-making. For example, behaviors such as reaching out to an object generally indicate a user's (e.g., a person with color vision diversity) request to identify the color of the object. Furthermore, behaviors such as picking up or changing the way an object is held generally indicate a user's request to obtain more detailed information beyond the object's color. Therefore, as described above, the level of information elements to be presented to the user switches in response to the type of user behavior.
[0125] In this example, if the behavior detection unit F2 detects the first type of behavior (the behavior of reaching out to an object), as follows: Figure 7 As shown, the presentation processing unit F4 presents color vision auxiliary information, which includes only a color image indicating auxiliary colors, in a display such as... Figure 7 On the display screen 17a shown.
[0126] In the following text, the information level that includes only information indicating colors as described above for color vision assistance is referred to as the "first element level" as the information element level of color vision assistance information.
[0127] On the other hand, if the behavior detection unit F2 detects the second type of behavior (the behavior of proposing an object), such as Figure 8 As shown, the presentation processing unit F4 displays color-aided information at a second element level that is higher than the first element level in terms of information element level on the display screen 17a.
[0128] Specifically, Figure 8 A shows in Figure 7The color image shown at the first element level displays the object's attribute information (including attribute information other than color) as an example of color vision auxiliary information at the second element level. This can be described as an example of displaying color information that indicates the auxiliary color of the object and other information besides the information indicating the color.
[0129] Here, the example shows that the character information indicating the color of the object, "purple," and the character information indicating the size, composition, and price of the shirt object are displayed as the object's attribute information.
[0130] Here, it can be envisioned that, in the case of an object such as a commodity in this example, the information about the RFID tag attached to the commodity read by the aforementioned RFID reader would be displayed as the object's attribute information.
[0131] Figure 8 B illustrates color vision assistance information at the second element level, corresponding to a situation where the user has previously inputted a desired color (requested color) relative to the object. Specifically, in this case, the color vision assistance information is obtained by displaying image information, wherein a color image indicating the color of the object ("viewed color" in the figure) and a color image indicating the auxiliary color requested by the user are arranged side by side.
[0132] In other words, a color image indicating the color of the object and a color image indicating the color corresponding to the color requested by the user are displayed side by side.
[0133] As described above, the information element levels of the color vision aids presented to the user switch in response to the type of user behavior. Therefore, information can be presented according to the information element levels implicitly requested by the user through their natural behavior, which can help the user understand objects not only from a color vision perspective but also from perspectives beyond color vision. Furthermore, the presentation of aids based on the user's natural behavior enables accessibility that is highly compatible with the user's daily life.
[0134] In particular, such as Figure 8 As shown in B, when color images indicating the color of an object and color images indicating the color corresponding to the color requested by the user are displayed side by side as color aid information at the second element level, it can appropriately assist the user in having a higher-order understanding of whether the object is the object the user expects in terms of hue (a higher-order understanding that goes beyond the primary understanding of only understanding the color of the object).
[0135] Note that in this example, such as Figure 8 The display of color vision aid information related to the color requested by the user, as shown in B, can be... Figure 8 The display of color vision aids shown in A is switched.
[0136] Specifically, in this example, upon detecting the second type of behavior, the presentation processing unit F4 first displays including... Figure 8 The color vision aid information shown in A is attribute information. After displaying the color vision aid information, including attribute information, in this way, if the user has previously entered the requested color, the color vision aid information will be displayed for the user. Figure 8 The operation to switch the display of color assist information related to the requested color, as shown in B, is received. If the switching operation has already been performed, the color assist information related to the requested color is displayed on display screen 17a. Thereafter, each time a display switching operation is performed, the display of color assist information, including attribute information, is switched to the display of color assist information related to the requested color.
[0137] Here, the example of displaying auxiliary colors of an object as an indicator of color vision has been described above, but it is also conceivable to display a magnified image of the object as color vision aid.
[0138] Figure 9 This is an explanatory diagram of a magnified image of the object.
[0139] Figure 9 A shows a captured image of the object before magnification. For example, in the case where the object includes a finely patterned color arrangement within a resistive element as shown in the figure, it is difficult for someone with diverse color vision to discern the colors of the patterned color arrangement. Therefore, as... Figure 9 As shown in B, a magnified image of the object is displayed on the display screen 17a as color vision aid information.
[0140] Note that in this case, the color vision aids, for example... Figure 9 As shown in B, the aiming mark M is displayed at a predetermined position in the image frame (such as the center of the image frame), and character information indicating the color of the part indicated by the aiming mark M can be displayed as part of the color vision aid information (e.g., the display of "red" in the figure).
[0141] Furthermore, for example, as with resistive elements, where the color arrangement pattern in the color arrangement pattern section represents the attribute information of the object (resistance value in the resistive element), the attribute information obtained through image analysis of the object can be displayed as part of the color vision aid information (e.g., the display of "1KΩ±5%" in the figure).
[0142] In this example, the color vision assistance information displayed by the magnified image described above is performed under the following conditions: the behavior detection unit F2 detects the user's first type of behavior, and the objects captured in the image captured by the imaging unit 2 include a finely colored pattern portion.
[0143] Specifically, when the behavior detection unit F2 detects a user's first type of behavior, the presentation processing unit F4 in this example analyzes the image captured by the imaging unit 21 and determines the presence or absence of a fine color arrangement pattern portion. It is conceivable, for example, to determine the presence or absence of a fine color arrangement pattern portion by detecting color arrangement pattern portions in which different colors are arranged in a predetermined pattern and determining that the arrangement interval of each color in the color arrangement pattern portion is equal to or less than a predetermined interval.
[0144] If a fine color arrangement pattern is determined to exist, the rendering processing unit F4 generates a magnified image of the image region including the fine color arrangement pattern in the captured image, and displays color vision assistance information including the magnified image on the display screen 17a.
[0145] Note that, although not shown, in the case of displaying a magnified image as described above, color vision aids, including information indicating the auxiliary colors of the object, can be displayed along with the magnified image. For example, in Figure 9 In example B, information about the auxiliary color (e.g., a color image or color character information) indicating the color of the part indicated by the aiming mark M is displayed along with a magnified image, etc.
[0146] Furthermore, as described above, examples including... Figure 8 The color vision auxiliary information of the attribute information of the object shown in A and the color vision auxiliary information of the object shown in A are as follows: Figure 8 The required color-related color vision auxiliary information shown in B is considered as color vision auxiliary information at the second element level, but various examples of color vision auxiliary information at the second element level can be envisioned. For example, it is also possible to envision... Figure 10 A and Figure 10 The example shown in B.
[0147] exist Figure 10 In A, when the object is a shirt, the following color vision aids are illustrated: information indicating the color of the object ("purple T-shirt" in the figure); and information suggesting the color of the pants, which is suitable for the shirt object ("black" and "white" in the figure).
[0148] In addition, Figure 10 In B, when the object is a tomato, the following color vision aids are illustrated: information indicating the color of the object ("ripe tomato" in the figure, that is, a red tomato); and information for suggesting recommended dishes to use the object ("pickled" and "curry" in the figure).
[0149] <3. Process>
[0150] Next, we will refer to Figure 11and Figure 12 The flowchart below illustrates a specific process example for implementing the color vision assistive method as described above.
[0151] Note that CPU 11 executes according to the program stored in ROM 12 or memory unit 19. Figure 11 and Figure 12 The processing shown.
[0152] Figure 11 This is a flowchart illustrating the process related to calibration for obtaining information about a user's color vision characteristics.
[0153] First, in step S101, CPU 11 sets the reference color identifier n to the initial value "1". The reference color identifier n is an identifier used to identify the reference color to be processed among a plurality of reference colors defined as calibration targets.
[0154] In step S102, following step S101, CPU 11 displays the start screen for the nth reference color. Specifically, for the nth reference color, this includes a color image of the reference color and, for example,... Figure 4 The start button B1 shown in A is displayed on the screen 17a.
[0155] In step S103 following step S102, as a waiting process to start the operation, CPU 11 waits until the operation start button B1 is pressed.
[0156] When the start button B1 is pressed and the start operation is confirmed, the CPU 11 proceeds to step S104 and begins the sequential change of the sample colors corresponding to the nth reference color. Specifically, the display process of sequentially changing the sample colors according to a predetermined pattern for the nth reference color begins.
[0157] In step S105, following step S104, CPU 11 waits for the operation of the stop button B2 as a wait process for the stop operation.
[0158] If it is confirmed that the stop button B2 has been pressed and a stop operation has been performed, CPU 11 proceeds to step S106 and displays the operation screen after the stop. Specifically, as follows... Figure 4 As shown in E, the screen displays a color image including a reference color (the nth reference color), a color image of the sample color at the time the operation stopped, a reset button B3, and a next button B4.
[0159] In step S107, following step S106, CPU 11 determines whether an operation for proceeding to the next step exists or not (i.e., whether the operation of the next step button B4 exists or not). If it is determined that no operation for proceeding to the next step has been performed, CPU 11 determines whether a reset operation exists or not (i.e., whether the operation of the reset button B3 exists or not) in step S108. If it is determined in step S108 that no reset operation has been performed, CPU 11 returns to step S107 and again determines whether an operation for proceeding to the next step exists or not. The loop of steps S107→S108→S107 causes a waiting process to be formed until an operation for proceeding to the next step or a reset operation is performed.
[0160] If it is determined in step S108 that a reset operation has been performed, CPU 11 returns to step S102. Therefore, for the nth reference color, a sample color that looks the same as the nth reference color can be selected again.
[0161] Furthermore, if it is determined in step S107 that the operation to proceed to the next step has been performed, the CPU 11 proceeds to step S109 and stores the sample color at the time of the stop as the perceived color corresponding to the nth reference color. Specifically, when it is determined that the stop operation in step S105 has been performed, the information of the sample color displayed on the display screen 17a is stored in a predetermined storage device (e.g., RAM 13, non-volatile memory unit 14, etc.) as information that is visually recognized by the user as the same as the nth reference color.
[0162] In step S110 following step S109, CPU 11 determines whether the reference color identifier n is greater than or equal to the upper limit value N. If the reference color identifier n is not greater than or equal to the upper limit value N, CPU 11 proceeds to step S111, increments the reference color identifier n by 1, and returns to step S102. Therefore, for the next reference color, processing is performed that allows the user to select a sample color that looks the same as the reference color.
[0163] On the other hand, if it is determined in step S110 that the reference color identifier n is greater than or equal to the upper limit value N, the CPU 11 proceeds to step S112, displays the end screen, and ends. Figure 11 The series of processes shown are illustrated. Note that in step S112, a screen is displayed that includes information to notify the user that the calibration is complete, such as "Calibration complete" as an end screen.
[0164] Figure 12 This is a flowchart illustrating the process for detecting user behavior and presenting color vision assistive information in response to the behavior detection results.
[0165] exist Figure 12 In step S201, CPU 11 determines whether a first type of behavior has been detected. As described above, in this example, the first type of behavior is the user extending his / her hand toward an object (see...). Figure 5 Furthermore, the behavior can be detected based on, for example, the image captured by the imaging unit 21 as described above and the detection signal from the motion sensor in the sensor unit 22.
[0166] If it is determined in step S201 that the first type of behavior has been detected, the CPU 11 proceeds to step S203 and generates and presents color vision auxiliary information at the first element level.
[0167] As described above, in this example, color vision auxiliary information relative to the first element level is performed based solely on whether the captured object in the image captured by imaging unit 21 includes a finely patterned color arrangement. Figure 7 The display of color vision aid information in the color image of the auxiliary colors shown includes, for example, the auxiliary colors shown. Figure 9 Switching between displaying color-aided information of a magnified image of the object shown in B. Specifically, in response to the detection of a first type of behavior in determination step S201, CPU 11 analyzes the image captured by imaging unit 21 and determines the presence or absence of a fine color arrangement pattern portion. As described above, it is conceivable, for example, to determine the presence or absence of a fine color arrangement pattern portion by detecting color arrangement pattern portions in which different colors are arranged in a predetermined pattern and determining whether the arrangement interval of each color in the color arrangement pattern portion is equal to or less than a predetermined interval.
[0168] If a fine color arrangement pattern is determined to exist, the CPU 11 generates a magnified image of the image region including the fine color arrangement pattern in the captured image, and displays color vision assistance information including the magnified image on the display screen 17a.
[0169] On the other hand, if it is determined that there is no fine color arrangement pattern, the CPU 11 assigns an auxiliary color to the color of the object specified in the captured image based on the user's color vision characteristic information obtained from the above calibration results, and displays color vision assistance information including the color image derived from the auxiliary color on the display screen 17a.
[0170] In response to the execution of the processing in step S203 as described above, CPU 11 advances the processing to step S205. Note that the processing in step S205 will be described later.
[0171] On the other hand, if it is determined in step S201 that no first type of behavior has been detected, CPU 11 proceeds to step S202 and determines whether a second type of behavior has been detected. In this example, as described above, the second type of behavior is the behavior in which the user raises an object (see...). Figure 6 Furthermore, the behavior can be detected based on, for example, detection signals from motion sensors and detection results of the first type of behavior as described above.
[0172] If it is determined in step S202 that the second type of behavior has been detected, the CPU 11 proceeds to step S204 and generates and presents color vision aid information at the second element level.
[0173] As described above, in this example, in response to the user's switching action, the display of the second element level of color vision aid information, which includes the object's attribute information, is performed. Figure 8 A) Switch to displaying the second element level of color vision aid information related to the color requested by the user. Specifically, in response to determining in step S202 that a second type of behavior has been detected, CPU 11 first includes Figure 8 Color assist information, as shown in A, is displayed on display screen 17a. Subsequently, if the user has previously entered a requested color, CPU 11 accepts the color information corresponding to... Figure 8 The operation shown in B involves switching the display of color vision assistive information related to the requested color, and if the switching operation has been performed, displaying the color vision assistive information related to the requested color on display screen 17a. Thereafter, each time a display switching operation is performed, the display of color vision assistive information, including attribute information, is switched to the display of color vision assistive information related to the requested color.
[0174] CPU 11 advances the process to step S205 in response to the execution of the processing in step S204.
[0175] Furthermore, if it is determined in step S202 that no second type of behavior is detected, CPU 11 skips the processing in step S204 and proceeds to step S205.
[0176] In step S205, the CPU 11 determines whether to end the process, that is, whether the process meets a predetermined condition. This predetermined condition is determined in advance as a condition under which the process should end, such as the power supply to the information processing device 1 being turned off.
[0177] In step S205, if the processing has not ended, CPU 11 will return to step S201. Therefore, for example, after detecting the first type of behavior in step S201, color vision auxiliary information at the first element level is presented in step S203; the second type of behavior is detected in step S202; and color vision auxiliary information at the second element level is presented in step S204. Furthermore, for example, after detecting the second type of behavior in step S202, color vision auxiliary information at the second element level is presented in step S204, and then, for example, if the first type of behavior towards another object is detected, color vision auxiliary information at the first element level can be presented for that other object in step S203.
[0178] Furthermore, in cases where neither type I behavior nor type II behavior is detected, color vision auxiliary information at the first element level or the second element level is not presented.
[0179] If the processing is determined to be complete in step S205, CPU 11 terminates. Figure 12 The series of processes shown.
[0180] <4. Modifications related to device construction>
[0181] As described above, an example of a wristwatch-type information processing device 1 including all the elements for implementing color vision assistance as in this embodiment has been described. Specifically, an information processing device 1 includes all of the following elements: a "characteristic information acquisition unit" that acquires color vision characteristic information of a user; an "imaging unit" that acquires a captured image of an object; an "assistance information generation unit" that generates color vision assistance information based on the captured image of the object and the color vision characteristic information; a "behavior detection unit" that detects the user's behavior relative to the object; a "sensor unit" for behavior detection that detects, for example, motion; a "presentation processing unit" that presents the color vision assistance information to the user; and a "presentation unit" that presents the color vision assistance information in response to the processing of the presentation processing unit.
[0182] However, the form of the information processing device according to the embodiment is not limited to a wristwatch. Furthermore, an information processing device need not include every element described above. In the following, modifications to the construction of the device for achieving color vision assistance as an embodiment will be described.
[0183] Figure 13 This is an explanatory diagram of the first modification related to the device's construction.
[0184] In the first modification, a wristwatch-type device 30 and an eyeglasses-type information processing device 1A are provided. In this case, the device 30 serves as a means of detecting the movement of a user's hand on one side of the wearable device 30. Furthermore, the information processing device 1A includes an imaging unit 21 and a display unit 17, and can capture an image of an object and display color vision assistive information.
[0185] In this case, the device 30 includes the aforementioned "sensor unit" for detecting the movement of the user's hand, and the information processing device 1A includes the aforementioned "characteristic information acquisition unit", "imaging unit" (imaging unit 21), "auxiliary information generation unit", "behavior detection unit", "presentation processing unit" and "presentation unit" (display unit 17), and performs calibration for obtaining color vision auxiliary information as described above, detects the user's behavior based on the detection signal of the hand movement obtained by the device 30, generates color vision auxiliary information based on color vision characteristic information, presents color vision auxiliary information based on the user's behavior detection result, and presents color vision auxiliary information through the presentation unit.
[0186] Note that in the eyeglasses-type information processing device 1A, the display unit 17 can be configured as, for example, a projector device that projects images onto the lens portion of the eyeglasses or the user's retina.
[0187] Figure 14 This is an explanatory diagram of the second modification related to the device construction.
[0188] The second modification provides a wristwatch device 30, an information processing device 1B, and a server device 32 capable of communicating with the information processing device 1B via a network 31.
[0189] Device 30 serves as a means of detecting hand movements in a manner similar to the first modification. Information processing device 1B includes an imaging unit 21 and a display unit 17.
[0190] In this configuration, the information processing device 1B may include at least the "characteristic information acquisition unit," "imaging unit," "presentation processing unit," and "presentation unit" described above. At least one of the processes in the aforementioned "auxiliary information generation unit" and "behavior detection unit" is performed by the server device 32 instead of the information processing device 1B. When the "behavior detection unit" is provided in the server device 32, the information processing device 1B transmits the detection signal of the user's movement obtained by the device 30 to the server device 32 and acquires information indicating the detection result of the user's behavior obtained by the server device 32. Furthermore, when the "auxiliary information generation unit" is provided in the server device 32, the information processing device 1B transmits the image captured by the imaging unit and color vision characteristic information obtained through calibration to the server device 32, acquires color vision auxiliary information generated by the server device 32 based on this information, and the "presentation processing unit" causes the "presentation unit" to present the color vision auxiliary information.
[0191] Figure 15 This is an explanatory diagram of the third modification related to the device's construction.
[0192] The third modification is an example of an eyeglass-type information processing device 1C that includes each of the elements described above.
[0193] In this context, it is conceivable to provide myoelectric sensors for detecting muscle movement near the user's eyes, and gaze sensors for detecting gaze direction, as "sensor units" for detecting user behavior. In this case, user behavior, for example, can be detected in relation to how the user views an object (e.g., the state in which the user is viewing an object or the state in which the user is gazing at an object). It is conceivable that the information element level of the color vision assistive information to be presented switches in response to the type of behavior.
[0194] Note that even in the case of the eyeglass-type information processing device 1C, behavior can be detected based on the user's hand movements. For example, if the user's hand can be captured in the image captured by the imaging unit 21, behavior related to the user's hand movements (e.g., the user extending his / her hand toward an object, lifting an object, etc.) can be detected by image analysis of the image captured by the imaging unit 21. Note that in this case, the imaging unit 21 (image sensor) can also be considered as a "sensor unit" for detecting the user's behavior.
[0195] Figure 16 This is an explanatory diagram of the fourth modification related to equipment configuration.
[0196] The fourth modification provides a chopstick-shaped device 30D and an eyeglasses-shaped information processing device 1D. In this case, the device 30D is provided with a "sensor unit" (motion sensor) for detecting the movement of the chopstick-shaped device 30D. In addition, the information processing device 1D is provided with an "imaging unit" (imaging unit 21) and a "presentation unit" (display unit 17), and is also provided with a "characteristic information acquisition unit", an "auxiliary information generation unit", an "behavior detection unit", a "presentation processing unit", and a "presentation unit".
[0197] For example, it is difficult for dichroic individuals to perceive the doneness of meat, and there is a possibility that they might eat undercooked meat. In the fourth modification, a solution to this problem is envisioned. In this case, the "behavior detection unit" in the information processing device 1D detects the user's action of holding an object (in this case, meat) using the chopstick-type device 30D based on detection signals from the motion sensor installed in the device 30D. Then, in response to the detection of the behavior, the "presentation processing unit" causes the "presentation unit" (display unit 17) to present color vision auxiliary information generated by the "auxiliary information generation unit".
[0198] Note that, in this context, it is also conceivable to switch the information element level in response to the type of behavior. For example, it is conceivable to present a first-element level of color-aided information (e.g., a color image of the auxiliary color of the object) for the behavior of a user holding an object with a chopstick-type device 30D, and a second-element level of color-aided information (e.g., information other than the color image, such as the estimated cooking time of the meat) for behaviors such as the object being inverted (meat being flipped).
[0199] Figure 17 This is an explanatory diagram of the fifth modification related to the device's construction.
[0200] The fifth modification is an example of a mirror-type information processing device 1E. The information processing device 1E includes all the elements of the aforementioned "characteristic information acquisition unit", "imaging unit", "auxiliary information generation unit", "behavior detection unit", "sensor unit", "presentation processing unit" and "presentation unit".
[0201] The information processing device 1E is equipped with an imaging unit 21 for capturing images in the same direction as the mirror surface. Therefore, it is possible to capture an image of a user standing in front of the mirror. In this case, the mirror surface also functions as a display screen capable of showing images and presenting information to the user standing in front of the mirror.
[0202] Examples of applications for the information processing device 1E include applications that assist users in selecting clothing. For instance, in the case of a user standing in front of a mirror performing the action of putting a shirt on their chest, color-aiding information is presented on the surface of the mirror. For example, as in... Figure 7 In the example, one could imagine presenting a color image as an auxiliary color indicating the color of the shirt as an object, or other color vision aids.
[0203] In the fifth revision, it is conceivable that the information element level can be switched in response to the type of behavior. For example, for the behavior of a user putting a shirt on their chest, a color image of an auxiliary color indicating the color of the shirt is presented as color vision aid information at the first element level as described above. If the user puts the shirt on their chest and simultaneously turns to the right or left, color vision aid information including information suggesting the color of trousers that match the shirt is presented as color vision aid information at the second element level. Alternatively, it is conceivable that an image of trousers matching the shirt is displayed on the user's legs in a superimposed mode in a mirror as color vision aid information at the second element level in this case.
[0204] Note that in this case, it is conceivable to detect user behavior by performing image analysis on the captured images of imaging unit 21. In this case, imaging unit 21 is used not only as a sensor for detecting the color of an object, but also as a "sensor unit" for detecting user behavior.
[0205] <5. Modifications related to the presented information>
[0206] Although examples of presenting only visual information as color vision aids have been described above, at least one of auditory, tactile, and olfactory information can also be presented as color vision aids in addition to visual information.
[0207] Examples of units for presenting tactile information include vibrators that generate vibrations, blowers that provide tactile stimulation via wind, and devices that provide electrotactile stimulation. Furthermore, examples of units for presenting olfactory information include perfume diffusers.
[0208] Here, one could envision switching the level of information elements related to auditory, tactile, and olfactory information to be presented in response to the type of user behavior.
[0209] The following will describe examples of how the above-mentioned auditory, tactile, and olfactory information is presented in response to the first type of behavior and the second type of behavior.
[0210] The time of first type of behavior
[0211] • Hearing: The sounds that evoke associations with an object.
[0212] • Touch: weak vibrations, long vibration intervals
[0213] • Smell: The scents associated with an object.
[0214] The second type of behavior at the time
[0215] • Hearing: The sound that indicates an object
[0216] • Touch: Strong vibrations, short vibration intervals
[0217] • Smell: the scent of an object
[0218] In terms of hearing, one could imagine that the "sound that evokes associations with an object" is, for example, the sound of the label "clothes" for an object = shirt, or the sound of the label "vegetable" for an object = tomato. On the other hand, one could imagine that the "sound that indicates an object" is, for example, the sound of the label "shirt" for an object = shirt, or the sound of the label "tomato" for an object = tomato.
[0219] Furthermore, regarding the sense of smell, examples of the aforementioned "smell associated with the object" include the "sweet" smell, the "grass-like" smell, etc., when the object is a tomato. On the other hand, examples of "smells that indicate the object" include the smell of the tomato itself when the object is a tomato.
[0220] Note that, in terms of the presentation of tactile information, for example, it is conceivable to allow users to perceive characters that indicate the form, color, name, etc. of an object in Morse code.
[0221] <6. Other modifications>
[0222] Here, the implementation is not limited to the specific examples described so far, and various modified configurations may be adopted.
[0223] For example, the example of displaying a magnified image of an object as color vision aid information has been described above, but the magnification ratio of the image at that time can be determined based on information about the magnification ratio set by the user's past magnification operations of the displayed object.
[0224] Furthermore, when a color image is displayed as color vision aid information, it is conceivable that differences in color appearance due to ambient light be taken into account when determining the color of the color image. That is, for example, the type of ambient light is specified by performing image analysis on the image captured by imaging unit 21, and the color of the color image is determined based on the specified type of ambient light.
[0225] In addition, it is conceivable that, in addition to motion sensors and gaze sensors as exemplified above, distance measurement sensors such as time-of-flight (ToF) sensors and illuminance sensors are also used as sensors to detect user behavior.
[0226] <7. Overview of Implementation Methods>
[0227] As described above, each of the information processing apparatuses (information processing apparatuses 1, 1A, 1B, 1C, 1D and 1E) according to the embodiments includes a presentation processing unit (presentation processing unit F4), which presents color vision assist information generated based on the captured image of the object and the color vision characteristic information of the user to the user.
[0228] As described above, color vision aids generated based on the captured image of the object and the user's color vision characteristics are presented to the user, thus helping the user to easily identify the color of the object they actually see.
[0229] Therefore, even if the user has a variety of color vision, it can enable the user to accurately identify the color of the object actually seen.
[0230] Furthermore, in the information processing apparatus according to the embodiment, the presentation processing unit presents information indicating the color as color vision aid information.
[0231] As described above, color information is presented to the user as color vision aids, and thus, in cases where the user has color vision diversity, the user can recall the actual color of the object they are viewing.
[0232] Therefore, compared to users with diverse color vision, it enables users to accurately identify the colors of objects they actually see.
[0233] Furthermore, in the information processing apparatus according to the embodiment, the presentation processing unit presents information indicating a color different from the color of the object as color vision aid information.
[0234] The color of an object perceived by a user (such as someone with color vision diversity) may differ from the actual color of the object.
[0235] Therefore, as described above, information about colors different from the object's color is presented, and thus it can make it easier for users (such as people with color vision diversity) to identify the object's actual color.
[0236] Furthermore, in the information processing apparatus according to the embodiment, the presentation processing unit presents information of an indicator color determined based on the difference between the color of the object and the perceived color when the user perceives the color of the object as color vision aid information.
[0237] Therefore, color information that is determined to compensate for the difference between the actual color of an object and the color perceived by a user (such as someone with color vision diversity) can be presented.
[0238] Therefore, it can make it easier for users (such as people with color vision diversity) to identify the actual color of an object.
[0239] Furthermore, in the information processing apparatus according to the embodiment, the presentation processing unit presents a color image as color vision aid information.
[0240] Using image information as color vision aids allows users to intuitively understand the actual color of an object.
[0241] Furthermore, in the information processing apparatus according to the embodiment, the presentation processing unit presents a magnified image of the object as color vision aid information.
[0242] For example, when the color pattern of an object is intricate, or when the object is small, it can be difficult for someone with a diverse color vision to discern the colors in the pattern when viewing the object with the naked eye. Therefore, presenting a magnified image of the object serves as supplementary color vision information.
[0243] Therefore, even if the color arrangement pattern in an object is intricate, users (such as those with diverse color vision) can easily identify the actual colors of the color arrangement pattern.
[0244] Furthermore, the information processing apparatus according to the embodiment includes a behavior detection unit (behavior detection unit F2) configured to detect the user's behavior relative to an object, wherein the presentation processing unit presents information element levels as color vision aid information in response to a type switching of the behavior.
[0245] Therefore, information can be presented at the level of information elements implicitly requested by users through their natural behavior, which can help users understand objects not only from the perspective of color perception but also from perspectives other than color perception. Furthermore, the presentation of auxiliary information based on users' natural behavior enables accessibility that is highly compatible with users' daily lives.
[0246] Furthermore, in the information processing apparatus according to the embodiment, when the type of behavior detected by the behavior detection unit is a first type, the presentation processing unit only presents the information indicating the color as color vision aid information, and when the type of behavior detected by the behavior detection unit is a second type different from the first type, the presentation processing unit presents the information indicating the color and other information besides the information indicating the color as presentation.
[0247] The presentation processing unit can present appropriate information corresponding to each of the following: the user expects to identify the color of the object, and the user expects to identify elements other than the color of the object.
[0248] Therefore, information can be presented according to the level of information elements implicitly required by users through their natural behavior, and it can help users understand objects not only from the perspective of color vision but also from perspectives other than color vision.
[0249] Furthermore, in the information processing apparatus according to the embodiment, when the type of behavior detected by the behavior detection unit is a first type, the presentation processing unit only presents a color image of the color of the indicated object as color vision aid information; and when the type of behavior detected by the behavior detection unit is a second type different from the first type, the presentation processing unit displays a color image of the color of the indicated object and a color image indicating the color corresponding to the color requested by the user side by side as presentation.
[0250] Therefore, for example, in cases where there is a color requested by the user in an object such as a commodity, it can facilitate a comparison between the color requested by the user and the color of the object.
[0251] Therefore, it can appropriately assist users in having a higher-level understanding of whether an object is the object the user expects in terms of hue.
[0252] Furthermore, the information processing apparatus according to the embodiment includes a characteristic information acquisition unit (characteristic information acquisition unit F1), which acquires color vision characteristic information based on information input by the user.
[0253] Therefore, it is possible to receive information input from the user specifying color vision characteristics and obtain the user's color vision characteristic information from the information.
[0254] Therefore, color vision assistance based on the individual user's color vision characteristics can be implemented, and the accuracy of color vision assistance can be improved.
[0255] Furthermore, in the information processing apparatus according to the embodiment, the characteristic information acquisition unit displays a reference color and a sample color on the display screen, allows the user to select a sample color that looks the same as the reference color, and obtains color vision characteristic information based on the information of the selected sample color.
[0256] As described above, users can select a sample color that looks the same as the reference color, and thus information about the type of color vision diversity can be obtained from the difference between the sample color and the reference color as information about the user's color vision characteristics.
[0257] Therefore, as a color vision aid, it is possible to perform appropriate assistance based on the user's color vision characteristics.
[0258] Furthermore, in the information processing apparatus according to the embodiment, the feature information acquisition unit allows the user to select a sample color that looks the same as the reference color, while sequentially changing the sample colors displayed side by side with the reference color on the display screen.
[0259] This eliminates the need for the user to change the sample color during calibration to obtain information about the user's color vision. Furthermore, since the sample color and reference color are displayed side-by-side, it is easy for the user to compare the reference color with the sample color and to obtain accurate color vision information.
[0260] Therefore, the accuracy of calibration can be improved, while reducing the burden of user operations related to calibration, and ultimately improving the precision of color vision assistance.
[0261] Furthermore, the information processing apparatus according to the embodiment includes an auxiliary information generation unit (auxiliary information generation unit F3), which can generate color vision auxiliary information based on color vision characteristic information.
[0262] Therefore, the processes of generating and presenting color vision auxiliary information can be performed by a single device.
[0263] Furthermore, the information processing apparatus according to the embodiment includes a presentation unit (presentation unit 17) that can present color vision assist information in response to the processing of the presentation unit.
[0264] Therefore, the processes of generating and presenting color vision auxiliary information can be performed by a single device.
[0265] Furthermore, the information processing apparatus according to the embodiment includes an imaging unit (imaging unit 21) that acquires a captured image of the object.
[0266] Therefore, a series of operations, including capturing an image of an object, generating color vision aids associated with the object, and presenting the generated color vision aids, can be accomplished by a single device.
[0267] Furthermore, the information processing method according to the embodiment is as follows: in this information processing method, the information processing device presents color vision assist information generated based on the captured image of the object and the user's color vision characteristic information to the user.
[0268] Using this information processing method, an information processing apparatus as described above in the embodiments can be implemented.
[0269] Furthermore, the color vision assist system according to the embodiments includes: an imaging unit configured to capture an image of an object; an assist information generation unit configured to generate color vision assist information based on the captured image of the object obtained by the imaging unit and the user's color vision characteristic information; and a presentation processing unit configured to present the color vision assist information generated by the assist information generation unit to the user.
[0270] As described above, color vision aids generated based on the captured image of the object and the user's color vision characteristics are presented to the user, thus helping the user to easily identify the color of the object they actually see.
[0271] Therefore, even when the user has a variety of color vision, it enables the user to accurately identify the color of the object actually seen.
[0272] Note that the effects described in this manual are merely examples and are not limited thereto; other effects may also exist.
[0273] <8. This technology>
[0274] Note that this technology can also have the following configurations: (1)
[0276] An information processing apparatus, comprising:
[0277] A presentation processing unit is configured to present color vision aid information generated based on a captured image of an object and the user's color vision characteristics to the user. (2)
[0279] According to the information processing device described in (1) above, wherein
[0280] The presentation processing unit presents information indicating the color as the color vision aid information. (3)
[0282] According to the information processing device described in (2) above, wherein
[0283] The presentation processing unit presents information indicating a color different from the color of the object as the color vision aid information. (4)
[0285] According to the information processing device described in (2) or (3) above, wherein
[0286] The presentation processing unit presents information indicating the color determined based on the difference between the color of the object and the color perceived by the user when perceiving the color of the object as the color vision aid information. (5)
[0288] The information processing apparatus according to any one of (2) to (4) above, wherein
[0289] The presentation processing unit presents a color image as the color vision aid information. (6)
[0291] The information processing apparatus according to any one of (1) to (5) above, wherein
[0292] The presentation processing unit presents a magnified image of the object as the presentation of the color vision auxiliary information. (7)
[0294] The information processing apparatus according to any one of (1) to (6) further includes
[0295] A behavior detection unit, configured to detect the user's behavior relative to the object.
[0296] The presentation processing unit switches the information element level of the information presented as color vision aid information in response to the type of the behavior. (8)
[0298] According to the information processing apparatus described in (7), wherein
[0299] When the behavior detected by the behavior detection unit is of the first type, the presentation processing unit only presents information indicating the color as the presentation of the color vision aid information; and when the behavior detected by the behavior detection unit is of the second type, which is different from the first type, the presentation processing unit presents information indicating the color and different information other than information indicating the color as the presentation. (9)
[0301] According to the information processing apparatus described in (7), wherein
[0302] When the behavior detected by the behavior detection unit is of the first type, the presentation processing unit only presents a color image indicating the color of the object as the presentation of the color vision aid information; and when the behavior detected by the behavior detection unit is of the second type, which is different from the first type, the presentation processing unit displays a color image indicating the color of the object and a color image indicating the color corresponding to the color requested by the user as presentation. (10)
[0304] The information processing apparatus according to any one of (1) to (9) above further includes
[0305] A characteristic information acquisition unit is configured to acquire the color vision characteristic information based on information input by the user. (11)
[0307] According to the information processing device described above (10), wherein
[0308] The characteristic information acquisition unit causes the display screen to show a reference color and a sample color, allowing the user to select a sample color that looks the same as the reference color, and obtain the color vision characteristic information based on the information of the selected sample color. (12)
[0310] According to the information processing device described above (11), wherein
[0311] The feature information acquisition unit allows the user to select a sample color that looks the same as the reference color by sequentially changing the sample colors displayed side by side with the reference color on the display screen. (13)
[0313] The information processing apparatus according to any one of (1) to (12), wherein
[0314] An auxiliary information generation unit is configured to generate color vision auxiliary information based on the color vision characteristic information. (14)
[0316] The information processing device described in (13) above also includes
[0317] A presentation unit configured to present the color vision assist information in response to presentation by the presentation processing unit. (15)
[0319] The information processing device described in (14) above also includes
[0320] An imaging unit configured to acquire a captured image of the object. (16)
[0322] An information processing method, the information processing method being configured to:
[0323] The information processing device presents color vision aids generated based on the captured image of the object and the user's color vision characteristics to the user.
[0324] (17) A color vision assist system, comprising
[0325] An imaging unit configured to capture an image of an object;
[0326] An auxiliary information generation unit is configured to generate color vision auxiliary information based on a captured image of an object obtained by the imaging unit and color vision characteristic information of a user; and
[0327] A presentation processing unit is configured to present the color vision assist information generated by the assist information generation unit to the user.
[0328] List of reference numerals
[0329] 1, 1A, 1B, 1C, 1D, 1E Information processing devices
[0330] 1a Main body
[0331] 1b with part
[0332] 11 CPU
[0333] 17 Display Units
[0334] 17a display screen
[0335] 18 Voice output units
[0336] 21 imaging units
[0337] 22 sensor units
[0338] F1 Feature Information Acquisition Unit
[0339] F2 Behavior Detection Unit
[0340] F3 Auxiliary Information Generation Unit
[0341] F4 Presentation Processing Unit
[0342] B1 Start button
[0343] B2 Stop button
[0344] B3 Reset button
[0345] B4 Next button
[0346] M aiming mark
[0347] 30, 30D device
[0348] 31 Network
[0349] 32 server devices
Claims
1. An information processing apparatus, comprising: A presentation processing unit is configured to present color vision aid information generated based on a captured image of an object and the user's color vision characteristics to the user; as well as A behavior detection unit, configured to detect the user's behavior relative to the object. The presentation processing unit switches the information element level of the information presented as color vision aid information according to the type of behavior.
2. The information processing device according to claim 1, wherein, The presentation processing unit presents information indicating the color as the presentation of the color vision auxiliary information.
3. The information processing device according to claim 2, wherein The presentation processing unit presents information indicating a color different from the color of the object as the presentation of color vision aid information.
4. The information processing device according to claim 2, wherein, The presentation processing unit presents information indicating the color determined based on the difference between the color of the object and the color perceived by the user when perceiving the color of the object as the color vision aid information.
5. The information processing device according to claim 2, in, The presentation processing unit presents a color image as the color vision aid information.
6. The information processing apparatus according to claim 1, in, The presentation processing unit presents a magnified image of the object as the presentation of the color vision auxiliary information.
7. The information processing apparatus according to claim 1, in, When the behavior detected by the behavior detection unit is of the first type, the presentation processing unit only presents the information indicating the color as the presentation of the color vision auxiliary information; And when the behavior detected by the behavior detection unit is of a second type that is different from the first type, the presentation processing unit presents information indicating the color and different information other than the information indicating the color as presentation.
8. The information processing apparatus according to claim 1, in, When the behavior detected by the behavior detection unit is of the first type, the presentation processing unit only presents a color image indicating the color of the object as the presentation of the color vision auxiliary information; And if the behavior detected by the behavior detection unit is of a second type that is different from the first type, the presentation processing unit displays side by side a color image indicating the color of the object and a color image indicating the color corresponding to the color requested by the user as a presentation.
9. The information processing apparatus according to claim 1, further comprising: A characteristic information acquisition unit is configured to acquire the color vision characteristic information based on information input by the user.
10. The information processing apparatus according to claim 9, in, The characteristic information acquisition unit causes the display screen to show a reference color and a sample color, allowing the user to select a sample color that looks the same as the reference color, and obtain the color vision characteristic information based on the information of the selected sample color.
11. The information processing apparatus according to claim 10, in, The feature information acquisition unit allows the user to select a sample color that looks the same as the reference color by sequentially changing the sample colors displayed side by side with the reference color on the display screen.
12. The information processing apparatus according to claim 1, further comprising: An auxiliary information generation unit is configured to generate color vision auxiliary information based on the color vision characteristic information.
13. The information processing apparatus according to claim 12, further comprising: A presentation unit configured to present the color vision assist information in response to the presentation by the presentation processing unit.
14. The information processing apparatus according to claim 13, further comprising: An imaging unit configured to acquire a captured image of the object.
15. An information processing method, the information processing method being configured to: The information processing device presents color vision aids generated based on the captured image of the object and the user's color vision characteristics to the user; and The information processing device detects the user's behavior relative to the object and switches the information element level of the information presented as color vision aid information according to the type of detected behavior.
16. A color vision assist system, comprising: An imaging unit configured to capture an image of an object; An auxiliary information generation unit is configured to generate color vision auxiliary information based on a captured image of an object obtained by the imaging unit and color vision characteristic information of a user. A presentation processing unit is configured to present the color vision assistance information generated by the assistance information generation unit to the user. as well as A behavior detection unit, configured to detect the user's behavior relative to the object. The presentation processing unit switches the information element level of the information presented as color vision aid information according to the type of behavior.
Citation Information
Patent Citations
Image processor, image processing method, program, and recording medium
JP2009071541A
Apparatus, method and program for color vision support
JP2005177144A
Color conversion device, color conversion system and program
JP2017085461A
