Color blindness assistance technology systems and methods

Abstract

A system and method for providing a colorblind-accessible version of a color image are disclosed. For a given image, the pixel values ​​that a colorblind user would see are determined. Using these values, a corrected rendered image is generated and presented to the colorblind user, the corrected rendered image appearing to the colorblind user as depicting a color scheme similar to the original image. The proposed system can be implemented using previously captured image data, real-time video, and additional features for other applications.

Description

[0001] This application is a divisional application of Chinese patent application No. 202180055045.7, filed on May 20, 2021, entitled "Color Blindness Assistive Technology System and Method". Background Technology

[0002] It is estimated that 15% of the world's population has a disability. In many cases, people with disabilities are marginalized in the socioeconomic activities of their communities. People with disabilities are also considered less likely to participate in sports, recreation, and leisure activities than those without disabilities. A subset of people with disabilities includes those who have difficulty distinguishing one color from another. This condition is called "color vision deficiency," commonly known as "color blindness." There are several different forms of color vision deficiency, including red-green dichromaticism (red-blindness, green-blindness), aberrant red-green trichromaticism (red-weakness and green-weakness), blue-yellow dichromaticism (blue-blindness), and aberrant blue-yellow trichromaticism (blue-weakness). Each form is caused by the expression of a recessive genetic trait that reduces the diversity of retinal cones in the eyes of those affected, or makes specific cones less sensitive and / or more sensitive to changes in the wavelength range. These traits are primarily carried on the Y chromosome and may affect 7%–10% of the male population and about 0.5% of the female population. Total color blindness (achromatopsia) is also considered a color vision deficiency associated with impairment.

[0003] In daily life, color vision deficiency may be associated with some degree of disability. For example, it may impair an affected person's ability to decipher information in images and media or other color-based content. It may disqualify them from employment in fields requiring keen color vision. Furthermore, color vision deficiency may hinder an affected person's overall perception and enjoyment of the visual world. Unfortunately, there is currently no cure or treatment for color vision deficiency.

[0004] Currently, thousands of assistive technology products are available on the market to help people with disabilities with a wide range of needs, from simple to complex. However, improvements in accessibility for colorblind individuals remain limited, and the challenges of facilitating the differentiation of different colors have not been adequately addressed. For example, many conventional solutions modify user interface color schemes to use high-contrast colors to allow colorblind individuals to more easily distinguish different displayed items. However, such solutions have significant drawbacks in allowing colorblind individuals to fully and accurately appreciate the more complex and subtle differences in colors presented in realistic image content. Conversely, colorblind individuals are unable to perform color-related tasks that non-colorblind individuals can easily perform, such as recognizing color differences and accurately identifying colors across the color spectrum. Therefore, technical challenges remain in assistive technology, and new technological solutions and improvements will enable colorblind individuals to perceive the world around them more comprehensively. Summary of the Invention

[0005] According to a first aspect of this disclosure, a system suitable for improving image rendering for users with color vision deficiencies includes a processor and a computer-readable medium including instructions that, when executed by the processor, cause the processor to obtain a source image comprising a plurality of source pixels. The instructions also cause the processor, for each render pixel included in the plurality of render pixels, to generate a render image comprising the plurality of render pixels by causing the processor to perform the following operations: selecting a source pixel included in the plurality of source pixels that corresponds to the render pixel; obtaining a first color value for the selected source pixel; identifying a first render color value, the first render color value being estimated to be perceived by the user as having the first color value when rendered by a display device viewed by a user with a first color vision deficiency; and determining the color of the render pixel based on the identified first render color value. Finally, the instructions cause the processor to cause the generated render image to be rendered by a display device.

[0006] According to a second aspect of this disclosure, a method for improving image rendering for users with color vision deficiencies includes a first step: obtaining a source image comprising a plurality of source pixels. The method further includes a second step: generating a rendered image comprising the plurality of rendered pixels by performing a series of sub-steps for each rendered pixel included in the plurality of rendered pixels. These sub-steps include: selecting a source pixel included in the plurality of source pixels that corresponds to a rendered pixel; obtaining a first color value for the selected source pixel; identifying a first rendered color value, the first rendered color value being estimated to be perceived by the user as having the first color value when rendered by a display device viewed by a user with a first color vision deficiency; and determining the color of the rendered pixel based on the identified first rendered color value. Furthermore, the method includes a third step: rendering the generated rendered image by a first display device.

[0007] According to a third aspect of this disclosure, a method for generating a color vision deficiency image conversion model includes a first step: selecting a plurality of source color values. The method also includes a second step: generating a plurality of estimated color values ​​associated with the source color values ​​through a series of sub-steps performed for each source color value included in the plurality of source color values. These sub-steps include: simulating the source color values ​​such that the source color values ​​rendered by a display device would be perceived by a user with a first color vision deficiency as having a first estimated color value included in the plurality of estimates; and associating the source color values ​​with the estimated color values. The method further includes a third step: selecting a plurality of target color values. Additionally, for each target color value included in the plurality of target color values, the method includes: selecting a second estimated color value from the plurality of estimated color values ​​that matches the target color value; selecting a rendered color value from the plurality of source color values ​​that is associated with the second estimated color value; and storing an association between the target color value and the rendered color value.

[0008] This summary is provided to introduce a selection of concepts in a simplified form, which are further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that address any or all of the shortcomings mentioned in any part of this disclosure. Attached Figure Description

[0009] The accompanying drawings illustrate one or more implementations of this teaching by way of example only and not as a limitation. In the drawings, similar reference numerals refer to the same or similar elements. Furthermore, it should be understood that the drawings are not necessarily drawn to scale. Figure 1A and Figure 1B An example of a color correction system and method is illustrated;

[0010] Figure 1A and Figure 1B An example of a color correction system and method is illustrated;

[0011] Figure 2 It is a conceptual diagram illustrating the appearance of a chart as seen by both people with normal color vision and those with color blindness.

[0012] Figures 3A-3C It is a flowchart showing a method for rendering color-accessible images for colorblind users to view;

[0013] Figures 4A-4C The illustration shows an example of a color correction tool implemented during a dynamic video capture session;

[0014] Figure 5A and Figure 5B The illustration shows an example of a color correction tool implemented using a static digital image;

[0015] Figures 6A-6C This is an example of a color adjustment tool used in conjunction with the implementation of a color correction system;

[0016] Figure 7 It is a flowchart illustrating one implementation method of the operation of a color correction tool;

[0017] Figure 8 This is a block diagram illustrating an exemplary software architecture, the various parts of which can be used in conjunction with various hardware architectures described herein, which can implement any of the features described herein; and

[0018] Figure 9 This is a block diagram illustrating components of an exemplary machine configured to read instructions from a machine-readable medium and execute any of the features described herein. Detailed Implementation

[0019] In the detailed description below, numerous specific details are illustrated by way of examples to provide a thorough understanding of the teachings. However, it will be clear that these teachings can be practiced without these details. In other instances, well-known methods, processes, components, and / or circuits have been described at a relatively high level without detailed explanation to avoid unnecessarily obscuring aspects of these teachings. In the material below, directional indications, such as “top” or “left”, are merely to provide a frame of reference during the discussion below and are not intended to indicate the desired, expected, or anticipated orientation of the described objects.

[0020] The detailed description below illustrates many specific details by way of examples to provide a thorough understanding of the relevant teachings. However, it will be clear that the current teachings can be practiced without these details. In other cases, well-known methods, processes, components, and / or circuits have already been described at a relatively high level without detailed explanation to avoid unnecessarily obscuring aspects of this teaching.

[0021] Color blindness, formally known as color vision deficiency (CVD), affects approximately 8% of men and 0.8% of women worldwide. Color blindness causes affected individuals to have difficulty distinguishing specific color combinations and differences. Generally, colorblind viewers lack the necessary physical components that enable them to differentiate and detect specific colors. Due to the loss of color information, many visual objects, such as images and videos, which appear to have high color quality to unaffected viewers, are often not fully appreciated by colorblind individuals. As mentioned above, red-blind and green-blind individuals have difficulty distinguishing between red and green hues, while blue-blind individuals have difficulty distinguishing between blue and yellow hues. Regardless of the specific type of color vision deficiency, colorblind viewers may experience difficulty searching for parts of an image that contain a specific color (such as a red apple). For example, a colorblind viewer may not be able to distinguish whether an apple in an image is red or green.

[0022] The following description presents various implementations of technical solutions and improvements in the form of methods and systems for promoting inclusion and accessibility for color-impaired individuals in perceiving colors and distinguishing objects or patterns with multiple colors or hues. These implementations can be used to convert an original first image into a recolored and optimized rendered second image for viewing by a color-blind user. For example, the color values ​​of the first image are automatically adjusted with reference to a color conversion model (in some implementations, this model may be implemented using one or more lookup tables) to allow the color-blind user to better distinguish and / or identify specific colors in the image. In one implementation, the image can be modified to produce a perception for the color-blind user that is close to that perceived by a user with normal color vision. The systems described herein are applicable to a wide range of applications and industries, including military, education, entertainment, research, and healthcare.

[0023] In various implementations, the proposed techniques can be used to transform or “convert” the appearance of digital images or other electronic content intended for viewing by colorblind users. Generally, the terms “electronic content” or “image” include any digital data or information that can be visually represented, including but not limited to: electronic documents, media streams, real-time video capture, real-time image display, documents, web pages, hypertext documents, any image, digital video or video recording, animation, and other digital data. As an example, the electronic content may include images captured by a photography application or other software configured to provide users with tools for viewing digital images. Therefore, the use of the term “image” should be understood to encompass all types of dynamic and static electronic content that can be visually presented to an end user. Furthermore, references to the “realistic,” “original,” or “normal” appearance of a real-world scene or image describe how visual information would be rendered to a person with normal color vision, while “rendered,” “recolored,” “virtual,” or “accessible” images describe images altered for viewing by colorblind users using embodiments of the techniques described herein.

[0024] Furthermore, end users include colorblind users of the application, as well as the devices and systems described herein. For the purposes of this description, the terms "software application," "software," or "application" refer to a computer program that performs useful work and is generally independent of the computer itself. Some non-limiting examples of software applications include photography software, image capture / editing applications, word processors, spreadsheets, PowerPoint presentations, presentation design applications, billing systems, and telecommunications programs, as well as game software, utilities and productivity tools, mobile applications, presentation graphics, and other productivity software. Some embodiments of the proposed examples can be implemented as standalone applications, while others can be combined with or integrated with another program. In other words, a user can access websites, documents, or other electronic content via a first application and use the proposed system to convert one or more colors in the electronic content into different colors that a colorblind person would perceive.

[0025] In some implementations, software applications incorporating the disclosed features can be installed on client devices or associated with third-party applications, such as web browser applications configured to communicate with the device. These devices can include, for example, desktop computers, mobile computers, mobile communication devices (such as mobile phones, smartphones, tablets, etc.), smart TVs, gaming devices, set-top boxes, and / or any other computing device including a camera and / or image display capabilities. Typically, such applications allow end users to capture or scan documents, presentations, real-world objects, and other objects using images received by a camera or stored or accessed from memory. Furthermore, in some implementations, camera-based scanning applications can be configured to implement the CVD color correction techniques described herein. Image transformation can occur in real time (e.g., when the camera is pointed at a scene or(s) of an object) and / or after an image is captured, generated, or stored in memory.

[0026] For simplicity, the following implementations discuss the use of the system within the context of mobile computing devices such as mobile phones and tablets. However, any electronic device with a camera can benefit from the use of these systems. These devices can provide the user with several input mechanisms, including a home button, power button, mode button, and / or camera shutter (image capture) button, which can be installed as hardware or available via a touchscreen display capable of displaying the touchscreen camera shutter button. In some cases, the user can choose to use the touchscreen camera shutter button instead of the mechanical camera shutter button. When input mechanisms are provided via a touchscreen display, additional options can also be used to control a subset of the image capture functionality. In different implementations, such controls can include still image capture mode buttons, video capture mode buttons, automatic image capture mode buttons, zoom in and out controls, and option or setting controls.

[0027] It is recognized that the use of color is ubiquitous in modem interfaces and graphical displays. Color is used to represent a variety of meanings, including data categories, highlighting, continuums, and specific values. While color is a valuable tool for representation and visualization, many users struggle to distinguish the colors used on a screen in many different situations. The inability to distinguish between two colors can have substantial consequences. These problems can range from annoyance and frustration (e.g., if the color of an "access link" in a web browser is indistinguishable from a normal link) to serious errors or security issues (e.g., matching colors between a bar chart and its legend, or identifying warning colors in a background). The implementations described herein promote the social inclusion of colorblind users. For the purposes of this application, the term "social inclusion" will refer to processes that improve the conditions for individuals and groups to participate in society—improving the capabilities, opportunities, and dignity of those disadvantaged by their identity. One of the most influential intrinsic barriers stems from the attitudes and motivations of visually impaired individuals, specifically self-awareness and low levels of self-confidence. Color vision deficiency is an adversarial condition that may be less noticeable to others but has a significant impact on those affected. For the purposes of the colorblindness simulation and rendering techniques described in this paper, colorblindness accessibility of images or real-world objects should be understood as referring to the degree to which a colorblind individual can perceive an image.

[0028] Therefore, the colorblind accessibility of an image can be understood as generally depending on the degree to which important information is lost in colorblind perception. In general, even high-quality images can have low accessibility due to the loss of color information perceived by colorblind individuals, often caused by the inability to distinguish colors. Although most interface design guidelines recommend using redundant coding in addition to color, many examples in information visualization and graphical interface design fail to follow this principle. Since up to ten percent of the world's population suffers from CVD to some degree, addressing the color discrimination problem could significantly improve usability for a wide range of users.

[0029] Technical problems, technical solutions, and technical improvements can be addressed through various means. Figure 1A-9 The examples shown in the illustrations are used to describe this. As a general overview, Figure 1A and Figure 1B This paper presents one implementation of a system and method for applying virtual color blindness correction technology in a real-world setting. For illustrative purposes, colors in the accompanying drawings are represented by different styles or types of cross-shading and dotting. Figure 1A In the image, a visitor 110 at museum 100 is shown looking at painting 120. Painting 120 comprises several circular objects colored in a specific pattern. However, for visitor 110 who is red-green colorblind, the complete or fully colored pattern is not apparent (see [reference needed]). Figure 2 ). Figure 1AThe painting 120 is shown as perceived by visitor 110, and accordingly, when three colors are actually present, only two types of perceived color are included. Figure 1A In the painting, the first perceptual color 122 (shown using first-type cross shading) is correctly perceived as blue by visitor 110. However, the second perceptual color 124 (shown using first-type dotted lines) is incorrectly perceived as dark brown by visitor 110. However, in reality, when viewed by someone with normal color vision, the portion of painting 120 perceived as the second perceptual color 124 actually comprises two distinct colors—red and green. Therefore, for visitor 110, the actual pattern of red, green, and blue units of approximately equal shape and size within each circle is perceived by visitor 110 as smaller blue units adjacent to larger dark brown U-shaped segments.

[0030] Now for reference Figure 1B Visitor 110 has accessed an implementation of CVD-assisted technology, including a color-modification application (“app”) 130 for CVD users executed on mobile device 140. Visitor 110 captures an image of painting 120 using a camera included in mobile device 140, which is received by app 130 for processing. App 130 then generates and displays a virtual view 150 comprising a color-modified version of the captured image, which has been modified such that, as perceived by visitor 110, red and green can now be distinguished by visitor 110 as a third perceptual color 126 (shown using a second type of cross shading) and a fourth perceptual color 128 (shown using a third type of cross shading), respectively. In some examples, the third perceptual color 126 is perceived by visitor 110 as a red similar to actual red in the painting, as would be seen through normal color vision, and the fourth perceptual color 128 is perceived by visitor 110 as a green similar to actual green in the painting, as would be seen through normal color vision. The brown secondary perceptual color 124, which the visitor 110 previously perceived when directly viewing the actual painting 120, is no longer perceived by the visitor 110 within the virtual view 150 (rendered image) of the same painting 120. Therefore, the visitor 110 is now able to recognize the presence of three distinct colors in the painting 120, instead of the two perceptual colors 122 and 124 when directly viewing the painting, the true nature of these colors (e.g., the visitor 110 can perceive the third perceptual color 126 as having a reddish hue and the fourth perceptual color 128 as having a greenish hue), and appreciate the complete pattern revealed by the arrangement of these three colors.

[0031] As mentioned above, although a large portion of the general population has some form of color vision, color discrimination remains a common means of presenting information. Since normal visual processing in the human visual system typically allows for rapid color recognition, using color to label objects—at least for those with normal color vision—allows for the quick and efficient identification and differentiation of categorical information. As an example, in categorical coding, a unique color is assigned to each category of data, and then all representations of that category in a visualization will use that color as a distinguishing feature. Color as a category is used for many information displays, including charts in spreadsheets, "link capture" encoding in web browsers, syntax highlighting in text editors, and tagged messages in email clients.

[0032] To more clearly emphasize some of the impacts of color blindness on accessing and utilizing color-coded information Figure 2 The illustration compares the perception of information based on the same image between a user with normal color vision and a CVD user. First image 210 depicts the original "real" image of the pie chart as perceived by a user with normal color vision, while second image 220 depicts how the pie chart would look to a CVD user (e.g., someone with green-green color blindness or a red-green color defect). In this example, first image 210 includes four "slices," wedges, or triangles within a large circle, which are recognizable because four different colors have been used (in...). Figure 2 (This is represented using four different cross-shading styles). Therefore, the boundaries between the first segment 212 (with red), the second segment 214 (with green), the third segment 216 (with light blue), and the fourth segment 218 (with dark blue) are easily distinguishable for those with normal color vision because each segment is a different color. In other words, there are no outlines around the segments; the only indication of the boundaries between segments is based on the use of different colors. In some cases, each segment can correspond to a corresponding hyperlink or other clickable command, where the user's selection and / or actuation of each colored segment is configured to trigger the presentation of information associated with that segment.

[0033] While this may be a simple and clear infographic tool for most people with normal vision, it can lead to the loss of important information and comprehension for those with CVD. Figure 2The second image 220 is a simplified demonstration of this information loss (represented by three different dotted patterns). Because the first image 210 comprises two adjacent segments 212 and 214 in red and green, there is a noticeable loss of division or boundary between the two segments when viewed by a CVD user, making the first segment 212 and the second segment 214 appear to the CVD user as merged into a single fifth segment 222. The perceptual coloring of the sixth segment 224 and the seventh segment 226 is also affected, appearing different from their corresponding portions in the first image 210 (the third segment 216 and the fourth segment 218, respectively). Changes in the blue of the sixth segment 224 and the seventh segment 226 may have little effect on the information being communicated, as the two segments maintain their distinct appearance; however, in other cases, CVD visual perception can lead to basic information not being properly conveyed and understood.

[0034] More specifically, red-green colorblind viewers will mistakenly perceive the pie chart as limited to three segments (instead of four), and one of these segments will appear to be significantly larger than it actually is. Furthermore, colorblind users will be unaware of the additional hyperlinks available to them, and depending on the mouse cursor position, selecting the fifth segment 222 might trigger information for either the first segment 212 or the second segment 214 without realizing that this information is ambiguous. Moreover, the differences in color intensity from the third segment 216 to the sixth segment 224, and also from the fourth segment 218 to the seventh segment 226, can lead to misinterpretation of information or otherwise hinder the viewer's ability to understand it.

[0035] In many cases, color can also be used to draw the user's attention to specific items or data points, rather than carrying the data itself. In these situations, those with CVD (Color Vision Disorder) are also at a disadvantage. For example, objects considered "special" might be colored differently to distinguish them within a larger group (e.g., color pops). The popped color must be sufficiently different from the other colors in the visualization for the effect to work. Typically, saturated, bright primary colors are used to replace established element colors. Pops allow for quick identification and location of important items. Similarly, highlighting uses color to focus attention on an element or area of ​​visualization. Unlike pops, highlighting does not replace element colors in the visualization but rather surrounds the element of interest. Therefore, desaturated colors are often used to prevent highlighting from obscuring the highlighted item. Color is also frequently used as a means of encoding univariate or multivariate data. For example, the depth of a body of water can be encoded using shades of blue, where darker blue indicates deep water and lighter blue indicates shallow water. Another challenge with these applications of tonal scales is simultaneous contrast, which occurs when the perception of color is influenced by surrounding colors. In the case of creators of such interfaces with normal color vision, insufficient understanding of CVD-related issues can lead to an inability to convey the intended information to CVD users.

[0036] Therefore, it is recognized that the ability to distinguish and / or accurately identify colors is of paramount importance. As will be described in more detail below, the virtual color correction technology described herein provides adaptation to the limitations of color perception experienced by CVD users. This technology processes image or video input and modifies color presentation to facilitate color distinction and / or recognition for individuals with CVD. Figures 3A-3C Three phases 310, 320, and 330 of an exemplary method for generating a rendered image for a source image, which improves color perception for colorblind users, are presented. Figure 3A In the first stage 310 shown, the perception of color values ​​by a viewer with color vision deficiency is simulated. This first stage 310 may include a first step 312 of selecting multiple source color values ​​(which may be referred to as "source color values"). In some examples, the color values ​​are multi-component colors. Examples of multi-component color values ​​include, but are not limited to: three-component RGB color values ​​(specifying values ​​for the red, green, and blue components), two-component RG color values ​​(specifying values ​​for the red and green components), and two-component BG color values ​​(specifying values ​​for the blue and green components). In some examples, multi-component color values ​​may also be referred to as multi-channel color values. In some implementations, the color values ​​of pixels in the source image may be used to identify the source color values. In some implementations, the source color values ​​may be a generally applicable range of color values ​​(e.g., all possible combinations of color component values ​​or sparse combinations of color component values).

[0037] The first stage 310 may include a second step 314, which simulates the source color value of the first step 312, as it will be perceived by a viewer with color vision deficiencies to obtain a corresponding estimated color value (which may be referred to as the "simulated color value"). Various different color vision deficiencies exist that can be simulated. In some implementations, the simulation of the perception of the source color value includes the simulation of the color rendering characteristics of the display device when rendering the source color value. For example, this may take into account nonlinearities in the color component intensities and / or spectral characteristics of the display device. Below is a list of exemplary pseudocode examples for several types of color vision deficiencies that generate simulated color component values ​​"red_sim", "green_sim", and "blue_sim" from the source color component values ​​"red", "green", and "blue". Pseudocode List 1: Simulation of Green Amblyopia Anomaly Pseudocode List 2: Simulation of Red-Colored Amblyopia Pseudocode List 3: Green-Blindness Simulation Pseudocode List 4: Red-Blindness Simulation

[0038] The first stage 310 may include a third step 316 of associating each estimated color value with its respective source color value. For example, these associations may be stored in memory as an array or list of estimated color values ​​and / or source color values. An exemplary implementation of the first stage 310 for three-component RGB color values ​​is shown below in pseudocode listing 5. Pseudocode listing 6 shows another exemplary implementation of the first stage 310 for two-component RG and BG values, which requires significantly less memory than the example in pseudocode listing 5. In both pseudocode listings 5 ​​and 6, 8-bit color component values ​​ranging from 0 to 255 are used, and all possible combinations of color component values ​​are simulated. It is understood that although pseudocode listings 5 ​​and 6 are shown for green amblyopia, as illustrated in pseudocode listing 1, pseudocode listings 5 ​​and 6 can be modified for other CVD simulations, such as those shown in pseudocode listings 2, 3, and 4. Pseudocode Listing 5: Generation of a Single RGB Simulation Table Pseudocode Listing 6: Generation of RG and BG Color Conversion Tables

[0039] See next. Figure 3B The second stage 320 of the method is presented as an implementation whereby simulated color values ​​are used to generate a transformation from target color values ​​(which may be multi-component color values) to corresponding rendered color values ​​(which may be multi-component color values) for color vision deficiencies. The second stage 320 may include a fourth step 322 of selecting multiple target multi-component color values ​​(which may be multi-component color values), similar to the selection of source color values ​​in the first step 312. In some implementations, sparse combinations of target color values ​​may be used in conjunction with interpolation combinations for color values ​​not included in the selected target color values. The second stage 320 may include a fifth step 324: for each target color value included in the target color values ​​from the fourth step 322, determining a corresponding color distance between the target color value and each estimated color value obtained in the second step 314. Various schemes can be used to calculate the color distance, such as, but not limited to, the squared difference or sum of squared differences of some or all color component values, or the absolute difference or sum of absolute differences of some or all color component values. In the examples shown in pseudocode listings 7 and 8 below, the color distance is calculated as the sum of the squared differences of all color component values.

[0040] The second stage 320 may include a sixth step 326: for each target color value included in the target color values, identifying a source color value associated with the estimated color value having the minimum color distance determined in the fifth step 324. The second stage 320 may include a seventh step 328: storing the association between each target color value and its respective source color value (which may be referred to as a "render color value") identified in the sixth step 326. For example, these associations may be stored in memory as an array or list of target and / or render color values. In some implementations, these associations may be stored in a texture for use by a segmented shading procedure executed by the GPU (Graphics Processing Unit).

[0041] In pseudocode list 7 below (which corresponds to pseudocode list 5 above), an exemplary implementation of the second stage 320 for three-component RGB color values ​​is shown. Pseudocode list 8 (which corresponds to pseudocode list 6 above) shows another exemplary implementation of the second stage 320 for two-component RG and BG values. Pseudocode Listing 7: Generation of a Single RGB Color Conversion Table Pseudocode List 8: Generation of RG and BG Color Conversion Tables

[0042] Finally, Figure 3C The document presents an implementation of the third stage 330 of the method, in which a rendered image is generated to improve the color perception of a CVD user for a source image. The third stage 330 includes an eighth step 332, in which the source image is obtained and color values ​​for the pixels of the source image are acquired. The third stage 330 includes a ninth step 334: identifying CVD-specific rendered color values ​​corresponding to the color values ​​obtained from the source image in the eighth step 332. For example, the rendered color values ​​can be determined from the target color value association made in the seventh step 328. The third stage 330 includes a tenth step 336: replacing the source image color values ​​with their respective rendered color values ​​(identified in the ninth step 334) to generate a rendered image (within the limits allowed by the specific CVD involved) that helps the CVD user perceive colors in the rendered image when a user with normal color vision receives colors from the original source image. The rendered image assists the CVD user in perceiving colors and / or differences between colors.

[0043] In pseudocode list 9 below (which corresponds to pseudocode lists 5 and 7 above), an exemplary implementation of the third stage 330 for three-component RGB color values ​​is shown. Pseudocode list 10 (which corresponds to pseudocode lists 6 and 8 above) shows another exemplary implementation of the third stage 330 for two-component RG and BG values. Pseudocode Listing 9: Conversion using a single RGB color conversion table Pseudocode List 10: Conversion using RG and BG color conversion tables

[0044] In different implementations, the system can be configured to shift the colors of an image to colors that are easier for a user to distinguish. For example, it can make green objects appear bluer, and a controllable dimming filter arranged in the device can be configured to controllably reduce the brightness of the actual image to achieve the desired overall brightness level. In some implementations, the system can be configured to increase the brightness of colors in an image without altering the hue. By increasing the brightness of these colors, mildly impaired users may find them easier to distinguish.

[0045] Now for reference Figures 4A-4CFor illustrative purposes, an example of an environment 400 is depicted, in which color correction processing for CVD users is applied to a real-time or live video stream input. In various implementations, the environment can include one or more computing device end users, or simply a “user” capable of capturing or viewing images or other electronic content. The user can interact with or manipulate the images presented via the user device. In many cases, a “live preview” of a real-world landscape viewed through a camera can be presented to the user, although in other cases, such a preview can occur when the user opens or accesses a previously created image file or media. In various implementations, color correction applications can be incorporated into the functionality of the computing device to implement the camera-based capture techniques described herein. These interfaces are typically presented in conjunction with a “viewfinder” capable of presenting a dynamic, current (real-time) image received by the camera optics and / or switching to presenting captured images (i.e., still photographs or videos stored in memory) when the image has been taken or is being accessed from a storage device. Both dynamic and still image data can be the subject of the proposed color correction techniques, as well as stored image data or live (real-time) received image data.

[0046] exist Figure 4A The diagram shows a normal color view of environment 400, which can be understood as being used as a classroom or meeting room. Presenter 410 stands near projector screen 462, on which map 470 (in this particular example, a map of North America) is displayed. Map 470 is segmented or divided by alternating colors (e.g., red and green, or yellow and blue) to indicate the locations of first country 472, second country 474, third country 476, and fourth country 478. For simplicity, the colors are represented by two types of cross-shading, such that first country 472 and third country 476 are filled with a first (same) color, such as green, while second country 474 and fourth country 478 are filled with a second (same) color, such as red.

[0047] While some regions or objects in an image may correspond to non-critical information, such as purely aesthetic information, some of these regions or objects may contain highly critical information. For example, colored regions or objects in an image may contain information necessary for the viewer's understanding. As discussed above, this type of scenario can be fraught with information loss for colorblind users. Figure 4B This demonstrates how colorblind users would perceive things. Figure 4A An example of the same scenario shown. Figure 4BIn this map, map 470 no longer distinguishes between adjacent countries. Instead, users (such as students) are able to see the entire continent (represented by first-type dots) without knowing where the boundaries might be. Furthermore, when the presenter 410 describes the various regions on map 470, information is preserved within a graphical context otherwise available to students with normal vision.

[0048] Then, in Figure 4C The figure illustrates how the benefits of the proposed system are realized, wherein a first computing device (“first device”) 402 is shown as being in the process of... Figure 4A The process of receiving a live first image 460 of the real-world scene shown is described. The image is displayed via a touchscreen display 450, which also serves as the user interface. A colorblind student 452 and... Figure 4A and Figure 4B The presenter is seated at desk 456 in the same classroom. Student 452 has propped the first device 402 on a book 454 and is watching the lecture in real time through the display 450 of the first device 402. The first device 402 is running a color correction app (“app”) 404, which transforms the color palette shown in the first image 460 into a different color palette that is more fully perceptible to student 452.

[0049] In some, but not all, implementations, display 450 can be configured to present a live preview of items or objects in the camera's field of view via app 404. In one implementation, app 404 can also combine the image preview to provide a graphical user interface (not shown here), referred to herein as an image content viewing interface ("interface"). In some implementations, the interface can be presented "full-screen" on display 450 or only on a portion of display 450. Additionally, in some implementations, portions of the interface can be substantially transparent or semi-transparent, allowing user interaction with the screen or image to be received as input by the application, while the image itself remains largely visible without overlaying additional interface graphics that would otherwise obstruct viewing of the image. However, in other implementations, the color correction application can present various graphical elements associated with, overlaying on, or adjacent to the image, such as menus, settings, or other options.

[0050] Generally, the term "interface" should be understood to refer to the mechanisms used to deliver content to application users through a client application. For example, an interface can include pop-ups, controls, executable interfaces, interactive buttons, or other objects that can be presented to the user via a native application user interface (UI), as well as application-specific native mechanisms for presenting content associated with those native controls. Furthermore, "actuation" or "actuation event" refers to an event (or a specific sequence of events) associated with a specific input or use of the application via the interface, such as a finger tap, keyboard command, voice command, or mouse click, which can trigger a change in the display or functionality of the application or device.

[0051] Therefore, as in Figure 4C As shown, for users with normal color vision (such as in...) Figure 4A The countries that can be identified in map 470 (as shown) are now fully identified by student 452 in the first image 460 (represented by two different types of cross shading). Colors that can be perceived by student 452 can, in some cases, be substantially the same as those perceived by a normal color-vision observer. In different implementations, the system proposed herein can include provisions for detecting changes in the scene and triggering automatic updates of the color palette for the changing scene during real-time or previously stored video. For example, the system can continuously receive image data of a real-world scene when student 452 orients or positions the camera lens of first device 402 in a direction typically facing or toward the projector screen. As the presenter continues their discussion, they may move from one slide to another. This change can be detected by app 404 and trigger another color correction event to maintain a colorblind-accessible view of the scene. Thus, when slides or other objects in the scene change, student 452 can remain hands-free, safely knowing that app 404 will continue receiving image data from the camera and automatically reprocess the incoming data when a change in the scene is detected. In other implementations, reprocessing can occur as the result of a manual command input by the user, or it can be configured to occur periodically after a pre-selected time interval. Such reprocessing is unnecessary for still images such as photographs.

[0052] For clarity, in Figures 5A-6C The sequence shown provides another example of a color correction process in which adjustment tools are provided. First, refer to... Figure 5A User 550 holds a second computing device (“second device”) 500 and is pointing the camera lens of the device at illustration 520 in book 590. Figure 5B middle, Figure 5AThe perspective view has been altered to allow the reader to observe the display 506 of the second device 500 while the user 550 is engaged in the color correction application. The display 506 presents an implementation of the color correction graphical user interface (GUI) for the image capture application (“application”) 504. In different implementations, the application 504 can be configured to display the GUI and various user interface elements, features, and controls to facilitate the capture or loading of image data, scanning and / or processing of image data, and in some cases, to initiate a live preview of a scene or the presentation of stored images. Although the illustration 520 viewed using normal color vision includes a series of overlapping colors that distinguish each star from its adjacent stars, this pattern is not apparent to the user 550, who only sees a single color (represented by the first dot type) within the larger multi-lobed shape while she is reading the book 590. Once the illustration 520 has been acquired by the second device 500, it can be displayed on the display 506 as a static, rendered second image 590 via the application 504. As described above, by applying 504, user 550 can then view an illustration with its correct color pattern.

[0053] In some cases, users may wish to adjust the color hue to better suit their own preferences or color deficiency types. In different implementations, the system can include provisions for receiving user input and selections in order to establish appropriate viewing settings for application 504. For example, in Figures 6A-6C As shown in the example, the user has accessed the interactive tool 610 of application 504, which is shown here as a slider. The user can slide or move the bar 620 across tool 610 to adjust the degree or scale of the rendered color correction. Figure 6A In the image, bar 620 is located at the first position closest to the lowest (smallest) end of the available color spectrum, which alters the appearance of the second image 590 so that a colorblind user can only see a single color (represented by the first type of dotted lines). Figure 6B In the middle, bar 620 is located at a second position towards the midpoint of the available colors, which changes the appearance of the second image 590 to now include a pair of colors (represented by the first and second types of cross shading) that are now visible to colorblind users. Finally, in Figure 6C In the middle, bar 620 is located in the third position, closest to the top (largest) end of the available color. Two colors continue to appear, but the hue in the second image 590 has been modified (represented by cross shadows of types third and fourth) to provide sharper or greater contrast. In different implementations, when the bar is in... Figure 6B and Figure 6CAs the scale moves up or down, the primary hue (primary color) of each of the two colors remains the same as perceived by a CVD viewer, while the characteristics of the hue can change in their eyes, including changes in color hue, tone, and / or shadow. Although a sliding scale is presented here for illustrative purposes, in other implementations, other optional options can be used to perform such adjustments to the appearance of the image, such as manual input of numerical values ​​representing the desired color, volume-type up / down buttons, or other input types. For illustrative purposes, pseudocode listing 11 below (corresponding to the examples shown in pseudocode listings 6, 8, and 10) shows an example where the aforementioned degree or scale (provided as the variable "percentage") is used to provide a user-selectable CVD color conversion. Pseudocode List 11: Scalable Color Transition

[0054] It should be understood that other customization options and settings are also available via application 504, including several capture or operation modes, such as live capture mode (see application 504). Figures 4A-4C ), such as in Figures 5A-6C The static image mode shown, as well as saved video or image modes, can be provided to the user upon application 504 startup and / or set as the default mode. In different implementations, the user can initiate the color correction process by viewing or capturing real-world scenes (e.g., taking pictures or photos), by capturing or viewing images that can include virtually generated objects (e.g., digital media, screenshots, cropping tools), and / or by loading any previously captured or saved image or media files for viewing through the application. Additionally, as previously mentioned, in some implementations, the application can be configured to operate in conjunction with other software instances, for example, by accessing the color correction app as an additional tool while using another software application (such as a web browser, productivity application, or other applications, particularly those that rely on color to convey information). In one implementation, the proposed system can be used to render some or all of a portion of a displayed "page," such as a document or website, by transforming problematic pixels using a reference base table.

[0055] In different implementations, app 504 can be configured to receive input(s) from a user to help determine how to adapt or adjust the user's color perception. The input can directly or indirectly control or influence the colors the system selects. For example, in one implementation, the user can specify a particular type of color vision deficiency they experience. Optionally, the input can also specify the degree or severity of the color vision deficiency. In another implementation, the user can specify one or more colors in a real image that should be corrected; for example, the specified colors could be colors that are difficult for the user to distinguish with the naked eye. In some other implementations, in response to an initial visual test presented to the user, the system can receive one or more user inputs to allow the system to determine the type of color correction that should be applied. Based on the results of such a color test, the system can be calibrated to accommodate one or more color vision deficiencies in a particular user.

[0056] Figure 7 This is a flowchart illustrating the implementation of method 700 for improving image rendering for users with color vision deficiencies. (For example...) Figure 7 As shown in the example, the first step 710 includes obtaining a source image comprising a plurality of source pixels. The second step 720 includes generating a rendered image comprising a plurality of rendered pixels. The second step 720 occurs by repeating a process for each rendered pixel 721 contained in the plurality of rendered pixels, the process including a first sub-step 722, a second sub-step 723, a third sub-step 724, and a fourth sub-step 725. The first sub-step 722 includes selecting a source pixel corresponding to the rendered pixel contained in the plurality of source pixels, and the second sub-step 723 includes obtaining a first color value (which may be a multi-component color value) for the selected source pixel. The third sub-step 724 includes identifying the first rendered color value (which may be a multi-component color value) which is estimated to be perceived by the user as having the first color value when rendered by a display device viewed by a user with a first color vision deficiency. The fourth sub-step 725 includes determining the color of the rendered pixel based on the identified first rendered color value. Finally, the method 700 includes a third step 730: causing the generated rendered image to be rendered by a first display device.

[0057] In other implementations, additional steps may be included. For example, in some implementations, the step of identifying the first rendered color value includes: obtaining a second rendered color value stored at a memory location determined based on the first color value, and identifying the first rendered color value based on the obtained second rendered color value.

[0058] In another example, the first color value can be a dual-component color value consisting of a red component and a first green component. In such a case, generating the rendered image further includes, for each rendered pixel included in a plurality of rendered pixels: obtaining a second color value for a selected source pixel, wherein the second color value is a dual-component color value including a blue component and a second green component; and identifying a second rendered color value, which is estimated to be perceived by the user as having the second color value when rendered by a display device viewed by a user with a first color vision deficiency. Furthermore, the color of the rendered pixel can be determined based on the identified first rendered color value and the identified second rendered color value.

[0059] In some implementations, the step of identifying the first rendered color value includes: obtaining a third rendered color value stored at a memory location determined based on the first color value, and identifying the first rendered color value based on the obtained third rendered color value. Furthermore, in such a case, identifying the second rendered color value includes: obtaining a fourth rendered color value stored at a memory location determined based on the second color value, and identifying the second rendered color value based on the obtained fourth rendered color value.

[0060] In some implementations, the primary color vision deficiency is green amblyopia or red amblyopia, while in others, it is green blindness or red blindness. In one implementation, a source image is obtained based on image frames contained in a series of image frames captured by a camera, and the rendered image is rendered in real time by a display device with respect to the image frames captured by the camera.

[0061] In another example, the method further includes the steps of: presenting a user interface via the first display device that allows the user to select the degree of color adjustment, and storing the degree of color adjustment. In such a case, determining the color of the rendered pixel includes identifying a color value between a first color value and a first rendered color value based on the stored degree of color adjustment.

[0062] Other methods are conceivable within the scope of this disclosure. For example, in some embodiments, a method for generating a conversion model for a color vision deficiency image includes a first step of selecting a plurality of source color values ​​(which may be multi-component color values). The method further includes a second step of generating a plurality of estimated color values ​​(which may be multi-component color values) associated with the source color values. The second step is performed for each source color value included in the plurality of source color values ​​by repeating a process. The process includes: simulating that source color values, as rendered by a display device, would be perceived by a user with a first color vision deficiency as having a first estimated color value included in the plurality of estimates, and associating the source color values ​​with the estimated color values. The method also includes a third step of selecting a plurality of target color values ​​(which may be multi-component color values). Additionally, the method includes a fourth step, which includes a process performed for each target color value included in the plurality of target color values. This process includes: selecting a second estimated color value from the plurality of estimates that matches the target color value; selecting a rendered color value (which may be multi-component color value) associated with the second estimated color value from the plurality of source color values; and storing an association between the target color value and the rendered color value.

[0063] In other implementations, additional steps may be included. For example, in some implementations, selecting the second estimated color value includes identifying the minimum color distance between the target color value and each of the plurality of estimates. In another example, the simulation of the user's perception of the source color value includes the simulation of the color rendering characteristics of the display device.

[0064] The detailed examples of systems, devices, and technologies described in conjunction with Figures 1-7 are presented herein to illustrate the present disclosure and its benefits. Such examples of use should not be construed as limiting the implementation of the logical processes of the present disclosure, nor should variations based on the user interface methods described herein be considered outside the scope of the disclosure. In some implementations, the various features described in Figures 1-7 are implemented in corresponding modules, which may also be referred to as and / or include logic, components, units, and / or mechanisms. Modules may constitute software modules (e.g., code embodied on a machine-readable medium) or hardware modules.

[0065] In some examples, hardware modules can be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic configured to perform certain operations. For instance, a hardware module may include a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware module may also include programmable logic or circuitry temporarily configured by software to perform certain operations, and may include a portion of machine-readable medium data and / or instructions for such configuration. For example, a hardware module may include software contained within a programmable processor configured to execute a software instruction set. It will be understood that the decision to mechanically implement a hardware module in dedicated and permanently configured circuitry or in temporarily configured circuitry (e.g., software-configured) may be driven by cost, time, support, and engineering considerations.

[0066] Therefore, the phrase "hardware module" should be understood to include tangible entities capable of performing certain operations, and entities that can be physically configured or arranged, i.e., physically constructed, permanently configured (e.g., hardwired), and / or temporarily configured (e.g., programmed) to operate or perform certain operations described herein. As used herein, "hardware-implemented module" refers to a hardware module. Considering examples where hardware modules are temporarily configured (e.g., programmed), each hardware module does not need to be configured or instantiated at any one time instance. For example, in cases where the hardware module includes a programmable processor configured by software to be a dedicated processor, the programmable processor can be configured at different times as distinct dedicated processors (e.g., including different hardware modules). The software can accordingly configure one or more particular processors, for example, to constitute a particular hardware module at one time instance and different hardware modules at different time instances. A hardware module implemented using a processor can be referred to as "processor-implemented" or "computer-implemented."

[0067] Hardware modules can provide information to and receive information from other hardware modules. Therefore, the described hardware modules can be considered communicationally coupled. In the presence of multiple hardware modules, communication can be achieved through signal transmission between two or more hardware modules (e.g., via appropriate circuitry and buses). In implementations where multiple hardware modules are configured or instantiated at different times, communication between these hardware modules can be achieved, for example, by storing and retrieving information from memory devices accessible to the multiple hardware modules. For instance, one hardware module can perform an operation and store the output in a memory device, and then another hardware module can access the memory device to retrieve and process the stored output.

[0068] In some examples, at least some operations of the method can be performed by one or more processors or modules implemented by processors. Furthermore, one or more processors can operate to support the execution of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS). For example, at least some operations can be performed by multiple computers (as an example of a machine including processors) and / or across multiple computers, and these operations can be accessed via a network (e.g., the Internet) and / or through one or more software interfaces (e.g., application programming interfaces (APIs)). The execution of some operations can be distributed among processors, residing not only within a single machine but also deployed across multiple machines. Processors or modules implemented by processors can reside in a single geographic location (e.g., in a home or office environment, or in a server farm) or can be distributed across multiple geographic locations.

[0069] Figure 8 This is a block diagram 800 illustrating an example software architecture 802, the various parts of which can be used in conjunction with various hardware architectures described herein, which can implement any of the features described above. Figure 8 This is a non-limiting example of a software architecture, and it will be understood that many other architectures can be implemented to facilitate the functionality described herein. Software architecture 802 can be implemented on hardware including document storage device 870, a processor, memory, and input / output (I / O) components, etc. A representative hardware layer 804 is shown. Representative hardware layer 804 includes a processing unit 806 and associated executable instructions 808. Executable instructions 808 represent executable instructions of software architecture 802, including implementations of the methods, modules, etc., described herein. Hardware layer 804 also includes a memory / storage device 810, which also includes the executable instructions 808 and accompanying data. Hardware layer 804 may also include other hardware modules 812. Instructions 808 held by processing unit 808 may be portions of instructions 808 held by memory / storage device 810.

[0070] The exemplary software architecture 802 can be conceptualized as layers, each providing various functionalities. For example, software architecture 802 may include layers and components such as an operating system (OS) 814, libraries 816, frameworks 818, applications 820, and a presentation layer 844. Operationally, applications 820 and / or other components within a layer can invoke API calls 824 to other layers and receive corresponding results 826. The layers shown are representative in nature, and other software architectures may include additional or different layers. For example, some mobile or dedicated operating systems may not provide a framework / middleware 818.

[0071] OS 814 can manage hardware resources and provide public services. OS 814 may include, for example, a kernel 828, services 830, and drivers 832. The kernel 828 can act as an abstraction layer between the hardware layer 804 and other software layers. For example, the kernel 828 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 830 can provide other public services to other software layers. Drivers 832 can be responsible for controlling or interfacing with the underlying hardware layer 804. For example, depending on the hardware and / or software configuration, drivers 832 may include display drivers, camera drivers, memory / storage device drivers, peripheral device drivers (e.g., via Universal Serial Bus (USB)), network and / or wireless communication drivers, audio drivers, etc.

[0072] Library 816 can provide common infrastructure that can be used by application 820 and / or other components and / or layers. Library 816 typically provides functionality used by other software modules to perform tasks, rather than interacting directly with OS 814. Library 816 may include system libraries 834 (e.g., the C standard library), which provide functions such as memory allocation, string manipulation, and file operations. Furthermore, library 816 may include API libraries 836, such as media libraries (e.g., those supporting the rendering and manipulation of image, sound, and / or video data formats), graphics libraries (e.g., OpenGL libraries for rendering 2D and 3D graphics on a display), database libraries (e.g., SQLite or other relational database functions), and web libraries (e.g., WebKit, which provides web browsing functionality). Library 816 may also include various other libraries 838 to provide numerous functionalities for application 820 and other software modules.

[0073] Framework 818 (sometimes also called middleware) provides a higher level of common infrastructure that can be used by Application 820 and / or other software modules. For example, Framework 818 can provide various graphical user interface (GUI) functions, advanced resource management, or advanced location services. Framework 818 can provide a wide range of other APIs for Application 820 and / or other software modules.

[0074] Application 820 includes built-in application 840 and / or third-party application 842. Examples of built-in application 840 may include, but are not limited to, contact applications, browser applications, location applications, media applications, messaging applications, and / or game applications. Third-party application 842 may include any application developed by an entity other than a vendor for a specific platform. Application 820 may use the functionality available via OS 814, library 816, framework 818, and presentation layer 844 to create a user interface for interaction with the user.

[0075] Some software architectures use virtual machines, such as Virtual Machine 848. Virtual Machine 848 provides an execution environment where applications / modules can run as if on a hardware machine (e.g., ...). Figure 8 The virtual machine 848 executes in the same way as on the host OS (e.g., OS 814) or hypervisor, and may have a virtual machine monitor 846 that manages the operation of the virtual machine 848 and its interoperability with the host operating system. Software architectures different from the external software architecture 802 execute within the virtual machine 848, such as OS 850, libraries 852, frameworks 854, applications 856, and / or presentation layers 858.

[0076] Figure 9 This is a block diagram illustrating components of an exemplary machine 900 configured to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any of the features described herein. The exemplary machine 900 is in the form of a computer system in which instructions 916 (e.g., in the form of a software component) for causing the machine 900 to perform any of the features described herein can be executed. Therefore, instructions 916 can be used to implement the modules or components described herein. Instructions 916 cause an unprogrammed and / or unconfigured machine 900 to operate as a specific machine configured to perform the described features. The machine 900 can be configured to operate as a standalone device or can be coupled (e.g., networked) to other machines. In a network deployment, the machine 900 can operate as a server machine or a client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. Machine 900 can be embodied as, for example, a server computer, client computer, personal computer (PC), tablet computer, laptop computer, netbook, set-top box (STB), gaming and / or entertainment system, smartphone, mobile device, wearable device (e.g., smartwatch), and Internet of Things (IoT) device. Furthermore, although only a single machine 900 is shown, the term "machine" includes a collection of machines that execute instructions 916 individually or jointly.

[0077] Machine 900 may include processor 910, memory 930, and I / O components 950, which may be communicatively coupled via, for example, bus 902. Bus 902 may include multiple buses coupling various components of machine 900 via various bus technologies and protocols. In examples, processor 910 (including, for example, a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), ASIC, or suitable combinations thereof) may include one or more processors 912a to 912n, which can execute instructions 916 and process data. In some examples, one or more processors 910 may execute instructions provided or recognized by one or more other processors 910. The term "processor" includes multi-core processors, which include cores capable of executing instructions simultaneously. Although Figure 9 Multiple processors are shown, but machine 900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, machine 900 may include multiple processors distributed across multiple machines.

[0078] Memory / storage device 930 may include main memory 932, static memory 934 or other memory, and memory cell 936, both of which can be accessed by processor 910, for example, via bus 902. Memory cell 936 and memories 932, 934 store instructions 916 embodying any one or more of the functions described herein. Memory / storage device 930 may also store temporary, intermediate, and / or long-term data for processor 910. Instructions 916 may also reside wholly or partially in memories 932, 934, memory cell 936, at least one processor 910 (e.g., in a command buffer or cache memory), at least one of I / O components 950, or any suitable combination thereof, during their execution. Thus, memories 932, 934, memory cell 936, memory in processor 910, and memory in I / O components 950 are examples of machine-readable media.

[0079] As used herein, "machine-readable medium" means a device capable of temporarily or permanently storing instructions and data that cause machine 900 to operate in a particular manner. The term "machine-readable medium" as used herein does not itself include transient electrical or electromagnetic signals (e.g., on a carrier wave propagating through the medium). Therefore, the term "machine-readable medium" can be considered tangible and non-transient. Non-limiting examples of non-transitory, tangible machine-readable media may include, but are not limited to, non-volatile memory (e.g., flash or read-only memory (ROM), volatile memory (e.g., static random access memory (RAM) or dynamic RAM), buffer memory, cache memory, optical storage media, magnetic storage media and devices, network-accessible or cloud storage, other types of storage devices and / or any suitable combination thereof. The term "machine-readable medium" applies to a single medium or combination of media for storing instructions (e.g., instruction 916) that are executed by machine 900, such that when executed by one or more processors 910 of machine 900, the instructions cause machine 900 to perform one or more features described herein. Therefore, "machine-readable medium" can refer to a single storage device, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or equipment.

[0080] I / O component 950 may include various hardware components suitable for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O component 950 included in a particular machine will depend on the type and / or function of the machine. For example, mobile devices such as mobile phones may include touch input devices, while headless servers or IoT devices may not include such touch input devices. Figure 9 The specific examples of I / O components shown are by no means limiting, and other types of components may be included in machine 900. The grouping of I / O components 950 is merely for the sake of simplicity in this discussion, and the grouping is by no means limiting. In various examples, I / O components 950 may include user output components 952 and user input components 954. User output components 952 may include, for example, display components for displaying information (e.g., liquid crystal display (LCD) or projector), acoustic components (e.g., speakers), haptic components (e.g., vibration motors or force feedback devices), and / or other signal generators. User input components 954 may include, for example, alphanumeric input components (e.g., keyboards or touchscreens), pointing components (e.g., mouse devices, touchpads, or other pointing instruments), and / or haptic input components (e.g., physical buttons or touchscreens that provide the position and / or force of touch or touch gestures), configured to receive various user inputs, such as user commands and / or selections.

[0081] In some examples, I / O component 950 may include a biostatistical component 956 and / or a location component 962 in a broad array of other environmental sensor components. Biostatistical component 956 may include components for detecting body expressions (e.g., facial expressions, vocal expressions, hand or body gestures, or eye tracking), measuring biosignals (e.g., heart rate or brain waves), and identifying people (e.g., via voice-based, retinal, and / or facial recognition). Location component 962 may include, for example, a location sensor (e.g., a Global Positioning System (GPS) receiver), an altitude sensor (e.g., a barometric pressure sensor from which altitude can be derived), and / or an orientation sensor (e.g., a magnetometer).

[0082] I / O component 950 may include communication component 964, which implements various technologies operable to couple machine 900 to network 970 and / or device 980 via respective communication couplers 972 and 982. Communication component 964 may include one or more network interface components or other suitable devices interface with network 970. Communication component 964 may include, for example, components adapted to provide wired communication, wireless communication, cellular communication, near field communication (NFC), Bluetooth communication, Wi-Fi, and / or communication via other modes. Device 980 may include other machines or various peripheral devices (e.g., coupled via USB).

[0083] In some examples, communication component 964 may detect identifiers or include components suitable for detecting identifiers. For example, communication component 964 may include a radio frequency identification (RFID) tag reader, an NFC detector, an optical sensor (e.g., a one-dimensional or multi-dimensional barcode, or other optical code), and / or an acoustic detector (e.g., a microphone for identifying audio signals of the tag). In some examples, location information may be determined based on information from communication component 962, such as, but not limited to, geolocation via Internet Protocol (IP) address, location identified via Wi-Fi, cellular, NFC, Bluetooth, or other wireless stations, and / or signal triangulation.

[0084] Although various embodiments have been described, the description is intended to be exemplary and not limiting, and it should be understood that many more embodiments and implementations are possible within the scope of the embodiments. While many possible combinations of features are shown in the drawings and discussed in this detailed description, many other combinations of the disclosed features are possible. Unless specifically limited, any feature of any embodiment may be used in combination with or in lieu of any other feature or element in any other embodiment. Therefore, it will be understood that any feature shown and / or discussed in this disclosure can be implemented together in any suitable combination. Thus, the embodiments are not limited except as provided in the appended claims and their equivalents. Furthermore, various modifications and variations may be made within the scope of the appended claims.

[0085] While the foregoing has described what is considered the best model and / or other examples, it should be understood that various modifications can be made therein, and the subject matter disclosed herein can be implemented in various forms and examples. The teachings can be applied in many applications, only some of which are described herein. The following claims are intended to claim any and all applications, modifications, and variations that fall within the true scope of these teachings.

[0086] Unless otherwise stated, all measurements, values, ratings, positions, amplitudes, dimensions, and other specifications set forth in this specification and included in the following claims are approximate, not precise. They are intended to have a reasonable range that is consistent with the functions they relate to and the custom in the field to which they relate.

[0087] The scope of protection is limited only by the claims now presented. This scope is intended and should be interpreted as consistent with the ordinary meaning of the language used in the claims as interpreted in accordance with this specification and the subsequent prosecution history, and includes all structural and functional equivalences. Nevertheless, no claim is intended to contain, nor should it be interpreted in this way, any subject matter that does not meet the requirements of Sections 101, 102, or 103 of the Patent Act. Any content unintentionally containing such subject matter is hereby declared.

[0088] Except as described above, nothing described or shown is intended or should not be construed as resulting in the offering of any component, step, feature, object, benefit, advantage, or equivalent to the public, whether or not it is described in the claims.

[0089] It should be understood that the terms and expressions used herein have the general meanings assigned to them in relation to their respective fields of investigation and research, unless otherwise specified herein. Relational terms such as "first" and "second" may be used merely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between those entities or actions. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements may include not only those elements but also other elements not expressly listed or inherent to such a process, method, article, or apparatus. An element preceded by "a" or "an" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element, without further limitation.

[0090] An abstract of this disclosure is provided to allow the reader to quickly determine the nature of the technical disclosure. It is understood at the time of submission that this disclosure is not intended to interpret or limit the scope or meaning of the claims. Furthermore, as can be seen in the foregoing detailed description, various features have been grouped together in various examples for the purpose of simplification. This approach should not be construed as reflecting an intention to represent more features claimed in the claims than those explicitly stated in each claim. Rather, as reflected in the following claims, the subject matter of the invention does not lie in all features of a single disclosed example. Therefore, the following claims are incorporated herein by reference to the detailed description, wherein each claim is considered a separate subject matter claim.

Claims

1. A method for generating a color conversion table for color vision deficiency (CVD) correction, comprising: Select multiple source color values; For each of the source color values, a corresponding estimated color value associated with the source color value is identified, the estimated color value being a simulation of the source color value perceived by a user with CVD; Associate each estimated color value with its corresponding source color value; The color conversion table is generated using the following operations: Select multiple target color values; For each target color value included in the plurality of target color values, determine the corresponding color distance to each estimated color value in the estimated color values; For each of the plurality of target color values, identify a source color value among the source color values ​​that is associated with an estimated color value among the estimated color values ​​that has the minimum color distance to the target color value; as well as The association between each of the target color values ​​and the corresponding identified source color value is stored to provide a color conversion table between the source color value and the target color value used to replace the source color value in a display on a display device calibrated for the user with CVD; as well as The colors in the source image are transformed into corrected colors using the color conversion table for display on the display device for the user with CVD.

2. The method of claim 1, wherein, The source color value is identified from the color values ​​of the pixels in the source image.

3. The method according to claim 1, wherein, The source color value is identified from the combination of color component values.

4. The method according to claim 1, wherein, The target color value is a multi-component color value.

5. The method according to claim 1, wherein, The simulation of the user's perception of the source color value includes the simulation of the color rendering characteristics of the display device.

6. The method according to claim 1, further comprising: The minimum color distance is identified by using the squared difference or sum of squared differences of at least a portion of the estimated color values.

7. The method according to claim 3, wherein, The minimum color distance is determined by using the squared difference or sum of squared differences of all estimated color values ​​in the estimated color values.

8. The method according to claim 1, wherein, The source image from which the plurality of source color values ​​are obtained is based on image frames included in a series of image frames captured by the camera.

9. The method according to claim 8, wherein, A rendered image with rendered color values ​​is rendered into rendered pixels in real time by the display device relative to the camera capturing the image frame.

10. The method according to claim 1, further comprising: The display device is used to present a user interface that allows users to select the degree of color adjustment; as well as Store the degree of color adjustment.

11. The method according to claim 1, further comprising: Obtain the source image comprising a plurality of source pixels having the plurality of source color values; A rendered image comprising multiple rendered pixels to be displayed on a display device is generated using the following operations, for each rendered pixel included in the plurality of rendered pixels: Select the source pixel that corresponds to the rendered pixel and is included among the plurality of source pixels. Obtain the first color value for the selected source pixel. Identify a first rendered color value, the first rendered color value being estimated to be perceived by the user as having the first color value when rendered by the display device viewed by a user with a first color vision deficiency, and The color of the rendered pixel is determined based on the first rendered color value identified; as well as This causes the generated rendered image to be rendered by the display device.

12. A machine-readable medium including instructions, which, when executed by a processor, cause the processor, alone or in combination with other processors, to perform the following functions for generating a color conversion table for color vision deficiency (CVD) correction: Select multiple source color values; For each of the source color values, a corresponding estimated color value associated with the source color value is identified, the estimated color value being a simulation of the source color value perceived by a user with CVD; Associate each estimated color value with its corresponding source color value; The color conversion table is generated using the following operations: Select multiple target color values; For each target color value included in the plurality of target color values, determine the corresponding color distance to each estimated color value in the estimated color values; For each of the plurality of target color values, identify a source color value among the source color values ​​that is associated with an estimated color value among the estimated color values ​​that has the minimum color distance to the target color value; as well as The association between each of the target color values ​​and the corresponding identified source color value is stored to provide a color conversion table between the source color value and the target color value used to replace the source color value in a display on a display device calibrated for the user with CVD; as well as The colors in the source image are transformed into corrected colors using the color conversion table for display on the display device for the user with CVD.

13. The machine-readable medium according to claim 12, wherein, The source color value is identified from the color values ​​of the pixels in the source image.

14. The machine-readable medium according to claim 12, wherein, The source color value is identified from the combination of color component values.

15. The machine-readable medium according to claim 12, wherein, The minimum color distance is determined by using the squared difference or sum of squared differences of all estimated color values ​​in the estimated color values.

16. The machine-readable medium of claim 12, wherein, The source image from which the plurality of source color values ​​are obtained is based on image frames included in a series of image frames captured by the camera.

17. The machine-readable medium of claim 16, wherein, A rendered image with rendered color values ​​is rendered into rendered pixels in real time by the display device relative to the camera capturing the image frame.

18. The machine-readable medium of claim 17, further comprising: The display device is used to present a user interface that allows users to select the degree of color adjustment; as well as Store the degree of color adjustment.

19. The machine-readable medium of claim 12, further comprising: Obtain a source image comprising multiple source pixels having the multiple source color values; A rendered image comprising multiple rendered pixels to be displayed on a display device is generated using the following operations, for each rendered pixel included in the plurality of rendered pixels: Select the source pixel that corresponds to the rendered pixel and is included among the plurality of source pixels. Obtain the first color value for the selected source pixel. Identify a first rendered color value, the first rendered color value being estimated to be perceived by the user as having the first color value when rendered by the display device viewed by a user with a first color vision deficiency, and The color of the rendered pixel is determined based on the first rendered color value identified; as well as This causes the generated rendered image to be rendered by the display device.

20. A system for generating a color conversion table for color vision deficiency (CVD) correction, comprising: processor; A memory coupled to the processor, the memory being configured to store executable instructions that, when executed by the processor alone or in conjunction with other processors, cause the processor to perform the following operations: Select multiple source color values; For each of the source color values, a corresponding estimated color value associated with the source color value is identified, the estimated color value being a simulation of the source color value perceived by a user with CVD; Associate each estimated color value with its corresponding source color value; The color conversion table is generated using the following operations: Select multiple target color values; For each target color value included in the plurality of target color values, determine the corresponding color distance to each estimated color value in the estimated color values; For each of the plurality of target color values, identify a source color value among the source color values ​​that is associated with an estimated color value among the estimated color values ​​that has the minimum color distance to the target color value; as well as The association between each of the target color values ​​and the corresponding identified source color value is stored to provide a color conversion table between the source color value and the target color value used to replace the source color value in a display on a display device calibrated for the user with CVD; as well as The colors in the source image are transformed into corrected colors using the color conversion table for display on the display device for the user with CVD.

Images