Visual enhancement method for color vision disorder, electronic equipment and medium

Abstract

The embodiment of the invention provides a visual enhancement method for color vision disorder, electronic equipment and a medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing visual enhancement on an original training image through an original visual enhancement model to obtain a visual enhancement image aiming at the performance of color vision disorder; performing color discrimination on the vision enhancement image through a color vision color simulation model so as to perform model training on the original vision enhancement model according to the calculated target loss values of the vision enhancement image, the original training image, the target color discrimination name and the real color name; and performing visual enhancement on the target image through the trained visual enhancement model. According to the embodiment of the invention, color name discrimination is carried out on the vision enhancement image through the color vision color simulation model so as to simulate a color naming process perceived by people with color vision disorder, and the vision enhancement model is trained in combination with the target loss value, so that the people with color vision disorder are effectively helped to calibrate the consistency of color naming and public language standards.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a visual enhancement method, electronic device, and medium for color vision disorders. Background Technology

[0002] Traditional visual enhancement methods typically rely on color transformation simulation and compensation (such as the Brettel model). This involves linearly transforming RGB (Red, Green, Blue) images into an LMS (Long / Medium / Shortwavelength-sensitive cone cells) color space for color blindness, and then replacing colors within the confusing hue range with non-confusing hues. For example, in traffic sign optimization, the Brettel model can simulate an LMS image of a green cone and replace its hue with blue (a non-confusing color), enabling color-impaired individuals to distinguish between green and red cones. However, this method only improves color discrimination through simple hue adjustment, neglecting the cognitive process from visual signal to color naming for color-impaired individuals. This means they can only recognize differences in object color but cannot accurately describe the correct object color using common language, making accurate color naming difficult. Therefore, helping color-impaired individuals align their color naming with common language standards is a pressing issue. Summary of the Invention

[0003] The main objective of this application is to propose a visual enhancement method, electronic device, and medium for color vision impairment, aiming to help people with color vision impairment calibrate the consistency of color naming with public language standards.

[0004] To achieve the above objectives, a first aspect of this application proposes a visual enhancement method for color vision impairment, the method comprising: Obtain the original visual enhancement model and target training set, and obtain the pre-trained color vision simulation model; wherein, the visual enhancement model is used to perform at least one of color, brightness and contrast enhancement based on the input image to address the manifestation of color vision impairment; the target training set includes the original training image and the real color name corresponding to the original training image; The original training image is visually enhanced using the aforementioned visual enhancement model to obtain a visually enhanced image that addresses the manifestations of color vision deficiency. The color perception simulation model is used to perform color discrimination on the visually enhanced image to obtain the target color discrimination name; The target loss value is obtained by calculating the target loss based on the visually enhanced image, the original training image, the target color discrimination name, and the real color name. The visual enhancement model is updated based on the target loss value to obtain the updated visual enhancement model. Based on the updated visual enhancement model, the original training image is visually enhanced using the visual enhancement model until the target loss value meets the preset target loss condition, thus obtaining the trained visual enhancement model. The target image is acquired, and the target image is visually enhanced using the trained visual enhancement model to obtain a target visually enhanced image.

[0005] In some embodiments, prior to obtaining the pre-trained color vision simulation model, the method further includes pre-training the color vision simulation model, specifically including: Obtain the original color vision simulation model, and use the color vision simulation model to simulate the original training image to obtain a color vision impairment simulation image; Color discrimination is performed on the simulated image of color vision deficiency, and the simulated color discrimination name is determined; Color loss is calculated based on the simulated color discrimination name and the real color name to obtain the color perception loss value; The color perception loss value is used to adjust the model parameters of the color vision simulation model in order to update the color vision simulation model. Based on the updated color perception simulation model, the original training image is simulated using the color perception simulation model until the color perception loss value meets the preset expected loss condition, thus obtaining the pre-trained color perception simulation model.

[0006] In some embodiments, the color vision simulation model includes a color vision impairment simulation sub-model; wherein the color vision impairment simulation sub-model is used to simulate human spatial perception in response to color vision impairment. The step of simulating the original training image using the color vision simulation model to obtain a color vision impairment simulation image includes simulating the original training image using the color vision impairment simulation sub-model to obtain a color vision impairment simulation image, specifically including: Obtain the training color vision deficiency type corresponding to the target training set; Obtain the human eye perception spatial data corresponding to the type of color vision deficiency being trained; Obtain random noise data; Based on the training color vision disorder type, the human eye perception space data, and the random noise data, the original training image is subjected to perceptual space transformation to obtain the color vision disorder simulation image.

[0007] In some embodiments, the color vision simulation model includes a color perception simulation sub-model; wherein the color perception simulation sub-model is used to perform color semantic discrimination based on the manifestations of color vision impairment. The step of performing color discrimination on the simulated image of color vision impairment and determining the simulated color discrimination name includes performing color discrimination on the simulated image of color vision impairment through the color perception simulation sub-model and determining the simulated color discrimination name, specifically including: The color vision impairment simulation image is feature-encoded to obtain the impairment image features; Based on a preset color name embedding vocabulary, the candidate color embedding vector corresponding to the color vision impairment simulation image is determined; The similarity between the obstacle image features and the candidate color embedding vector is calculated to obtain the image color similarity. The simulated color discrimination name corresponding to the simulated image of color vision deficiency is determined based on the color similarity of the images.

[0008] In some embodiments, the step of calculating the color loss based on the simulated color discrimination name and the real color name to obtain a color perception loss value includes: Based on the simulated color discrimination name, obtain the image block color discrimination name corresponding to each image block in the color vision impairment simulation image, and obtain the probability distribution of the image block color name corresponding to the image block discrimination name; The image patch color loss is calculated based on the probability distribution of the image patch color names and the real color names to obtain the image patch color loss value; The color perception loss value is determined based on the color loss value of the image patch.

[0009] In some embodiments, the step of calculating the target loss value based on the visually enhanced image, the original training image, the target color discrimination name, and the real color name includes: The image structure loss is calculated based on the visually enhanced image and the original training image to obtain the image structure loss value; The color loss is calculated based on the target color discrimination name and the real color name to obtain the enhanced color loss value; The target loss value is determined based on the image structure loss value and the enhanced color loss value.

[0010] In some embodiments, the step of calculating image structure loss based on the visually enhanced image and the original training image to obtain an image structure loss value includes: Determine the obstacle image pixel distribution in the visually enhanced image; Determine the original image pixel distribution of the original training image; The image structure loss value is obtained by calculating the loss based on the pixel distribution of the obstacle image and the pixel distribution of the original image.

[0011] In some embodiments, the step of visually enhancing the target image using the trained visual enhancement model to obtain a target visually enhanced image includes: The target color vision deficiency type of the target image is obtained, and the severity of the deficiency corresponding to the target color vision deficiency type is obtained; wherein, the target image is derived from an image file, video frame, display input frame, or light field; Based on the pre-trained color vision simulation model, the target color vision impairment type, and the severity of the impairment, the target image is color-classified to obtain a color classification name for the target image; The visual optimization constraints of the target image are determined based on the color discrimination name of the target image, and the visual enhancement model trained according to the visual optimization constraints is used to adjust the image pixels of the target image to obtain the target visual enhancement image.

[0012] To achieve the above objectives, a second aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0013] To achieve the above objectives, a third aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of the first aspect described above.

[0014] This application proposes a visual enhancement method, electronic device, and medium for color vision impairment. First, it acquires a pre-trained color vision simulation model and a target training set containing real color names, providing a color naming cognitive benchmark for subsequent model training. Second, it generates visually enhanced images based on the original training images, targeting the color vision impairment, achieving an initial conversion from normal images to color vision impairment-friendly images. The color vision simulation model then performs color discrimination on the original training images, simulating the real color naming behavior of people with color vision impairment and effectively capturing their cognitive process from visual signals to standard public language naming. Furthermore, it calculates the target loss value based on the visually enhanced images, the original training images, the target color discrimination names, and the real color names, enabling... The accuracy of color naming is used as an optimization objective to ensure that the generated images of color vision impairment can guide people with color vision impairment to make correct color names. Loss calculation is performed by combining the original training images and visually enhanced images, which effectively avoids the distortion of the generated images caused by pursuing high discrimination. Finally, the visual enhancement model is iteratively updated until convergence based on the target loss value. It can accurately adjust the image colors according to the public color language standard, so that the trained visual enhancement model can enhance the target image in at least one of color, brightness and contrast. This ensures that the trained visual enhancement model generates visually enhanced images that people with color vision impairment can clearly distinguish different colors, accurately name different colors, and maintain a natural visual appearance. This effectively helps people with color vision impairment to calibrate the consistency of color naming with public language standards. Attached Figure Description

[0015] Figure 1 This is a flowchart of a visual enhancement method for color vision deficiency provided in an embodiment of this application; Figure 2 This is another flowchart of a visual enhancement method for color vision impairment provided in an embodiment of this application; Figure 3 yes Figure 2 The flowchart of step S201 in the text; Figure 4 yes Figure 2 The flowchart of step S202 in the document; Figure 5 yes Figure 2 The flowchart of step S203 in the process; Figure 6 yes Figure 1 The flowchart of step S104 in the process; Figure 7 yes Figure 6 The flowchart of step S601 in the process; Figure 8 yes Figure 1 The flowchart of step S107 in the process; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0017] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0019] First, let's analyze some of the terms used in this application: Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0020] This application provides a visual enhancement method, electronic device, and medium for color vision disorders, aiming to help people with color vision disorders calibrate the consistency of color naming with public language standards.

[0021] The visual enhancement method, electronic device, and medium for color vision impairment provided in this application are specifically described through the following embodiments. First, the visual enhancement method for color vision impairment in this application is described.

[0022] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0023] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0024] The visual enhancement method for color vision impairment provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the visual enhancement method for color vision impairment, but is not limited to the above forms.

[0025] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] Figure 1This is an optional flowchart of a visual enhancement method for color vision deficiency provided in an embodiment of this application, and Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0027] Step S101: Obtain the original visual enhancement model and target training set, and obtain the pre-trained color vision simulation model; wherein, the visual enhancement model is used to perform at least one visual enhancement of color, brightness and contrast based on the input image for the performance of color vision impairment; the target training set includes the original training image and the real color name corresponding to the original training image.

[0028] Step S102: Visually enhance the original training image using a visual enhancement model to obtain a visually enhanced image for color vision impairment.

[0029] Step S103: The color of the visually enhanced image is determined by the color vision simulation model to obtain the target color name.

[0030] Step S104: Calculate the target loss based on the visually enhanced image, the original training image, the target color discrimination name, and the real color name to obtain the target loss value.

[0031] Step S105: Update the visual enhancement model based on the target loss value to obtain the updated visual enhancement model.

[0032] Step S106: Based on the updated visual enhancement model, return to perform visual enhancement on the original training image through the visual enhancement model until the target loss value meets the preset target loss condition, and obtain the trained visual enhancement model.

[0033] Step S107: Obtain the target image and perform visual enhancement on the target image using the trained visual enhancement model to obtain the target visually enhanced image.

[0034] Steps S101 to S107 of this embodiment firstly obtain a pre-trained color vision simulation model and a target training set containing real color names, providing a color naming cognitive benchmark for the subsequent model training process. Secondly, a visual enhancement image targeting color vision impairment is generated based on the original training image through a visual enhancement model, achieving an initial conversion from normal images to color vision impairment-friendly images. The color vision simulation model is then used to perform color discrimination on the original training image, simulating the real color naming behavior of people with color vision impairment and effectively capturing their cognitive process from visual signals to standard naming in common language. Furthermore, a target loss is calculated based on the visual enhancement image, the original training image, the target color discrimination name, and the real color name to obtain the target loss value, which can effectively transform color perception into color perception. Naming accuracy is used as an optimization objective to ensure that the generated color vision impairment images can guide color vision impairment individuals to make correct color names. Loss calculations are performed by combining the original training images and visually enhanced images, effectively avoiding distortion caused by pursuing high discriminative power in the generated color vision impairment images. Finally, the visual enhancement model is iteratively updated until convergence based on the target loss value. This allows for precise adjustment of image colors according to common color language standards. The trained visual enhancement model enhances the target image in at least one of color, brightness, and contrast, ensuring that the generated visually enhanced images allow color vision impairment individuals to clearly distinguish different colors, accurately name them, and maintain a natural visual appearance. This effectively helps color vision impairment individuals align their color naming with common language standards.

[0035] In step S101 of some embodiments, specifically, the original visual enhancement model is a neural network model to be trained. The model includes an encoder and a decoder, and the visual enhancement model is used to perform at least one of color, brightness and contrast enhancements based on the input image for color vision impairment. With color naming accuracy as the core optimization goal, it is used to transform the RGB image seen by normal people into an RGB image (i.e., a visually enhanced image) that is friendly to people with color vision impairment, so that people with color vision impairment can identify the corresponding correct color name based on the visually enhanced image.

[0036] Specifically, the target training set includes multiple original training images and their corresponding true color names; among them, the original training images refer to RGB images perceived by normal people; the true color names refer to color category labels defined according to the common language standard of people with normal color vision, which are the benchmarks used to train and evaluate the accuracy of color naming in the model. The common language standard refers to color category names that are commonly agreed upon, widely used, and achieve unambiguous communication among people with normal color vision, such as color category labels such as red, green, yellow, and brown.

[0037] Furthermore, the original training images can be obtained from image files, video frames, display input frames, or light fields.

[0038] For example, for an RGB image containing red flowers and green leaves, the corresponding real color names could be red and green.

[0039] Please see Figure 2 In some embodiments, the visual enhancement method for color vision impairment also includes, but is not limited to, steps S201 to S205: Step S201: Obtain the original color vision simulation model, and simulate the original training image using the color vision simulation model to obtain a color vision impairment simulation image.

[0040] Step S202: Perform color discrimination on the simulated image of color vision deficiency and determine the name of the simulated color discrimination.

[0041] Step S203: Calculate the color loss based on the simulated color discrimination name and the real color name to obtain the color perception loss value.

[0042] Step S204: Adjust the model parameters of the color perception simulation model based on the color perception loss value to update the color perception simulation model.

[0043] Step S205: Based on the updated color vision simulation model, simulate the original training image using the color vision simulation model until the color perception loss value meets the preset expected loss condition, and obtain the pre-trained color vision simulation model.

[0044] Please see Figure 3 In some embodiments, the color vision simulation model includes a color vision impairment simulation sub-model; wherein, the color vision impairment simulation sub-model is used to simulate human spatial perception in response to color vision impairment, and step S201 includes, but is not limited to, steps S301 to S304: Step S301: Obtain the training color vision deficiency type corresponding to the target training set.

[0045] Step S302: Obtain the human eye perception space data corresponding to the training color vision deficiency type.

[0046] Step S303: Obtain random noise data.

[0047] Step S304: Perform perceptual space transformation on the original training image based on the training color vision deficiency type, human eye perception space data, and random noise data to obtain a color vision deficiency simulation image.

[0048] In step S301 of some embodiments, specifically, the original color vision simulation model is a neural network model to be trained, which includes a color vision impairment simulation sub-model and a color perception simulation sub-model; wherein, the color vision impairment simulation sub-model is a neural network model built based on the LMS cone response mechanism of biological vision, and the color vision impairment simulation sub-model is used to simulate human eye spatial perception for color vision impairment (such as red-green color blindness); the color perception simulation sub-model is a neural network model built based on vision and color language (such as visual Transformer and Text-Embedding-3-Large, the third-generation large model of text embedding), which is used to perform color semantic discrimination for color vision impairment to help people with color vision impairment name the colors they perceive in the color vision impairment simulation images.

[0049] Specifically, the training focuses on color vision impairment types defined as visual perception disorders caused by a shift or confusion in the photosensitivity of any one of the three types of cone cells (L, M, S). These types include, but are not limited to, impairments caused by dysfunction of type I cone cells (i.e., L, long-wavelength sensitive cone cells) (such as red-blindness or red-weakness), impairments caused by dysfunction of type II cone cells (i.e., M, medium-wavelength sensitive cone cells) (such as green-blindness or green-weakness), impairments caused by dysfunction of type III cone cells (i.e., S, short-wavelength sensitive cone cells) (such as blue-blindness or blue-weakness), and cone cell-related achromatopsia. Color vision impairment can manifest as a shift in the sensitivity curve of any one of the L, M, or S cone cells towards the sensitivity curve of another cone cell. In terms of response values, this can be manifested as the response value of any channel corresponding to L, M, or S being mixed with the response value of another channel, or as a significant decrease in the response value of any channel corresponding to L, M, or S.

[0050] Furthermore, the training color vision impairment type can be converted into a corresponding impairment type transformation matrix through the color vision impairment simulation sub-model, which can then be used for subsequent perceptual space transformation.

[0051] Specifically, the transformation matrix of the impairment type refers to the transformation matrix related to the type of color vision impairment and its severity, and is used to simulate individual differences in different types of color vision impairment and their severity.

[0052] The specific obstacle type transformation matrix is ​​determined based on the actual application scenario.

[0053] For example, for a moderate level of green weakness, the obstacle type transformation matrix can be:

[0054] The second row of the aforementioned transformation matrix indicates that the response value of the M cone cell is the sum of the response values ​​of the L cone cell and the M cone cell under normal conditions, and It can be 0.56. It can be 0.37. and The ratio of M and L cone cells is used to correct for differences in value range across cone cell channels.

[0055] Similarly, for mild green-blindness, the obstacle type transformation matrix can be:

[0056] Similarly, for a moderate level of red weakness, the obstacle type transformation matrix can be:

[0057] Similarly, for mild red-blindness, the obstacle type transformation matrix can be:

[0058] Similarly, for a moderate level of blue weakness, the obstacle type transformation matrix can be:

[0059] Similarly, for moderate blue-blindness, the obstacle type transformation matrix can be:

[0060] In step S302 of some embodiments, specifically, the human eye perception space data refers to the transformation matrix of the conventional RGB color space into a color space that conforms to the physiological perception of normal color vision populations (such as the LMS cone cell response space). This human eye perception space data takes into account the visual sensitivity curve characteristics of a single cone cell (i.e., the response functions of L, M, and S cone cells to different wavelengths of light), the distribution characteristics of the number of cone cells (i.e., the relative proportions of L, M, and S cone cells on the retina), and the number of incident photons (i.e., physical quantities related to light intensity and spectral energy distribution).

[0061] For example, spatial data perceived by the human eye can be:

[0062] in, This represents spatial data perceived by the human eye; the first row of the matrix represents the total response coefficient of L cone cells, which is the effective stimulus weight of the light from the R, G, and B channels to L cone cells from left to right; the second row of the matrix represents the total response coefficient of S cone cells; and the third row of the matrix represents the total response coefficient of M cone cells. This represents the scaling of the perceptual space values, used to adjust the range of values ​​represented in the perceptual space, without affecting the correspondence between RGB and LMS.

[0063] In step S303 of some embodiments, specifically, the random noise data is a random perturbation signal introduced during the simulation process to simulate Gaussian noise present in the cone cell perception process of the human eye visual system.

[0064] Specifically, the noisy obstacle space feature refers to the color-damaged and perturbed cone cell response value represented in the LMS color space after the obstacle type transformation matrix is ​​applied and random noise is superimposed.

[0065] In step S304 of some embodiments, specifically, the color vision impairment simulation image is the LMS impairment image perceived by the human eye of a person with color vision impairment.

[0066] Specifically, the obstacle type transformation matrix, random noise data, and human eye perception space data can be transformed into obstacle perception space through the color vision impairment simulation sub-model to obtain the noisy obstacle perception space features, and then output the color vision impairment simulation image.

[0067] Furthermore, the spatial data for sensing noisy obstacles can be determined using the following formula:

[0068] in, Represents the noisy perceptual spatial characteristics of the i-th image patch in a simulated image of color vision impairment; This represents the transformation matrix of the color vision deficiency type corresponding to the CVD training. This represents spatial data perceived by the human eye. This represents the color value of the i-th image patch in the original training image in the RGB color space, and n represents random noise data.

[0069] Specifically, the red, green, and blue channels in the RGB space are mapped to the response values ​​of L-cone cells, M-cone cells, and S-cone cells in the noisy impairment perception space features. The outputs of the three LMS channels are then fused to determine the simulated image of color vision impairment.

[0070] It should be understood that in this embodiment, perceptual space transformation can be achieved based on human eye perceptual space data, random noise data, and training color vision disorder types, but there is no difference in the order of the steps of obtaining human eye perceptual space data, random noise data, and training color vision disorder types.

[0071] Through steps S301 to S304, the perceptual space transformation of the original training image can be achieved by combining the training color vision disorder type, human eye perception space data and random noise data. This effectively simulates the neural noise and individual perceptual fluctuations existing in the real human eye visual system, generating a simulated image that not only realistically reflects the visual confusion characteristics of people with color vision disorders, but also includes natural perceptual fluctuations of the human eye, significantly improving the accuracy of color vision disorder image simulation.

[0072] Please see Figure 4 In some embodiments, the color vision simulation model includes a color perception simulation sub-model; wherein, the color perception simulation sub-model is used to perform color semantic discrimination based on the manifestations of color vision impairment, and step S202 includes, but is not limited to, steps S401 to S404: Step S401: Perform feature encoding on the color vision impairment simulation image to obtain the impairment image features.

[0073] Step S402: Determine the candidate color embedding vector corresponding to the color vision impairment simulation image according to the preset color name embedding vocabulary.

[0074] Step S403: Calculate the similarity between the obstacle image features and the candidate color embedding vectors to obtain the image color similarity.

[0075] Step S404: Determine the simulated color discrimination name corresponding to the simulated image of color vision impairment based on the image color similarity.

[0076] In step S401 of some embodiments, specifically, the obstacle image features refer to the visual vector representation extracted from the color vision impairment simulation image.

[0077] Specifically, the simulated image of color vision impairment can be divided into image blocks to obtain simulated image blocks of color vision impairment. The simulated image blocks of color vision impairment can be embedded using a visual Transformer model to obtain image block embedding vectors. The image block embedding vectors can then be positionally encoded to obtain image block embedding position vectors. Furthermore, the color features of the image block embedding position vectors can be captured through a self-attention mechanism to output the features of the impairment image blocks.

[0078] In step S402 of some embodiments, specifically, the color name embedding vocabulary refers to a lookup table that stores the real color names and the corresponding color name semantic vectors.

[0079] Specifically, the candidate color embedding vector refers to the set of color name semantic vectors corresponding to all simulated image patches with color vision impairment selected from the color name embedding vocabulary.

[0080] Specifically, the Text-Embedding-3-Large model can be used to perform word embedding processing on all real color names stored in the color name embedding vocabulary to obtain the color name semantic vector corresponding to each real color name.

[0081] For example, for a simulated image patch of color vision impairment in orange-red, the candidate color embedding vector corresponding to the image patch may be the semantic vector of the color name corresponding to brownish-red, red, vermilion, brown, and orange, etc.

[0082] In step S403 of some embodiments, specifically, the image color similarity is the degree of closeness between the obstacle image features and the candidate color embedding vector in the vector space, used to measure the degree of matching between the color vision impairment simulation image and its corresponding color name.

[0083] Specifically, the similarity between each obstacle image patch feature and each candidate color embedding vector in the obstacle image features can be calculated using the cosine similarity algorithm to obtain the image patch color similarity between the obstacle image patch features and each candidate color embedding vector, thereby determining the image color similarity.

[0084] For example, for brownish-red obstacle image patch features, the similarity with brownish-red candidate color embedding vectors is high, while the similarity with orange candidate color embedding vectors is low.

[0085] In step S404 of some embodiments, specifically, the simulated color discrimination name refers to the color naming label perceived by a simulated color vision disorder person in response to a simulated color vision disorder image, and the color naming label is consistent with the description of the real color name.

[0086] Specifically, the color distribution of the candidate color embedding vector corresponding to the feature of each obstacle image block can be calculated based on the image color similarity to obtain the probability distribution of the image block color name. The image block color discrimination name can be determined based on the probability distribution of the image block color name, and the simulated color discrimination name of the color vision impairment simulation image can be determined based on the image block color discrimination name.

[0087] The image patch color similarity can be normalized using the Softmax function to convert the similarity values ​​into a probability distribution of image patch color names.

[0088] Specifically, a greedy decoding strategy can be used to directly select the color name with the highest probability as the image block color discrimination name corresponding to the color vision impairment simulation image block.

[0089] For example, if the probability distribution of image patch colors is: red is 0.65, orange is 0.25, and yellow is 0.10, then the image patch color classification name corresponding to this image patch is red.

[0090] Furthermore, the frequency of occurrence of color discrimination names for all image blocks can be statistically analyzed, and the simulated color discrimination names for color vision impairment simulation images can be determined according to preset rules (such as selecting the top K most frequent color names).

[0091] For example, if a simulated image of color vision impairment contains multiple color regions such as red, green, and brown, and the frequency of the color discrimination names of the image patches is from high to low as red, green, brown, yellow, and blue, then the set of the three most frequent color names, namely red, green, and brown, is selected as the simulated color discrimination name corresponding to the image.

[0092] Please see Figure 5 In some embodiments, step S203 includes, but is not limited to, steps S501 to S503: Step S501: Obtain the image block color discrimination name corresponding to each image block in the color vision impairment simulation image according to the simulated color discrimination name, and obtain the probability distribution of the image block color name corresponding to the image block discrimination name.

[0093] Step S502: Calculate the image patch loss based on the probability distribution of image patch color names and the true color names to obtain the image patch color loss value.

[0094] Step S503: Determine the color perception loss value based on the color loss value of the image patch.

[0095] In step S501 of some embodiments, specifically, the image patch color discrimination name refers to the color naming label perceived by the simulated color vision disorder group for each color vision disorder simulation image patch, and the color naming label is consistent with the expression of the real color name.

[0096] Specifically, the probability distribution of color names in an image patch represents the probability distribution of each color vision impairment simulation image patch belonging to each color name in the color name embedding vocabulary.

[0097] For example, the color probability distribution for a brownish-red image patch could be: brownish-red 0.75, red 0.20, brown 0.10, and orange 0.05, etc., meaning that the color name probability distribution of this image patch indicates that the image patch has a 75% probability of being identified as red.

[0098] In step S502 of some embodiments, specifically, the image patch color loss value refers to the degree of deviation between the color image patch color discrimination name and the real color name of each color vision impairment simulation image patch.

[0099] Specifically, the probability distribution of image patch color names and the true color names can be effectively identified according to the preset indicator function to obtain the effective image patch color probability, and the image patch color loss value can be obtained by calculating the effective image patch color probability.

[0100] The indicator function is a mathematical function that selects an index based on the real color name. It is used to extract the single probability value of the corresponding real color name from different category probability distributions.

[0101] Specifically, the effective image patch color probability refers to the single probability value extracted from the probability distribution of image patch color names that corresponds to the true color name of the image patch. It is used to characterize the confidence level of the color vision simulation model in predicting the correct color of the image patch.

[0102] Specifically, the image patch color discrimination name corresponding to the probability distribution of the image patch color name is compared with the real color name through the indicator function. If and only if the image patch color discrimination name is consistent with the real color name, the image patch color discrimination name is determined to be a valid color discrimination name, and the image patch color probability value corresponding to the valid color discrimination name is 1, that is, the image patch color probability value corresponding to the valid color discrimination name is a valid image patch color probability; otherwise, it is 0.

[0103] For example, if the true color name is red, and the image patch color name discrimination name corresponding to the probability distribution of image patch color names can be red (0.7), orange (0.2), and yellow (0.1), then the indicator function is only 1 at the red position, thus the probability of extracting the effective image patch color is 0.7.

[0104] Specifically, the higher the probability of a valid image patch color, the closer its corresponding negative logarithm value is to 0, indicating a smaller image patch color loss value; conversely, the lower the probability of a valid image patch color, the larger its corresponding negative logarithm value, indicating a larger image patch color loss value.

[0105] For example, if the effective image patch color probability is 0.7, the image patch color loss value can be -log(0.7)≈0.36. If the effective image patch color probability is 0.1, the image patch color loss value jumps to -log(0.1)≈2.3, indicating that there is a large deviation between the image patch color discrimination name and the real color name.

[0106] In step S503 of some embodiments, specifically, the color perception loss value refers to the degree of deviation between the simulated color discrimination name and the real color name.

[0107] Specifically, the color loss values ​​of all image blocks in the color vision impairment simulation image are summed to obtain the global image color loss value. The global image color loss value is then divided by the total number of color vision impairment simulation image blocks to obtain the color loss value of the color vision impairment simulation image.

[0108] Through steps S501 to S503, the indicator function can determine the probability of valid image block colors corresponding to the real color names, avoid interference from irrelevant color names, and use negative logarithmic processing to amplify the low confidence penalty, thereby improving the accuracy of color name discrimination. Furthermore, the global color loss value is obtained by summing and averaging, which provides direction for the optimization of color vision simulation models and effectively helps people with color vision disorders to calibrate the consistency of color naming with public language standards.

[0109] In some specific embodiments of this application, the color perception loss value can be determined using the following formula:

[0110] in, Let represent the color perception loss value, i represent the patch index of the simulated image of color vision impairment, and j represent the index of the real color name. This represents the color discrimination name of the image block corresponding to the i-th simulated image block with color vision deficiency. This represents the name of the j-th real color. The indicator function (i.e., the indicator function value is 1 when the color discrimination name of the image block corresponding to the i-th color vision impairment simulation image block is the same as the j-th real color name) This represents a simulated image of color vision deficiency (CVD). This represents the noisy spatial features of the i-th image patch in a simulated image of color vision deficiency. Let represent the probability that the i-th image patch belongs to the j-th real color name in the color vision impairment simulation image.

[0111] In step S204 of some embodiments, specifically, the color perception loss value can be calculated as the optimization target through the backpropagation algorithm. The gradient of the weights and bias parameters of the color vision simulation model with respect to the loss value can be determined based on the color perception loss value. The weights and bias parameters can be fine-tuned along the gradient in the opposite direction through the gradient descent algorithm to obtain the updated color vision simulation model.

[0112] For example, if the color vision simulation model misclassifies the name "green" in a color vision impairment simulation image as "brown", the color loss value will be high. The weights and bias parameters of the name "green" can be adjusted according to the gradient guidance of the color vision impairment simulation image to enhance the color vision simulation model's ability to distinguish between the names "green" and "brown".

[0113] In step S205 of some embodiments, specifically, the pre-trained color vision simulation model is a trained neural network model used to identify color names for the original training image.

[0114] Specifically, the preset expected loss condition can be a preset color loss threshold. When the color loss value is lower than the color loss threshold, it means that the color vision simulation model has been trained, that is, the model can accurately simulate color vision impairment images and distinguish color names.

[0115] For example, the color loss threshold can be 0.4.

[0116] Through steps S201 to S205, the color vision simulation model is iteratively trained, which effectively improves the accuracy of the model in simulating color vision impairment images and color name discrimination. This ensures that the simulated color vision impairment images conform to the visual perception of people with color vision impairment and guides them in color naming. It further helps people with color vision impairment to calibrate the consistency of color naming with public language standards.

[0117] In step S102 of some embodiments, specifically, the visual enhancement image refers to the RGB obstacle image perceived by a person with color vision impairment in relation to the original training image.

[0118] Specifically, visual enhancement of the original training image using a visual enhancement model to obtain a visually enhanced image can include: encoding the original training image using a visual enhancement model to obtain image visual features; performing visual space transformation on the image visual features to obtain visual optimization features; and reconstructing the image using the visual optimization features to obtain the visually enhanced image. The visual optimization features include at least one of color optimization features, brightness optimization features, and contrast optimization features. The specific visual space transformation is determined based on the type and severity of the color vision impairment during training, and the adjustment of image pixels in the RGB channels is not limited here.

[0119] For example, if the original training image is an RGB image containing red flowers and green leaves, the encoder of the visual enhancement model can extract the image visual features such as color, texture, and structure of the red flowers and green leaves. Then, the image pixels are adjusted to meet the color optimization features perceived by people with color vision impairment. The decoder further reconstructs the image based on the color optimization features to make the original red flowers more yellow and the green leaves more blue, thus enabling people with color vision impairment to more clearly distinguish the color differences between flowers and leaves and correctly name the colors of objects in the image.

[0120] Specifically, the visual enhancement process can be represented by the following formula:

[0121] in, This represents training images for color vision deficiency. Indicates the enhancement of model parameters through vision. Defined visual enhancement model Convert the original training image I into a visually enhanced image. .

[0122] In this embodiment, the original training images are visually enhanced using a visual enhancement model, achieving a preliminary conversion from normal images to color vision impairment-friendly images, providing image data that is closer to the visual perception of people with color vision impairment for subsequent color discrimination.

[0123] In step S103 of some embodiments, specifically, the target color discrimination name refers to the color naming label that simulates the perception of a person with color vision impairment in response to a visually enhanced image, and the color naming label is also consistent with the expression of the real color name.

[0124] Specifically, the method for obtaining the target color name by performing color discrimination on visually enhanced images through color vision simulation models is the same as the method for determining the simulation color name by performing color discrimination on simulated images of color vision impairment, and will not be elaborated here.

[0125] In this embodiment, the color perception simulation model is used to perform color discrimination on the visual enhancement image, which can effectively simulate the color naming behavior of people with color vision impairment, provide accurate target color names for the optimization of the visual enhancement model, and improve the accuracy of the visual enhancement model in generating images of people with color vision impairment.

[0126] Please see Figure 6 In some embodiments, step S104 includes, but is not limited to, steps S601 to S603: Step S601: Calculate the image structure loss based on the visual enhancement image and the original training image to obtain the image structure loss value.

[0127] Step S602: Calculate the color loss based on the target color discrimination name and the real color name to obtain the enhanced color loss value.

[0128] Step S603: Perform weighted processing based on the image structure loss value and the enhanced color loss value to obtain the target loss value.

[0129] Please see Figure 7 In some embodiments, step S601 includes, but is not limited to, steps S701 to S703: Step S701: Determine the visual enhancement pixel distribution data of the visual enhancement image.

[0130] Step S702: Determine the original pixel distribution data of the original training image.

[0131] Step S703: Perform loss calculation based on the visual enhancement pixel distribution data and the original pixel distribution data to obtain the image structure loss value.

[0132] In step S701 of some embodiments, specifically, the visual enhancement pixel distribution data refers to the pixel value distribution data of the visual enhancement image within a preset calculation window. The visual enhancement pixel distribution data includes the enhanced image mean and the enhanced image variance. The enhanced image mean refers to the average pixel value of the visual enhancement image in each color channel in the RGB space, which is used to characterize the overall brightness of the visual enhancement image. The enhanced image variance represents the degree of dispersion of the pixel value distribution in each color channel of the visual enhancement image in the RGB space, which is used to characterize the contrast of the visual enhancement image.

[0133] Specifically, for example, the average pixel value on the red channel of a visually enhanced image containing red flowers and green leaves can be 120, and the variance of the green channel of the visually enhanced image can be 50.

[0134] In step S702 of some embodiments, specifically, the original pixel distribution data refers to the pixel value distribution data of the original training image within a preset calculation window. The original pixel distribution data includes the original image mean and the original image variance. The original image mean refers to the average pixel value of the original training image in each color channel in the RGB space, which is used to characterize the overall brightness of the original training image. The original image variance represents the degree of dispersion of the pixel value distribution in each color channel of the original training image in the RGB space, which is used to characterize the contrast of the original training image.

[0135] For example, the average pixel value on the red channel of the original training image containing red flowers and green leaves can be 255, and the variance of the green channel of the original training image can be 30.

[0136] In step S703 of some embodiments, specifically, the image structure loss value refers to the degree of deviation between the visually enhanced image and the original training image in terms of image structure, contrast, and brightness.

[0137] Specifically, the covariance can be calculated based on the variance of the enhanced image and the variance of the original image to obtain the image covariance. The image mean constant and image variance constant of the visually enhanced image can then be obtained. Based on the image covariance, image mean constant, image variance constant, visually enhanced pixel distribution data, and original pixel distribution data, the loss can be calculated to obtain the image structure loss value.

[0138] Image covariance describes the similarity between the original training image and the visually enhanced image in terms of structural features (such as texture, edges, and shape). The larger the image covariance, the more similar the structures between the original training image and the visually enhanced image. Image mean constant and image variance constant are hyperparameters used to adjust the calculation of image structure loss. They are usually set according to experimental experience, such as 0.0001 for image mean constant and 0.0009 for image variance constant.

[0139] It should be noted that the statistical distribution data and constant values ​​used in the above image structure loss calculation are for illustrative purposes only and are not intended to be limiting.

[0140] Specifically, the image structure loss value can be determined using the following formula:

[0141] in, Represents the original training image I and the visually enhanced image. Image structure loss values ​​between This represents the original image mean of the original training image I. The mean of the augmented image represents the visually enhanced image. Represents the constant of the image mean. Represents the original training image I and the visually enhanced image. Image covariance between This represents the original image variance of the original training image I. This represents the variance of the visually enhanced image.

[0142] Through steps S701 to S703, the differences in image structure, contrast, and brightness between the visually enhanced image and the original training image can be measured using visually enhanced pixel distribution data and original pixel distribution data. This avoids the situation where excessive adjustment of image hue to improve color discrimination leads to image distortion. It effectively enhances the discrimination of color vision impairment images while retaining the key structural features of the original training images. This ensures that the generated color vision impairment images can meet the visual needs of people with color vision impairment without affecting the visual experience of people with normal color vision, thus improving the accuracy of visual enhancement.

[0143] In step S602 of some embodiments, specifically, the enhanced color loss value refers to the degree of deviation between the target color discrimination name and the true color name.

[0144] Specifically, the method for calculating the enhanced color loss value based on the target color discrimination name and the real color name is the same as the method for calculating the color loss value based on the simulated color discrimination name and the real color name, and will not be repeated here.

[0145] In step S603 of some embodiments, specifically, the target loss value is the overall loss that combines the image structure loss value and the enhanced color loss value. It is used to characterize the naturalness of the color vision impairment image generated by the visual enhancement model under the color constraint that the color vision simulation model predicts the real color name more likely than other color names.

[0146] Specifically, the color constraints can be:

[0147] Where i represents the patch index of the simulated image for color vision deficiency, and j represents the index of the actual color name. This indicates the parameters of the color vision simulation model. A defined color vision simulation model is used to simulate the empirical estimation process from LMS to stimulus values ​​to color naming perception. This indicates the target color discrimination name corresponding to the visually enhanced image. This represents the name of the j-th real color. This refers to visually enhanced images that represent color enhancement for individuals with color vision deficiencies. Image representing visual enhancement after color enhancement The noisy obstacle spatial features of the i-th image patch.

[0148] Specifically, the target loss value can be determined using the following formula:

[0149] in, Indicates the target loss value; This represents the balance coefficient, which can take a value of 0.7. Indicates visually enhanced images The enhanced color loss value is used to characterize the probability of the predicted color distribution of the real color name corresponding to the visually enhanced image under the pre-trained color vision simulation model. Represents the original training image I and the visually enhanced image. The image structure loss value between.

[0150] By combining image structure loss and color enhancement loss through steps S601 to S603, it is possible to ensure that when optimizing the visual enhancement model, not only is the accuracy of color name recognition improved for people with color vision impairment, but also the structural naturalness of the generated color vision impairment image is ensured, avoiding image distortion caused by over-adjustment of the hue of the impairment image. While helping people with color vision impairment to calibrate the consistency of color naming with public language standards, the accuracy of visual enhancement is also improved.

[0151] In step S105 of some embodiments, specifically, the calculated target loss value can be used as the global optimization target through the backpropagation algorithm. The gradient of the weights and bias parameters of the visual enhancement model relative to the loss value is determined based on the target loss value. Subsequently, the gradient descent algorithm is used to fine-tune the weights and bias parameters of the visual enhancement along the opposite direction of the gradient to obtain the updated visual enhancement model.

[0152] For example, if the visual enhancement model over-adjusts the contrast of the color vision impairment image to improve color discrimination, resulting in an increase in structural similarity loss, or if the color vision simulation model still fails to correctly identify color names after adjustment, resulting in persistently high color loss, then the target loss value will be high. The gradient signal will integrate the errors of image structure loss and color loss to guide the visual enhancement to adjust the weights and bias parameters of the encoder and decoder, so as to learn a generation strategy that achieves a better balance between enhancing color guidance effect and maintaining the natural structure of the image.

[0153] In step S106 of some embodiments, specifically, the trained visual enhancement model is a trained neural network model used to generate RGB color vision impairment images that are friendly to people with color vision impairment and can guide them to correctly name colors.

[0154] Specifically, the preset target loss condition can be the comprehensive loss threshold. If the target loss value is lower than the comprehensive loss threshold, it can be determined that the image visual enhancement for color vision impairment is completed, indicating that the visual enhancement model has learned to generate color vision impairment images that achieve a balance between the accuracy of color naming guidance and the naturalness of image structure.

[0155] For example, if the overall loss threshold is 0.3, a trained visual augmentation model is obtained when the target loss value is below 0.3.

[0156] Please see Figure 8 In some embodiments, step S107 includes, but is not limited to, steps S801 to S803: Step S801: Obtain the target color vision impairment type of the target image and obtain the severity of the impairment corresponding to the target color vision impairment type; wherein, the target image is derived from an image file, video frame, display input frame or light field.

[0157] Step S802: Based on the pre-trained color vision simulation model, the type and severity of the target color vision impairment, the target image is color-discriminated to obtain the target image color name.

[0158] Step S803: Determine the visual optimization constraints of the target image based on the color discrimination name of the target image, and adjust the image pixels of the target image according to the visual optimization constraints to obtain the target visual enhancement image.

[0159] In step S801 of some embodiments, specifically, the target image refers to the RGB image to be converted for color vision deficiency as perceived by normal people, wherein the target image may be derived from an image file, video frame, display input frame or light field.

[0160] For example, a landscape image containing red, green, and brown.

[0161] Specifically, the target color vision disorder type refers to the visual perception disorder category determined by the shift or confusion in the light sensitivity of any of the three types of cone cells (L, M, S) in a person with color vision disorder. It also includes disorder types with abnormal function of type I cone cells (L), type II cone cells (M), and type III cone cells (S).

[0162] Specifically, the severity of the target color vision deficiency type describes the degree of color vision deficiency, including mild, moderate, or severe.

[0163] In step S802 of some embodiments, specifically, the target image color discrimination name refers to the color naming label perceived by people with color vision impairment for the target image, and the color naming label is also consistent with the expression of the real color name.

[0164] Specifically, the target image can be downsampled to obtain dimensionality-reduced image features; where dimensionality-reduced image features refer to low-dimensional vector representations that retain the core structure and semantic information of the image after reducing the spatial resolution of the target image.

[0165] For example, max pooling can be used to process a 1920x1080 pixel RGB image to obtain a dimensionality-reduced image feature of 480x270 pixels.

[0166] In this embodiment, the method of color discrimination of the reduced image features based on the pre-trained color vision simulation model, the type and severity of the target color vision impairment, and obtaining the target image color discrimination name is the same as the method of color discrimination of the visually enhanced image through the color vision simulation model and obtaining the target color discrimination name, and will not be described again here.

[0167] In this embodiment, by downsampling the target image, the computational resource requirements of the visual enhancement model can be reduced, while preserving sufficient image structure and semantic information for subsequent effective visual enhancement.

[0168] In step S803 of some embodiments, specifically, the visual optimization constraint means that the pre-trained color vision simulation model predicts the true color name of the target image with a higher probability than other color names.

[0169] Specifically, target visual enhancement images refer to RGB visual enhancement images perceived by people with color vision deficiencies in relation to target images.

[0170] Specifically, adjusting the image pixels of the target image using a visual enhancement model trained according to visual optimization constraints to obtain a target visually enhanced image can include: encoding the dimensionality-reduced image features using the visual enhancement model trained according to visual optimization constraints to obtain visual features of the target image; performing a visual space transformation on the visual features of the target image to obtain target visual enhancement features; and reconstructing the target visual enhancement features to obtain the target visually enhanced image, thereby converting the target image into an RGB visually enhanced image perceptible to people with color vision deficiencies, and accurately identifying the color names corresponding to the target visually enhanced image; wherein the target visual enhancement features include at least one of target color enhancement features, target brightness enhancement features, and target contrast enhancement features.

[0171] For example, given an original landscape image containing red, green, and brown, the color vision simulation model, upon receiving the input "weak green, intensity 0.6", outputs the color discrimination names as red, green, and brown. Based on the green weakness categories of red, green, and brown, the visual enhancement model adjusts the hue, brightness, and contrast of the red and green pixels in the original landscape image, enhancing the differentiation of green and yellow tones in the visually enhanced landscape image, so that people with color vision impairments can correctly distinguish red, green, and brown from the visually enhanced landscape image.

[0172] Through steps S801 to S803, the target image can be analyzed based on the type and degree of color vision impairment and the pre-trained color vision simulation model to generate visual optimization constraints for the performance of color vision impairment. The trained visual enhancement model generates a target visual enhancement image for the performance of color vision impairment, enabling people with color vision impairment to clearly distinguish different colors, accurately name different colors, and maintain a visually natural visual enhancement image. This significantly helps people with color vision impairment to calibrate the consistency of color naming with public language standards.

[0173] It should be understood that the training steps, exemplary training dataset, and implementation method involved in the above-mentioned visual enhancement method for color vision disorders are exemplary implementation descriptions and are not limited here. Even if a third party changes the training step order, modifies, or implements a training dataset for the same purpose, it will not depart from the protection scope of this application.

[0174] This application first obtains a pre-trained color vision simulation model and a target training set containing real color names, providing a color naming cognitive benchmark for the subsequent model training process. Second, a visual enhancement model generates visually enhanced images based on the original training images, targeting the manifestations of color vision impairment. This achieves an initial conversion from normal images to color vision-friendly images. The color vision simulation model then performs color discrimination on the original training images, simulating the real color naming behavior of people with color vision impairment and effectively capturing their cognitive process from visual signals to standard naming in common language. Furthermore, a target loss is calculated based on the visually enhanced images, the original training images, the target color discrimination names, and the real color names, obtaining a target loss value that allows color naming accuracy to be used as an optimization metric. The goal is to ensure that the generated color vision impairment images can guide people with color vision impairment to make correct color names. Loss calculations are performed by combining the original training images and visually enhanced images, effectively avoiding distortion caused by pursuing high discrimination. Finally, the visual enhancement model is iteratively updated until convergence based on the target loss value. This allows for precise adjustment of image colors according to common color language standards. The trained visual enhancement model enhances the target image in at least one of color, brightness, and contrast, ensuring that the generated visually enhanced images allow people with color vision impairment to clearly distinguish different colors, accurately name them, and maintain a natural visual appearance. This effectively helps people with color vision impairment align their color naming with common language standards.

[0175] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned visual enhancement method for color vision impairment. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0176] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the processing system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the visual enhancement method for color vision impairment according to the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0177] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual enhancement method for color vision impairment.

[0178] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0179] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0180] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0181] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0183] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0184] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0186] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0188] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method of visual enhancement for color vision impairment, characterized by, The method comprises: obtaining an original visual enhancement model and a target training set, and obtaining a pre-trained color vision color simulation model; wherein the visual enhancement model is used to perform visual enhancement on at least one of color, brightness and contrast for color vision impairment performance based on an input image; the target training set comprises an original training image and a true color name corresponding to the original training image; performing visual enhancement on the original training image through the visual enhancement model to obtain a visual enhancement image for color vision impairment performance; performing color discrimination on the visual enhancement image through the color vision color simulation model to obtain a target color discrimination name; performing target loss calculation according to the visual enhancement image, the original training image, the target color discrimination name and the true color name to obtain a target loss value; updating and detecting the visual enhancement model according to the target loss value to obtain an updated visual enhancement model; based on the updated visual enhancement model, returning to perform visual enhancement on the original training image through the visual enhancement model until the target loss value meets a preset target loss condition, and obtaining a trained visual enhancement model; obtaining a target image and performing visual enhancement on the target image through the trained visual enhancement model to obtain a target visual enhancement image.

2. The method of claim 1, wherein, Before the pre-trained color vision color simulation model is obtained, the method further comprises pre-training the color vision color simulation model, specifically comprising: obtaining an original color vision color simulation model and performing simulation image simulation on the original training image through the color vision color simulation model to obtain a color vision impairment simulation image; performing color discrimination on the color vision impairment simulation image to determine a simulation color discrimination name; performing color loss calculation according to the simulation color discrimination name and the true color name to obtain a color perception loss value; adjusting model parameters of the color vision color simulation model based on the color perception loss value to update the color vision color simulation model; based on the updated color vision color simulation model, returning to perform simulation image simulation on the original training image through the color vision color simulation model until the color perception loss value meets a preset expected loss condition, and obtaining the pre-trained color vision color simulation model.

3. The method of claim 2, wherein, The color vision color simulation model comprises a color vision impairment simulation sub-model; wherein the color vision impairment simulation sub-model is used to simulate human eye spatial perception for color vision impairment performance; the simulation image simulation on the original training image through the color vision color simulation model to obtain a color vision impairment simulation image comprises simulation image simulation on the original training image through the color vision impairment simulation sub-model to obtain a color vision impairment simulation image, specifically comprising: obtaining a training color vision impairment type corresponding to the target training set; obtaining human eye perception space data corresponding to the training color vision impairment type; obtaining random noise data; According to the training color vision disorder type, the human eye perception space data and the random noise data, the original training image is subjected to a perception space transformation to obtain the color vision disorder simulation image.

4. The method of claim 2, wherein, The color vision color simulation model comprises a color perception simulation sub-model; wherein the color perception simulation sub-model is used for color semantic discrimination for color vision disorder performance; The color discrimination for the color vision disorder simulation image determines the simulation color discrimination name, which specifically comprises: feature coding is performed on the color vision disorder simulation image to obtain a disorder image feature; According to a preset color name embedding word table, a candidate color embedding vector corresponding to the color vision disorder simulation image is determined; The similarity of the disorder image feature and the candidate color embedding vector is calculated to obtain an image color similarity; According to the image color similarity, the simulation color discrimination name corresponding to the color vision disorder simulation image is determined.

5. The method of claim 2, wherein, The color loss calculation according to the simulation color discrimination name and the real color name obtains a color perception loss value, which comprises: According to the simulation color discrimination name, an image block color discrimination name corresponding to each image block in the color vision disorder simulation image is obtained, and an image block color name probability distribution corresponding to the image block discrimination name is obtained; According to the image block color name probability distribution and the real color name, an image block loss value is calculated. According to the image block color loss value, the color perception loss value is determined.

6. The method of claim 1, wherein, The target loss calculation according to the visual enhancement image, the original training image, the target color discrimination name and the real color name obtains a target loss value, which comprises: According to the visual enhancement image and the original training image, an image structure loss value is calculated. According to the target color discrimination name and the real color name, an enhanced color loss value is calculated. According to the image structure loss value and the enhanced color loss value, the target loss value is determined.

7. The method of claim 6, wherein, The image structure loss calculation according to the visual enhancement image and the original training image obtains an image structure loss value, which comprises: determining the visual enhancement pixel distribution data of the visual enhancement image; determining the original pixel distribution data of the original training image; According to the visual enhancement pixel distribution data and the original pixel distribution data, a loss value is calculated to obtain the image structure loss value.

8. The method according to any one of claims 1 to 7, characterized in that, The visual enhancement model trained is used to perform visual enhancement on the target image to obtain a target visual enhancement image, which comprises: obtaining a target color vision disorder type of the target image, and obtaining a disorder severity corresponding to the target color vision disorder type; wherein the target image is derived from an image file, a video frame, a display input frame or a light field; According to the pre-trained color vision color simulation model, the target color vision disorder type and the disorder severity, color discrimination is performed on the target image to obtain a target image color discrimination name; According to the target image color discrimination name, a visual optimization constraint condition of the target image is determined, and the trained visual enhancement model is instructed to perform image pixel adjustment on the target image according to the visual optimization constraint condition to obtain a target visual enhancement image.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the visual enhancement method for color vision disorders according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the visual enhancement method for color vision disorders according to any one of claims 1 to 8.

Images