Visual enhancement method for color vision impairment, electronic device, and medium

Abstract

The embodiment of the application provides a visual enhancement method for color vision impairment, an electronic device and a medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing visual enhancement on an original training image through an original visual enhancement model to obtain a visual enhancement image for color vision impairment performance; performing color discrimination on the visual enhancement image through a color vision color simulation model to perform model training on the original visual enhancement model according to a target loss value of the visual enhancement image, the original training image, a target color discrimination name and a real color name, so as to perform visual enhancement on a target image through the trained visual enhancement model. According to the embodiment of the application, the color vision color simulation model is used to discriminate the color name of the visual enhancement image to simulate the color naming process perceived by the color vision impaired population, and then the visual enhancement model is trained in combination with the target loss value, so that the consistency of the color naming of the color vision impaired population and the public language standard is effectively helped.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a visual enhancement method for color vision impairment, an electronic device and a medium. BACKGROUND

[0002] Traditional visual enhancement methods usually simulate and compensate color conversion (such as the Brettel model), linearly transform the RGB (Red, Green, Blue) image into a space mapping, map the RGB image into the color space of LMS (Long / Medium / Short wavelength-sensitive cone cells), and replace the color in the confusion color range with the non-confusion color. For example, in the traffic sign optimization scenario, the RGB image of the green cone cylinder can be simulated into an LMS image by the Brettel model, and the hue of the green cone cylinder is replaced with blue (non-confusion color), so that color vision impaired people can distinguish between green and red cone cylinders. However, this method only improves the color discrimination degree by simply adjusting the hue, and does not consider the cognitive process of color vision impaired people from visual signals to color naming, so that color vision impaired people can only recognize the difference between object colors, and cannot accurately describe the correct object color using public language, making it difficult for color vision impaired people to achieve correct color naming. Therefore, how to help color vision impaired people calibrate the consistency of color naming and public language standards has become a problem to be solved. SUMMARY

[0003] The main purpose of the embodiments of the present application is to propose a visual enhancement method for color vision impairment, an electronic device and a medium, which aims to help color vision impaired people calibrate the consistency of color naming and public language standards.

[0004] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application proposes a visual enhancement method for color vision impairment, which comprises:

[0005] An original visual enhancement model and a target training set are obtained, and a pre-trained color vision color simulation model is obtained; wherein the visual enhancement model is used to perform at least one of color, brightness and contrast visual enhancement for color vision impairment performance based on an input image; the target training set includes an original training image and a real color name corresponding to the original training image;

[0006] The original training image is visually enhanced by the visual enhancement model to obtain a visually enhanced image for color vision impairment performance;

[0007] color discrimination of the visual enhanced image is performed through the color vision color simulation model to obtain a target color discrimination name;

[0008] Target loss calculation is performed according to the visual enhanced image, the original training image, the target color discrimination name and the real color name to obtain a target loss value;

[0009] The visual enhanced model is updated and detected according to the target loss value to obtain an updated visual enhanced model;

[0010] Based on the updated visual enhanced model, visual enhancement of the original training image is performed through the visual enhanced model until the target loss value meets a preset target loss condition, and a trained visual enhanced model is obtained.

[0011] A target image is obtained, and the target image is visually enhanced through the trained visual enhanced model to obtain a target visual enhanced image.

[0012] In some embodiments, before the pre-trained color vision color simulation model is obtained, the color vision color simulation model is pre-trained, specifically including:

[0013] An original color vision color simulation model is obtained, and a simulation image of the original training image is simulated through the color vision color simulation model to obtain a color vision impairment simulation image;

[0014] Color discrimination is performed on the color vision impairment simulation image to determine a simulation color discrimination name;

[0015] Color loss calculation is performed according to the simulation color discrimination name and the real color name to obtain a color perception loss value;

[0016] Based on the color perception loss value, model parameter adjustment is performed on the color vision color simulation model to update the color vision color simulation model;

[0017] Based on the updated color vision color simulation model, simulation image simulation of the original training image is performed through the color vision color simulation model until the color perception loss value meets a preset expected loss condition, and the pre-trained color vision color simulation model is obtained.

[0018] In some embodiments, the color vision color simulation model includes a color vision impairment simulation sub-model; wherein the color vision impairment simulation sub-model is used to simulate spatial perception of the human eye for color vision impairment performance;

[0019] The simulation image simulation of the original training image by the color vision color simulation model obtains a color vision impairment simulation image, and specifically includes:

[0020] Obtaining a training color vision impairment type corresponding to the target training set;

[0021] Obtaining human eye perception space data corresponding to the training color vision impairment type;

[0022] Obtaining random noise data;

[0023] According to the training color vision impairment type, the human eye perception space data and the random noise data, the original training image is subjected to a perception space transformation to obtain the color vision impairment simulation image.

[0024] In some embodiments, the color vision color simulation model includes a color perception simulation sub-model; wherein the color perception simulation sub-model is used for color semantic discrimination for color vision impairment performance;

[0025] The color discrimination for the color vision impairment simulation image determines a simulation color discrimination name, which includes color discrimination for the color vision impairment simulation image by the color perception simulation sub-model to determine a simulation color discrimination name, and specifically includes:

[0026] Feature encoding is performed on the color vision impairment simulation image to obtain an impairment image feature;

[0027] According to a preset color name embedding word table, a candidate color embedding vector corresponding to the color vision impairment simulation image is determined;

[0028] The impairment image feature and the candidate color embedding vector are subjected to similarity calculation to obtain an image color similarity;

[0029] According to the image color similarity, the simulation color discrimination name corresponding to the color vision impairment simulation image is determined.

[0030] In some embodiments, the color loss calculation according to the simulation color discrimination name and the real color name obtains a color perception loss value, which includes:

[0031] According to the simulation color discrimination name, an image block color discrimination name corresponding to each image block in the color vision impairment simulation image is obtained, and an image block color name probability distribution corresponding to the image block discrimination name is obtained;

[0032] According to the image block color name probability distribution and the real color name, an image block loss value is obtained by image block loss calculation.

[0033] determine the color perception loss value according to the image block color loss value.

[0034] In some embodiments, the target loss value is obtained by performing target loss calculation according to the visually enhanced image, the original training image, the target color discrimination name and the true color name.

[0035] perform image structure loss calculation according to the visually enhanced image and the original training image to obtain an image structure loss value;

[0036] perform color loss calculation according to the target color discrimination name and the true color name to obtain an enhanced color loss value;

[0037] determine the target loss value according to the image structure loss value and the enhanced color loss value.

[0038] In some embodiments, the image structure loss value is obtained by performing image structure loss calculation according to the visually enhanced image and the original training image, including:

[0039] determine an obstacle image pixel distribution of the visually enhanced image;

[0040] determine an original image pixel distribution of the original training image;

[0041] perform loss calculation according to the obstacle image pixel distribution and the original image pixel distribution to obtain the image structure loss value.

[0042] In some embodiments, the target visually enhanced image is obtained by performing visual enhancement on the target image by the trained visually enhanced model, including:

[0043] obtain a target color vision obstacle type of the target image, and obtain an obstacle severity corresponding to the target color vision obstacle type; wherein the target image is derived from an image file, a video frame, a display input frame or a light field;

[0044] perform color discrimination on the target image according to the pre-trained color vision color simulation model, the target color vision obstacle type and the obstacle severity to obtain a target image color discrimination name;

[0045] determine a visual optimization constraint condition of the target image according to the target image color discrimination name, and instruct the trained visually enhanced model to perform image pixel adjustment on the target image according to the visual optimization constraint condition to obtain a target visually enhanced image.

[0046] To achieve the above object, a second aspect of the embodiment of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0047] To achieve the above object, a third aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0048] The visual enhancement method for color vision impairment, the electronic device and the medium provided by the present application firstly provide a color naming cognition benchmark for subsequent model training process by obtaining a pre-trained color vision color simulation model and a target training set containing real color names; secondly, a visual enhancement image for color vision impairment performance based on the original training image is generated through the visual enhancement model, realizing the preliminary conversion from normal image to color vision impairment friendly image, and the color discrimination of the original training image is performed by using the color vision color simulation model, which can simulate the real color naming behavior of the color vision impairment population and effectively capture the cognitive process from visual signal to public language standard naming of the color vision impairment population; further, target loss calculation is performed according to the visual enhancement image, the original training image, the target color discrimination name and the real color name to obtain a target loss value, which can take color naming accuracy as an optimization target to ensure that the generated color vision impairment image can guide the color vision impairment population to make correct color naming, and the loss calculation is also performed in combination with the original training image and the visual enhancement image, which effectively avoids the distortion of the generated color vision impairment image caused by the pursuit of discrimination; finally, the visual enhancement model is iteratively updated based on the target loss value until convergence, which can accurately adjust the image color based on the public color language standard, so as to perform at least one of color, brightness and contrast visual enhancement on the target image through the trained visual enhancement model, thereby ensuring that the trained visual enhancement model generates a visual enhancement image that can clearly distinguish different colors, accurately name different colors and maintain visual naturalness, effectively helping the color vision impairment population to calibrate the consistency of color naming and public language standard. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of the visual enhancement method for color vision impairment provided by the embodiment of the present application;

[0050] Figure 2 is another flowchart of the visual enhancement method for color vision impairment provided by the embodiment of the present application;

[0051] Figure 3 is Figure 2 the flowchart of step S201 in

[0052] Figure 4 is Figure 2 a flowchart of step S202 in

[0053] Figure 5 is Figure 2 a flowchart of step S203 in

[0054] Figure 6 is Figure 1 a flowchart of step S104 in

[0055] Figure 7 is Figure 6 a flowchart of step S601 in

[0056] Figure 8 is Figure 1 a flowchart of step S107 in

[0057] Figure 9 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0059] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0061] First, several terms involved in the present application are analyzed:

[0062] Artificial intelligence (AI): is a new technical science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, artificial intelligence attempts to understand the essence of intelligence, and produce a new intelligent machine that can react in a similar way to human intelligence, including robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0063] The embodiment of the present application provides a visual enhancement method for color vision impairment, an electronic device and a medium, aiming to help the color vision impaired population to calibrate the consistency of color naming and public language standard.

[0064] The visual enhancement method for color vision impairment, the electronic device and the medium provided by the embodiment of the present application are specifically described through the following embodiments. First, the visual enhancement method for color vision impairment in the embodiment of the present application is described.

[0065] The embodiment of the present application can acquire and process related data based on artificial intelligence technology. Wherein, artificial intelligence (Artificial Intelligence, AI) is a theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0066] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0067] The method for visual enhancement for color vision impairment provided in the embodiments of the present application relates to the field of artificial intelligence. The method for visual enhancement for color vision impairment provided in the embodiments of the present application can be applied in a terminal, can also be applied in a server end, and can also be software running in a terminal or a server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as a separate physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application implementing the method for visual enhancement for color vision impairment, but is not limited to the above forms.

[0068] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0069] Figure 1 is an optional flowchart of the method for visual enhancement for color vision impairment provided in the embodiments of the present application, and Figure 1 The method in can include but is not limited to including steps S101 to S107.

[0070] Step S101, obtaining an original visual enhancement model and a target training set, and obtaining a pre-trained color vision color simulation model; wherein the visual enhancement model is used to perform at least one of color, brightness and contrast visual enhancement for color vision impairment performance based on an input image; the target training set includes an original training image and a real color name corresponding to the original training image.

[0071] Step S102, performing visual enhancement on the original training image by the visual enhancement model to obtain a visual enhancement image for color vision impairment performance.

[0072] Step S103, color discrimination of the visual enhancement image is performed through the color vision color simulation model to obtain a target color discrimination name.

[0073] Step S104, target loss calculation is performed according to the visual enhancement image, the original training image, the target color discrimination name and the real color name to obtain a target loss value.

[0074] Step S105, the visual enhancement model is updated and detected according to the target loss value to obtain an updated visual enhancement model.

[0075] Step S106, based on the updated visual enhancement model, visual enhancement of the original training image is performed through the visual enhancement model until the target loss value meets a preset target loss condition to obtain a trained visual enhancement model.

[0076] Step S107, a target image is obtained, and visual enhancement of the target image is performed through the trained visual enhancement model to obtain a target visual enhancement image.

[0077] The steps S101 to S107 shown in the embodiments of the present application first obtain a pre-trained color vision color simulation model and a target training set containing real color names to provide a color naming cognition benchmark for subsequent model training process; secondly, a visual enhancement image for color vision impairment performance based on the original training image is generated through the visual enhancement model to realize preliminary conversion from a normal image to a color vision impairment friendly image, and color discrimination of the original training image is performed through the color vision color simulation model to simulate real color naming behavior of the color vision impairment population and effectively capture the cognitive process of the color vision impairment population from visual signals to public language standard naming; further, target loss calculation is performed according to the visual enhancement image, the original training image, the target color discrimination name and the real color name to obtain a target loss value, which can take color naming accuracy as an optimization target to ensure that the generated color vision impairment image can guide the color vision impairment population to make correct color naming, and loss calculation is performed in combination with the original training image and the visual enhancement image to effectively avoid distortion of the generated color vision impairment image caused by pursuit of discrimination; finally, the visual enhancement model is iteratively updated based on the target loss value until convergence, which can accurately adjust the image color according to the public color language standard to perform at least one of color, brightness and contrast visual enhancement of the target image through the trained visual enhancement model, thereby ensuring that the trained visual enhancement model generates a visual enhancement image that enables the color vision impairment population to clearly distinguish different colors, accurately name different colors and maintain visual naturalness, and effectively helps the color vision impairment population to calibrate consistency of color naming with the public language standard.

[0078] In step S101 of some embodiments, specifically, the original visual enhancement model is a neural network model to be trained, which includes an encoder and a decoder, and the model visual enhancement model is used to perform at least one of color, brightness and contrast visual enhancement for color vision deficiency performance based on an input image, with color naming accuracy as the core optimization target, for converting an RGB image seen by a normal population into an RGB image friendly to a color vision deficiency population (i.e., a visual enhancement image), so that the color vision deficiency population can distinguish the corresponding correct color name based on the visual enhancement image.

[0079] Specifically, the target training set includes a plurality of original training images and their corresponding true color names; wherein the original training image refers to an RGB image perceived by a normal population; the true color name refers to a color category label defined according to a public language standard of a color vision normal population, which is a benchmark for training and evaluating the color naming accuracy of the model, wherein the public language standard refers to color category names commonly agreed upon, widely used, and achieving unambiguous communication by a color vision normal population, such as red, green, yellow, and brown color category labels.

[0080] Further, the original training image can be obtained from an image file, a video frame, a display input frame, or a light field.

[0081] For example, for an RGB image containing red flowers and green leaves, its corresponding true color name can be red and green.

[0082] Please refer to Figure 2 In some embodiments, the visual enhancement method for color vision deficiency further includes but is not limited to steps S201 to S205:

[0083] Step S201, obtaining an original color vision color simulation model, and simulating an original training image to obtain a color vision deficiency simulation image through the color vision color simulation model.

[0084] Step S202, color discrimination for the color vision deficiency simulation image to determine a simulation color discrimination name.

[0085] Step S203, color loss calculation according to the simulation color discrimination name and the true color name to obtain a color perception loss value.

[0086] Step S204, model parameter adjustment of the color vision color simulation model based on the color perception loss value to update the color vision color simulation model.

[0087] Step S205, based on the updated color vision color simulation model, return the simulation image simulation of the original training image through the color vision color simulation model until the color perception loss value meets the preset expected loss condition, and obtain the pre-trained color vision color simulation model.

[0088] Please refer to Figure 3 In some embodiments, the color vision color simulation model includes a color vision impairment simulation sub-model; wherein the color vision impairment simulation sub-model is used to simulate the spatial perception of the human eye for color vision impairment performance, and step S201 includes but is not limited to steps S301 to S304:

[0089] Step S301, obtaining the training color vision impairment type corresponding to the target training set.

[0090] Step S302, obtaining the human eye perception space data corresponding to the training color vision impairment type.

[0091] Step S303, obtaining random noise data.

[0092] Step S304, performing perception space transformation on the original training image according to the training color vision impairment type, the human eye perception space data and the random noise data, and obtaining the color vision impairment simulation image.

[0093] In step S301 of some embodiments, specifically, the original color vision color simulation model is a neural network model to be trained, which includes a color vision impairment simulation sub-model and a color perception simulation sub-model; wherein the color vision impairment simulation sub-model is a neural network model constructed based on the LMS cone response mechanism of biological vision, wherein the color vision impairment simulation sub-model is used to simulate the spatial perception of the human eye for color vision impairment (such as red-green color blindness); the color perception simulation sub-model is a neural network model (such as visual Transformer and Text-Embedding-3-Large, text embedding third generation large model) constructed based on vision and color language, which is used for color semantic discrimination for color vision impairment performance, to help color vision impaired people to name the perceived color of the color vision impairment simulation image.

[0094] Specifically, the training color vision disorder type is a visual perception disorder determined by the shift or confusion of the photosensitive sensitivity of any one of the three types of cone cells (L, M, S), which includes but is not limited to the disorder type of the first type of cone cell (i.e. L, long-wave sensitive cone cell) functional abnormality (such as red blindness, red weakness), the disorder type of the second type of cone cell (i.e. M, medium-wave sensitive cone cell) functional abnormality (such as green blindness, green weakness), the disorder type of the third type of cone cell (i.e. S, short-wave sensitive cone cell) functional abnormality (such as blue blindness, blue weakness) and cone cell achromatopsia, etc. Wherein, the color vision disorder can be manifested as the sensitivity curve of any one of L, M and S shifting to the sensitivity curve of another cone cell, which is manifested as the response value of any one of L, M and S corresponding to the channel mixing the response value of another channel, or as the response value of any one of L, M and S corresponding to the channel greatly reducing.

[0095] Further, the training color vision disorder type can be converted into a corresponding disorder type transformation matrix by a color vision disorder simulation sub-model, which is used for subsequent perception space transformation.

[0096] Specifically, the disorder type transformation matrix refers to a transformation matrix related to the color vision disorder type and the disorder severity, which is used to simulate the individual differences of different color vision disorder types and disorder severity.

[0097] Specifically, the disorder type transformation matrix is determined based on the actual application scenario.

[0098] For example, for moderate green weakness, the disorder type transformation matrix can be:

[0099]

[0100] The second row of the above transformation matrix indicates that the M cone cell response value is the sum of the L cone cell and M cone cell response values under normal circumstances, and may be 0.56, may be 0.37, and is the number ratio of M and L cone cells, and the value range difference caused by cross-cone cell channels can be corrected according to this ratio value.

[0101] Similarly, for mild green blindness, the disorder type transformation matrix can be:

[0102]

[0103] Similarly, for moderate red weakness, the disorder type transformation matrix can be:

[0104]

[0105] Similarly, for mild red blindness, the obstacle type conversion matrix can be:

[0106]

[0107] Similarly, for moderate blue weakness, the obstacle type conversion matrix can be:

[0108]

[0109] Similarly, for moderate blue blindness, the obstacle type conversion matrix can be:

[0110]

[0111] In step S302 of some embodiments, specifically, the human eye perception space data refers to a color space conversion matrix that converts the conventional RGB color space into a color space that is consistent with the physiological perception of the normal color vision population (such as the LMS cone cell response space), which takes into account the visual sensitivity curve characteristics of a single cone cell (i.e. the response functions of L, M, and S cone cells to different wavelengths of light), the number distribution characteristics of cone cells (i.e. the relative proportions of L, M, and S cone cells on the retina), and the number of incident photons (i.e. a physical quantity related to light intensity and spectral energy distribution).

[0112] For example, the human eye perception space data can be:

[0113]

[0114] wherein, represents the human eye perception space data; the first row of the matrix represents the total response coefficient of L cone cells, i.e. the effective stimulus weight of light from left to right in the R, G, and B channels to L cone cells; the second row of the matrix represents the total response coefficient of S cone cells; and the third row of the matrix represents the total response coefficient of M cone cells. represents the scaling of the perception space value, which is used to adjust the representation range of the perception space value without affecting the corresponding relationship between RGB and LMS.

[0115] In step S303 of some embodiments, specifically, the random noise data is a random disturbance signal introduced in the simulation process, which is used to simulate the Gaussian noise existing in the cone cell perception process of the human visual system.

[0116] Specifically, the noisy obstacle space feature refers to the color-impaired and disturbed cone cell response value in the LMS color space after the obstacle type conversion matrix and the superimposed random noise.

[0117] In step S304 of some embodiments, specifically, the color vision impairment simulation image is an LMS impairment image perceived by the human eye of the color vision impaired population.

[0118] Specifically, the impairment type transformation matrix, random noise data and human eye perception space data can be transformed in the impairment perception space by the color vision impairment simulation sub-model to obtain the noisy impairment perception space feature, so as to output the color vision impairment simulation image.

[0119] Further, the noisy impairment perception space data can be determined by the following formula:

[0120]

[0121] Wherein, represents the noisy impairment perception space feature of the i th image block in the color vision impairment simulation image; represents the impairment type transformation matrix corresponding to the training color vision impairment type CVD, represents the human eye perception space data, represents the color value of the i th image block in the original training image in the RGB color space, and n represents the random noise data.

[0122] Specifically, the red, green and blue channels in the RGB space are mapped to the response values of the L cone cells, M cone cells and S cone cells in the noisy impairment perception space feature, and the LMS three channel outputs are fused to determine the color vision impairment simulation image.

[0123] It should be understood that in the present embodiment, the perception space transformation can be realized based on the human eye perception space data, random noise data and training color vision impairment type, but there is no order difference in the acquisition steps of the human eye perception space data, random noise data and training color vision impairment type.

[0124] Through steps S301 to S304, the perception space conversion of the original training image can be realized by combining the training color vision impairment type, human eye perception space data and random noise data, the neural noise and individual perception fluctuation existing in the real human eye visual system are effectively simulated, the simulation image which not only truly reflects the human eye visual confusion characteristics of the color vision impaired population, but also contains the natural perception fluctuation of the human eye is generated, and the accuracy of the color vision impairment image simulation is significantly improved.

[0125] Please refer to Figure 4 In some embodiments, the color vision color simulation model includes a color perception simulation sub-model; wherein the color perception simulation sub-model is used for color semantic discrimination for color vision impairment performance, and step S202 includes but is not limited to steps S401 to S404:

[0126] Step S401, feature encoding is performed on the color vision disorder simulation image to obtain a disorder image feature.

[0127] Step S402, a candidate color embedding vector corresponding to the color vision disorder simulation image is determined according to a preset color name embedding word table.

[0128] Step S403, similarity calculation is performed on the disorder image feature and the candidate color embedding vector to obtain an image color similarity.

[0129] Step S404, a simulation color judgment name corresponding to the color vision disorder simulation image is determined according to the image color similarity.

[0130] In step S401 of some embodiments, specifically, the disorder image feature refers to a visual vector representation extracted from the color vision disorder simulation image.

[0131] Specifically, the color vision disorder simulation image can be divided into image blocks to obtain color vision disorder simulation image blocks, the color vision disorder simulation image blocks are subjected to embedding (i.e., Embedding) processing through a visual Transformer model to obtain image block embedding vectors, the image block embedding vectors are subjected to position encoding to obtain image block embedding position vectors, and the color features of the image block embedding position vectors are further captured through a self-attention mechanism (Self-Attention) to output disorder image block features.

[0132] In step S402 of some embodiments, specifically, the color name embedding word table refers to a query table storing real color names and color name semantic vectors corresponding to the real color names.

[0133] Specifically, the candidate color embedding vector refers to a set of color name semantic vectors corresponding to all color vision disorder simulation image blocks selected from the color name embedding word table.

[0134] Specifically, all real color names stored in the color name embedding word table can be subjected to word embedding processing through a Text-Embedding-3-Large model to obtain color name semantic vectors corresponding to the real color names.

[0135] For example, for an orange-red color vision disorder simulation image block, the candidate color embedding vector corresponding to the image block can be color name semantic vectors corresponding to brown-red, red, vermilion, brown, and orange.

[0136] In step S403 of some embodiments, specifically, the image color similarity is the closeness of the disorder image feature and the candidate color embedding vector in the vector space, and is used to measure the matching degree of the color vision disorder simulation image and the color name corresponding thereto.

[0137] Specifically, the similarity between each obstacle image block feature and each candidate color embedding vector can be calculated by a cosine similarity algorithm to obtain the image block color similarity between the obstacle image block feature and each candidate color embedding vector, so as to determine the image color similarity.

[0138] For example, the similarity between the obstacle image block feature of brown-red color and the candidate color embedding vector of brown-red color is high, while the similarity between the obstacle image block feature of brown-red color and the candidate color embedding vector of orange color is low.

[0139] In step S404 of some embodiments, specifically, the simulation color discriminative name refers to the color naming label perceived by the color vision impaired population for the color vision impairment simulation image, which is consistent with the expression of the true color name.

[0140] Specifically, the color distribution of each candidate color embedding vector corresponding to the obstacle image block feature can be calculated according to the image color similarity to obtain the image block color name probability distribution, and the image block color discriminative name is determined according to the image block color name probability distribution, and the simulation color discriminative name of the color vision impairment simulation image is determined according to the image block color discriminative name.

[0141] The image block color similarity can be normalized by a Softmax function to convert the similarity value into the image block color name probability distribution.

[0142] Specifically, the color name with the highest probability can be directly selected as the image block color discriminative name corresponding to the color vision impairment simulation image block by using a greedy decoding strategy.

[0143] For example, if the image block color probability distribution is: red color 0.65, orange color 0.25, and yellow color 0.10, then the image block color discriminative name corresponding to the image block is red color.

[0144] Further, the appearance frequency of all image block color discriminative names can be counted, and the simulation color discriminative name of the color vision impairment simulation image is determined according to a preset rule (such as selecting the top K color names with the highest frequency).

[0145] For example, a color vision impairment simulation image contains multiple color regions such as red, green, and brown, and the appearance frequency of the image block color discriminative name is in descending order of red, green, brown, yellow, and blue. The top 3 color names with the highest frequency, i.e., the set of red, green, and brown, are selected as the simulation color discriminative name corresponding to the image.

[0146] Please refer to Figure 5 In some embodiments, step S203 includes but is not limited to steps S501 to S503:

[0147] In step S501, the image block color discriminant name corresponding to each image block in the color vision disorder simulation image is obtained according to the simulation color discriminant name, and an image block color name probability distribution corresponding to the image block discriminant name is obtained.

[0148] In step S502, an image block color loss value is obtained by calculating the image block loss according to the image block color name probability distribution and the real color name.

[0149] In step S503, the color perception loss value is determined according to the image block color loss value.

[0150] In step S501 of some embodiments, specifically, the image block color discriminant name refers to the color naming label perceived by the simulated color vision disorder population for each color vision disorder simulation image block, which is consistent with the expression of the real color name.

[0151] Specifically, the image block color name probability distribution is the distribution of the possibility of each color vision disorder simulation image block belonging to each color name in the color name embedding word table.

[0152] For example, the color probability distribution of the brown-red image block can be: brown-red is 0.75, red is 0.20, brown is 0.10, and orange is 0.05, etc. That is, the color image block color name probability distribution indicates that the image block has a 75% possibility of being identified as red.

[0153] In step S502 of some embodiments, specifically, the image block color loss value refers to the deviation between the color image block discriminant name of each color vision disorder simulation image block and the real color name.

[0154] Specifically, the effective image block color probability can be obtained by performing effective probability recognition on the image block color name probability distribution and the real color name according to a preset indicator function, and the image block color loss value can be obtained by calculating the effective image block color probability.

[0155] The indicator function refers to a mathematical function based on index selection of the real color name, which is used to extract a single probability value corresponding to the real color name from different category probability distributions.

[0156] Specifically, the effective image block color probability refers to a single probability value corresponding to the real color name of the image block extracted from the image block color name probability distribution, which is used to represent the prediction confidence of the color vision color simulation model for the correct color of the image block.

[0157] Specifically, the image block color discriminant name corresponding to the image block color name probability distribution is compared with the real color name by the indicator function. When and only when the image block color discriminant name is consistent with the real color name, the image block color discriminant name is determined as an effective color discriminant name. The image block color probability value corresponding to the effective color discriminant name is 1, that is, the image block color probability value corresponding to the effective color discriminant name is the effective image block color probability, otherwise it is 0.

[0158] For example, if the real color name is red, and the image block color discriminant name corresponding to the image block color name probability distribution can be red 0.7, orange 0.2, and yellow 0.1, the indicator function is only 1 at red, so that the effective image block color probability is 0.7.

[0159] Specifically, the greater the effective image block color probability, the closer the corresponding negative logarithm value approaches 0, indicating that the image block color loss value is smaller. The smaller the effective image block color probability, the greater the corresponding negative logarithm value, indicating that the image block color loss value is greater.

[0160] For example, if the effective image block color probability is 0.7, the image block color loss value can be -log(0.7)≈0.36, and if the effective image block color probability is 0.1, the image block color loss value jumps to -log(0.1)≈2.3, indicating that there is a large deviation between the image block color discriminant name and the real color name.

[0161] In step S503 of some embodiments, specifically, the color perception loss value refers to the deviation degree between the simulated color discriminant name and the real color name.

[0162] Specifically, the loss of all image block color loss values of the color vision impairment simulation image is accumulated to obtain a global image color loss value, and the global image color loss value is divided by the total number of color vision impairment simulation image blocks to obtain the color loss value of the color vision impairment simulation image.

[0163] Through steps S501 to S503, the effective image block color probability corresponding to the real color name can be determined by the indicator function, irrelevant color names are avoided, and the low confidence penalty is amplified by using negative logarithm processing, which improves the accuracy of color name discrimination. Further, the global color loss value is obtained by summation and averaging, which provides a direction for color vision color simulation model optimization and effectively helps color vision impaired people to calibrate the consistency of color naming and public language standards.

[0164] In some more specific embodiments of the application, the color perception loss value can be determined by the following formula:

[0165]

[0166] wherein, represents a color perception loss value, i represents a color vision disorder simulation image block index, j represents a real color name index, represents an image block color discrimination name corresponding to the i-th color vision disorder simulation image block, represents the j-th real color name, represents an indicator function (i.e., the indicator function value is 1 when the image block color discrimination name corresponding to the i-th color vision disorder simulation image block is consistent with the j-th real color name), represents a color vision disorder (CVD) simulation image, represents a noisy disorder space feature of the i-th image block in the color vision disorder simulation image, represents an image block color name probability of the i-th image block belonging to the j-th real color name in the color vision disorder simulation image.

[0167] In step S204 of some embodiments, specifically, the color perception loss value can be calculated as an optimization target through a back propagation algorithm, the gradient of the weight and bias parameters of the color vision color simulation model with respect to the loss value is determined according to the color perception loss value, and the weight and bias parameters are fine-tuned in the opposite direction of the gradient through a gradient descent algorithm to obtain an updated color vision color simulation model.

[0168] For example, if the color vision color simulation model misjudges the “green” name of the color vision disorder simulation image as the “brown” name, the color loss value will be higher, and the weight and bias parameters related to the “green” name are adjusted according to the gradient to enhance the discrimination ability of the color vision color simulation model for the “green” and “brown” names.

[0169] In step S205 of some embodiments, specifically, the pre-trained color vision color simulation model is a trained neural network model, which is used to realize color name discrimination for the original training image.

[0170] Specifically, the preset expected loss condition can be a preset color loss threshold, and when the color loss value is lower than the color loss threshold, it indicates that the color vision color simulation model is trained, i.e., the model can more accurately simulate the color vision disorder image and perform color name discrimination.

[0171] For example, the color loss threshold can be 0.4.

[0172] Through steps S201 to S205, the color vision color simulation model is trained through cyclic iteration, effectively improving the accuracy of the color vision color simulation model in simulating color vision impairment images and color name discrimination, ensuring that the simulated color vision impairment simulation image conforms to the visual perception of the color vision impairment population, and guiding the color vision impairment population to name colors, further helping the color vision impairment population to calibrate the consistency of color naming and public language standards.

[0173] In step S102 of some embodiments, specifically, the visual enhancement image refers to the RGB impairment image perceived by the color vision impairment population for the original training image.

[0174] Specifically, the visual enhancement image obtained by visual enhancement of the original training image by the visual enhancement model can include: encoding the original training image by the visual enhancement model to obtain image visual features; performing visual spatial transformation on the image visual features to obtain visual optimization features; and performing image reconstruction on the visual optimization features to obtain the visual enhancement image; wherein the visual optimization features at least include one of color optimization features, brightness optimization features and contrast optimization features, and the specific visual spatial transformation is based on the training color vision impairment type and severity to determine the image pixel adjustment of the RGB channel, which is not limited here.

[0175] For example, if the original training image is an RGB image containing red flowers and green leaves, the color, texture and structure of the red flowers and green leaves can be extracted by the encoder of the visual enhancement model, and the image pixel adjustment (such as adjusting the RGB value of the red flowers from (255, 0, 0) to (255, 128, 0), i.e. yellow, and adjusting the RGB value of the green leaves from (0, 255, 0) to (0, 128, 255), i.e. blue) is performed on the image visual features to meet the color optimization features perceived by the color vision impairment population, and further image reconstruction is performed on the color optimization features by the decoder to adjust the original red flowers to a more yellow visual enhancement image and the green leaves to a more blue visual enhancement image, so that the color vision impairment population can more clearly distinguish the color difference between the flowers and the leaves, and correctly name the colors of the objects in the image.

[0176] Specifically, the visual enhancement process can be represented by the following formula:

[0177]

[0178] wherein, represents the training color vision impairment image, represents the visual enhancement image obtained by the visual enhancement model parameter defined visual enhancement model converts the original training image I into a visual enhancement image .

[0179] In this embodiment, the original training image is visually enhanced by the visual enhancement model, realizing the preliminary conversion from normal image to color vision impairment friendly image, and providing image data closer to the visual perception of color vision impairment population for subsequent color discrimination.

[0180] In step S103 of some embodiments, specifically, the target color discrimination name refers to the color naming label perceived by the color vision impairment population for the visually enhanced image, which is consistent with the expression of the real color name.

[0181] Specifically, the method of color discrimination of the visually enhanced image by the color vision color simulation model is consistent with the method of color discrimination of the color vision impairment simulation image to determine the simulation color discrimination name, which will not be repeated here.

[0182] In this embodiment, the color discrimination of the visually enhanced image by the color vision color simulation model can effectively simulate the color naming behavior of the color vision impairment population, provide accurate target color name for the optimization of the visual enhancement model, and improve the accuracy of the color vision impairment image generated by the visual enhancement model.

[0183] Please refer to Figure 6 In some embodiments, step S104 includes but is not limited to steps S601 to S603:

[0184] Step S601, calculating image structure loss according to the visually enhanced image and the original training image to obtain an image structure loss value.

[0185] Step S602, calculating color loss according to the target color discrimination name and the real color name to obtain an enhanced color loss value.

[0186] Step S603, performing weighted processing according to the image structure loss value and the enhanced color loss value to obtain a target loss value.

[0187] Please refer to Figure 7 In some embodiments, step S601 includes but is not limited to steps S701 to S703:

[0188] Step S701, determining the visual enhancement pixel distribution data of the visually enhanced image.

[0189] Step S702, determining the original pixel distribution data of the original training image.

[0190] Step S703, calculating loss according to the visual enhancement pixel distribution data and the original pixel distribution data to obtain the image structure loss value.

[0191] In step S701 of some embodiments, specifically, the visual enhanced pixel distribution data refers to pixel value distribution data of pixel values in a pre-designed calculation window in the visual enhanced image, and the visual enhanced pixel distribution data includes an enhanced image mean and an enhanced image variance; wherein the enhanced image mean refers to a pixel average value of each color channel of the visual enhanced image in the RGB space, and is used to represent the overall brightness of the visual enhanced image; and the enhanced image variance represents a dispersion degree of pixel value distribution of each color channel of the visual enhanced image in the RGB space, and is used to represent the contrast of the visual enhanced image.

[0192] Specifically, for example, the pixel average value of the red channel of the visual enhanced image containing red flowers and green leaves can be 120, and the green channel variance of the visual enhanced image can be 50.

[0193] In step S702 of some embodiments, specifically, the original pixel distribution data refers to pixel value distribution data of pixel values in a pre-designed calculation window in the original training image, and the original pixel distribution data includes an original image mean and an original image variance; wherein the original image mean refers to a pixel average value of each color channel of the original training image in the RGB space, and is used to represent the overall brightness of the original training image; and the original image variance represents a dispersion degree of pixel value distribution of each color channel of the original training image in the RGB space, and is used to represent the contrast of the original training image.

[0194] For example, the pixel average value of the red channel corresponding to the original training image containing red flowers and green leaves can be 255, and the green channel variance of the original training image can be 30.

[0195] In step S703 of some embodiments, specifically, the image structure loss value refers to the deviation degree of the visual enhanced image and the original training image in image structure, contrast and brightness.

[0196] Specifically, the image covariance can be obtained by covariance calculation according to the enhanced image variance and the original image variance, and the image mean constant and the image variance constant of the visual enhanced image are obtained, so as to calculate the loss according to the image covariance, the image mean constant, the image variance constant, the visual enhanced pixel distribution data and the original pixel distribution data, and obtain the image structure loss value.

[0197] Wherein, the image covariance is used to describe the similarity degree of the original training image and the visual enhanced image in structural features (such as texture, edge and shape), and the greater the image covariance is, the more similar the structure of the original training image and the visual enhanced image is; the image mean constant and the image variance constant are used to adjust the hyperparameters of the image structure loss calculation, and are usually set according to experimental experience, such as the image mean constant is 0.0001 and the image variance constant is 0.0009.

[0198] It should be understood that the statistical distribution data and constant values of the image structure loss calculation above are exemplary and are not limited herein.

[0199] Specifically, the image structure loss value can be determined by the following formula:

[0200]

[0201] wherein, represents the image structure loss value between the original training image I and the visual enhancement image represents the original image mean of the original training image I, represents the enhanced image mean of the visual enhancement image, represents the image mean constant, represents the image covariance between the original training image I and the visual enhancement image represents the original image variance of the original training image I, represents the enhanced image variance of the visual enhancement image. Through steps S701 to S703, the differences in image structure, contrast and brightness between the visual enhancement image and the original training image can be measured by the visual enhancement pixel distribution data and the original pixel distribution data, avoiding the situation that the image is distorted due to excessive adjustment of the image hue to improve the color discrimination, effectively enhancing the color discrimination of the color vision impairment image while retaining the key structural features of the original training image, ensuring that the generated color vision impairment image can meet the visual needs of the color vision impairment population and will not affect the visual experience of the normal color vision population, improving the accuracy of visual enhancement.

[0202] In step S602 of some embodiments, specifically, the enhanced color loss value refers to the degree of deviation between the target color discrimination name and the real color name.

[0203] Specifically, the method of color loss calculation according to the target color discrimination name and the real color name to obtain the enhanced color loss value is consistent with the method of color loss calculation according to the simulated color discrimination name and the real color name to obtain the color loss value, which will not be repeated here.

[0204] In step S603 of some embodiments, specifically, the target loss value is the overall loss of the image structure loss value and the enhanced color loss value, which is used to represent the naturalness of the color vision impairment image generated by the visual enhancement model under the color constraint condition that the prediction probability of the color vision color simulation model for the real color name is higher than that of other color names.

[0205] In step S603 of some embodiments, specifically, the target loss value is the overall loss of the image structure loss value and the enhanced color loss value, which is used to represent the naturalness of the color vision impairment image generated by the visual enhancement model under the color constraint condition that the prediction probability of the color vision color simulation model for the real color name is higher than that of other color names.

[0206] ​Specifically, the color constraint condition can be:

[0207]

[0208] wherein i represents the index of the color vision simulation image block, j represents the index of the real color name, represents the color vision color simulation model parameter defined by the color vision color simulation model for simulating the experience estimation process from LMS to stimulus value to color naming perception, represents the target color discrimination name corresponding to the visual enhancement image, represents the jth real color name, represents the visual enhancement image after color enhancement for the color vision impaired population, represents the visual enhancement image after color enhancement of the ith image block in the noisy impaired space feature.

[0209] Specifically, the target loss value can be determined by the following formula:

[0210]

[0211] wherein, represents the target loss value; represents the balance coefficient, which can be 0.7; represents the enhanced color loss value based on the visual enhancement image , which is used to represent the maximum prediction color distribution probability of the real color name corresponding to the visual enhancement image under the pre-trained color vision color simulation model; represents the image structure loss value between the original training image I and the visual enhancement image .

[0212] By combining the image structure loss and the enhanced color loss through steps S601 to S603, it can be ensured that when optimizing the visual enhancement model, not only the color name recognition accuracy of the color vision impaired population is improved, but also the structure naturalness of the generated color vision impaired image is ensured, avoiding the distortion of the generated image caused by excessive adjustment of the impaired image hue. While realizing the consistency of helping the color vision impaired population to calibrate the color naming and the public language standard, the accuracy of visual enhancement is also improved.

[0213] In step S105 of some embodiments, specifically, the target loss value calculated can be taken as a global optimization target by using a back propagation algorithm, and the gradient of the weight and bias parameters of the visual enhancement model relative to the loss value is determined according to the target loss value. Subsequently, the gradient descent algorithm is used to fine-tune the weight and bias parameters of the visual enhancement in the opposite direction of the gradient, and the updated visual enhancement model is obtained.

[0214] For example, if the visual enhancement model excessively adjusts the contrast of the color vision impaired image to improve color discrimination, resulting in an increase in structural similarity loss, or the color discrimination model still cannot correctly identify the color name after adjustment, resulting in a high color loss, the target loss value will be high. The gradient signal will guide the adjustment of the weight and bias parameters of the encoder and decoder of the visual enhancement to learn a better balance between enhancing color guidance and maintaining the natural structure of the image.

[0215] In step S106 of some embodiments, specifically, the trained visual enhancement model is a trained neural network model for generating an RGB color vision impaired image that is friendly to the color vision impaired population and can guide the color vision impaired population to correctly name the color.

[0216] Specifically, the preset target loss condition can be a comprehensive loss threshold. If the target loss value is below the comprehensive loss threshold, it can be determined that the visual enhancement for color vision impairment is complete, indicating that the visual enhancement model has learned to generate color vision impaired images that balance color naming guidance accuracy and image structure naturalness.

[0217] For example, if the comprehensive loss threshold is 0.3, when the target loss value is below 0.3, the trained visual enhancement model is obtained.

[0218] Please refer to Figure 8 In some embodiments, step S107 includes but is not limited to steps S801 to S803:

[0219] Step S801, obtaining the target color vision impairment type of the target image, and obtaining the impairment severity corresponding to the target color vision impairment type; wherein the target image is derived from an image file, a video frame, a display input frame or a light field.

[0220] Step S802, color discrimination of the target image according to the pre-trained color vision color simulation model, the target color vision impairment type and the impairment severity, to obtain the target image color discrimination name.

[0221] Step S803, determining the visual optimization constraint condition of the target image according to the target image color discrimination name, and instructing the trained visual enhancement model to adjust the image pixels of the target image according to the visual optimization constraint condition, to obtain the target visual enhancement image.

[0222] In step S801 of some embodiments, specifically, the target image refers to an RGB image perceived by the normal population to be converted to color vision impairment, wherein the target image can be derived from an image file, a video frame, a display input frame or a light field.

[0223] For example, a landscape image containing red, green, and brown.

[0224] Specifically, the target color vision disorder type refers to the visual perception disorder category determined by the shift or confusion in the light sensitivity of any of the three types of cone cells (L, M, S) in a person with color vision disorder. It also includes disorder types with abnormal function of type I cone cells (L), type II cone cells (M), and type III cone cells (S).

[0225] Specifically, the severity of the target color vision deficiency type describes the degree of color vision deficiency, including mild, moderate, or severe.

[0226] In step S802 of some embodiments, specifically, the target image color discrimination name refers to the color naming label perceived by people with color vision impairment for the target image, and the color naming label is also consistent with the expression of the real color name.

[0227] Specifically, the target image can be downsampled to obtain dimensionality-reduced image features; where dimensionality-reduced image features refer to low-dimensional vector representations that retain the core structure and semantic information of the image after reducing the spatial resolution of the target image.

[0228] For example, max pooling can be used to process a 1920x1080 pixel RGB image to obtain a dimensionality-reduced image feature of 480x270 pixels.

[0229] In this embodiment, the method of color discrimination of the reduced image features based on the pre-trained color vision simulation model, the type and severity of the target color vision impairment, and obtaining the target image color discrimination name is the same as the method of color discrimination of the visually enhanced image through the color vision simulation model and obtaining the target color discrimination name, and will not be described again here.

[0230] In this embodiment, by downsampling the target image, the computational resource requirements of the visual enhancement model can be reduced, while preserving sufficient image structure and semantic information for subsequent effective visual enhancement.

[0231] In step S803 of some embodiments, specifically, the visual optimization constraint means that the pre-trained color vision simulation model predicts the true color name of the target image with a higher probability than other color names.

[0232] Specifically, target visual enhancement images refer to RGB visual enhancement images perceived by people with color vision disorders in relation to target images.

[0233] Specifically, the image pixel adjustment on the target image by the trained visual enhancement model according to the visual optimization constraint condition indication can include: encoding the dimension-reduced image feature according to the visual optimization constraint condition indication to obtain target image visual features; performing visual space transformation on the target image visual features to obtain target visual enhancement features; and performing image reconstruction on the target visual enhancement features to obtain the target visual enhancement image, so as to convert the target image into an RGB visual enhancement image perceived by the color vision impaired population, and accurately identify the color name corresponding to the target visual enhancement image; wherein the target visual enhancement features at least include one of target color enhancement features, target brightness enhancement features and target contrast enhancement features.

[0234] For example, for an original landscape image containing red, green and brown, after receiving the input of "weak green, degree 0.6", the color vision color simulation model outputs the color identification name as red, green and brown, and adjusts the hue, brightness and contrast of the red and green pixels of the original landscape image based on the weak green category of red, green and brown through the visual enhancement model to enhance the visual enhancement landscape image with enhanced green and yellow tone distinction, so that the color vision impaired population can correctly identify red, green and brown from the visual enhancement landscape image.

[0235] Through steps S801 to S803, the target image can be analyzed based on the color vision impairment type and degree of the target image and the pre-trained color vision simulation model to generate a visual optimization constraint for color vision impairment performance, and the target visual enhancement image for color vision impairment performance is generated by the trained visual enhancement model to generate a visual enhancement image for color vision impairment performance, so that the color vision impaired population can clearly distinguish different colors, accurately name different colors, and maintain a natural visual enhancement image, which significantly helps the color vision impaired population to calibrate the consistency of color naming and public language standards.

[0236] It should be understood that the above-mentioned visual enhancement method for color vision impairment involves a training step sequence, an exemplary training data set and its implementation, which is an exemplary implementation description, and is not limited here. Even if a third party replaces the training step sequence, modifies or implements a training data set for the same purpose, it does not deviate from the protection scope of the present application.

[0237] The embodiment of the application first obtains a pre-trained color vision color simulation model and a target training set containing real color names, thereby providing a color naming cognition benchmark for subsequent model training process; secondly, a visual enhancement model is used to generate a visual enhancement image based on the original training image for color vision impairment performance, thereby realizing preliminary conversion from a normal image to a color vision impairment friendly image, and using the color vision color simulation model to distinguish the color of the original training image, thereby simulating the real color naming behavior of the color vision impairment population and effectively capturing the cognitive process of the color vision impairment population from visual signals to public language standard naming; further, target loss calculation is performed according to the visual enhancement image, the original training image, the target color distinguishing name and the real color name, thereby obtaining a target loss value, which can take color naming accuracy as an optimization target to ensure that the generated color vision impairment image can guide the color vision impairment population to make correct color naming, and loss calculation is also performed in combination with the original training image and the visual enhancement image, thereby effectively avoiding distortion of the generated color vision impairment image caused by pursuit of discrimination; finally, the visual enhancement model is iteratively updated based on the target loss value until convergence, thereby accurately adjusting the image color based on the public color language standard, and at least one of color, brightness and contrast of the target image is enhanced by using the trained visual enhancement model, thereby ensuring that the trained visual enhancement model generates a visual enhancement image that enables the color vision impairment population to clearly distinguish different colors, accurately name different colors and maintain visual naturalness, thereby effectively helping the color vision impairment population to calibrate the consistency of color naming and the public language standard.

[0238] The embodiment of the application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned visual enhancement method for color vision impairment when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer and the like.

[0239] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0240] The processor 901 can be implemented in the form of a general-purpose CPU (Central Processing Unit, central processor), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiment of the application;

[0241] The memory 902 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 902 can store processing systems and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 902 and are called and executed by the processor 901 to implement the color vision deficiency visual enhancement method of the embodiments of the present application;

[0242] The input / output interface 903 is used to realize information input and output.

[0243] The communication interface 904 is used to realize the communication interaction between the device and other devices, and can realize communication through a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0244] The bus 905 transmits information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0245] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize communication connection between the device.

[0246] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the color vision deficiency visual enhancement method.

[0247] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0248] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0249] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0250] The apparatus embodiments described above are merely illustrative, and units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0251] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0252] The terms "first", "second", "third", "fourth" and the like used in the description of the present application and the above-described figures (if any) are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0253] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0254] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0255] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0256] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0257] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0258] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method of visual enhancement for color vision impairment, characterized by, The method comprises: obtaining an original visual enhancement model and a target training set, and obtaining a pre-trained color vision color simulation model; wherein the visual enhancement model is used to perform visual enhancement on at least one of color, brightness and contrast for color vision impairment performance based on an input image; the target training set comprises an original training image and a true color name corresponding to the original training image; performing visual enhancement on the original training image through the visual enhancement model to obtain a visual enhancement image for color vision impairment performance; performing color discrimination on the visual enhancement image through the color vision color simulation model to obtain a target color discrimination name; performing target loss calculation according to the visual enhancement image, the original training image, the target color discrimination name and the true color name to obtain a target loss value; updating and detecting the visual enhancement model according to the target loss value to obtain an updated visual enhancement model; based on the updated visual enhancement model, returning to perform visual enhancement on the original training image through the visual enhancement model until the target loss value meets a preset target loss condition, and obtaining a trained visual enhancement model; obtaining a target image and performing visual enhancement on the target image through the trained visual enhancement model to obtain a target visual enhancement image.

2. The method of claim 1, wherein, Before the pre-trained color vision color simulation model is obtained, the method further comprises pre-training the color vision color simulation model, specifically comprising: obtaining an original color vision color simulation model and performing simulation image simulation on the original training image through the color vision color simulation model to obtain a color vision impairment simulation image; performing color discrimination on the color vision impairment simulation image to determine a simulation color discrimination name; performing color loss calculation according to the simulation color discrimination name and the true color name to obtain a color perception loss value; adjusting model parameters of the color vision color simulation model based on the color perception loss value to update the color vision color simulation model; based on the updated color vision color simulation model, returning to perform simulation image simulation on the original training image through the color vision color simulation model until the color perception loss value meets a preset expected loss condition, and obtaining the pre-trained color vision color simulation model.

3. The method of claim 2, wherein, The color vision color simulation model comprises a color vision impairment simulation sub-model; wherein the color vision impairment simulation sub-model is used to simulate human eye spatial perception for color vision impairment performance; the simulation image simulation on the original training image through the color vision color simulation model to obtain a color vision impairment simulation image comprises simulation image simulation on the original training image through the color vision impairment simulation sub-model to obtain a color vision impairment simulation image, specifically comprising: obtaining a training color vision impairment type corresponding to the target training set; obtaining human eye perception space data corresponding to the training color vision impairment type; obtaining random noise data; According to the training color vision disorder type, the human eye perception space data and the random noise data, the original training image is subjected to a perception space transformation to obtain the color vision disorder simulation image.

4. The method of claim 2, wherein, The color vision color simulation model comprises a color perception simulation sub-model; wherein the color perception simulation sub-model is used for color semantic discrimination for color vision disorder performance; The color discrimination for the color vision disorder simulation image determines the simulation color discrimination name, which specifically comprises: The color vision disorder simulation image is subjected to feature coding to obtain a disorder image feature; According to a preset color name embedding word table, a candidate color embedding vector corresponding to the color vision disorder simulation image is determined; The similarity of the disorder image feature and the candidate color embedding vector is calculated to obtain an image color similarity; According to the image color similarity, the simulation color discrimination name corresponding to the color vision disorder simulation image is determined.

5. The method of claim 2, wherein, The color loss calculation according to the simulation color discrimination name and the real color name obtains a color perception loss value, which comprises: According to the simulation color discrimination name, an image block color discrimination name corresponding to each image block in the color vision disorder simulation image is obtained, and an image block color name probability distribution corresponding to the image block color discrimination name is obtained; According to the image block color name probability distribution and the real color name, an image block loss value is calculated by image block loss calculation. According to the image block color loss value, the color perception loss value is determined.

6. The method of claim 1, wherein, The target loss calculation according to the visual enhancement image, the original training image, the target color discrimination name and the real color name obtains a target loss value, which comprises: According to the visual enhancement image and the original training image, an image structure loss value is calculated by image structure loss calculation. According to the target color discrimination name and the real color name, an enhanced color loss value is calculated by color loss calculation. According to the image structure loss value and the enhanced color loss value, the target loss value is determined.

7. The method of claim 6, wherein, The image structure loss calculation according to the visual enhancement image and the original training image obtains an image structure loss value, which comprises: Determine the visual enhancement pixel distribution data of the visual enhancement image; Determine the original pixel distribution data of the original training image; According to the visual enhancement pixel distribution data and the original pixel distribution data, the image structure loss value is calculated by loss calculation.

8. The method according to any one of claims 1 to 7, characterized in that, The visual enhancement model trained is used to perform visual enhancement on the target image to obtain a target visual enhancement image, which comprises: Obtain the target color vision disorder type of the target image, and obtain the disorder severity corresponding to the target color vision disorder type; wherein the target image is derived from an image file, a video frame, a display input frame or a light field; According to the pre-trained color vision color simulation model, the target color vision disorder type and the disorder severity, color discrimination is performed on the target image to obtain a target image color discrimination name; According to the target image color discrimination name, a visual optimization constraint condition of the target image is determined, and the trained visual enhancement model is instructed to perform image pixel adjustment on the target image according to the visual optimization constraint condition to obtain a target visual enhancement image.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the visual enhancement method for color vision disorders according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the visual enhancement method for color vision disorders according to any one of claims 1 to 8.

Images