Automatic application of mapping function to video signal based on inferred parameter

Abstract

To provide devices and methods for automatic application of mapping functions to video signals based on inferred parameters.SOLUTION: In one example, a method is provided, including initiating display of content based on a video signal being processed by a device. The method may further include, in response to at least a first change in intensity of ambient light or a second change in color of the ambient light subsequent to the initiating of the display of the content based on the video signal, selecting a first mapping function applicable to pixels corresponding to frames of the video signal based at least on a first inferred parameter from a selected machine learning model. The method may further include automatically applying the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.SELECTED DRAWING: Figure 5

Description

[Background technology]

[0001] Devices with displays may be used across many ambient lighting conditions. Users of such devices may view different types of content in these widely differing ambient lighting conditions. Certain devices may include a light sensor or light meter to measure the amount of ambient light in the user's environment. Based on the measurement of the amount of ambient light, the device or user may adjust the brightness of the content displayed to the user. This adjustment may dissatisfy the user because the content may not be clearly visible or may otherwise remain inadequate despite the brightness adjustment.

[0002] Therefore, what is needed is a method and system for the automatic application of a mapping curve to adjust display parameters. Summary of the Invention

[0003] In one example, the disclosure relates to a method that includes initiating display of content based on a video signal processed by a device. The method may further include, following the initiation of display of the content based on the video signal, selecting a first mapping function applicable to pixels corresponding to frames of the video signal based at least on first estimated parameters from a selected machine learning model in response to at least a first change in an intensity of ambient light or a second change in a color of ambient light. The method may further include automatically applying the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.

[0004] In another example, the disclosure relates to a device including a processor configured to initiate display of content based on a video signal being processed by the device, the device may further include a memory including at least one module including instructions configured to: (1) select, in response to at least a first change in ambient light intensity or a second change in ambient light color following the initiation of display of the content based on the video signal, a first mapping function applicable to pixels corresponding to frames of the video signal based at least on first estimated parameters from a selected machine learning model, and (2) automatically apply the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.

[0005] In yet another example, the present disclosure relates to a computer-readable medium including a first set of instructions configured to initiate display of content to a user of a device based on a video signal. The computer-readable medium may further include a second set of instructions configured to select, in response to at least a first change in ambient light intensity or a second change in ambient light color following the initiation of display of the content based on the video signal, a first mapping function applicable to pixels corresponding to frames of the video signal based at least on first estimated parameters from a selected machine learning model. The computer-readable medium may further include a third set of instructions configured to automatically apply the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.

[0006] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. [Brief description of the drawings]

[0007] The present disclosure is illustrated by way of example and not limitation in the accompanying drawings, in which like reference numerals indicate like elements and in which elements are illustrated for simplicity and clarity and have not necessarily been drawn to scale.

[0008] [Figure 1] 1 illustrates a diagram of a system environment 100 for automatic application of a mapping function to a video signal based on estimated parameters, according to one example. [Diagram 2] 1 illustrates a memory containing instructions for automatic application of a mapping function to a video signal based on estimated parameters according to one example. [Diagram 3] FIG. 1 is a block diagram illustrating the collection of data for use as training data for a machine learning model, according to an example. [Figure 4] 4 shows a mapping curve corresponding to a mapping function according to an example. [Diagram 5] 1 shows a block diagram for automatic application of a mapping function to a video signal based on estimated parameters according to an example. [Figure 6] 4 shows a flowchart of a method for automatic application of a mapping function to a video signal based on estimated parameters according to an example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Examples described in this disclosure relate to automatically applying a mapping function to pixels corresponding to a signal, such as a video signal, based on estimated parameters. In a particular example, the estimated parameters may be dynamically generated by using a machine learning model and corresponding instructions. Devices with displays (e.g., smartphones, tablets, laptops, and other types of devices) may be used in a variety of ambient lighting conditions. As an example, a user may start watching a video inside their home, but then leave their home and continue watching the video outside. When a user seeks to improve the visibility of a video playback, a desktop-wide display setting may only get the user so far. As an example, in a bright outdoor environment, even if the display brightness is increased all the way, the user will be barely able to see any fine details in the video.

[0010] Certain devices may include a light sensor or light meter to measure the amount of ambient light in a user's environment. Based on the measurement of the amount of ambient light, the device or the user may adjust the brightness of the content displayed to the user. Despite the brightness adjustment, the adjustment may not satisfy the user because the content may not be clearly visible or may otherwise still be insufficient.

[0011] FIG. 1 shows a diagram of a system environment 100 for automatic application of a mapping function to a video signal based on estimated parameters according to an example. A user device 154 may be coupled via a network 110 to a remote computing facility, including a cloud computing infrastructure, which may store user feedback 156. A test device 152 may also be coupled via the network 110 such that feedback obtained via the test device 152 may be added to the user feedback 156. Details of one user device 120 are provided in FIG. 1. Other user devices may include similar functionality. The user device 120 may include a processor 122, a graphics processor 124, a memory 126, a sensor 128, a display 130, and a network interface 132, which may be interconnected via a bus system 140. The network 110 may be a wired network, a wireless network, or a combination of both wired and wireless networks. The processor 122 and the graphics processor 124 may execute instructions stored in the memory 126. The memory 126 may be any combination of non-volatile or volatile storage devices (e.g., flash memory, DRAM, SRAM, or other types of memory). The sensors 128 may include touch sensors, light intensity sensors, color temperature sensors, pressure sensors, proximity sensors, GPS sensors, and other types of sensors. The display 130 may be any type of display, such as an LCD, LED, or other type of display. The network interface 132 may include a communication interface, such as an Ethernet, cellular radio, Bluetooth radio, UWB radio, or other type of wireless or wired communication interface. Although FIG. 1 illustrates the system environment as including a number of components arranged and coupled in a certain manner, the system environment may include fewer or additional components arranged and coupled in a different manner.1 illustrates user device 120 as including a certain number of components arranged and coupled in a certain manner, user device 120 may include fewer or additional components arranged and coupled in a different manner. As an example, user device 120 may include other components, such as a keyboard, a mouse, a voice recognition processor, an artificial intelligence processor, or a touch screen.

[0012] 2 illustrates a memory 200 including instructions for automatic application of a mapping function to a video signal based on estimated parameters according to one example. In this example, the instructions may be organized in the memory 200 in the form of blocks or modules. The memory 200 may be the memory 126 of the user device 120. In this example, the memory 200 may include device model information 210, user feedback 220, machine learning (ML) models 230, a supervised learning-based analyzer (SLBA) 240, a mapping function 250, light conditions and measurements 260, and an operating system 270. Although FIG. 2 illustrates instruction modules organized in a certain way, the instructions may be combined or distributed in various ways.

[0013] Continuing with reference to FIG. 2, the Supervised Learning Based Analyzer (SLBA) 240 may implement a supervised learning algorithm that can be trained based on input data and, once trained, can make predictions or prescriptions based on the training. In this example, the SLBA 240 may implement techniques such as linear regression, support vector machines (SVMs) configured for regression, random forests configured for regression, gradient boosted trees configured for regression, and neural networks. Linear regression may include modeling past relationships between independent variables and dependent output variables. The neural network may include an input layer, one or more hidden layers, and artificial neurons used to create an output layer. Each layer may be encoded as a matrix or vector of weights expressed in the form of coefficients or constants that may have been obtained by offline training of the neural network. The neural network may be implemented as a recurrent neural network (RNN), a long short-term memory (LSTM) neural network, or a gated recurrent unit (GRU). All information required by the supervised learning based model may be converted into a vector representation corresponding to any of these techniques. In the LSTM example, the LSTM network may include a sequence of repeating RNN layers or other types of layers. Each layer of the LSTM network may consume input at a given time step, e.g., the state of the layer from the previous time step, and may generate a new set of outputs or states. When using LSTMs, a single chunk of content may be encoded into a single vector or multiple vectors. As an example, a word or combination of words (e.g., a phrase, sentence, or paragraph) may be encoded as a single vector. Each chunk may be encoded into an individual layer (e.g., a particular time step) of the LSTM network. An LSTM layer may be described using a set of equations, such as the following equations:

number

[0014] In this example, within each LSTM layer, the inputs and hidden states may be processed using vector operations (e.g., dot products, inner products, or vector addition) or a combination of nonlinear operations, as appropriate.

[0015] 2 illustrates the supervised learning based analyzer 240 as including instructions, which may be encoded as hardware corresponding to the A / I processor. In this case, some or all of the functionality associated with the supervised learning based latch-up analyzer may be hard-coded or provided as part of the A / I processor. As an example, the A / I processor may be implemented using an FPGA having the necessary functionality.

[0016] FIG. 3 is a block diagram 300 illustrating the collection of data for use as training data for a machine learning model according to an example. In an example, a user device and a test device may include instructions corresponding to modules that may enable the functionality described with respect to FIG. 3. As an example, the device 120 may begin displaying a video signal to a user of the device 120. The video signal may include an input video frame 310 that may be processed by a graphics processing pipeline 320 and then output as an output video frame 330. The video signal may be encoded into frames or other types of units using any of the video encoding standards including H.264, H.265, VP9, ​​or other video encoding standards. The frames corresponding to the video signal may include pixels. The pixels corresponding to the input frames may first be processed by a pre-processing block 322. The pixels may then be modified by application of at least one mapping function 324. The pixels may then be processed by a post-processing block 326 before being output as an output video frame 330. The output video frame may be used to display content corresponding to the video signal. Various mapping functions may be tested as part of creating training data for a machine learning model or as part of a research or testing device. As an example, a device similar to device 120 may include instructions corresponding to a light measurement block 342, which may be configured to receive ambient light intensity information from a light sensor (e.g., a light meter) and process the sensor-generated information. In this example, processing by light measurement block 342 may include classifying the light sensor-generated information into a particular class of ambient light. As an example, Table 1 below illustrates a classification of lighting conditions into nine categories.

[0017] [Table 1]

[0018] Although Table 1 shows nine categories of ambient light conditions, the light measurement block 342 may have instructions that may classify the ambient light conditions into fewer or more categories. The update mapping parameters 344 may include instructions configured to update the mapping parameters based on input from the light measurement block 342. The select mapping function 346 may include instructions configured to select a mapping function based on the updated mapping parameters. Although FIG. 3 shows certain blocks arranged in a particular manner to generate training data for a machine learning model, these blocks may be arranged differently. Table 2 below shows high level instructions for performing functions associated with the various blocks described with respect to FIG. 3.

[0019] [Table 2]

[0020] Continuing with reference to FIG. 3, various mapping parameters and corresponding mapping functions may be obtained by testing or performing other types of studies with devices similar to device 120. Data corresponding to these may then be used to train a neural network model or similar machine learning model to minimize the error function. In one example, minimization of the error function is obtained by obtaining user feedback on the various mapping parameters and corresponding mapping functions to determine appropriate weights for convolution or other types of operations to be performed as part of the machine-based learning. As an example, users of a testing environment may be provided with a set of preselected mapping functions with known parameters and requested to select a mapping function or mapping curve that they prefer. In one example, users may not have control over the mapping parameters, and instead, instructions associated with device 120 may determine an appropriate mapping curve based on user feedback. Alternatively, users may be allowed to select specific mapping parameters. The trained machine learning models may be stored in a remote computing facility, such as a cloud. Different machine learning models corresponding to different types of devices may be trained by respective users.

[0021] FIG. 4 shows mapping curves corresponding to mapping functions according to an example. In this example, mapping curves corresponding to different mapping functions are shown. In this example, a set of parameters (e.g., intensity levels of red, green, and blue pixels and luminance level of a display) may define the shape of the mapping curve. In this example, mapping curves are shown for various lighting conditions including "BELOW NORMAL_INDOORS", "BRIGHT_INDOORS", "DIM_OUTDOORS", "CLOUDY_OUTDOORS", and "DIRECT_SUNLIGHT". Each mapping curve corresponds to a mapping function that relates an input pixel luminance or intensity to an output pixel luminance or intensity. Although FIG. 4 shows only five mapping curves for five different lighting conditions for ambient light, additional or fewer mapping curves may be created. In addition, although not shown, similar mapping curves may be created that factor in the color temperature of the environment.

[0022] FIG. 5 is a block diagram for automatic application of a mapping function to a video signal based on estimated parameters according to an example. In an example, a user device (e.g., user device 120) may include instructions corresponding to modules that may enable the functionality described with respect to FIG. 5. As an example, device 120 may initiate display of a video signal to a user of device 120. The video signal may include an input video frame 510 that may be processed by a graphics processing pipeline 520 and then output as an output video frame 530. The video signal may be encoded into frames or other types of units using any of the video encoding standards, including H.264. The frame corresponding to the video signal may include pixels. The pixels corresponding to the input frame may first be processed by a pre-processing block 522. The pixels may then be modified by application of at least one mapping function 524. The pixels may then be processed by a post-processing block 526 before being output as an output video frame 530. The output video frame 530 may be used to display content corresponding to the video signal.

[0023] As an example, the device 120 may include instructions corresponding to a light / color measurement block 542 that may be configured to receive ambient light intensity information from a light sensor (e.g., a light meter) and process the sensor-generated information. In addition, the light / color measurement block 542 may process the color temperature of the ambient light. In this example, the processing by the light / color measurement block 542 may include classifying the light sensor-generated information into a particular type of ambient light. As an example, the light / color measurement block 542 may classify the ambient light conditions into nine categories of environmental light conditions, as shown in Table 1. The light / color measurement block 542 may have instructions that may also classify the ambient light conditions into color temperature categories. The color temperature categories may include cool and warm environments. The colors may be defined by chromacity values ​​associated with the color of the light. The histogram generation block 544 may process the input video frames 510 and generate histogram values ​​that may be used to characterize the type or nature of the content being processed as part of the video signal. In particular examples, this characterization of the content may include determining whether the video signal corresponds to a movie, a game, a sporting event, or some other type of content. Device model 546 may include information regarding the model number or model type of the device that processes the video signal. In this example, device model 546 may be stored in a memory associated with the device (e.g., device 120).

[0024] 5, machine learning (ML) model 550 may include an inference block 552 and a user feedback 556. Inference block 552 may include instructions configured to receive ambient light intensity values ​​(e.g., lux), ambient light color temperature, and content histogram values ​​as inputs. Training block 554 may include instructions configured to receive ambient light intensity values ​​(e.g., lux), ambient light color temperature, content histogram values, and device model information as inputs. In one example, instructions corresponding to training block 554 may execute the algorithm shown in Table 3 below:

[0025] [Table 3]

[0026] Still referring to Figure 5, build mapping function 560 may include instructions configured to build and select a mapping function based on estimated mapping parameters generated based on machine learning (ML) model 550. Although Figure 5 illustrates certain blocks configured in a particular way to generate training data for the machine learning model, these blocks may be configured differently. Table 4 below illustrates high level instructions for performing functions associated with the various blocks described with respect to Figure 5.

[0027] [Table 4] TIFF2025072399000007.tif181170

[0028] 6 shows a flowchart of a method for automatic application of a mapping function to a video signal based on estimated parameters according to an example. Step 610 may include starting to display content based on the video signal processed by the device. As mentioned above, a processor and corresponding instructions stored in a memory in the device 120 may perform this step.

[0029] Step 620 may include selecting a first mapping function applicable to pixels corresponding to a frame of the video signal based at least on a first estimated parameter from the selected machine learning model in response to at least a first change in the intensity of the ambient light or a second change in the color temperature of the ambient light following the start of displaying the content based on the video signal. As described above with respect to FIG. 5, instructions corresponding to a module including a configuration based on the machine learning model may be used to perform this step. Thus, sensors including an ambient light sensor and a color temperature sensor and corresponding instructions (e.g., a suitable application programming interface (API)) may detect a change in the intensity of the ambient light or a change in the color temperature of the ambient light. Additionally, the relevant portion of the pseudocode provided in Table 4, when converted into executable instructions and processed by a processor, may select a first mapping function applicable to pixels corresponding to a frame of the video signal based at least on the estimated parameter from the selected machine learning model.

[0030] Step 630 may include automatically applying a first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal. As discussed above with respect to Figure 5, the mapping function 524 may process the pixels after they have been preprocessed by the preprocessing block 522 of Figure 5. Thus, in this example, as part of this step, at least one mapping function may be applied to the pixels which may change the intensity of the pixels. Other values ​​corresponding to the pixels may also be changed, including hue, saturation, contrast, etc.

[0031] Step 640 may include selecting a second mapping function applicable to pixels corresponding to frames of the video signal in response to at least a third change in the intensity of the ambient light or a fourth change in the color temperature of the ambient light, the second mapping function being selected based on at least a second estimated parameter from the selected machine learning model, and automatically applying the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal. This step may be performed in a similar manner to step 620. In this manner, when the user's environment changes in terms of light / color conditions, device 120 may continuously apply the appropriate mapping function by detecting the change in the environment in real time.

[0032] In conclusion, in one example, the present disclosure relates to a method including initiating display of content based on a video signal being processed by a device. The method may further include selecting a first mapping function applicable to pixels corresponding to frames of the video signal based on at least a first estimated parameter from a selected machine learning model in response to a first change in ambient light intensity or a second change in ambient light color following the initiation of display of the content based on the video signal. The method may further include automatically applying the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.

[0033] The method may further include selecting a second mapping function applicable to pixels corresponding to frames of the video signal in response to at least a third change in ambient light intensity or a fourth change in ambient light color temperature, the second mapping function selected based on at least second estimated parameters from the selected machine learning model, and automatically applying the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal.

[0034] The selected machine learning model may be selected based at least on a device model of the device. The method may further include requesting a user of the device to provide feedback corresponding to a change in perceived quality of the displayed content following at least one of a first application of the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal or a second application of the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal.

[0035] The method may further include transmitting information regarding the feedback, the device model, and the at least one estimated parameter from the device to a remote computing system.

[0036] The method may further include training the machine learning models, including the selected machine learning models, based on preference data related to perceived quality of content displayed by the devices. The method may further include training the machine learning models, including the selected machine learning models, based on training data, where at least a subset of the training data is generated based on feedback from users of each device having the device model.

[0037] In another example, the disclosure relates to a device including a processor configured to initiate display of content based on a video signal being processed by the device, the device may further include a memory having at least one module including instructions configured to: (1) select, in response to at least a first change in ambient light intensity or a second change in ambient light color subsequent to the initiation of display of the content based on the video signal, a first mapping function applicable to pixels corresponding to frames of the video signal based at least on first estimated parameters from a selected machine learning model, and (2) automatically apply the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.

[0038] The device may further include a second set of instructions configured to select a second mapping function applicable to pixels corresponding to frames of the video signal in response to at least a third change in an intensity of the ambient light or a fourth change in a color temperature of the ambient light, the second mapping function selected based at least on second estimated parameters from the selected machine learning model, and automatically apply the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal. The selected machine learning model may be selected based at least on a device model of the device.

[0039] The at least one module may further include a third set of instructions configured to request a user of the device to provide feedback corresponding to a change in the perceived quality of content displayed following at least one of a first application of the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal, or a second application of the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal.

[0040] The device may further include a third set of instructions configured to transmit information regarding the feedback, the device model, and the at least one estimated parameter from the device to a remote computing system. The device may further include a fourth set of instructions configured to train machine learning models, including the selected machine learning model, based on the preference data regarding the perceived quality of content displayed by the device.

[0041] The at least one module may further include an inference block and a training block configured to generate training data for use by the inference block, at least a subset of the training data being generated based on feedback from users of each device having the device model.

[0042] In yet another example, the present disclosure relates to a computer-readable medium including a first set of instructions configured to initiate display of content based on the video signal to a user of a device. The computer-readable medium may further include a second set of instructions configured to select, in response to at least a first change in ambient light intensity or a second change in ambient light color following the initiation of display of the content based on the video signal, a first mapping function applicable to pixels corresponding to frames of the video signal based at least on first estimated parameters from a selected machine learning model. The computer-readable medium may further include a third set of instructions configured to automatically apply the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal.

[0043] The computer-readable medium may further include a fourth set of instructions configured to select a second mapping function applicable to pixels corresponding to frames of the video signal in response to at least a third change in ambient light intensity or a fourth change in color temperature of the ambient light, the second mapping function selected based at least on second estimated parameters from the selected machine learning model, and automatically applying the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal.

[0044] The selected machine learning model may be selected based at least on a device model of the device.

[0045] The computer-readable medium may further include a fifth set of instructions configured to request a user of the device to provide feedback corresponding to a change in perceived quality of content displayed following at least one of a first application of the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal or a second application of the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal.

[0046] The computer-readable medium may further include a sixth set of instructions configured to transmit information regarding the feedback, the device model, and the at least one estimated parameter from the device to a remote computing system. The computer-readable medium may further include a fourth set of instructions configured to train machine learning models, including the selected machine learning model, based on preference data regarding a perceived quality of content displayed by the device.

[0047] It should be understood that the methods, modules, and components shown herein are merely exemplary. Alternatively, or in addition, the functions described herein can be performed, at least in part, by one or more hardware logic components. For example, but not limited to, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like. In the abstract, yet clear sense, an arrangement of components to achieve the same function is substantially "associated" such that the desired function is achieved. Thus, any two components combined herein to achieve a particular function can be considered to be "associated" with each other such that the desired function is achieved, regardless of architecture or intermediate components. Similarly, any two components so associated can also be considered to be "operably connected" or "coupled" with each other to achieve the desired function.

[0048] Functions related to some examples described in this disclosure may also include instructions stored on non-transitory media. The term "non-transitory media" as used herein refers to any medium that stores data and / or instructions that cause a machine to operate in a particular manner. Exemplary non-transitory media include non-volatile media and / or volatile media. Non-volatile media include, for example, hard disks, solid state drives, magnetic disks or tapes, optical disks or tapes, flash memory, EPROM, NVRAM, PRAM, or other such media, or networked versions of such media. Volatile media include, for example, dynamic memory such as DRAM, SRAM, cache, or other such media. Non-transitory media is distinct from, but can be used in conjunction with, transmission media. Transmission media is used to transfer data and / or instructions to and from a machine. Examples of transmission media include coaxial cable, fiber optic cable, copper wire, and wireless media such as radio waves.

[0049] Moreover, those skilled in the art will recognize that boundaries between the functionality of the operations described above are merely exemplary. The functionality of multiple operations may be combined into a single operation and / or the functionality of a single operation may be distributed among additional operations. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments.

[0050] While the present disclosure provides specific examples, various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the following claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure. Benefits, advantages, or solutions to problems described herein with respect to specific examples are not intended to be construed as critical, required, or essential features or elements of any or all claims.

[0051] Moreover, the terms "a" or "an," as used herein, are defined as one or more than one. Also, the use of introductory phrases such as "at least one" and "one or more" in a claim should not be construed as meaning that the introduction of another claim element with the indefinite article "a" or "an" limits a particular claim that includes the claim element so introduced to an invention that includes only one such element, even if the claim includes an introductory phrase such as "one or more" or "at least one" and "a" or "an."

[0052] Unless otherwise indicated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements they represent, and thus do not necessarily imply a temporal or other priority between the elements.

Claims

1. commencing display of content based on a video signal processed by the device; In response to either a first change in the intensity of ambient light, a second change in the color temperature of the ambient light, or a third change in the type or nature of the content being displayed: analyzing a histogram generated based on ambient light intensity information and the video signal to infer a first inference parameter using a trained machine learning model, the machine learning model being trained based on a first set of values ​​corresponding to the intensity of the ambient light, a second set of values ​​corresponding to the color temperature of the ambient light, and a third set of values ​​corresponding to content histogram values ​​characterizing the type or nature of the content in the video signal; selecting a first mapping function applicable to pixels corresponding to a frame of the video signal, said selection being based on the first estimation parameters; adjusting brightness of the content by automatically applying the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal; Including, method.

2. selecting a second mapping function applicable to pixels corresponding to a frame of the video signal in response to detecting a change in the intensity of the ambient light or the color temperature of the ambient light, the second mapping function being selected based on at least second estimated parameters output by the trained machine learning model; and automatically applying the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal. The method of claim 1.

3. The trained machine learning model is selected based on a device model that includes at least information regarding a model type of the device. The method of claim 2.

4. and requesting feedback from a user of the device, the feedback being associated with a change in a perceived quality of the content displayed following at least one of a first application of the first mapping function to the first plurality of pixels corresponding to the first set of frames of the video signal or a second application of the second mapping function to the second plurality of pixels corresponding to the second set of frames of the video signal. The method according to claim 3.

5. transmitting information regarding the feedback, the device model, and at least one of the first and second estimated parameters from the device to a remote computing system. The method according to claim 4.

6. a subset of training data used to train the trained machine learning model is generated based on feedback from users of a plurality of devices having the device model; The method according to claim 3.

7. The trained machine learning model is one of a plurality of machine learning models, and the method further comprises: training the plurality of machine learning models based on preference data associated with a perceived quality of content displayed by the device. The method of claim 1.

8. A device comprising: a processor configured to initiate display of content based on a video signal processed by the device; a machine learning model configured to analyze ambient light intensity information and a histogram generated based on the video signal to infer a first inference parameter, the machine learning model being trained based on a first set of values ​​corresponding to an intensity of the ambient light, a second set of values ​​corresponding to a color temperature of the ambient light, and a third set of values ​​corresponding to content histogram values ​​characterizing a type of the content in the video signal; and Computer executable instructions stored in a memory: selecting a first mapping function applicable to pixels corresponding to a frame of the video signal based on the first estimation parameters; and adjusting a brightness of the content by automatically applying the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal; A computer-executable instruction comprising: having device.

9. The computer executable instructions further include: select a second mapping function applicable to pixels corresponding to a frame of the video signal in response to detecting a change in the intensity of the ambient light or the color temperature of the ambient light, the second mapping function selected based on at least second estimated parameters output by the trained machine learning model; automatically applying the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal. It is configured as follows: The device of claim 8.

10. The trained machine learning model is selected based on a device model that includes at least information regarding a model type of the device. The device of claim 9.

11. the computer-executable instructions are further configured to request feedback from a user of the device, the feedback relating to a change in a perceived quality of the content displayed following at least one of a first application of the first mapping function to the first plurality of pixels corresponding to the first set of frames of the video signal or a second application of the second mapping function to the second plurality of pixels corresponding to the second set of frames of the video signal. The device of claim 10.

12. The computer-executable instructions are further configured to transmit information regarding the feedback, the device model, and at least one of the first and second estimated parameters from the device to a remote computing system. The device of claim 11.

13. the computer-executable instructions are implemented by an inference block and a training block configured to generate training data for use by the inference block, at least a subset of the training data being generated based on feedback from users of a plurality of devices having the device model; The device of claim 10.

14. the trained machine learning model is trained based on preference data regarding perceived quality of content displayed by the device; The device of claim 9.

15. 1. A non-transitory computer-readable medium encoding computer-executable instructions for executing a computer process, the computer process comprising: commencing display of content based on the video signal to a user of the device; analyzing ambient light intensity information and a histogram generated based on the video signal to infer a first inference parameter using a trained machine learning model, the machine learning model being trained based on at least a first set of values ​​corresponding to an intensity of ambient light, a second set of values ​​corresponding to a color temperature of the ambient light, and a third set of values ​​corresponding to content histogram values ​​characterizing a type or nature of the content in the video signal; selecting a first mapping function applicable to pixels corresponding to a frame of the video signal based on the first estimated parameters estimated by the machine learning model; adjusting brightness of the content by automatically applying the first mapping function to a first plurality of pixels corresponding to a first set of frames of the video signal; Including, Non-transitory computer-readable medium.

16. The computer process further comprises: selecting a second mapping function applicable to pixels corresponding to a frame of the video signal in response to detecting a change in the intensity of the ambient light or the color temperature of the ambient light, the second mapping function being selected based on at least second estimated parameters output by the trained machine learning model; automatically applying the second mapping function to a second plurality of pixels corresponding to a second set of frames of the video signal; Including, 16. The non-transitory computer-readable medium of claim 15.

17. The trained machine learning model is selected based on a device model that includes at least information regarding a model type of the device.

20. The non-transitory computer-readable medium of claim 16.

18. The computer process further includes receiving feedback from a user of the device, the feedback relating to a change in a perceived quality of the content displayed following at least one of a first application of the first mapping function to the first plurality of pixels corresponding to the first set of frames of the video signal or a second application of the second mapping function to the second plurality of pixels corresponding to the second set of frames of the video signal.

20. The non-transitory computer-readable medium of claim 17.

19. The computer process: transmitting information regarding the feedback, the device model, and at least one of the first and second estimated parameters to a remote computing system.

20. The non-transitory computer-readable medium of claim 18.

20. the trained machine learning model is trained based on preference data regarding perceived quality of content displayed by the device; 16. The non-transitory computer-readable medium of claim 15.

Images