Training method of video brightness prediction model, and intelligent brightness adjustment method and device

Abstract

The invention provides a training method of a video brightness prediction model, and an intelligent brightness adjustment method and device, and relates to machine learning in the technical field of artificial intelligence. The training method comprises the steps that a training data set is acquired, the training data set comprises multiple sets of training data, each set of training data comprises picture visual information of a terminal historical playing video and terminal data, and the terminal position comprises that a terminal is located indoors, outdoors or in moving; acquiring a simulation matrix based on the training data set; wherein the simulation matrix is used for simulating the missing probability of target type data in the training data set; and training the video brightness prediction model based on the training data set, and in the training process, processing an output matrix of an attention layer in the video brightness prediction model based on the simulation matrix to obtain an input matrix of a next layer. According to the technical scheme provided by the invention, the video brightness prediction model with high prediction accuracy can be trained, so that the video brightness corresponding to different scenes can be accurately predicted.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method for a video brightness prediction model, an intelligent brightness adjustment method and a device. Background Art

[0002] With the rapid development of Internet technology, more and more users watch videos based on computer devices. Therefore, the requirements for video services provided by computer devices are getting higher and higher. In response to this, major video service operators improve users' video viewing experience by adjusting the video brightness in video services.

[0003] In the related art, a computer device detects the brightness of ambient light through a light sensor or a camera, and adjusts the brightness of a video currently being played by the computer device based on the obtained ambient light brightness.

[0004] However, in the above method, the brightness of the ambient light is easily affected by the light emitted by the device itself, which results in poor adjustment accuracy and reduces the user's video viewing experience. Summary of the invention

[0005] The embodiment of the present application provides a video brightness prediction model training method, intelligent brightness adjustment method and device, which can train a video brightness prediction model with high prediction accuracy, and then accurately predict the video brightness corresponding to different scenes. The technical solution is as follows:

[0006] According to a first aspect of an embodiment of the present application, a method for training a video brightness prediction model is provided, the method comprising:

[0007] Acquire a training data set, the training data set includes multiple sets of training data, each set of training data includes visual information of a video played by a terminal in history and terminal data, the terminal data includes at least one of a play time, ambient light brightness during play, and a location of the terminal, the terminal location includes whether the terminal is indoors, outdoors, or moving;

[0008] Based on the training data set, a simulation matrix is ​​obtained; wherein the simulation matrix is ​​used to simulate the missing probability of target type data in the training data set, and the target type data includes ambient light brightness and terminal position;

[0009] Based on the training data set, the video brightness prediction model is trained to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

[0010] According to a second aspect of an embodiment of the present application, there is provided an intelligent brightness adjustment method, the method comprising:

[0011] In response to the video brightness adjustment instruction, obtaining visual information of the video currently being played by the terminal and terminal data, the terminal data including at least one of the playing time, the ambient light brightness during the playing, and the terminal location, the terminal location including whether the terminal is indoors, outdoors, or moving;

[0012] Inputting the picture visual information and the terminal data into a trained video brightness prediction model to obtain a target video brightness;

[0013] Adjust the brightness of the video currently being played on the terminal to the target video brightness.

[0014] According to a third aspect of an embodiment of the present application, there is provided an intelligent brightness adjustment device, the device comprising:

[0015] A first acquisition module is used to acquire a training data set, the training data set includes multiple sets of training data, each set of training data includes visual information of a video played by a terminal in history and terminal data, the terminal data includes at least one of a play time, ambient light brightness during play, and a terminal location, the terminal location includes whether the terminal is indoors, outdoors, or moving;

[0016] A second acquisition module is used to acquire a simulation matrix based on the training data set; wherein the simulation matrix is ​​used to simulate the missing probability of target type data in the training data set, and the target type data includes ambient light brightness and terminal position;

[0017] The training module is used to train the video brightness prediction model based on the training data set to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

[0018] In a possible implementation, the second acquisition module includes:

[0019] A determination unit, configured to determine a first probability based on the number of training data sets in which terminal positions are missing from the multiple training data sets, and to determine a second probability based on the number of training data sets in which ambient light brightness is missing from the multiple training data sets;

[0020] The adjustment unit is used to obtain a unit matrix, and adjust the unit matrix based on the first probability and the second probability to obtain the simulation matrix.

[0021] In a possible implementation, the adjustment unit includes:

[0022] A first updating subunit updates a first element in the identity matrix based on the first probability, wherein the probability that the first element is 0 is equal to the missing probability of the terminal position;

[0023] The second updating subunit updates the second element in the unit matrix based on the second probability, so that the probability that the second element is 0 is equal to the missing probability of the ambient light brightness.

[0024] In a possible implementation, the training module includes:

[0025] A computing unit is used to perform a dot multiplication of the simulated matrix with the output matrix of the first attention layer in the video brightness prediction model at the i-th iteration during the training process to obtain an input matrix of the next layer, wherein i is an integer greater than or equal to 1.

[0026] According to a fourth aspect of an embodiment of the present application, there is provided an intelligent brightness adjustment device, the device comprising:

[0027] an acquisition module, configured to acquire, in response to a video brightness adjustment instruction, visual information of a video currently being played by a terminal and terminal data, the terminal data including at least one of a play time, ambient light brightness during play, and a terminal location, the terminal location including whether the terminal is indoors, outdoors, or in motion;

[0028] A prediction module, used for inputting the picture visual information and the terminal data into a trained video brightness prediction model to obtain a target video brightness;

[0029] The adjustment module is used to adjust the brightness of the video currently played by the terminal to the target video brightness.

[0030] According to a fifth aspect of an embodiment of the present application, a computer device is provided, which includes a processor and a memory, the memory being used to store at least one program, the at least one program being loaded by the processor and executing the training method of the video brightness prediction model in the embodiment of the present application or the intelligent brightness adjustment method based on the video brightness prediction model.

[0031] According to the sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program is stored, and the at least one program is loaded and executed by a processor to implement a training method or an intelligent brightness adjustment method of a video brightness prediction model of an embodiment of the present application.

[0032] According to the seventh aspect of the embodiments of the present application, a computer program product is provided, which includes at least one program segment, and the at least one program segment is loaded and executed by a processor to implement the video brightness prediction model training method or intelligent brightness adjustment method in the embodiments of the present application.

[0033] In an embodiment of the present application, a simulation matrix is ​​used to simulate the missing probability of ambient light brightness and the missing probability of terminal position in various video playback scenarios, so that when training is performed based on a training data set and the simulation matrix, the video brightness prediction model can learn more video playback scenarios, and for each different video playback scenario, the corresponding video brightness can be learned. Therefore, a video brightness prediction model with high prediction accuracy can be trained, and the video brightness corresponding to different scenarios can be accurately predicted. At the same time, automatic adjustment of the video brightness is achieved, thereby improving the user's audio-visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application;

[0036] Figure 2 It is a flowchart of a method for training a video brightness prediction model provided in an embodiment of the present application;

[0037] Figure 3 is a structural schematic diagram of a video brightness prediction model provided according to an embodiment of the present application;

[0038] Figure 4 This is a schematic diagram of an example flow chart of a method for training a video brightness prediction model provided in an embodiment of the present application;

[0039] Figure 5 It is a flowchart of an intelligent brightness adjustment method provided according to an embodiment of the present application;

[0040] Figure 6 It is a structural schematic diagram of a training device for a video brightness prediction model provided according to an embodiment of the present application;

[0041] Figure 7 is a structural schematic diagram of an intelligent brightness adjustment device provided according to an embodiment of the present application;

[0042] Figure 8 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;

[0043] Fig. 9 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0045] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0046] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0047] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0048] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0049] It should be noted that the data (including but not limited to training data and prediction data, such as terminal data, etc.) and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the training data involved in this application are all obtained with full authorization.

[0050] In some embodiments, the video brightness prediction model training method or intelligent brightness adjustment method provided in the embodiments of the present application can be applied to the field of artificial intelligence technology. Next, artificial intelligence technology is introduced.

[0051] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0052] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, the pre-trained model is also called a large model or a basic model. After fine-tuning, it can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes several major directions, such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The training method for video brightness prediction and the intelligent brightness adjustment method provided in the embodiments of the present application involve computer vision technology and machine learning technology. Next, computer vision technology and machine learning are introduced.

[0053] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, follow and measure targets, and further process images so that the computer processes them into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Large model technology has brought important changes to the development of computer vision technology. Pre-trained models in the visual field such as swin-transformer (transformer is a neural network model based on attention mechanism), ViT, V-MOE, MAE, etc. can be quickly and widely applied to downstream specific tasks after fine tuning. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and common biometric recognition technologies such as face recognition and fingerprint recognition.

[0054] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0055] Figure 1 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application. The implementation environment includes: a terminal 101 and a server 102. The terminal 101 and the server 102 can be directly or indirectly connected via a wired network or a wireless network, and the present application does not limit this.

[0056] A video application is installed in the terminal 101, and a brightness adjustment function is provided in the video application, that is, the video application in the terminal 101 can play the video, and the video application can also adjust the brightness of the currently playing video. The terminal 101 includes but is not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, a smart car device, etc.

[0057] The server 102 is used to provide a video playback function to the terminal 101. When the terminal 101 plays a video, the video data is obtained from the server 102 for playback. The server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The number of the above-mentioned servers 102 can be more or less, and the embodiments of the present application are not limited to this. Of course, the server 102 can also include other functional servers to provide more comprehensive and diversified services.

[0058] Optionally, server 102 is used to train a video brightness prediction model and provide the trained video brightness prediction model to terminal 101 for use in predicting video brightness. The terminal can adjust the brightness of the currently playing video based on the predicted video brightness to improve the video playback effect and enhance the user's audio-visual experience.

[0059] In some embodiments, the wired or wireless network described above uses standard communication technology and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network, or any combination of a virtual private network. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0060] Figure 2 FIG. 1 is a flow chart of a method for training a video brightness prediction model according to an embodiment of the present application. Figure 2 As shown, in the embodiment of the present application, the method is described by taking the method being executed by a computer device as an example, and the computer device can be the above Figure 1 For the server or terminal shown, the method includes the following steps 201 to 203.

[0061] In step 201, a training data set is obtained, and the training data set includes multiple groups of training data. Each group of training data includes visual information of the terminal's historical video playback and terminal data. The terminal data includes at least one of the playback time, ambient light brightness during playback, and terminal location. The terminal location includes whether the terminal is indoors, outdoors, or moving.

[0062] Wherein, each set of training data includes the visual information of any frame of video in the video that has been played by the terminal in history and the terminal data; optionally, the arbitrary frame of video is the first frame of video in the video. The visual information of the picture is used to describe the factors that affect the visual effect in the video picture, for example, including the average pixel value of the video picture and the contrast brightness of the video picture; wherein the average pixel value of the video picture is the average value of all pixel values ​​in the video picture; the contrast of the video picture is the ratio of the pixel value at the brightest part of the video picture to the pixel value at the darkest part of the video picture; the playback time includes the moment when the video plays the video picture and the time attribute when it is played, such as holidays (or non-holidays) and days of the week. Different moments and / or holidays (non-holidays) and / or days of the week have different corresponding video playback scenes. For example, when the terminal plays the video at night, it may be necessary to reduce the video brightness to avoid glare; or when the terminal plays the video during the day, it may be necessary to increase the video brightness to adapt to the environment; therefore, the terminal data contained in the training data set can enable the video brightness prediction model to learn more playback scenes.

[0063] In some embodiments, the time of the video screen is stored in the form of a floating point number, with 0:00 as the starting point and the number of minutes that have passed. For example, the time corresponding to 1:12 is 72.

[0064] In some embodiments, holidays (or non-holidays), days of the week, and terminal locations of video images are stored in the form of N-dimensional unique hot coding; where N is an integer greater than or equal to 1; for example, two-dimensional unique hot coding is used to represent holidays (or non-holidays), [1, 0] is used to represent holidays, [0, 1] is used to represent non-holidays, etc.; seven-dimensional unique hot coding is used to represent the day of the week; [1,0,0,0,0,0,0] is used to represent Monday, and [0,0,0,0,0,0,1] is used to represent Sunday, etc.; three-dimensional unique hot coding is used to represent the terminal location, [1,0,0] is used to represent indoors, [0,1,0] is used to represent outdoors, and [0,0,1] is used to represent moving.

[0065] In some embodiments, the ambient light brightness of the video picture is stored in the form of floating point numbers, with the unit being nits.

[0066] In some embodiments, the terminal location is obtained by the terminal based on a map tool. For example, the terminal can determine whether the terminal is indoors or outdoors based on information related to the terminal location provided by GPS (Global Positioning System) and other map tools; or, based on information related to the terminal location provided by GPS, it can determine whether the terminal location continues to change within the current time period, and if so, it is determined that the terminal is moving.

[0067] In some embodiments, the terminal obtains the ambient light brightness of the environment in which the terminal is located based on the configured ambient light sensor, and stores the ambient light brightness as a type of terminal data to the server. The embodiment of the present application does not limit the ambient light sensor.

[0068] In some embodiments, when the terminal plays any video, the visual information and terminal data of the video screen of the video are obtained, and the visual information and terminal data of the video screen of the video are stored as a set of training data to the server, and for the server, for each terminal, a training data set within a cycle can be created periodically based on the visual information and terminal data from the terminal, that is, for a terminal, the server obtains a training data set for the terminal every preset time period. The embodiment of the present application does not limit the duration of the preset time period. Optionally, the terminal periodically collects visual information and terminal data, and each time a cycle is reached, the visual information and terminal data collected in the cycle are stored as a training data set to the server, and maintained by the server. Of course, the length of the above cycle can be set according to the real-time requirements, and the embodiment of the present application does not limit this.

[0069] In some embodiments, the training data set only includes training data from one terminal, so that a personalized video brightness prediction model can be generated for the terminal, so that the prediction results are more in line with the terminal user's own audio-visual habits. In some cases, with full authorization by the user, the training data set may also include training data from multiple terminals, so that a video brightness prediction model that can be shared by the platform can be trained. In the cold start phase, the terminal can use the video brightness prediction model shared by the platform to adjust the brightness, and during the use of the terminal, continuously accumulate training data unique to the terminal to train separately or train the terminal's own video brightness prediction model based on the existing model, thereby ensuring that the brightness adjustment can be applied to all stages of the user's use of the application, and providing users with personalized models that are more in line with user habits.

[0070] In some embodiments, the video brightness is divided into multiple levels. When each group of training data is stored, the video brightness in each group of training data is stored as a u×1-dimensional vector, where u is the preset number of video brightness levels, and u is an integer greater than or equal to 2. For example, any group of training data is stored as a u×1-dimensional vector (0, 0, 1, 0…, 0, 0), indicating that the number of video brightness levels in the group of training data is 3.

[0071] It should be noted that the acquisition of the above-mentioned types of information requires the user's separate authorization. For example, a query box is popped up to ask the user whether to authorize the terminal to obtain the information. After the authorization operation is detected, information can be obtained based on the authorization scope.

[0072] In step 202, a simulation matrix is ​​obtained based on the training data set; wherein the simulation matrix is ​​used to simulate the missing probability of target type data in the training data set, and the target type data includes ambient light brightness and terminal position.

[0073] The terminal may not have an environmental sensor installed or the terminal is not authorized, and may not be able to obtain the terminal location and / or ambient light brightness, resulting in missing terminal location and / or missing ambient light brightness. In order to enable the trained video brightness prediction model to be applicable to such scenarios, a simulation matrix is ​​used to simulate the missing probability of target type data in the training data set.

[0074] In step 203, the video brightness prediction model is trained based on the training data set to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

[0075] In some embodiments, the video brightness prediction model may be an encoder in a transformer, wherein the transformer is provided with multiple encoders, the multiple encoders are sequentially connected, the output of the previous encoder is used as the input of the next encoder, and each encoder is provided with an attention layer. Figure 3 As shown in FIG, there are 10 encoders in the transformer. When the attention layer in an encoder outputs an output matrix, the encoder processes the output matrix based on the simulation matrix to obtain the input matrix of the attention layer in the next encoder.

[0076] In the embodiments of the present application, a simulation matrix is ​​used to simulate the missing probability of ambient light brightness and the missing probability of terminal position in various video playback scenarios, so that when training is performed based on a training data set and the simulation matrix, the video brightness prediction model can learn more video playback scenarios, and for each different video playback scenario, the corresponding video brightness can be learned. Therefore, a video brightness prediction model with high prediction accuracy can be trained, and the video brightness corresponding to different scenarios can be accurately predicted. At the same time, automatic adjustment of the video brightness is achieved, thereby improving the user's audio-visual experience.

[0077] Figure 2 The embodiment shown is a brief process of the embodiment of the present application. Below is an example process diagram of a training method of a video brightness prediction model provided by the embodiment of the present application. Figure 4 As shown, in the embodiment of the present application, the training method is described by taking the execution of a computer device as an example. The computer device can be the above Figure 1 The server or terminal shown in the figure, the method includes the following steps 401 to 405.

[0078] In step 401, a training data set is obtained; the training data set includes multiple groups of training data, each group of training data includes visual information of the terminal's historical video playback and terminal data, the terminal data includes at least one of the playback time, ambient light brightness during playback, and terminal location, the terminal location includes whether the terminal is indoors, outdoors, or moving.

[0079] In some embodiments, an initial data set is obtained, and the data in the initial data set is preprocessed to obtain a training data set. The preprocessing includes processing the average pixel value and contrast of the video picture in the picture visual information to obtain a more stable target pixel value and target contrast of the video picture, so as to reduce the variance of the video picture changes and prevent the video brightness prediction model from overfitting.

[0080] For example, the target pixel value of the previous frame of a video frame is obtained, and the average pixel value of the frame of the video frame and the target pixel value of the previous frame of the video frame are weighted averaged to obtain the target pixel value of the video frame; wherein the above-mentioned weighted average process is implemented based on the exponential sliding average method.

[0081] The above method is implemented based on the following formula (1):

[0082] X t ←τX t-1 +(1-τ)x t (1)

[0083] Where: t represents the tth group of training data in the training data set, t is greater than or equal to 1 and is an integer; τ represents the preset hyperparameter, τ is greater than 0 and less than 1; X t represents the target pixel value of the video screen in the tth group of training data; X t-1 represents the target pixel value of the video screen in the t-1th group of training data; x t Represents the average pixel value of the video screen corresponding to the tth group of training data.

[0084] It should be noted that the method for obtaining the target contrast of any frame of video is the same as the method for obtaining the target pixel value of the video, and will not be repeated in the embodiments of the present application.

[0085] In the embodiment of the present application, a video brightness prediction model is trained based on a more stable target pixel value and target contrast of a video picture, thereby improving the stability and applicability of the target video brightness output by the video brightness prediction model. Even if the video brightness is unstable during playback, a relatively stable target video brightness can be predicted, thereby avoiding the problem of flickering video brightness during adjustment, which is beneficial to improving the user's video viewing experience.

[0086] In some embodiments, the above preprocessing also includes: performing matrix transformation on holidays (non-holidays), terminal locations and days of the week represented in one-hot encoding form based on the embedded matrix to obtain multiple e×1 dimensional vectors, where e is an integer greater than or equal to 2, so that each element in any e×1 dimensional vector is a floating point number, and e is determined based on the parameters of the video brightness prediction model.

[0087] In step 402, a first probability is determined based on the number of training data sets with missing terminal positions among the multiple training data sets, and a second probability is determined based on the number of training data sets with missing ambient light brightness among the multiple training data sets.

[0088] In some embodiments, multiple groups of training data in a training data set are detected to obtain the number of groups of training data missing at the terminal location and the number of groups of training data missing at the ambient light brightness. The ratio between the number of groups of training data missing at the terminal location and the total number of groups of training data is used as the first probability, and the ratio between the number of groups of training data missing at the ambient light brightness and the total number of groups of training data is used as the second probability.

[0089] In step 403, the identity matrix is ​​obtained.

[0090] The dimension of the unit matrix (all elements on the diagonal are 1, and all other elements are 0) is e×e, where e is an integer greater than or equal to 2, and e is determined based on the parameters of the video brightness prediction model.

[0091] In step 404, the unit matrix is ​​adjusted based on the first probability and the second probability to obtain a simulation matrix, wherein the simulation matrix is ​​used to simulate the missing probability of the target type data in the training data set.

[0092] In some embodiments, the above step 404 includes:

[0093] Based on the first probability, the first element in the unit matrix is ​​updated, and the probability that the first element is 0 is equal to the missing probability of the terminal position; based on the second probability, the second element in the unit matrix is ​​updated, so that the probability that the second element is 0 is equal to the missing probability of the ambient light brightness.

[0094] In some embodiments, the dimension of the output matrix is ​​e×512, wherein when the terminal position corresponding to each frame of video is mapped to the kth (k is an integer greater than or equal to 1 and less than or equal to e) row of the output matrix, the terminal position of each frame of video is matched with the element 1 of the kth row and kth column in the unit matrix. k，k Determined as the first element, the first probability represents 1 k，k When the ambient light brightness corresponding to each frame of video is mapped to the jth (j is an integer greater than or equal to 1, less than or equal to e, k≠j) row of the output matrix, the ambient light brightness of each frame of video is compared with the element 1 in the jth row and jth column of the unit matrix. j，j Determined as the second element, the second probability represents 1 j，j The probability of being 0.

[0095] In the embodiment of the present application, through the adjustment process of steps 403 to 404, the simulation matrix can simulate the missing probability of the ambient light brightness of the training data set and the missing probability of the terminal position.

[0096] In step 405, the video brightness prediction model is trained based on the training data set to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

[0097] In some embodiments, the above step 405 includes: at the i-th iteration during the training process, the simulation matrix is ​​dot-multiplied with the output matrix of the first attention layer in the video brightness prediction model to obtain the input matrix of the next layer, where i is an integer greater than or equal to 1.

[0098] In some embodiments, when the video brightness prediction model is trained based on the training data set, the simulation matrix of the training data set is input into the first encoder in the video brightness prediction model, so that the output matrix of the attention layer of the encoder, that is, the first attention layer of the video brightness prediction model, is multiplied by points to obtain the input matrix of the next encoder, that is, the next attention layer. For example: when i=1, in the first iteration during the training process, any set of training data and the simulation matrix in the training data set are input into the first attention layer, and the set of training data is processed to obtain the output matrix of the first attention layer; the first encoder multiplies the output matrix of the first attention layer with the simulation matrix to obtain the input matrix of the second attention layer (that is, the attention layer in the second encoder), and multiplies the output matrix of the first attention layer with the simulation matrix, realizing the simulation of the missing probability of the ambient light brightness of the input arbitrary set of training data and the missing probability of the terminal position based on the simulation matrix; wherein, the first attention layer processes the set of training data based on the multi-head attention mechanism. In the subsequent multiple iterations, the processing process is the same as the above process.

[0099] In some embodiments, each set of training data includes multiple e×1 dimensional vectors. For any e×1 dimensional vector, a query value weight matrix, a key value weight matrix, and an eigenvalue weight matrix are obtained, wherein the query value weight matrix, the key value weight matrix, and the eigenvalue weight matrix are all e×512 dimensional matrices; the e×1 dimensional vector is point-multiplied with the query value weight matrix, the key value weight matrix, and the feature weight matrix, respectively, to obtain the query value vector, the key value vector, and the eigenvalue vector; the query value vector of each e×1 dimensional vector is point-multiplied with each key value vector, respectively, to obtain the weight vector corresponding to the arbitrary set of training data; the eigenvalue vector of each e×1 dimensional vector is weighted summed based on the weight vector, and the result of the weighted summation is normalized to obtain the output matrix of the first attention layer. Among them, the result of the weighted summation can be normalized by the softmax function. The embodiment of the present application does not specifically limit the query value weight matrix, the key value weight matrix, and the eigenvalue weight matrix.

[0100] For example, the output matrix of the first encoder, which is also the input matrix of the second attention layer, can be obtained based on the following formula (2):

[0101]

[0102] Wherein, SA represents the output matrix of the first encoder; M represents the simulation matrix; Q represents the query value vector, K represents the key value vector, V represents the eigenvalue vector; K T represents the transpose of K, d a represents the dimension of the vector, QKT Represents the similarity between any Q and any K.

[0103] In some embodiments, taking the video brightness prediction model including multiple encoders as an example, from the second attention layer to the last attention layer, the output matrix of the previous attention layer is used as the input matrix of this layer, and the input matrix of this layer is processed respectively until the output matrix of the last attention layer is obtained.

[0104] For example, from the second attention layer to the last attention layer, the acquired input matrix can be processed based on the following formula (3).

[0105]

[0106] Among them, TA represents the output matrix of any layer from the second attention layer to the last attention layer; Q represents the query value vector, K represents the key value vector, and V represents the eigenvalue vector; K T represents the transpose of K, d a Represents the dimension of the vector.

[0107] In some embodiments, the video brightness is divided into multiple levels according to the degree of brightness. After obtaining the output matrix of the last attention layer, based on the fully connected weight matrix of the fully connected layer, the output matrix of the last attention layer is multiplied by the fully connected weight matrix to obtain a u×1-dimensional result vector. The result vector including u elements is mapped to a mapping vector including u elements whose sum is 1. Each element in the mapping vector is associated with a level of video brightness. For example, the third element in the mapping vector is associated with the third level of video brightness. The level number associated with the element with the largest value is used as the predicted level number of the video brightness. For example, the mapping vector is (p1, p2, p3, p4, ..., p s , ..., p u ), s is an integer less than or equal to u.

[0108] For example, the result vector is mapped to a mapping vector including u elements whose sum is 1 based on the following formula (4).

[0109]

[0110] Among them, p f represents the fth element in the mapping vector; L f represents the fth element in the result vector, L y The yth element in the result vector, and d represents the number of elements in the result vector.

[0111] In some embodiments, after obtaining the mapping vector of any set of training data, the number of video brightness levels represented by the vector in the training data is obtained, and the elements associated with the number of video brightness levels in the mapping vector are calculated to obtain the prediction accuracy of the video brightness prediction model. If the prediction accuracy does not meet the preset stop condition, the current query value weight matrix, key value weight matrix, and eigenvalue weight matrix are updated, and training continues; if the prediction accuracy meets the preset stop condition, the training is stopped to obtain a trained video brightness prediction model. The prediction accuracy is determined based on the cross entropy loss function. The smaller the value of the cross entropy loss function, the closer the predicted video brightness is to the actual video brightness. For example, when u=10, the mapping vector of any set of training data is (0.1, 0.1, 0.05, 0.25, 0.15, 0.1, 0.05, 0.1, 0.05, 0.05), the element with the largest value in the mapping vector is 0.25, and 0.25 corresponds to the fourth gear of video brightness. The fourth gear of video brightness is determined as the predicted video brightness, and the gear of the video brightness of this set of training data is (0, 1, 0, 0, 0, 0, 0, 0, 0), that is, the gear is the second gear; based on the element 0.1 corresponding to the second gear in the mapping vector, the cross entropy loss function value is calculated, that is, the prediction accuracy. The embodiment of the present application does not limit the preset stop condition.

[0112] For example, the prediction accuracy can be determined based on the following formula (5):

[0113] ρ=-log(p m ) (5)

[0114] Among them, ρ represents the value of the cross entropy loss function, m represents the number of levels corresponding to the video brightness in the training data, and ρ m Represents the element in the mapping vector associated with the number of levels corresponding to the video brightness in the training data.

[0115] In the embodiment of the present application, the output matrix of the first attention layer is processed based on the simulation matrix, thereby realizing the simulation of the training data set, and further being able to simulate the situation in the video playback scenario where a certain type of data cannot be obtained, so that the trained video brightness prediction model can adapt to more video playback scenarios, and for different video playback scenarios, the corresponding target video brightness can be accurately predicted, thereby improving the user's video viewing experience.

[0116] After the video brightness prediction model is obtained based on the above step 405, the brightness of the video currently played by the terminal can be adjusted based on the video brightness prediction model. Figure 5 The technical solution of this application is further explained. Figure 51 is a flow chart of an intelligent brightness adjustment method provided according to an embodiment of the present application. In the embodiment of the present application, the intelligent brightness adjustment method is described by taking the execution of a computer device as an example. The computer device may be the above-mentioned Figure 1 The server shown in the figure comprises the following steps 501 to 503:

[0117] In step 501, in response to the video brightness adjustment instruction, the server obtains the visual information of the video currently playing by the terminal and the terminal data, the terminal data including the playing time, the ambient light brightness during the playing and at least one of the terminal position, the terminal position including whether the terminal is indoors, outdoors or in motion.

[0118] In some embodiments, the terminal sends a video brightness adjustment instruction based on a preset adjustment frequency, so that the server obtains the video brightness adjustment instruction once every period of time, or the server automatically triggers the video brightness adjustment instruction based on the preset adjustment frequency. The preset adjustment frequency can be set by the terminal itself or a default frequency of the server.

[0119] In step 502, the server inputs the picture visual information and the terminal data into the trained video brightness prediction model to obtain the target video brightness.

[0120] In some embodiments, the trained video brightness prediction model processes the picture visual information and terminal data based on the attention layers in multiple encoders to obtain the target video brightness.

[0121] In step 503, the server adjusts the brightness of the video currently being played by the terminal to the target video brightness.

[0122] In some embodiments, the server performs the video brightness prediction process, and the terminal performs the brightness adjustment, that is, after predicting the target video brightness, the server sends the target video brightness to the terminal, and the terminal performs brightness adjustment based on the received target video brightness. In other embodiments, the terminal performs the above prediction and adjustment process, that is, the terminal downloads a trained video brightness prediction model from the server, predicts the target video brightness based on the video brightness prediction model, and then adjusts the brightness of the video currently playing on the terminal based on the target video brightness.

[0123] In some embodiments, after obtaining the target video brightness, if the target video brightness is too high, a prompt is given to the terminal to prompt the terminal that the target video brightness to be adjusted is too high, which is not conducive to eye health, so that the user can make an operation suitable for himself according to the prompt. Optionally, the ratio of the target video brightness to the corresponding ambient light brightness is obtained, and the ratio is compared with a preset ratio. If the ratio is greater than the preset ratio, a prompt is given. The preset ratio is not specifically limited in the present embodiment of the application.

[0124] It should be noted that for the video brightness prediction model trained for user personalization, when the brightness adjustment function of any terminal is activated, the corresponding video brightness prediction model can be obtained or called according to the terminal or terminal identifier to perform brightness prediction and subsequent processing.

[0125] In the embodiment of the present application, the target video brightness of the video currently being played by the terminal is predicted based on the terminal data of the video screen being played in the video currently being played by the terminal and the visual information of the screen, so as to realize the prediction of the target video brightness of the video played by the terminal in different scenarios for different users, and automatically adjust the brightness of the video currently being played by the terminal according to the target video brightness through the terminal, thereby improving the user's video viewing experience.

[0126] Figure 6 : is a structural schematic diagram of a training device for a video brightness prediction model provided according to an embodiment of the present application, the device comprising:

[0127] A first acquisition module 601 is used to acquire a training data set, where the training data set includes multiple sets of training data, each set of training data includes visual information of a video played in history by a terminal and terminal data, where the terminal data includes at least one of time, ambient light brightness, and terminal location;

[0128] The second acquisition module 602 is used to acquire a simulation matrix based on the training data set; wherein the simulation matrix is ​​used to simulate the missing probability of target type data in the training data set, and the target type data includes ambient light brightness and terminal position;

[0129] The training module 603 is used to train the video brightness prediction model based on the training data set to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

[0130] In a possible implementation, the second acquisition module 602 includes:

[0131] A determination unit, configured to determine a first probability based on the number of training data sets in which terminal positions are missing from the multiple training data sets, and to determine a second probability based on the number of training data sets in which ambient light brightness is missing from the multiple training data sets;

[0132] The adjustment unit is used to obtain a unit matrix, and adjust the unit matrix based on the first probability and the second probability to obtain a simulation matrix.

[0133] In a possible implementation, the adjustment unit includes:

[0134] A first updating subunit updates a first element in the identity matrix based on the first probability, wherein the probability that the first element is 0 is equal to the missing probability of the terminal position;

[0135] The second updating subunit updates the second element in the unit matrix based on the second probability, so that the probability that the second element is 0 is equal to the missing probability of the ambient light brightness.

[0136] In a possible implementation, the training module 603 includes:

[0137] A computing unit is used to perform a dot multiplication of the simulated matrix with the output matrix of the attention layer in the video brightness prediction model at the i-th iteration during the training process to obtain an input matrix of the next layer, where i is an integer greater than or equal to 1.

[0138] It should be noted that: when executing the corresponding steps, the training device for the video brightness prediction model provided in the above embodiment only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the training device for the video brightness prediction model provided in the above embodiment and the training method embodiment of the video brightness prediction model belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0139] In the embodiments of the present application, a simulation matrix is ​​used to simulate the missing probability of ambient light brightness and the missing probability of terminal position in various video playback scenarios, so that when training is performed based on a training data set and the simulation matrix, the video brightness prediction model can learn more video playback scenarios, and for each different video playback scenario, the corresponding video brightness can be learned. Therefore, a video brightness prediction model with high prediction accuracy can be trained, and the video brightness corresponding to different scenarios can be accurately predicted, thereby realizing automatic adjustment of video brightness and improving the user's audio-visual experience.

[0140] Figure 7: is a structural schematic diagram of an intelligent brightness adjustment device provided according to an embodiment of the present application, the device comprising:

[0141] The acquisition module 701 acquires, in response to the video brightness adjustment instruction, the visual information of the video currently being played by the terminal and the terminal data, where the terminal data includes at least one of the playing time, the ambient light brightness during the playing, and the terminal position, where the terminal position includes whether the terminal is indoors, outdoors, or moving;

[0142] Prediction module 702, used to input the picture visual information and terminal data into the trained video brightness prediction model to obtain the target video brightness;

[0143] The adjustment module 703 is used to adjust the brightness of the video currently played by the terminal to the target video brightness.

[0144] It should be noted that: when the intelligent brightness adjustment device provided in the above embodiment executes the corresponding steps, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the intelligent brightness adjustment device provided in the above embodiment and the intelligent brightness adjustment method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0145] In the embodiment of the present application, the target video brightness of the video currently being played by the terminal is predicted based on the terminal data of the video screen being played in the video currently being played by the terminal and the visual information of the screen, so as to realize the prediction of the target video brightness of the video played by the terminal in different scenarios for different users, and automatically adjust the brightness of the video currently being played by the terminal according to the target video brightness through the terminal, thereby improving the user's video viewing experience.

[0146] In an exemplary embodiment, a computer device is also provided, which includes a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the video brightness prediction model training method or intelligent brightness adjustment method in the embodiment of the present application.

[0147] In an exemplary embodiment, a computer device is also provided. Taking the computer device as a terminal as an example,

[0148] When the computer device is configured as a terminal, Figure 8 A structural block diagram of a terminal 800 provided by an exemplary embodiment of the present application is shown.

[0149] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0150] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0151] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one program code, which is used to be executed by the processor 801 to implement the training method of the video brightness prediction model or the intelligent brightness adjustment method provided in the method embodiment of the present application.

[0152] In some embodiments, the terminal 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808 and a power supply 809.

[0153] The peripheral device interface 803 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0154] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0155] The display screen 805 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on the surface or above the surface of the display screen 805. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 805 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 805 can be one, set on the front panel of the terminal 800; in other embodiments, the display screen 805 can be at least two, respectively set on different surfaces of the terminal 800 or in a folding design; in other embodiments, the display screen 805 can be a flexible display screen, set on the curved surface or folding surface of the terminal 800. Even, the display screen 805 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0156] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0157] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing, or input them into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0158] The positioning component 808 is used to locate the current user position of the terminal 800 to implement navigation or LBS (Location Based Service).

[0159] The power supply 809 is used to power various components in the terminal 800. The power supply 809 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 809 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0160] In some embodiments, the terminal 800 further includes one or more sensors 810 . The one or more sensors 810 include, but are not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , an optical sensor 814 , and a proximity sensor 815 .

[0161] The acceleration sensor 811 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 800. For example, the acceleration sensor 811 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 811. The acceleration sensor 811 can also be used to collect game or user motion data.

[0162] The gyroscope sensor 812 can detect the body direction and rotation angle of the terminal 800, and the gyroscope sensor 812 can cooperate with the acceleration sensor 811 to collect the user's 3D actions on the terminal 800. The processor 801 can implement the following functions based on the data collected by the gyroscope sensor 812: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0163] The pressure sensor 813 can be set on the side frame of the terminal 800 and / or the lower layer of the display screen 805. When the pressure sensor 813 is set on the side frame of the terminal 800, it can detect the user's holding signal of the terminal 800, and the processor 801 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 813. When the pressure sensor 813 is set on the lower layer of the display screen 805, the processor 801 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 805. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0164] The optical sensor 814 is used to collect the ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 according to the ambient light intensity collected by the optical sensor 814. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is reduced. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 according to the ambient light intensity collected by the optical sensor 814.

[0165] The proximity sensor 815, also called a distance sensor, is usually arranged on the front panel of the terminal 800. The proximity sensor 815 is used to collect the distance between the user and the front of the terminal 800. In one embodiment, when the proximity sensor 815 detects that the distance between the user and the front of the terminal 800 is gradually decreasing, the processor 801 controls the display screen 805 to switch from the screen-on state to the screen-off state; when the proximity sensor 815 detects that the distance between the user and the front of the terminal 800 is gradually increasing, the processor 801 controls the display screen 805 to switch from the screen-off state to the screen-on state.

[0166] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0167] Take the computer device as a server as an example. Fig. 9It is a structural diagram of a server provided according to an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 901 and one or more memories 902, wherein the memory 902 stores at least one program, and the at least one program is loaded and executed by the processor 901 to implement the training method of the video brightness prediction model or the intelligent brightness adjustment method provided by the above-mentioned various method embodiments. Of course, the server can also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server can also include other components for implementing device functions, which will not be repeated here.

[0168] An embodiment of the present application also provides a computer-readable storage medium, which is applied to a computer device, and stores at least one program, which is loaded and executed by a processor to implement the training method of the video brightness prediction model or the intelligent brightness adjustment method in the above-mentioned embodiment.

[0169] The embodiment of the present application also provides a computer program product, which includes at least one program, and the at least one program is stored in a computer-readable storage medium. The processor of the computer device reads the at least one program from the computer-readable storage medium, and the processor executes the at least one program, so that the computer device executes to implement the video brightness prediction model training method or intelligent brightness adjustment method in the above embodiment.

[0170] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0171] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a video brightness prediction model, characterized in that: The method comprises: Acquire a training data set, the training data set including multiple sets of training data, each set of training data including visual information of a video played historically by a terminal and terminal data, the terminal data including at least one of a play time, ambient light brightness during play, and a terminal location, the terminal location including whether the terminal is indoors, outdoors, or moving; Based on the training data set, a simulation matrix is ​​obtained; wherein the simulation matrix is ​​used to simulate the missing probability of target type data in the training data set, and the target type data includes ambient light brightness and terminal position; Based on the training data set, the video brightness prediction model is trained to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

2. The method according to claim 1, characterized in that The step of obtaining a simulation matrix based on the training data set includes: Determine a first probability based on the number of training data sets in which terminal positions are missing among the multiple sets of training data, and determine a second probability based on the number of training data sets in which ambient light brightness is missing among the multiple sets of training data; Obtain a unit matrix, and adjust the unit matrix based on the first probability and the second probability to obtain the simulation matrix.

3. The method according to claim 2, characterized in that The adjusting the unit matrix based on the first probability and the second probability to obtain the simulation matrix includes: Based on the first probability, a first element in the identity matrix is ​​updated, and the probability that the first element is 0 is equal to the missing probability of the terminal position; The second element in the identity matrix is ​​updated based on the second probability so that the probability that the second element is 0 is equal to the probability of missing the ambient light brightness.

4. The method according to claim 3, characterized in that: In the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer, including: At the i-th iteration during the training process, the simulation matrix is ​​dot-multiplied with the output matrix of the first attention layer in the video brightness prediction model to obtain the input matrix of the next layer, where i is an integer greater than or equal to 1.

5. The method according to claim 1, characterized in that The prediction model is an encoder in a transformer, and the encoder is used to process the output matrix of the attention layer in the video brightness prediction model based on the simulation matrix to obtain the input matrix of the next layer.

6. An intelligent brightness adjustment method, characterized in that: The method comprises: In response to the video brightness adjustment instruction, obtaining visual information of the video currently being played by the terminal and terminal data, wherein the terminal data includes at least one of the playing time, the ambient light brightness during the playing, and the terminal location, wherein the terminal location includes whether the terminal is indoors, outdoors, or in motion; Inputting the picture visual information and the terminal data into a trained video brightness prediction model to obtain a target video brightness; The brightness of the video currently being played by the terminal is adjusted to the target video brightness.

7. A training device for a video brightness prediction model, characterized in that: The device comprises: A first acquisition module is used to acquire a training data set, wherein the training data set includes multiple sets of training data, each set of training data includes visual information of a video played by a terminal in history and terminal data, wherein the terminal data includes at least one of a play time, ambient light brightness during play, and a terminal location, wherein the terminal location includes whether the terminal is indoors, outdoors, or moving; A second acquisition module is used to acquire a simulation matrix based on the training data set; wherein the simulation matrix is ​​used to simulate the missing probability of target type data in the training data set, and the target type data includes ambient light brightness and terminal position; A training module is used to train the video brightness prediction model based on the training data set to obtain the trained video brightness prediction model. During the training process, the output matrix of the attention layer in the video brightness prediction model is processed based on the simulation matrix to obtain the input matrix of the next layer.

8. An intelligent brightness adjustment device, characterized in that: The device comprises: an acquisition module, configured to acquire, in response to a video brightness adjustment instruction, visual information of a video currently being played by a terminal and terminal data, wherein the terminal data includes at least one of a play time, ambient light brightness during play, and a terminal location, wherein the terminal location includes whether the terminal is indoors, outdoors, or in motion; A prediction module, used for inputting the picture visual information and the terminal data into a trained video brightness prediction model to obtain a target video brightness; The adjustment module is used to adjust the brightness of the video currently played by the terminal to the target video brightness.

9. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one program, and the at least one program is loaded by the processor and executed according to the training method of the video brightness prediction model described in any one of claims 1 to 5 or the intelligent brightness adjustment method described in claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the training method of the video brightness prediction model according to any one of claims 1 to 5 or the intelligent brightness adjustment method according to claim 6.

11. A computer program product, characterized in that The computer program product includes at least one program, which is loaded and executed by a processor to implement the training method of the video brightness prediction model according to any one of claims 1 to 5 or the intelligent brightness adjustment method according to claim 6.

Images