Film data processing method, device, medium and electronic equipment

Abstract

The present invention provides a film data processing method, comprising: obtaining a first film data sample, wherein the first film data sample includes historical film data; using the first film data sample, iteratively updating parameters in a deep learning training model of each GPU in each GPU block, and obtaining local update parameters of the deep learning training model of each GPU in each GPU block; determining block synchronization parameters of the deep learning training model in each GPU block according to the local update parameters of the deep learning training model of each GPU in each GPU block; and determining global synchronization parameters of the deep learning training models in all GPU blocks according to the block synchronization parameters of the deep learning training model in each GPU block. Training the model in this manner can improve communication efficiency and meet the requirements of computing efficiency during actual operation, effectively avoiding the defect that the communication efficiency of each device is constrained during parallel operation.

Description

Technical Field

[0001] The present invention relates to the technical field of film data processing, and in particular to a film data processing method, device, medium and electronic equipment. Background Art

[0002] Stochastic Gradient Descent (SGD) is extremely efficient, making it the most common training method in most distributed machine learning applications. For example, when considering computational efficiency, mini-batch SGD can be used to train models. However, while mini-batch SGD can calculate multiple stochastic gradients, it does not meet the actual computational efficiency requirements. Another example is when considering reducing training time, large-batch SGD can be used to train models. However, due to the lack of proper consideration of local parallelism between devices and inter-device communication efficiency, inter-device communication efficiency is low. In particular, as the number of devices increases, the degree of parallelism on each device severely constrains communication efficiency.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a film data processing method, device, medium and electronic device, which can at least improve communication efficiency to a certain extent, meet the requirements of computing efficiency during actual operation, and effectively avoid the defect that the communication efficiency of each device is constrained when running in parallel.

[0005] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0006] According to a first aspect of an embodiment of the present invention, a film data processing method is provided, which is applied to a film system, wherein the film system includes a deep learning training model, and the deep learning training model is deployed in a GPU block of each server in multiple servers. The method includes: using a hierarchical local stochastic gradient descent method to obtain global synchronization parameters of the deep learning training model in all GPU blocks; wherein using a hierarchical local stochastic gradient descent method to obtain the global synchronization parameters of the deep learning training model in all GPU blocks includes: obtaining a first film data sample, the first film data sample including historical film data; using the first film data sample, iteratively updating the parameters in the deep learning training model of each GPU in each GPU block to obtain local update parameters of the deep learning training model of each GPU in each GPU block; determining the block synchronization parameters of the deep learning training model in each GPU block based on the local update parameters of the deep learning training model of each GPU in each GPU block; and determining the global synchronization parameters of the deep learning training model in all GPU blocks based on the block synchronization parameters of the deep learning training model in each GPU block.

[0007] In some embodiments of the present invention, the expression for the local update parameter of the deep learning training model of each GPU in each GPU block includes:

[0008]

[0009] in, It represents the local updated parameters of the deep learning training model of the kth GPU in the lth GPU block during the tth round of training after H local iterative updates. represents the parameters of the deep learning training model of the kth GPU in the lth GPU block before the local iterative update in the tth round of training, γ represents the learning rate, and B loc represents the number of the first movie data samples required in one local iteration, i represents the first movie data sample, represents the set of first movie data samples required by the kth GPU in the first GPU block during the h-1th local iteration of the tth round of training, f i represents the loss function of the i-th sample of the first movie data.

[0010] In some embodiments of the present invention, the expression of the block synchronization parameter of the deep learning training model in each GPU block includes:

[0011]

[0012] in, Defines the block synchronization parameters of the deep learning training model in the lth GPU block during the first block synchronization process in the tth round of training. K represents the parameters of the deep learning training model in the lth GPU block before block synchronization during the tth round of training. l Indicates the total number of GPUs in the lth GPU block, It represents the local updated parameters of the deep learning training model of the k-th GPU in the l-th GPU block during the t-th round of training after H local iterative updates.

[0013] In some embodiments of the present invention, the expression of the global synchronization parameter of the deep learning training model in all GPU blocks includes:

[0014]

[0015] in, Represents the global synchronization parameters of the deep learning training model in all GPU blocks during the t+1 round of training. represents the global synchronization parameter of the deep learning training model in all GPU blocks during the t-th round of training. K is the total number of GPUs in all GPU blocks. Indicates the deep learning training model of each GPU block in all GPU blocks during the t-th round of training. b Block synchronization parameters during the secondary block synchronization process.

[0016] In some embodiments of the present invention, the method further includes: obtaining a second film data sample, the second film data sample being different from the first film data sample, the second film data sample comprising historical film data and new film data; using the historical film data to train the global synchronized deep learning training model to obtain a category sequence of the historical film data, and using the new film data to train the global synchronized deep learning training model to obtain a category sequence of the new film data; obtaining a loss function for this training based on the category sequence of the historical film data and the category sequence of the new film data; and looping the above steps to obtain a prediction model, wherein the prediction model comprises historical category labels and newly added category labels.

[0017] In some embodiments of the present invention, the expression of the category sequence of the historical film data includes:

[0018] q d =o d 1≤d≤n

[0019] Among them, q drepresents the category sequence of the historical film data, d represents the category label in the category sequence of the historical film data, and n represents the total number of category labels in the deep learning training model after global synchronization.

[0020] In some embodiments of the present invention, the expression of the category sequence of the new movie data includes:

[0021] q d =αo d +βn+1≤d≤n+m

[0022] Wherein, qd represents the category sequence of the new movie data, d represents the category label in the category sequence of the new movie data, α and β represent the bias parameters in the deep learning training model after global synchronization, n represents the total number of category labels in the deep learning training model after global synchronization, and n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new movie data.

[0023] In some embodiments of the present invention, the formula for calculating the loss function is:

[0024]

[0025] Among them, n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new video data, and q d Represents the category sequence of the historical video data and / or the category sequence of the new video data.

[0026] In some embodiments of the present invention, the method further includes: storing the historical film data and / or the new film data in a distributed search engine; receiving film keywords input by a user; based on the film keywords, calling an interface corresponding to the film keywords from the distributed search engine, and using the interface to call film data matching the film keywords; and sending the film data matching the film keywords to the terminal.

[0027] According to a second aspect of an embodiment of the present invention, a film data processing device is provided, which is applied to a film system, wherein the film system includes a deep learning training model, and the deep learning training model is deployed in a GPU block of each server in multiple servers. The device includes: a first acquisition module, used to obtain a first film data sample, wherein the first film data sample includes historical film data; a local iterative update module, which uses the first film data sample to iteratively update the parameters in the deep learning training model of each GPU in each GPU block to obtain the local updated parameters of the deep learning training model of each GPU in each GPU block; a block synchronization module, used to determine the block synchronization parameters of the deep learning training model in each GPU block according to the local updated parameters of the deep learning training model of each GPU in each GPU block; and a global synchronization module, used to determine the global synchronization parameters of the deep learning training models in all GPU blocks according to the block synchronization parameters of the deep learning training model in each GPU block.

[0028] In some embodiments of the present invention, the expression for the local update parameter of the deep learning training model of each GPU in each GPU block includes:

[0029]

[0030] in, It represents the local updated parameters of the deep learning training model of the kth GPU in the lth GPU block during the tth round of training after H local iterative updates. represents the parameters of the deep learning training model of the kth GPU in the lth GPU block before the local iterative update in the tth round of training, γ represents the learning rate, and B loc represents the number of the first movie data samples required in one local iteration, i represents the first movie data sample, I represents the set of first movie data samples required by the kth GPU in the first GPU block during the h-1th local iteration during the tth round of training, and f i represents the loss function of the i-th sample of the first movie data.

[0031] In some embodiments of the present invention, the expression of the block synchronization parameter of the deep learning training model in each GPU block includes:

[0032]

[0033] in, Indicates the block synchronization parameters of the deep learning training model in the lth GPU block during the first block synchronization process in the tth round of training. K represents the parameters of the deep learning training model in the lth GPU block before block synchronization during the tth round of training. l Indicates the total number of GPUs in the lth GPU block, It represents the local updated parameters of the deep learning training model of the k-th GPU in the l-th GPU block during the t-th round of training after H local iterative updates.

[0034] In some embodiments of the present invention, the expression of the global synchronization parameter of the deep learning training model in all GPU blocks includes:

[0035]

[0036] in, Represents the global synchronization parameters of the deep learning training model in all GPU blocks during the t+1 round of training. represents the global synchronization parameter of the deep learning training model in all GPU blocks during the t-th round of training. K is the total number of GPUs in all GPU blocks. Indicates the deep learning training model of each GPU block in all GPU blocks during the t-th round of training. b Block synchronization parameters during the secondary block synchronization process.

[0037] In some embodiments of the present invention, the device also includes: a second acquisition module, used to obtain a second film data sample, the second film data sample is different from the first film data sample, and the second film data sample includes historical film data and new film data; a category label adding module, using the historical film data to train the global synchronized deep learning training model to obtain the category sequence of the historical film data, and using the new film data to train the global synchronized deep learning training model to obtain the category sequence of the new film data; a third acquisition module, used to obtain the loss function of this training based on the category sequence of the historical film data and the category sequence of the new film data; a loop module, used to loop the above steps to obtain a prediction model, which includes historical category labels and newly added category labels.

[0038] In some embodiments of the present invention, the expression of the category sequence of the historical film data includes:

[0039] q d =o d 1≤d≤n

[0040] Among them, q drepresents the category sequence of the historical film data, d represents the category label in the category sequence of the historical film data, and n represents the total number of category labels in the deep learning training model after global synchronization.

[0041] In some embodiments of the present invention, the expression of the category sequence of the new movie data includes:

[0042] q d =αo d +β n+1≤d≤n+m

[0043] Among them, q d represents the category sequence of the new movie data, d represents the category label in the category sequence of the new movie data, α and β represent the bias parameters in the deep learning training model after global synchronization, n represents the total number of category labels in the deep learning training model after global synchronization, and n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new movie data.

[0044] In some embodiments of the present invention, the formula for calculating the loss function is:

[0045]

[0046] Among them, n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new video data, and q d Represents the category sequence of the historical video data and / or the category sequence of the new video data.

[0047] In some embodiments of the present invention, the device also includes: a storage module for storing the historical film data and / or the new film data in a distributed search engine; a receiving module for receiving film keywords input by a user; a matching module for calling an interface corresponding to the film keyword from the distributed search engine based on the film keyword, and using the interface to call film data that matches the film keyword; and a sending module for sending film data that matches the film keyword to a terminal.

[0048] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the film data processing method as described in the first aspect of the above embodiment.

[0049] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the film data processing method as described in the first aspect of the above embodiment is implemented.

[0050] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0051] In the technical solutions provided by some embodiments of the present invention, a hierarchical local stochastic gradient descent method is adopted to obtain the global synchronization parameters of the deep learning training model in all GPU blocks. Training the model in this way can improve communication efficiency and meet the requirements of computing efficiency during actual operation, effectively avoiding the defect that the communication efficiency of each device is constrained during parallel operation.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0054] Figure 1 A schematic diagram showing an exemplary system architecture to which a film data processing method or a film data processing apparatus according to an embodiment of the present invention may be applied;

[0055] Figure 2 A flowchart of a method for processing film data according to an embodiment of the present invention is schematically shown;

[0056] Figure 3 A flowchart schematically shows a method for processing film data according to another embodiment of the present invention;

[0057] Figure 4 A flowchart schematically shows a method for processing film data according to another embodiment of the present invention;

[0058] Figure 5 A schematic diagram schematically illustrates an iterative update parameter within a GPU according to an embodiment of the present invention;

[0059] Figure 6 A schematic diagram illustrating updating parameters using a hierarchical stochastic gradient descent method according to an embodiment of the present invention is shown;

[0060] Figure 7 A block diagram schematically shows a method for processing film data according to an embodiment of the present invention;

[0061] Figure 8Schematically shows a block diagram of a film data processing device according to another embodiment of the present invention;

[0062] Figure 9 Schematically shows a block diagram of a film data processing device according to another embodiment of the present invention;

[0063] Figure 10 A schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0064] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0065] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, it will be appreciated by those skilled in the art that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0066] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0067] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0068] Figure 1A schematic diagram of an exemplary system architecture to which the film data processing method or film data processing device according to an embodiment of the present invention can be applied is shown. In some embodiments of the present invention, the entire system architecture mainly includes three parts: a data preparation part, a distributed training part, and an application part. The data preparation part includes crawling film data, preprocessing the data, feature engineering, and data labeling; the distributed training part mainly includes computing nodes for distributed training of film data, parameter service nodes, a deep learning training model, a hierarchical local SGD optimization algorithm and an online learning algorithm for incremental learning used in the deep learning training model, and Elasticsearch for storing and retrieving film data; the application part mainly includes a front-end page for detecting and testing film data, as well as a recommendation service for related films.

[0069] like Figure 1 As shown, system architecture 100 may include application 110, film system 120, and data processing tool 130. Application 110 may be an application associated with film system 120. For example, a user may enter a film keyword on the front-end page of a terminal device (e.g., the user interface of application 110). Film system 120 may receive the film keyword entered by the user and, based on the film keyword, call an interface corresponding to the film keyword from the distributed search engine of film system 120. Using the interface, film data matching the film keyword may be called up, and the film data matching the film keyword may be sent to the terminal so that the film data matching the film keyword can be displayed on the user interface of application 110 of the terminal. Furthermore, in this embodiment, film system 120 may also recommend services to application 110 of the terminal, such as a new film promotion service within film system 120.

[0070] The film system 120 includes a deep learning training model, which is deployed on multiple computing nodes (e.g., GPUs) and uses a hierarchical local stochastic gradient descent (i.e., hierarchical local SGD) approach to obtain global synchronization parameters of the deep learning training model. Specifically, the terminal data or server data is processed by the data processing tool 130. For example, the terminal data or server data is obtained by using a crawler tool, and the data is preprocessed and annotated to obtain a first film data sample. The first film data sample includes historical film data. Using the first film data sample, the parameters in the deep learning training model of each computing node in each computing node block are iteratively updated to obtain the local update parameters of the deep learning training model of each computing node in each computing node block, and based on the local update parameters of the deep learning training model of each computing node in each computing node block, the block synchronization parameters of the deep learning training model in each computing node block are determined. Then, based on the block synchronization parameters of the deep learning training model in each computing node block, the global synchronization parameters of the deep learning training models in all computing node blocks are determined. In this way, the global synchronization parameters of the deep learning training model are obtained by adopting a hierarchical local SGD method. Training the model in this way can improve communication efficiency and meet the requirements of computing efficiency during actual operation, effectively avoiding the defect that the communication efficiency of each device is constrained during parallel operation. In this embodiment, the parameter service node can be used to calculate and obtain the global synchronization parameters of the deep learning training model.

[0071] In some embodiments of the present invention, after obtaining the global synchronization parameters of the deep learning training model using a hierarchical local SGD method, an incremental learning method can be used to obtain new category labels in the deep learning training model. For example, a second film data sample is obtained, the second film data sample is different from the first film data sample, and the second film data sample contains historical film data and new film data. The deep learning training model after global synchronization is trained using the historical film data to obtain the category sequence of the historical film data, and the deep learning training model after global synchronization is trained using the new film data to obtain the category sequence of the new film data. According to the category sequence of the historical film data and the category sequence of the new film data, the loss function of this training is obtained, and the above steps are repeated to obtain a prediction model, which contains historical category labels and newly added category labels.

[0072] In some embodiments of the present invention, the historical video data and / or new video data are stored in a distributed search engine (eg, Elastic search).

[0073] Figure 2A flowchart of a method for processing film data according to an embodiment of the present invention is schematically shown. In this embodiment, the method for processing film data is applied to a film system, wherein the film system includes a deep learning training model deployed on a GPU block of each of multiple servers.

[0074] The above-mentioned film data processing method may include using a hierarchical local stochastic gradient descent method to obtain the global synchronization parameters of the deep learning training model in all GPU blocks. Wherein, using a hierarchical local stochastic gradient descent method to obtain the global synchronization parameters of the deep learning training model in all GPU blocks specifically includes steps S210 to S240, such as Figure 2 shown.

[0075] In step S210 , a first film data sample is obtained, where the first film data sample includes historical film data.

[0076] In step S220, the first film data sample is used to iteratively update the parameters in the deep learning training model of each GPU in each GPU block to obtain local updated parameters of the deep learning training model of each GPU in each GPU block.

[0077] In step S230 , block synchronization parameters of the deep learning training model in each GPU block are determined based on the local update parameters of the deep learning training model of each GPU in each GPU block.

[0078] In step S240 , global synchronization parameters of the deep learning training models in all GPU blocks are determined based on the block synchronization parameters of the deep learning training model in each GPU block.

[0079] This method can use hierarchical local stochastic gradient descent to obtain the global synchronization parameters of the deep learning training model in all GPU blocks. Training the model in this way can improve communication efficiency and meet the requirements of computing efficiency in actual operation, effectively avoiding the defect of constrained communication efficiency of each device during parallel operation.

[0080] In one embodiment of the present invention, the film system includes a deep learning training model, which is deployed on a GPU block within each of multiple servers. For example, a large number of GPUs are grouped across the servers, each group being referred to as a GPU block. A GPU block includes at least two GPUs, and the deep learning training model is distributed across each GPU.

[0081] In one embodiment of the present invention, the first film data sample includes historical film data. For example, the historical film data may include the name, category, introduction, etc. of the historical film, but is not limited thereto.

[0082] In one embodiment of the present invention, by Figure 1 The data processing tool 130 shown in the figure processes the film data to obtain a first film data sample. For example, before obtaining the first film data sample, the method may further include: using web crawler technology to crawl film data from various relevant film and television websites, using a data preprocessing method to preliminarily clean the crawled film data, and then using a labeling tool to label the processed film data to obtain the first film data sample, so that the results obtained when the first film data sample is subsequently used to train the model are more accurate.

[0083] In one embodiment of the present invention, a hierarchical local stochastic gradient descent training method is adopted to update the local model on each GPU (i.e., the above-mentioned deep learning training model). Furthermore, after the local update, a fast synchronization will be performed inside the GPU block. After the block is updated, all GPU blocks will be globally synchronized, thereby completing a global parameter update. For example, a hierarchical local stochastic gradient descent method is adopted to obtain the global synchronization parameters of the deep learning training model in all GPU blocks. Specifically, the parameters of the deep learning training model in the GPU are iteratively updated, and then the internal GPU block is synchronously updated based on the iteratively updated parameters. Finally, the parameters of the synchronously updated parameters within all GPU blocks are synchronously updated based on the synchronously updated parameters within all GPU blocks. In this way, the hierarchical local update of the parameters in the deep learning training model is achieved.

[0084] In one embodiment of the present invention, the expression (1) for the local update parameters of the deep learning training model of each GPU in each GPU block includes:

[0085]

[0086] in, It represents the local updated parameters of the deep learning training model of the kth GPU in the lth GPU block during the tth round of training after H local iterative updates. represents the parameters of the deep learning training model of the kth GPU in the lth GPU block before the local iterative update in the tth round of training, γ represents the learning rate, and B loc represents the number of the first movie data samples required in one local iteration, i represents the first movie data sample, represents the set of first movie data samples required by the kth GPU in the first GPU block during the h-1th local iteration of the tth round of training, f i represents the loss function of the i-th sample of the first movie data.

[0087] refer to Figure 5 , node block 1 can be the above-mentioned GPU block, and the node block 1 includes computing node 1...·computing node c, and the computing node can be the above-mentioned GPU. A deep learning training model is deployed on each computing node in node block 1. In this embodiment, a fixed number of first movie data samples are obtained from the set of first movie data samples, and the parameters of the deep learning training model on each computing node are iteratively updated multiple times. For example, for the deep learning training model on computing node 1, the result of the first iterative update is w1. After multiple iterative updates, the result of the last iterative update is g1. For the deep learning training model on computing node c, the result of the first iterative update is w c After multiple iterations, the result of the last iteration is g c In this embodiment, the result of the last iteration update of each computing node is output to the parameter service node d, and the parameter service node d is used to update the last iteration update of each computing node (i.e. g1...g c ), use the above expression (1) to perform synchronization updates inside node block 1 to obtain the block synchronization parameter w of the deep learning training model.

[0088] In one embodiment of the present invention, the expression (2) for the block synchronization parameter of the deep learning training model in each GPU block includes:

[0089]

[0090] in, Indicates the block synchronization parameters of the deep learning training model in the lth GPU block during the first block synchronization process in the tth round of training. K represents the parameters of the deep learning training model in the lth GPU block before block synchronization during the tth round of training. l Indicates the total number of GPUs in the lth GPU block, It represents the local updated parameters of the deep learning training model of the k-th GPU in the l-th GPU block during the t-th round of training after H local iterative updates.

[0091] In one embodiment of the present invention, the expression (3) for the global synchronization parameters of the deep learning training model in all GPU blocks includes:

[0092]

[0093] in, Represents the global synchronization parameters of the deep learning training model in all GPU blocks during the t+1 round of training. represents the global synchronization parameter of the deep learning training model in all GPU blocks during the t-th round of training. K is the total number of GPUs in all GPU blocks. Indicates the deep learning training model of each GPU block in all GPU blocks during the t-th round of training. b Block synchronization parameters during the secondary block synchronization process.

[0094] refer to Figure 6 , D1 and D2 represent GPU block 1 and GPU block 2 respectively. GPU block 1 contains two GPUs, namely GPU1 and GPU2, and GPU block 2 contains two GPUs, namely GPU3 and GPU4. Deep learning training models are deployed in GPU1, GPU2, GPU3, and GPU4 respectively. In this embodiment, first movie data samples of the same batch size are extracted from the set of first movie data samples, and the first movie data samples of the same batch size are input into the deep learning training model in each GPU. The parameters of the deep learning training model in each GPU are iteratively updated multiple times using multiple first movie data samples of the same batch size. For example, Figure 6 As shown, for the deep learning training model in each GPU, the batch size is 2 (i.e., the B loc =2, such as Figure 6 The number of first movie data samples required in one local iteration is 2. The parameters of the deep learning training model are locally iterated using two first movie data samples to obtain the parameter Δω). The number of iterative updates is 3 (i.e., H=3 above). That is, one iterative update requires two first movie data samples, and the number of iterative updates in the entire local iterative update process is 3. The number of synchronizations within the GPU block is 2 (i.e., H b =2). The above expression (1) can be used to obtain the local update parameter ω′ of the deep learning training model in each GPU after three local iterative updates. In addition, Figure 6 The Δω shown in the above expression (1) represents That is, the local update parameters of the deep learning training model of the kth GPU in the lth GPU block during the tth round of training are obtained after h-1 local iterative updates. For D1 and D2, the block synchronization parameter ω1 of the deep learning training model can be obtained through the above expression (2). That is, the block synchronization parameter ω1 of the deep learning training model is calculated using the local update parameter ω′ of the deep learning training model in GPU1 and GPU2 after three local iterative updates, and the block synchronization parameter ω2 of the deep learning training model is calculated using the local update parameter ω′ of the deep learning training model in GPU3 and GPU4 after three local iterative updates. For D1 and D2, the global synchronization parameter ω3 of the deep learning training model can be obtained through the above expression (3), that is, the block synchronization parameter ω1 of the deep learning training model is calculated using the local update parameter ω′ of the deep learning training model in GPU1 and GPU2 after three local iterative updates, and the block synchronization parameter ω2 of the deep learning training model is calculated using the local update parameter ω′ of the deep learning training model in GPU3 and GPU4 after three local iterative updates, and the global synchronization parameter ω3 of the deep learning training model is calculated.

[0095] Figure 6 The figure shows the process of obtaining the global synchronization parameter ω3 of the deep learning training model. In actual situations, the number of global synchronizations and the batch size B in the local iteration process can be set according to business needs. loc , the number of local iterations H, the number of internal synchronizations of the block H b etc.

[0096] In one embodiment of the present invention, accurate model parameters, namely, the global synchronization parameter ω3 of the deep learning training model, are obtained by looping steps S210 to S240.

[0097] Figure 3 The following schematically shows a flow chart of a method for processing film data according to another embodiment of the present invention.

[0098] like Figure 3 As shown, the above method may further include steps S310 to S340.

[0099] In step S310 , a second film data sample is obtained. The second film data sample is different from the first film data sample, and the second film data sample includes historical film data and new film data.

[0100] In step S320, the globally synchronized deep learning training model is trained using the historical film data to obtain a category sequence of the historical film data, and the globally synchronized deep learning training model is trained using the new film data to obtain a category sequence of the new film data.

[0101] In step S330 , a loss function for this training is obtained according to the category sequence of the historical film data and the category sequence of the new film data.

[0102] In step S340, steps S310 to S330 are looped to obtain a prediction model, wherein the prediction model includes historical category labels and newly added category labels.

[0103] This method can add new category labels based on new movie data, effectively solving the classification preference problem that exists in related technologies when classifying movie data.

[0104] In one embodiment of the present invention, incremental learning technology is used to eliminate the classification bias problem that occurs when fitting the fully connected layer in the deep learning training model when training newly input samples (for example, new movie data input to the global synchronized deep learning training model). For example, when training the deep learning training model based on new movie data, the bias parameters α and β in the deep learning training model can be used to add new category labels. Specifically, the new movie data and historical movie data are input into the deep learning training model as a validation set to train the deep learning training model, and the training results are output as follows:

[0105]

[0106] Among them, the expression of the category sequence of historical film data is q d =o d 1≤d≤n. qd represents the category sequence of the historical video data, d represents the category labels in the category sequence of the historical video data, and n represents the total number of category labels in the deep learning training model after global synchronization. In this example, the category labels in the category sequence of the historical video data are one or more of the n category labels.

[0107] The expression of the category sequence of the new movie data is q d =αo d+β n+1≤d≤n+m. qd represents the category sequence of the new video data, d represents the category labels in the category sequence of the new video data, α and β represent the bias parameters in the deep learning training model after global synchronization, n represents the total number of category labels in the deep learning training model after global synchronization, and n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new video data. In this example, the category labels in the category sequence of the new video data are one or more of the n+m category labels.

[0108] In one embodiment of the present invention, when the globally synchronized deep learning training model is trained using the second film data sample, the parameters of the fully connected classifier and the model layer in the globally synchronized deep learning training model need to be frozen to prevent them from being updated again. When the globally synchronized deep learning training model is trained using the validation set, a loss function may be used. For example, the loss function is cross entropy, and the formula for calculating the loss function is:

[0109]

[0110] Among them, n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new video data, and q d represents the category sequence of the historical video data and / or the category sequence of the new video data. d is the result predicted by the global synchronized deep learning training model, and y = d is the actual result manually annotated. The loss function is the cross entropy derived from the maximum likelihood estimate.

[0111] In one embodiment of the present invention, by looping steps S310 to S330, a prediction model can be obtained, wherein the prediction model includes historical category labels and newly added category labels. For example, by looping steps S310 to S330, a prediction model is obtained, and after obtaining the prediction model, the loss function L is determined. b Whether it has converged. If it has converged, no more movie data will be obtained to train the model. If it has not converged, continue to obtain movie data to train the model.

[0112] In one embodiment of the present invention, b After convergence, when the prediction model is used to classify new movie data, new category labels related to the new movie data can be added through the bias parameters in the prediction model to make the classification results more accurate.

[0113] Figure 4 The following schematically shows a flow chart of a method for processing film data according to another embodiment of the present invention.

[0114] like Figure 4 As shown, the above step S230 may specifically include steps S410 to S440.

[0115] In step S410, the historical video data and / or the new video data are stored in a distributed search engine.

[0116] In step S420 , a movie keyword input by a user is received.

[0117] In step S430, based on the movie keyword, an interface corresponding to the movie keyword is called from the distributed search engine, and movie data matching the movie keyword is called using the interface.

[0118] In step S440, the movie data matching the movie keyword is sent to the terminal.

[0119] The method can retrieve film data matching the film keywords input by the user, and send the film data matching the film keywords to the terminal, so that the user can browse the film data on the application interface of the terminal.

[0120] In one embodiment of the present invention, an HTML5 page can be created for a preset model, and different API interfaces can be called for movie query and search according to output needs.

[0121] In one embodiment of the present invention, a recommendation service page can be created for a preset model, and different API interfaces can be called to provide a movie recommendation service based on output needs.

[0122] For current large-scale film data, the above-mentioned film data processing method, through the use of a hierarchical extended training framework in the deep learning distributed data parallelization process, can further improve the adaptability of local stochastic gradient descent to heterogeneous distribution systems in practical applications. While ensuring accuracy, the hierarchical SGD improves communication efficiency. By using the bias parameter introduced after full connection to minimize classification bias, this incremental learning method improves the online learning capability of the deep learning distributed system for film data. Therefore, by introducing new optimization methods and training methods into the film deep learning distributed system, the present invention improves the accuracy, communication efficiency, and online learning capability of the entire distributed film system device, solves the time-consuming problem of model training, and saves labor costs.

[0123] Figure 7A block diagram of a film data processing device according to an embodiment of the present invention is schematically shown. In this embodiment, the film data processing device 700 is applied to a film system, wherein the film system includes a deep learning training model deployed on a GPU block of each of multiple servers.

[0124] like Figure 7 As shown, the video data processing device 700 includes a first acquisition module 702 , a local iterative update module 704 , a block synchronization module 706 and a global synchronization module 708 .

[0125] Specifically, the first acquisition module 702 is configured to acquire a first film data sample, where the first film data sample includes historical film data.

[0126] The local iterative update module 704 uses the first movie data sample to iteratively update the parameters in the deep learning training model of each GPU in each GPU block to obtain the local updated parameters of the deep learning training model of each GPU in each GPU block.

[0127] The block synchronization module 706 is used to determine the block synchronization parameters of the deep learning training model in each GPU block based on the local update parameters of the deep learning training model of each GPU in each GPU block.

[0128] The global synchronization module 708 is used to determine the global synchronization parameters of the deep learning training models in all GPU blocks based on the block synchronization parameters of the deep learning training models in each GPU block.

[0129] The film data processing device 700 can adopt a hierarchical local stochastic gradient descent method to obtain the global synchronization parameters of the deep learning training model in all GPU blocks. Training the model in this way can improve communication efficiency and meet the requirements of computing efficiency during actual operation, effectively avoiding the defect of constrained communication efficiency of each device during parallel operation.

[0130] According to an embodiment of the present invention, the film data processing device 700 can be used to implement Figure 2 The embodiment describes a method for processing film data.

[0131] Figure 8 The figure schematically shows a block diagram of a film data processing device according to another embodiment of the present invention.

[0132] like Figure 8 As shown, the above-mentioned film data processing device 700 may further include a second acquisition module 710 , a category label adding module 712 , a third acquisition module 714 and a loop module 716 .

[0133] Specifically, the second acquisition module 710 is configured to acquire a second film data sample, where the second film data sample is different from the first film data sample, and the second film data sample includes historical film data and new film data.

[0134] The category label adding module 712 uses the historical film data to train the globally synchronized deep learning training model to obtain the category sequence of the historical film data, and uses the new film data to train the globally synchronized deep learning training model to obtain the category sequence of the new film data.

[0135] The third acquisition module 714 is used to acquire the loss function of this training according to the category sequence of the historical video data and the category sequence of the new video data.

[0136] The loop module 716 is used to loop the above steps to obtain a prediction model, which includes historical category labels and newly added category labels.

[0137] The film data processing device 700 can add new category labels based on new film data, effectively solving the classification preference problem existing in the related art when classifying film data.

[0138] According to an embodiment of the present invention, the film data processing device 700 can be used to implement Figure 3 The embodiment describes a method for processing film data.

[0139] Figure 9 The figure schematically shows a block diagram of a film data processing device according to another embodiment of the present invention.

[0140] like Figure 9 As shown, the above-mentioned film data processing device 700 may further include a storage module 718 , a receiving module 720 , a matching module 722 and a sending module 724 .

[0141] Specifically, the storage module 718 is configured to store the historical video data and / or the new video data in a distributed search engine.

[0142] The receiving module 720 is configured to receive movie keywords input by a user.

[0143] The matching module 722 calls an interface corresponding to the movie keyword from the distributed search engine based on the movie keyword, and uses the interface to call movie data that matches the movie keyword.

[0144] The sending module 724 is configured to send the movie data matching the movie keyword to the terminal.

[0145] The film data processing device 700 can retrieve film data matching the film keywords input by the user, and send the film data matching the film keywords to the terminal so that the user can browse the film data on the application interface of the terminal.

[0146] In the embodiment of the present invention, the film data processing device 700 can be used to implement Figure 4 The embodiment describes a method for processing film data.

[0147] Since the modules of the film data processing apparatus 700 according to the exemplary embodiment of the present invention can be used to implement the above 2. Figure 4 The steps of the exemplary embodiment of the film data processing method are described above. Therefore, for details not disclosed in the embodiment of the device of the present invention, please refer to the above-mentioned embodiment of the film data processing method of the present invention.

[0148] It is understood that the first acquisition module 702, the local iterative update module 704, the block synchronization module 706, the global synchronization module 708, the second acquisition module 710, the category label addition module 712, the third acquisition module 714, the loop module 716, the storage module 718, the receiving module 720, the matching module 722, and the sending module 724 can be implemented in a single module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. According to an embodiment of the present invention, at least one of the first acquisition module 702, the local iterative update module 704, the block synchronization module 706, the global synchronization module 708, the second acquisition module 710, the category label addition module 712, the third acquisition module 714, the loop module 716, the storage module 718, the receiving module 720, the matching module 722, and the sending module 724 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware in any other reasonable way of integrating or packaging circuits, or in an appropriate combination of software, hardware, and firmware. Alternatively, at least one of the first acquisition module 702, the local iterative update module 704, the block synchronization module 706, the global synchronization module 708, the second acquisition module 710, the category label addition module 712, the third acquisition module 714, the loop module 716, the storage module 718, the receiving module 720, the matching module 722, and the sending module 724 can be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module can be executed.

[0149] Reference below Figure 10 , which shows a schematic structural diagram of a computer system 1000 of an electronic device suitable for implementing an embodiment of the present invention. Figure 10 The computer system 1000 of the electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0150] like Figure 10 As shown, computer system 1000 includes a central processing unit (GPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for system operation are also stored in RAM 1003. GPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004.

[0151] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN card or a modem. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1010 as needed, so that computer programs read therefrom can be installed into the storage section 1008 as needed.

[0152] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1009, and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit (GPU) 1001, the above-mentioned functions defined in the system of the present application are performed.

[0153] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0155] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not limit the units themselves.

[0156] As another aspect, the present application further provides a computer-readable medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements the film data processing method described in the above embodiments.

[0157] For example, the electronic device can implement Figure 2 As shown in: In step S210, a first movie data sample is obtained, and the first movie data sample includes historical movie data. In step S220, the parameters in the deep learning training model of each GPU in each GPU block are iteratively updated using the first movie data sample to obtain local update parameters of the deep learning training model of each GPU in each GPU block. In step S230, the block synchronization parameters of the deep learning training model in each GPU block are determined based on the local update parameters of the deep learning training model of each GPU in each GPU block. In step S240, the global synchronization parameters of the deep learning training models in all GPU blocks are determined based on the block synchronization parameters of the deep learning training model in each GPU block.

[0158] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0159] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present invention.

[0160] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0161] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for processing video data, applied to a video system, wherein the video system includes a deep learning training model deployed on a GPU block of each of a plurality of servers, the method comprising: Using a hierarchical local stochastic gradient descent approach, global synchronization parameters of the deep learning training model in all GPU blocks are obtained for use in training the deep learning training model; The method of using hierarchical local stochastic gradient descent to obtain the global synchronization parameters of the deep learning training model in all GPU blocks includes: Acquire a first film data sample, where the first film data sample includes historical film data; Iteratively updating parameters in the deep learning training model of each GPU in each GPU block using the first movie data sample to obtain local updated parameters of the deep learning training model of each GPU in each GPU block; Determine block synchronization parameters for the deep learning training model in each GPU block based on the local update parameters of the deep learning training model of each GPU in each GPU block; According to the block synchronization parameters of the deep learning training model in each GPU block, the global synchronization parameters of the deep learning training model in all GPU blocks are determined.

2. The method according to claim 1, characterized in that The expressions for the local update parameters of the deep learning training model for each GPU in each GPU block include: in, It represents the local updated parameters of the deep learning training model of the kth GPU in the lth GPU block during the tth round of training after H local iterative updates. represents the parameters of the deep learning training model of the kth GPU in the lth GPU block before the local iterative update in the tth round of training, γ represents the learning rate, and B loc represents the number of the first movie data samples required in one local iteration, i represents the first movie data sample, represents the set of first movie data samples required by the kth GPU in the lth GPU block during the h-1th local iteration during the tth round of training, f i represents the loss function of the i-th sample of the first movie data, It represents the local updated parameters of the deep learning training model of the k-th GPU in the l-th GPU block during the t-th round of training after (h-1) local iterative updates.

3. The method according to claim 1, characterized in that The expressions for the block synchronization parameters of the deep learning training model in each GPU block include: in, Indicates the block synchronization parameters of the deep learning training model in the lth GPU block during the first block synchronization process in the tth round of training. K represents the parameters of the deep learning training model in the lth GPU block before block synchronization during the tth round of training. l Indicates the total number of GPUs in the lth GPU block, It represents the local updated parameters of the deep learning training model of the k-th GPU in the l-th GPU block during the t-th round of training after H local iterative updates.

4. The method according to claim 1, wherein The expressions for the global synchronization parameters of the deep learning training model in all GPU blocks include: in, Represents the global synchronization parameters of the deep learning training model in all GPU blocks during the t+1 round of training. represents the global synchronization parameter of the deep learning training model in all GPU blocks during the t-th round of training. K is the total number of GPUs in all GPU blocks. Indicates the deep learning training model of each GPU block in all GPU blocks during the t-th round of training. b Block synchronization parameters during the secondary block synchronization process.

5. The method according to claim 1, characterized in that The method further includes: Acquire a second video data sample, where the second video data sample is different from the first video data sample, and the second video data sample includes historical video data and new video data; Using the historical film data, training the globally synchronized deep learning training model to obtain a category sequence of the historical film data; and using the new film data, training the globally synchronized deep learning training model to obtain a category sequence of the new film data; Obtaining a loss function for this training according to the category sequence of the historical film data and the category sequence of the new film data; The above steps are repeated to obtain a prediction model, which includes historical category labels and newly added category labels.

6. The method according to claim 5, characterized in that The expression of the category sequence of the historical film data includes: q d =o d 1≤d≤n Among them, q d represents the category sequence of the historical film data, d represents the category label in the category sequence of the historical film data, and n represents the total number of category labels in the deep learning training model after global synchronization.

7. The method according to claim 5, characterized in that The expression of the category sequence of the new movie data includes: what d =αo d +β n+1≤d≤n+m Among them, q d represents the category sequence of the new movie data, d represents the category label in the category sequence of the new movie data, α and β represent the bias parameters in the deep learning training model after global synchronization, n represents the total number of category labels in the deep learning training model after global synchronization, and n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new movie data.

8. The method according to claim 5, characterized in that Calculate the loss function L b The formula is: Among them, n+m represents the total number of categories after the deep learning training model after global synchronization adds categories based on the new video data, and q d Represents the category sequence of the historical video data and / or the category sequence of the new video data.

9. The method according to claim 1, characterized in that The method further includes: storing the historical video data and / or the new video data in a distributed search engine; Receive movie keywords input by users; Based on the movie keywords, calling an interface corresponding to the movie keywords from the distributed search engine, and using the interface to call movie data matching the movie keywords; Sending the movie data matching the movie keyword to the terminal.

10. The method according to claim 5, characterized in that The method further includes: storing the historical video data and / or the new video data in a distributed search engine; Receive movie keywords input by users; Based on the movie keywords, calling an interface corresponding to the movie keywords from the distributed search engine, and using the interface to call movie data matching the movie keywords; Sending the movie data matching the movie keyword to the terminal.

11. A film data processing device, applied to a film system, wherein the film system includes a deep learning training model, wherein the deep learning training model is deployed on a GPU block of each of a plurality of servers, the device comprising: A first acquisition module is configured to acquire a first film data sample, wherein the first film data sample includes historical film data; a local iterative update module, which uses the first movie data sample to iteratively update parameters in the deep learning training model of each GPU in each GPU block to obtain local updated parameters of the deep learning training model of each GPU in each GPU block; A block synchronization module for determining block synchronization parameters of the deep learning training model in each GPU block based on the local update parameters of the deep learning training model of each GPU in each GPU block; A global synchronization module is used to determine the global synchronization parameters of the deep learning training models in all GPU blocks based on the block synchronization parameters of the deep learning training models in each GPU block, so as to be used for training the deep learning training models.

12. An electronic device comprising: one or more processors; as well as A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.

13. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images