Method and electronic device for initializing algorithm parameters

Abstract

The present invention discloses a method and electronic device for initializing algorithm parameters, which are used to initialize the algorithm parameters of editing software, thereby saving labor costs. The method includes: obtaining a video frame and an editing type corresponding to the video frame, determining a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model; inputting the video frame into the parameter prediction model, and outputting an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample set corresponding to the same editing type includes sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames; and using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set to edit the video frame.

Description

Technical Field

[0001] The present invention relates to the technical field of image / video editing software, and in particular to a method for initializing algorithm parameters and electronic equipment. Background Art

[0002] For image / video editing software, the parameters of each algorithm module (noise reduction, brightness, contrast, etc.) are all factory settings. Users need to adjust the parameters of various algorithms from scratch every time they use the software, which consumes a lot of manpower costs.

[0003] Furthermore, when processing videos, even if the algorithm parameters for the same scene can be directly copied, once the scene switches to a completely different one, the user still needs to adjust the algorithm parameters from scratch, thus requiring a lot of manpower to adjust the image. Summary of the Invention

[0004] The present invention provides a method and electronic device for initializing algorithm parameters, which are used to automatically initialize the algorithm parameters of editing software and initialize different algorithm parameters for input video frames of different editing types and different scenes, which can greatly save labor costs.

[0005] In a first aspect, an embodiment of the present invention provides a method for initializing algorithm parameters, the method comprising:

[0006] Obtaining a video frame and an editing type corresponding to the video frame, and determining a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0007] Inputting the video frame into the parameter prediction model and outputting an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0008] The video frame is edited using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0009] The method for initializing the algorithm parameters provided in this embodiment uses a deep learning model to predict the algorithm parameters of each algorithm module to achieve the adjustment effect of the input video frame, and uses the predicted algorithm parameters as the initialization algorithm parameters of each algorithm module. Even when the scene switches, the model can automatically predict the algorithm parameters based on the current input video frame to achieve dynamic algorithm parameter initialization. Through dynamic algorithm parameter initialization, it can be achieved that most scenes can use automatically initialized algorithm parameters to present usable effects, and a small number of scenes only need to fine-tune the initialized algorithm parameters, which can greatly save labor costs.

[0010] As an optional implementation manner, the acquired video frame includes multiple frames, and the method further includes:

[0011] Grouping the video frames containing the same type of scenes in the plurality of video frames into a video frame group;

[0012] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N ≥ 1, where N is an integer;

[0013] An algorithm parameter set corresponding to the video frame group is determined according to the N prediction algorithm parameter sets.

[0014] As an optional implementation manner, determining the algorithm parameter set corresponding to the video frame group according to the N prediction algorithm parameter sets includes:

[0015] For each prediction algorithm parameter in the N prediction algorithm parameter sets, using the weights corresponding to the N prediction algorithm parameter sets, performing weighted summation on the prediction algorithm parameters to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0016] An algorithm parameter set corresponding to the video frame group is determined according to algorithm parameters corresponding to various prediction algorithm parameters.

[0017] As an optional implementation, the N prediction algorithm parameter sets each have the same corresponding weight.

[0018] As an optional implementation, the sample set is obtained in the following manner:

[0019] determining a type of editing performed by a user on a sample video frame using editing software;

[0020] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0021] The sample set is determined according to the editing type of the sample video frame, the type of the scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0022] As an optional implementation manner, determining the editing type of the sample video frame edited by the user using editing software includes:

[0023] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0024] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0025] As an optional implementation, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; the order of the multiple algorithm parameters is determined by:

[0026] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0027] The order of the algorithm parameters corresponding to the algorithm modules is determined according to the usage frequency of the algorithm modules corresponding to each sample video frame in the sample set.

[0028] As an optional implementation manner, determining the order of algorithm parameters corresponding to each algorithm module according to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set includes:

[0029] Counting the algorithm modules used in each operation step of the sample video frame;

[0030] In each operation step, determining the algorithm module most frequently used in the operation step according to the algorithm modules used in each sample video frame;

[0031] The order of the algorithm parameters corresponding to each algorithm module is determined based on the algorithm module most frequently used in each operation step.

[0032] As an optional implementation manner, determining the order of algorithm parameters corresponding to each algorithm module according to the algorithm module most frequently used in each operation step includes:

[0033] Determine a first algorithm module according to the algorithm module most frequently used in the first operation step for each sample video frame;

[0034] For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used by each sample video frame in the current operation step;

[0035] According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

[0036] As an optional implementation, the parameter prediction model is trained in the following manner:

[0037] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function based on an output result and a sample parameter set corresponding to the sample video frame;

[0038] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0039] As an optional implementation,

[0040] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0041] As an optional implementation, the method further includes:

[0042] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0043] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and each algorithm parameter corresponds to one algorithm module; the method further includes:

[0044] The multiple algorithm parameters are displayed in sequence;

[0045] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0046] As an optional implementation, the method further includes:

[0047] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0048] In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

[0049] As an optional implementation, the algorithm parameters and algorithm modules are displayed in sequence; further comprising:

[0050] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0051] In a second aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0052] Obtaining a video frame and an editing type corresponding to the video frame, and determining a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0053] Inputting the video frame into the parameter prediction model and outputting an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0054] The video frame is edited using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0055] As an optional implementation manner, the acquired video frame includes multiple frames, and the processor is further configured to execute:

[0056] Grouping the video frames containing the same type of scenes in the plurality of video frames into a video frame group;

[0057] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N ≥ 1, where N is an integer;

[0058] An algorithm parameter set corresponding to the video frame group is determined according to the N prediction algorithm parameter sets.

[0059] As an optional implementation, the processor is specifically configured to execute:

[0060] For each prediction algorithm parameter in the N prediction algorithm parameter sets, using the weights corresponding to the N prediction algorithm parameter sets, performing weighted summation on the prediction algorithm parameters to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0061] An algorithm parameter set corresponding to the video frame group is determined according to algorithm parameters corresponding to various prediction algorithm parameters.

[0062] As an optional implementation, the N prediction algorithm parameter sets each have the same corresponding weight.

[0063] As an optional implementation manner, the processor is specifically configured to obtain the sample set in the following manner:

[0064] determining a type of editing performed by a user on a sample video frame using editing software;

[0065] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0066] The sample set is determined according to the editing type of the sample video frame, the type of the scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0067] As an optional implementation, the processor is specifically configured to execute:

[0068] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0069] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0070] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; and the processor is specifically configured to determine the order of the multiple algorithm parameters in the following manner:

[0071] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0072] The order of the algorithm parameters corresponding to the algorithm modules is determined according to the usage frequency of the algorithm modules corresponding to each sample video frame in the sample set.

[0073] As an optional implementation, the processor is specifically configured to execute:

[0074] Counting the algorithm modules used in each operation step of the sample video frame;

[0075] In each operation step, determining the algorithm module most frequently used in the operation step according to the algorithm modules used in each sample video frame;

[0076] The order of the algorithm parameters corresponding to each algorithm module is determined based on the algorithm module most frequently used in each operation step.

[0077] As an optional implementation, the processor is specifically configured to execute:

[0078] Determine a first algorithm module according to the algorithm module most frequently used in the first operation step for each sample video frame;

[0079] For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used by each sample video frame in the current operation step;

[0080] According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

[0081] As an optional implementation manner, the processor is specifically configured to train the parameter prediction model in the following manner:

[0082] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function based on an output result and a sample parameter set corresponding to the sample video frame;

[0083] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0084] As an optional implementation,

[0085] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0086] As an optional implementation manner, the processor is further configured to execute:

[0087] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0088] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to an algorithm module; and the processor is further configured to execute:

[0089] The multiple algorithm parameters are displayed in sequence;

[0090] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0091] As an optional implementation manner, the processor is further configured to execute:

[0092] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0093] In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

[0094] As an optional implementation, the algorithm parameters and algorithm modules are displayed in sequence; and the processor is further configured to execute:

[0095] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0096] In a third aspect, an embodiment of the present invention further provides a device for initializing algorithm parameters, the device comprising:

[0097] a prediction model determination unit, configured to obtain a video frame and an editing type corresponding to the video frame, and determine a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0098] an algorithm parameter prediction unit, configured to input the video frame into the parameter prediction model and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, the training set including sample sets corresponding to different editing types, and the sample set corresponding to the same editing type including sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0099] The video editing processing unit is used to edit the video frame using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0100] As an optional implementation manner, the acquired video frame includes multiple frames, and the prediction model determination unit is further configured to:

[0101] Grouping the video frames containing the same type of scenes in the plurality of video frames into a video frame group;

[0102] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N ≥ 1, where N is an integer;

[0103] An algorithm parameter set corresponding to the video frame group is determined according to the N prediction algorithm parameter sets.

[0104] As an optional implementation manner, the prediction model determination unit is specifically configured to:

[0105] For each prediction algorithm parameter in the N prediction algorithm parameter sets, using the weights corresponding to the N prediction algorithm parameter sets, performing weighted summation on the prediction algorithm parameters to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0106] An algorithm parameter set corresponding to the video frame group is determined according to algorithm parameters corresponding to various prediction algorithm parameters.

[0107] As an optional implementation, the N prediction algorithm parameter sets each have the same corresponding weight.

[0108] As an optional implementation manner, the algorithm parameter prediction unit is specifically configured to obtain a sample set in the following manner:

[0109] determining a type of editing performed by a user on a sample video frame using editing software;

[0110] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0111] The sample set is determined according to the editing type of the sample video frame, the type of the scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0112] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to:

[0113] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0114] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0115] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; the algorithm parameter prediction unit is further configured to determine the order of the multiple algorithm parameters in the following manner:

[0116] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0117] The order of the algorithm parameters corresponding to the algorithm modules is determined according to the usage frequency of the algorithm modules corresponding to each sample video frame in the sample set.

[0118] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to:

[0119] Counting the algorithm modules used in each operation step of the sample video frame;

[0120] In each operation step, determining the algorithm module most frequently used in the operation step according to the algorithm modules used in each sample video frame;

[0121] The order of the algorithm parameters corresponding to each algorithm module is determined based on the algorithm module most frequently used in each operation step.

[0122] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to:

[0123] Determine a first algorithm module according to the algorithm module most frequently used in the first operation step for each sample video frame;

[0124] For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used by each sample video frame in the current operation step;

[0125] According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

[0126] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to train the parameter prediction model in the following manner:

[0127] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function based on an output result and a sample parameter set corresponding to the sample video frame;

[0128] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0129] As an optional implementation,

[0130] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0131] As an optional embodiment, the device further includes a display unit specifically configured to:

[0132] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0133] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and each algorithm parameter corresponds to one algorithm module; the display unit is further configured to:

[0134] The multiple algorithm parameters are displayed in sequence;

[0135] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0136] As an optional implementation manner, the display unit is further configured to:

[0137] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0138] In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

[0139] As an optional implementation manner, the algorithm parameters and algorithm modules are displayed in sequence; the display unit is further configured to:

[0140] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0141] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, it is used to implement the steps of the method described in the first aspect above.

[0142] These and other aspects of the present application will be more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0143] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0144] Figure 1 A flowchart of a specific implementation of a method for initializing algorithm parameters provided by an embodiment of the present invention;

[0145] Figure 2 An architectural diagram of a parameter prediction model provided by an embodiment of the present invention;

[0146] Figure 3 A detailed structural diagram of a parameter prediction model provided by an embodiment of the present invention;

[0147] Figure 4 A flow chart of reasoning of a parameter prediction model provided by an embodiment of the present invention;

[0148] Figure 5 A schematic diagram of an application interface of editing software provided by an embodiment of the present invention;

[0149] Figure 6 An embodiment of the present invention provides an editing software display content refinement display interface diagram;

[0150] Figure 7 This is a display effect diagram provided by an embodiment of the present invention when the saturation algorithm is not used;

[0151] Figure 8 A schematic diagram of a parameter optimization option provided by an embodiment of the present invention;

[0152] Figure 9 A system flow chart for training and reasoning a parameter prediction model provided by an embodiment of the present invention;

[0153] Figure 10 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0154] Figure 11 A schematic diagram of a device for initializing algorithm parameters provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0155] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0156] In embodiments of the present invention, the term "and / or" describes the association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0157] The application scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Persons skilled in the art will appreciate that as new application scenarios emerge, the technical solutions provided by the embodiments of the present invention will also be applicable to similar technical problems. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0158] Image / video editing software is used for image and video editing, enabling adjustments including, but not limited to, noise reduction, super-resolution, brightness, contrast, shadows, and highlights. The parameters for each algorithm module (such as noise reduction, brightness, and contrast) are factory-set, requiring users to adjust these parameters from scratch each time they use the software. Software like Photoshop and DaVinci Resolve typically require users to manually adjust these parameters before outputting the final adjusted image / video. For example, HDR functionality requires at least BT2020 color gamut conversion, 10-bit bit depth conversion, 50fps frame rate conversion, as well as adjustments for super-resolution, noise reduction, brightness, contrast, hue, highlights, and shadows. Because the parameters for each algorithm module (such as noise reduction, brightness, and contrast) are factory-set, users must adjust these parameters from scratch each time they use the software. When processing videos, even if you can directly copy the algorithm parameters for the same scene, switching to a completely different scene will require users to adjust these parameters from scratch. Therefore, a lot of manpower is required to adjust the image.

[0159] The disclosed embodiment uses AI technology to use a deep learning model to predict the algorithm parameters of each algorithm module required to achieve the adjustment effect of the input video frame, and uses the predicted algorithm parameters as the initialization algorithm parameters of each algorithm module. Even when the scene switches, the model can automatically predict the algorithm parameters based on the current input video frame to achieve dynamic algorithm parameter initialization. Through dynamic algorithm parameter initialization, it can be achieved that most scenes can use automatically initialized algorithm parameters to present usable effects, and a small number of scenes only need to fine-tune the initialized algorithm parameters, which can greatly save labor costs.

[0160] like Figure 1 As shown, the specific implementation process of the method for initializing algorithm parameters provided in this embodiment is as follows:

[0161] Step 100: Obtain a video frame and an editing type corresponding to the video frame, and determine a parameter prediction model corresponding to the editing type. Different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions.

[0162] In practice, different editing types process video frames differently in the editing software. Therefore, in this application, the model parameters of the parameter prediction model corresponding to video frames of the same editing type are the same, while the model parameters of the parameter prediction model corresponding to video frames of different editing types are different. For example, if the editing type is to achieve HDR (High Dynamic Range) color grading, when HDR color grading is performed on both video frames, the parameter prediction model with the same model parameters is used to predict the algorithm parameters.

[0163] Optionally, the model structure of the parameter prediction model corresponding to different editing types is the same, but the model parameters are different. During implementation, the parameter prediction model is trained using sample sets corresponding to different editing types to obtain model parameters corresponding to different editing types. After determining the editing type of the input video frame, the corresponding model parameters can be called to use the parameter prediction model to predict (output) the algorithm parameters.

[0164] It is easy to understand that an editing type in this embodiment is used to represent an editing function of the editing software on the video frame.

[0165] Step 101: Input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame. The parameter prediction model is trained using a training set, and the training set includes sample sets corresponding to different editing types. The sample set corresponding to the same editing type includes sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames.

[0166] During implementation, since the parameter prediction model is trained using sample sets of different editing types, and the sample sets of the same editing type contain sample video frames of different types of scenes, the parameter prediction model implemented in this application can predict algorithm parameters for video frames of different editing types and video frames of different types of scenes, which is more targeted and improves the accuracy of prediction by distinguishing editing types and scenes.

[0167] In implementation, a sample parameter set includes one or more sample parameters, and one sample parameter corresponds to one algorithm module. Optionally, in this embodiment, one algorithm parameter includes one algorithm parameter, and one sample parameter includes one sample parameter; or one algorithm parameter includes multiple algorithm parameters, and one sample parameter includes multiple sample parameters.

[0168] It should be noted that the scene in this embodiment refers to the content contained in the video frame, and video frames of different types of scenes refer to video frames whose content similarity is less than or equal to a threshold. Video frames of the same type of scene include video frames with the same content and video frames with content similarity greater than the threshold.

[0169] In some embodiments, this embodiment obtains a sample set through the following steps:

[0170] Step 1a: determining the editing type of the sample video frame edited by the user using editing software;

[0171] Optionally, determine the edit type as follows:

[0172] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0173] Optionally, the video parameters (original video parameters or required video parameters) in this embodiment include but are not limited to: video frame resolution, color space (standard), video frame bit depth, dynamic range, encapsulation format, codec format, pixel format, etc.

[0174] During implementation, taking the editing type of achieving 4K HDR as an example, the original video parameters of the input 2K SDR video frame include a resolution of 1920×1080, a color space of bt709, and a bit depth of 8 bits; the 2K SDR video frame needs to be edited and output as a 4K HDR video frame. The required video parameters of the output video frame include a resolution of 3840×2160, a color space of bt2020, a bit depth of 10 bits, and an HDR curve of HLG.

[0175] Optionally, the video parameters of the video frame currently input to the editing software and the output video frame set by the user are recorded.

[0176] Step 1b: recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0177] Optionally, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, where the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value. In implementation, one algorithm module corresponds to one set of algorithm parameter information.

[0178] During implementation, the algorithm modules used by users when editing video frames using editing software are recorded in sequence; a user behavior record database is constructed, and the database should at least contain algorithm parameter information, as shown below: the original video parameters of the input video frame, the required video parameters of the output video frame, according to different types of scenes, for the same type of scenes, the names of the algorithm modules are recorded in the order of use of the algorithm modules, the algorithm parameter values ​​obtained after editing, the maximum algorithm parameter value, the minimum algorithm parameter value, and the normalized algorithm parameter value, as shown in Table 1 below.

[0179] Table 1 User behavior record database

[0180]

[0181]

[0182] Table 1 gives the editing record descriptions of video frames for three scenarios corresponding to the same editing type. The order of algorithm modules used in scenario 1 is: brightness, contrast, highlight, shadow, temporal noise reduction, and spatial noise reduction; the order of algorithm modules used in scenario 2 is: brightness, contrast, shadow, highlight, temporal noise reduction, and spatial noise reduction; the order of algorithm modules used in scenario 3 is: brightness, contrast, highlight, shadow, and temporal noise reduction.

[0183] Step 1c: Determine the sample set according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0184] During implementation, one sample set corresponds to one editing type; one sample set may include video frames of scenes of the same type, or may include video frames of scenes of different types.

[0185] This embodiment records the algorithm parameter adjustments made by users using editing software on input video frames across all algorithm modules. The recorded content primarily includes: 1) the algorithm modules used during the editing process; 2) the algorithm parameters corresponding to the used algorithm modules, with these algorithm parameters normalized from 0 to 1; and 3) the order in which the algorithm modules were used. By recording the algorithm parameter adjustments made by different users using editing software, a sample set is constructed for training the parameter prediction model.

[0186] Step 102: Edit the video frame using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0187] In some embodiments, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; the order of the multiple algorithm parameters is determined by any of the following methods:

[0188] Method 1: Determine the order of multiple algorithm parameters according to a pre-set order;

[0189] In implementation, according to the algorithm modules in the editing software, the execution order of each algorithm module is set, and the order of the corresponding algorithm parameters is determined according to the set execution order of each algorithm module. One algorithm module is used to determine one algorithm parameter.

[0190] Method 2: determining the order of the algorithm parameters corresponding to each algorithm module according to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set.

[0191] Optionally, determine the order of algorithm parameters corresponding to each algorithm module through the following steps:

[0192] Step 2a) counting the algorithm modules used in each operation step of the sample video frame;

[0193] Step 2b) In each operation step, based on the algorithm modules used in each sample video frame, determining the algorithm module most frequently used in the operation step;

[0194] Step 2c) Determine the order of algorithm parameters corresponding to each algorithm module according to the algorithm module most frequently used in each operation step.

[0195] Optionally, the order of algorithm parameters is determined based on frequency through the following steps:

[0196] Step 3a) determining a first algorithm module based on the algorithm module most frequently used in the first operation step for each sample video frame;

[0197] Step 3b) For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used in each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used in each sample video frame in the current operation step;

[0198] Step 3c) According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

[0199] During implementation, the order in which the algorithm modules are used is recorded and sorted by their frequency of use. The sorting rule is: the algorithm module most frequently used in the first operation step is selected as the first algorithm module for parameter prediction. After excluding the first algorithm module determined in the first operation step, the algorithm module most frequently used in the second operation step is selected as the second algorithm module for parameter prediction. This continues in this order until all algorithm modules are sorted. This order is ultimately used as the order of model parameters output by the parameter prediction model, and the manually adjusted model parameters serve as the true value data for parameter prediction model training.

[0200] For example, taking the editing type of achieving 4K HDR as an example, the original video parameters of the input 2K SDR video frame include a resolution of 1920×1080, a color space of bt709, and a bit depth of 8 bits; the 2K SDR video frame needs to be edited to output a 4K HDR video frame, and the required video parameters of the output video frame include a resolution of 3840×2160, a color space of bt2020, a bit depth of 10 bits, and an HDR curve of HLG. The recorded user behavior is shown in Table 1 above. When determining the order of algorithm parameters, first retrieve the algorithm modules used in the first operation step corresponding to all scenes, which are: brightness, brightness, brightness. Then, it is determined that under this editing type, the first algorithm module used by the parameter prediction module is brightness; after removing brightness, retrieve the algorithm modules used in the second operation step, which are: contrast, contrast, contrast. Then, it is determined that under this editing type, the second algorithm module used by the parameter prediction module is contrast; after removing brightness and contrast, retrieve the algorithm modules used in the third operation step, which are: highlights, shadows, highlights. Statistics are performed based on the algorithm modules used the most times to determine the parameter prediction module under this editing type. The third algorithm module used by the module is highlight. After removing brightness, contrast, and highlight, the algorithm modules used in the fourth operation step are retrieved as shadow, highlight, and shadow, respectively. Therefore, the fourth algorithm module used by the parameter prediction module for this edit type is shadow. The algorithm modules used in the fifth operation step are retrieved as temporal noise reduction, temporal noise reduction, and temporal noise reduction, respectively. Therefore, the fifth algorithm module used by the parameter prediction module for this edit type is temporal noise reduction. The algorithm nodes used in the sixth operation step are retrieved as spatial noise reduction, spatial noise reduction, and blank, respectively. Therefore, the sixth algorithm module used by the parameter prediction module for this edit type is spatial noise reduction. After determining the order of all algorithm modules, this order is used as the order of the algorithm parameters output by the final parameter prediction model. If a scene does not have this algorithm module, for example, scene 3 does not have spatial noise reduction, the spatial noise reduction parameter can be set to "0" (here, 0 does not necessarily mean the value 0, but rather a parameter that has no effect on the current algorithm module. For example, temporal noise reduction may have no effect if parameter 0 is set, and contrast may have no effect if parameter 1 is set).

[0201] It should be noted that the algorithm parameters output by the parameter prediction model in this embodiment are normalized algorithm parameters, that is, the output algorithm parameters are all between 0 and 1.

[0202] In some embodiments, the parameter prediction model is trained in the following manner:

[0203] The sample video frames in the sample set are input into the parameter prediction model, and a loss function is determined based on the output results and the sample parameter set corresponding to the sample video frames; the model parameters in the parameter prediction model are adjusted using the loss function, and when the loss function value meets the preset conditions or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0204] During implementation, a sample video frame is input and the corresponding algorithm parameter set is output. Each algorithm parameter in the output algorithm parameter set is compared with each sample parameter in the sample parameter set. The loss function is calculated using the output algorithm parameters and the corresponding sample parameters, and the model parameters are adjusted based on the loss function value to train the parameter prediction model.

[0205] Optionally, the parameter prediction model includes a feature extraction network and a parameter mapping network, the feature extraction network includes a convolutional layer and a residual network, and the feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer, and the parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0206] In implementation, the parameter prediction model is used to perform feature analysis on the input video frame (image) and output the predicted algorithm parameters of each algorithm module. Figure 2 As shown in FIG, this embodiment provides an architecture diagram of a parameter prediction model, which consists of two parts. The first part is a feature extraction network, which is used to extract and analyze features of input video frames (images); the second part is a parameter mapping network, which can map the extracted image features to the final output algorithm parameter sequence w1, w2, ..., wn. Figure 3 As shown, this embodiment provides a detailed structural diagram of a parameter prediction model, in which the feature extraction network consists of a convolutional layer and a residual network, and the parameter mapping network consists of a global average pooling network and a fully connected layer.

[0207] The function loss in this embodiment is used to calculate the difference between the algorithm parameters output by the parameter prediction model and the sample parameters (true value data) obtained by manual adjustment by the user in the sample set, and optimize the parameter prediction model based on this difference. The loss function in this embodiment includes but is not limited to L1 (mean absolute error) loss and MSE (mean square error) loss. The formula of the loss function is as follows:

[0208]

[0209]

[0210] In formula (1), Loss_l1 represents the L1 loss function value, n represents the number of algorithm parameters, and y irepresents the i-th sample parameter, f(x) i Represents the output of the i-th algorithm parameter.

[0211] In formula (2), Loss_mse represents the MSE loss function value, n represents the number of algorithm parameters, and y i represents the i-th sample parameter, f(x) i Represents the output of the i-th algorithm parameter.

[0212] In some embodiments, since the parameter prediction model in this embodiment can output different algorithm parameters according to different scenes, when the acquired video frames include multiple scenes, the scene can be segmented first, and the video frames of the same scene can be grouped as a group for parameter prediction. The specific implementation steps are as follows:

[0213] Step 4a) grouping the video frames containing scenes of the same type from the plurality of video frames into a video frame group;

[0214] During implementation, a video stream can be input, and the video frames in the video stream can be segmented according to scenes. Video frames of the same type of scene can be grouped into a video frame group, and the video stream can be segmented into multiple video frame groups, where the video frames contained in each video frame group belong to the same type of scene. In this embodiment, video frames of the same type of scene refer to video frames whose content similarity exceeds a threshold.

[0215] Step 4b) Select N video frames from the video frame group, input the N video frames into the parameter prediction model respectively, and output N prediction algorithm parameter sets; N≥1, N is an integer.

[0216] During implementation, N video frames can be randomly selected from the video frame group for parameter prediction. After completing the parameter prediction of N video frames, the weighted average of each prediction algorithm parameter in the N prediction algorithm parameter sets is calculated to obtain the algorithm parameter set corresponding to the scene.

[0217] Step 4c) Determine the algorithm parameter set corresponding to the video frame group based on the N prediction algorithm parameter sets.

[0218] Optionally, a final algorithm parameter set is determined based on N prediction algorithm parameter sets in the following manner:

[0219] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summation is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain the algorithm parameters corresponding to the prediction algorithm parameters; and the algorithm parameter set corresponding to the video frame group is determined based on the algorithm parameters corresponding to the various prediction algorithm parameters.

[0220] Optionally, the N prediction algorithm parameter sets in this embodiment each have the same corresponding weight, that is, the algorithm parameter set corresponding to the scenario is obtained by taking a weighted average of the prediction algorithm parameters in the N prediction algorithm parameter sets.

[0221] like Figure 4 As shown, this embodiment provides an inference process for a parameter prediction model, including steps such as scene segmentation, parameter estimation, and weighted averaging. During implementation, the video stream is input into the editing software, and scene segmentation is first performed. Video frames of scenes of the same type are divided into a video frame group, and the video stream is divided into multiple video frame groups. The scenes of the video frames contained in each video frame group belong to the same type. Secondly, N video frames are randomly selected from the video frame group for parameter prediction, and the N video frames are respectively input into the parameter prediction model, and N prediction algorithm parameter sets are output. Finally, the weighted average of each prediction algorithm parameter in the N prediction algorithm parameter sets is calculated to obtain the algorithm parameter set corresponding to the scene. For example, 5 video frames are randomly selected, and the brightness parameters corresponding to the first algorithm module of the 5 video frames are: 0.12, 0.13, 0.12, 0.14, and 0.11, respectively. Then, the brightness parameters are weighted averaged, and the final initialized brightness parameter corresponding to the 5 video frames of the same scene is w1 = (0.12 + 0.13 + 0.12 + 0.14 + 0.11) / 5 = 0.124.

[0222] In some embodiments, this embodiment further provides a display method, which is specifically as follows:

[0223] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0224] like Figure 5 As shown, this embodiment provides a schematic diagram of the application interface of editing software, including a file list area, a screen display area, a video scene segment selection area, an area for adjusting all algorithm module parameters, an oscilloscope display area, an area for processing the initialized / used algorithm modules, and an area for listing initialized algorithm modules. The area for processing the initialized / used algorithm modules is a display area for processing all automatically initialized algorithm modules, and the area for listing initialized algorithm modules is used to display the names of all initialized algorithm modules. A selection button is provided in front of the name. A black dot indicates that the algorithm module is selected for use, and no black dot indicates that the algorithm module is not used.

[0225] In some embodiments, the algorithm parameter set includes multiple algorithm parameters, one algorithm parameter corresponds to one algorithm module; the multiple algorithm parameters are displayed in sequence; and the algorithm modules corresponding to the multiple algorithm parameters are displayed in the order of the multiple algorithm parameters.

[0226] During implementation, algorithm parameters and algorithm module partitions are displayed. Figure 6As shown, this embodiment provides an interface for displaying detailed content of editing software. The initialized / used algorithm module processing flow area is a display area for all automatically initialized algorithm modules to be processed. Arrows are used to indicate the processing flow direction of the algorithm modules, and the arrow direction indicates the processing order of the algorithm modules. The algorithm modules displayed in this area have been initialized for the algorithm parameters of the current input video frame according to the algorithm parameters output by the parameter prediction model. The initialized algorithm module list area is used to display the names of all initialized algorithm modules. A selection button is set in front of the name. A black dot indicates that the algorithm module is selected for use, and no black dot indicates that the algorithm module is not used.

[0227] In some embodiments, after obtaining the initialized algorithm parameters using the parameter prediction model, the algorithm module and the corresponding algorithm parameters may be deleted according to actual needs by any one or more of the following methods:

[0228] (1) In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

[0229] by Figure 6 For example, if you do not want to use a certain algorithm module, you can directly delete the algorithm module. For example, the names of all initialized algorithm modules are displayed in the initialized algorithm module list area, and a selection button is set in front of the name. A black dot indicates that the algorithm module is selected for use, and no black dot indicates that the algorithm module is not used. The displayed algorithm module is used by default. Figure 7 As shown, this embodiment provides a display effect when the saturation algorithm is not used. The user can delete the saturation algorithm module and its corresponding parameters by clicking the button in front of the saturation algorithm (the second instruction).

[0230] (2) in response to a first instruction from the user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold;

[0231] Optionally, the initialized algorithm module list area also includes a parameter optimization option. The parameter optimization option is used to indicate whether to limit the numerical range of the algorithm parameters output by the parameter prediction model. For example, when the parameter optimization option is selected (the first instruction), the algorithm modules whose algorithm parameters are less than the threshold value can be deleted. If the algorithm parameters output by the parameter prediction model are less than the threshold value, it is considered that the algorithm module does not play a much improvement role. Optionally, the threshold value can be set at around 0.05, between 0 and 0.1. Not selecting the parameter optimization option means using the algorithm modules whose algorithm parameters are not 0. Selecting the parameter optimization option means deleting the algorithm modules whose algorithm parameters are less than the threshold value. Figure 8 As shown, this embodiment provides a schematic diagram of selecting parameter optimization options (excluding the two algorithm modules of hue and saturation).

[0232] Optionally, the algorithm parameters and algorithm modules are displayed in order; according to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated. Figure 7 For example, when the user does not select saturation, delete the saturation algorithm module in the initialized / used algorithm module processing flow area, and redraw the connection arrows between the algorithm modules in the order after deletion; Figure 8 For example, after selecting parameter optimization, delete the two algorithm modules of hue and saturation in the initialized / used algorithm module processing flow area and the initialized algorithm module list area, redraw the connection arrows between the algorithm modules in the order after deletion, and update the display order of the algorithm modules.

[0233] It should be noted that the video frames in this embodiment are also understood to be images. The editing software in this embodiment includes but is not limited to image / video editing software. The parameter prediction model in this embodiment can be an application independent of the editing software, or it can be a functional module integrated into the editing software, and this embodiment does not impose any additional restrictions on this.

[0234] like Figure 9 As shown, this embodiment provides a system process for training and inference of a parameter prediction model. Sample video frames are input into editing software. The behavior recording module records the user's behavior of manually adjusting algorithm parameters to obtain a sample set (including editing type, algorithm parameter information, scene type, and the order in which algorithm modules are used). The sample set is input into the initial parameter prediction model to train the parameter prediction model. The model parameters are adjusted through loss function calculation to obtain a trained parameter prediction model. Video frames are input into the trained parameter prediction model, and the initialized algorithm parameters are output. The editing software obtains the initialized algorithm parameters. The user can edit the input video frames based on the initialized algorithm parameters to obtain the corresponding effect.

[0235] This embodiment provides an automatic initialization solution for algorithm parameters of algorithm modules of image / video editing software. By designing a user behavior record database, matching input images / videos and algorithm parameter values ​​of each algorithm module are obtained, and a parameter prediction model is established. The algorithm module to be used and the optimal algorithm parameters of the algorithm module can be automatically predicted based on the input image / video. Even if different materials are constantly changed, different algorithm parameters can still be initialized according to different materials, which greatly saves labor costs.

[0236] Example 2. Based on the same inventive concept, the embodiment of the present invention also provides an electronic device. Since the electronic device is the electronic device in the method in the embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0237] like Figure 10 As shown, the electronic device includes a processor 1000 and a memory 1001, wherein the memory 1001 is used to store a program executable by the processor 1000, and the processor 1000 is used to read the program in the memory 1001 and perform the following steps:

[0238] Obtaining a video frame and an editing type corresponding to the video frame, and determining a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0239] Inputting the video frame into the parameter prediction model and outputting an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0240] The video frame is edited using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0241] As an optional implementation manner, the acquired video frame includes multiple frames, and the processor 1000 is further configured to execute:

[0242] Grouping the video frames containing the same type of scenes in the plurality of video frames into a video frame group;

[0243] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N ≥ 1, where N is an integer;

[0244] An algorithm parameter set corresponding to the video frame group is determined according to the N prediction algorithm parameter sets.

[0245] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0246] For each prediction algorithm parameter in the N prediction algorithm parameter sets, using the weights corresponding to the N prediction algorithm parameter sets, performing weighted summation on the prediction algorithm parameters to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0247] An algorithm parameter set corresponding to the video frame group is determined according to algorithm parameters corresponding to various prediction algorithm parameters.

[0248] As an optional implementation, the N prediction algorithm parameter sets each have the same corresponding weight.

[0249] As an optional implementation manner, the processor 1000 is specifically configured to obtain the sample set in the following manner:

[0250] determining a type of editing performed by a user on a sample video frame using editing software;

[0251] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0252] The sample set is determined according to the editing type of the sample video frame, the type of the scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0253] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0254] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0255] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0256] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; the processor 1000 is specifically configured to determine the order of the multiple algorithm parameters in the following manner:

[0257] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0258] The order of the algorithm parameters corresponding to the algorithm modules is determined according to the usage frequency of the algorithm modules corresponding to each sample video frame in the sample set.

[0259] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0260] Counting the algorithm modules used in each operation step of the sample video frame;

[0261] In each operation step, determining the algorithm module most frequently used in the operation step according to the algorithm modules used in each sample video frame;

[0262] The order of the algorithm parameters corresponding to each algorithm module is determined based on the algorithm module most frequently used in each operation step.

[0263] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0264] Determine a first algorithm module according to the algorithm module most frequently used in the first operation step for each sample video frame;

[0265] For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used by each sample video frame in the current operation step;

[0266] According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

[0267] As an optional implementation manner, the processor 1000 is specifically configured to train the parameter prediction model in the following manner:

[0268] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function based on an output result and a sample parameter set corresponding to the sample video frame;

[0269] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0270] As an optional implementation,

[0271] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0272] As an optional implementation manner, the processor 1000 is further configured to execute:

[0273] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0274] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and each algorithm parameter corresponds to an algorithm module; the processor 1000 is further configured to execute:

[0275] The multiple algorithm parameters are displayed in sequence;

[0276] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0277] As an optional implementation manner, the processor 1000 is further configured to execute:

[0278] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0279] In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

[0280] As an optional implementation, the algorithm parameters and algorithm modules are displayed in sequence; the processor 1000 is further configured to execute:

[0281] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0282] Based on the same inventive concept, an embodiment of the present invention also provides a device for initializing algorithm parameters. Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0283] like Figure 11 As shown, the device includes:

[0284] A prediction model determination unit 1100 is configured to obtain a video frame and an editing type corresponding to the video frame, and determine a parameter prediction model corresponding to the editing type. Different editing types have different model parameters corresponding to the parameter prediction model, and different editing types are used to implement different video editing functions.

[0285] An algorithm parameter prediction unit 1101 is configured to input the video frame into the parameter prediction model and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample set corresponding to the same editing type includes sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0286] The video editing processing unit 1102 is configured to edit the video frame using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0287] As an optional implementation manner, the acquired video frame includes multiple frames, and the prediction model determination unit 1100 is further configured to:

[0288] Grouping the video frames containing the same type of scenes in the plurality of video frames into a video frame group;

[0289] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N ≥ 1, where N is an integer;

[0290] An algorithm parameter set corresponding to the video frame group is determined according to the N prediction algorithm parameter sets.

[0291] As an optional implementation manner, the prediction model determination unit 1100 is specifically configured to:

[0292] For each prediction algorithm parameter in the N prediction algorithm parameter sets, using the weights corresponding to the N prediction algorithm parameter sets, performing weighted summation on the prediction algorithm parameters to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0293] An algorithm parameter set corresponding to the video frame group is determined according to algorithm parameters corresponding to various prediction algorithm parameters.

[0294] As an optional implementation, the N prediction algorithm parameter sets each have the same corresponding weight.

[0295] As an optional implementation, the algorithm parameter prediction unit 1101 is specifically configured to obtain a sample set in the following manner:

[0296] determining a type of editing performed by a user on a sample video frame using editing software;

[0297] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0298] The sample set is determined according to the editing type of the sample video frame, the type of the scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0299] As an optional implementation manner, the algorithm parameter prediction unit 1101 is specifically configured to:

[0300] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0301] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0302] As an optional implementation, the algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; the algorithm parameter prediction unit 1101 is further configured to determine the order of the multiple algorithm parameters in the following manner:

[0303] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0304] The order of the algorithm parameters corresponding to the algorithm modules is determined according to the usage frequency of the algorithm modules corresponding to each sample video frame in the sample set.

[0305] As an optional implementation manner, the algorithm parameter prediction unit 1101 is specifically configured to:

[0306] Counting the algorithm modules used in each operation step of the sample video frame;

[0307] In each operation step, determining the algorithm module most frequently used in the operation step according to the algorithm modules used in each sample video frame;

[0308] The order of the algorithm parameters corresponding to each algorithm module is determined based on the algorithm module most frequently used in each operation step.

[0309] As an optional implementation manner, the algorithm parameter prediction unit 1101 is specifically configured to:

[0310] Determine a first algorithm module according to the algorithm module most frequently used in the first operation step for each sample video frame;

[0311] For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used by each sample video frame in the current operation step;

[0312] According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

[0313] As an optional implementation, the algorithm parameter prediction unit 1101 is specifically configured to train the parameter prediction model in the following manner:

[0314] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function based on an output result and a sample parameter set corresponding to the sample video frame;

[0315] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0316] As an optional implementation,

[0317] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0318] As an optional embodiment, the device further includes a display unit specifically configured to:

[0319] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0320] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and each algorithm parameter corresponds to one algorithm module; the display unit is further configured to:

[0321] The multiple algorithm parameters are displayed in sequence;

[0322] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0323] As an optional implementation manner, the display unit is further configured to:

[0324] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0325] In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

[0326] As an optional implementation manner, the algorithm parameters and algorithm modules are displayed in sequence; the display unit is further configured to:

[0327] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0328] Based on the same inventive concept, embodiments of the present disclosure provide a computer storage medium comprising computer program code. When the computer program code is executed on a computer, the computer executes any of the algorithm parameter initialization methods discussed above. Because the principles underlying the problem solved by the computer storage medium are similar to those of the algorithm parameter initialization method, the implementation of the computer storage medium can be referenced to the implementation of the method, and any repetitions will not be repeated.

[0329] In a specific implementation process, computer storage media may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0330] Based on the same inventive concept, embodiments of the present disclosure further provide a computer program product, comprising: computer program code, which, when executed on a computer, causes the computer to execute any of the algorithm parameter initialization methods discussed above. Because the principles underlying the problems solved by the aforementioned computer program products are similar to those of the algorithm parameter initialization methods, the implementation of the aforementioned computer program products can be referenced to the implementation of the methods, and any repetitions will not be repeated.

[0331] The computer program product can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0332] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0333] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that performs the functions specified in one or more boxes.

[0334] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0335] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0336] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for initializing algorithm parameters, characterized in that: The method includes: Obtaining a video frame and an editing type corresponding to the video frame, and determining a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions; Inputting the video frame into the parameter prediction model and outputting an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is trained using a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames; The video frame is edited using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

2. The method according to claim 1, characterized in that The acquired video frames include a plurality of frames, and the method further includes: Grouping the video frames containing the same type of scenes in the plurality of video frames into a video frame group; Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N ≥ 1, where N is an integer; An algorithm parameter set corresponding to the video frame group is determined according to the N prediction algorithm parameter sets.

3. The method according to claim 2, characterized in that Determining the algorithm parameter set corresponding to the video frame group according to the N prediction algorithm parameter sets includes: For each prediction algorithm parameter in the N prediction algorithm parameter sets, using the weights corresponding to the N prediction algorithm parameter sets, performing weighted summation on the prediction algorithm parameters to obtain an algorithm parameter corresponding to the prediction algorithm parameter; An algorithm parameter set corresponding to the video frame group is determined according to algorithm parameters corresponding to various prediction algorithm parameters.

4. The method according to claim 3, characterized in that The N prediction algorithm parameter sets each have the same corresponding weight.

5. The method according to claim 1, wherein Obtain the sample set as follows: determining a type of editing performed by a user on a sample video frame using editing software; Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module; The sample set is determined according to the editing type of the sample video frame, the type of the scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

6. The method according to claim 5, characterized in that The determining of the editing type of the sample video frame edited by the user using the editing software includes: The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

7. The method according to claim 5, characterized in that The algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, where the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

8. The method according to claim 1, characterized in that The algorithm parameter set includes multiple algorithm parameters, each algorithm parameter corresponds to one algorithm module; the order of the multiple algorithm parameters is determined by: Determine the order of multiple algorithm parameters in a pre-set order; or, The order of the algorithm parameters corresponding to the algorithm modules is determined according to the usage frequency of the algorithm modules corresponding to each sample video frame in the sample set.

9. The method according to claim 8, characterized in that The determining, based on the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of algorithm parameters corresponding to each algorithm module includes: Counting the algorithm modules used in each operation step of the sample video frame; In each operation step, determining the algorithm module most frequently used in the operation step according to the algorithm modules used in each sample video frame; The order of the algorithm parameters corresponding to each algorithm module is determined based on the algorithm module most frequently used in each operation step.

10. The method according to claim 9, characterized in that The step of determining the order of algorithm parameters corresponding to each algorithm module according to the algorithm module most frequently used in each operation step includes: Determine a first algorithm module according to the algorithm module most frequently used in the first operation step for each sample video frame; For the second and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules based on the algorithm module most frequently used by each sample video frame in the current operation step; According to the determined order of the algorithm modules, the order of the algorithm parameters corresponding to the algorithm modules is determined.

11. The method according to claim 1, wherein The parameter prediction model is trained in the following manner: Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function based on an output result and a sample parameter set corresponding to the sample video frame; The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

12. The method according to claim 1, characterized in that The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

13. The method according to any one of claims 1 to 12, characterized in that: The method further includes: The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

14. The method according to claim 13, characterized in that The algorithm parameter set includes multiple algorithm parameters, and each algorithm parameter corresponds to one algorithm module; the method further includes: The multiple algorithm parameters are displayed in sequence; The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

15. The method according to claim 13, characterized in that The method further includes: In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or, In response to a second instruction from the user, the algorithm module and corresponding algorithm parameters indicated by the second instruction are deleted.

16. The method according to claim 15, characterized in that The algorithm parameters and algorithm modules are displayed in order; further comprising: According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

17. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and execute the steps of any one of the methods of claims 1 to 16.

18. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images