Algorithm parameter initialization method and electronic equipment

Abstract

The invention discloses an algorithm parameter initialization method and electronic equipment, which are used for initializing algorithm parameters of editing software and saving labor cost. The method comprises the steps that video frames and editing types corresponding to the video frames are acquired, parameter prediction models corresponding to the editing types are determined, and model parameters of the parameter prediction models corresponding to different editing types are different; the video frame is input into the parameter prediction model, an algorithm parameter set corresponding to a scene contained in the video frame is output, the parameter prediction model is obtained through training by using a training set, the training set comprises sample sets corresponding to different editing types, and the sample sets correspond to different editing types. The sample sets corresponding to the same editing type comprise sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames; and editing the video frame by using an algorithm module of editing software and algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

Description

Technical Field

[0001] The present invention relates to the technical field of image / video editing software, and in particular to a method for initializing algorithm parameters and electronic equipment. Background Art

[0002] For image / video editing software, the parameters of each algorithm module (noise reduction, brightness, contrast, etc.) are all parameter settings when the software leaves the factory. Users need to adjust the parameters of various algorithms from scratch every time they use the software, which consumes a lot of manpower costs.

[0003] Furthermore, when processing videos, even if the algorithm parameters of the same scene can be directly copied, once the scene is switched to a completely different scene, the user still needs to adjust the algorithm parameters from the beginning, which requires a lot of manpower to adjust the image. Summary of the invention

[0004] The present invention provides a method and electronic device for initializing algorithm parameters, which are used to automatically initialize algorithm parameters of editing software and initialize different algorithm parameters for input video frames of different editing types and different scenes, thereby greatly saving labor costs.

[0005] In a first aspect, an embodiment of the present invention provides a method for initializing algorithm parameters, the method comprising:

[0006] Acquire a video frame and an editing type corresponding to the video frame, determine a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0007] Input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0008] The video frame is edited using the algorithm module of the editing software and the algorithm parameters in the algorithm parameter set corresponding to the algorithm module.

[0009] The method for initializing the algorithm parameters provided in this embodiment uses a deep learning model to predict the algorithm parameters of each algorithm module to achieve the adjustment effect of the input video frame, and uses the predicted algorithm parameters as the initialization algorithm parameters of each algorithm module. Even when the scene is switched, the model can automatically predict the algorithm parameters based on the current input video frame to achieve dynamic algorithm parameter initialization. Through dynamic algorithm parameter initialization, it can be achieved that most scenes can present usable effects using automatically initialized algorithm parameters, and a small number of scenes only need to fine-tune the initialized algorithm parameters, which can greatly save labor costs.

[0010] As an optional implementation manner, the acquired video frame includes multiple frames, and the method further includes:

[0011] Grouping video frames containing scenes of the same type in a plurality of video frames into a video frame group;

[0012] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N≥1, N is an integer;

[0013] According to the N prediction algorithm parameter sets, an algorithm parameter set corresponding to the video frame group is determined.

[0014] As an optional implementation manner, determining the algorithm parameter set corresponding to the video frame group according to the N prediction algorithm parameter sets includes:

[0015] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summing is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0016] According to the algorithm parameters corresponding to the various prediction algorithm parameters, an algorithm parameter set corresponding to the video frame group is determined.

[0017] As an optional implementation, the weights corresponding to the N prediction algorithm parameter sets are the same.

[0018] As an optional implementation, the sample set is obtained in the following manner:

[0019] determining the type of editing performed by the user on the sample video frame using the editing software;

[0020] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0021] The sample set is determined according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0022] As an optional implementation manner, determining the editing type of the sample video frame edited by the user using editing software includes:

[0023] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0024] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0025] As an optional implementation, the algorithm parameter set includes multiple algorithm parameters, one algorithm parameter corresponds to one algorithm module; the order of the multiple algorithm parameters is determined in the following manner:

[0026] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0027] According to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0028] As an optional implementation manner, determining the order of algorithm parameters corresponding to each algorithm module according to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set includes:

[0029] Counting the algorithm modules used in each operation step of the sample video frame;

[0030] In each operation step, according to the algorithm modules used by each sample video frame, determining the algorithm module most frequently used in the operation step;

[0031] According to the algorithm module with the highest frequency of use in each operation step, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0032] As an optional implementation manner, determining the order of algorithm parameters corresponding to each algorithm module according to the algorithm module with the highest frequency of use in each operation step includes:

[0033] Determine a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step of each sample video frame;

[0034] For the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step;

[0035] According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0036] As an optional implementation, the parameter prediction model is trained in the following manner:

[0037] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function according to an output result and a sample parameter set corresponding to the sample video frame;

[0038] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0039] As an optional implementation,

[0040] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of an input video frame. The parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0041] As an optional implementation, the method further includes:

[0042] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0043] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the method further includes:

[0044] The multiple algorithm parameters are displayed in order;

[0045] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0046] As an optional implementation, the method further includes:

[0047] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0048] In response to a second instruction from the user, the algorithm module and the corresponding algorithm parameters indicated by the second instruction are deleted.

[0049] As an optional implementation, the algorithm parameters and algorithm modules are displayed in order; further comprising:

[0050] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0051] In a second aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0052] Acquire a video frame and an editing type corresponding to the video frame, determine a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0053] Input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0054] The video frame is edited using the algorithm module of the editing software and the algorithm parameters in the algorithm parameter set corresponding to the algorithm module.

[0055] As an optional implementation manner, the acquired video frame includes multiple frames, and the processor is specifically further configured to execute:

[0056] Grouping video frames containing scenes of the same type in a plurality of video frames into a video frame group;

[0057] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N≥1, N is an integer;

[0058] According to the N prediction algorithm parameter sets, an algorithm parameter set corresponding to the video frame group is determined.

[0059] As an optional implementation, the processor is specifically configured to execute:

[0060] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summing is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0061] According to the algorithm parameters corresponding to the various prediction algorithm parameters, an algorithm parameter set corresponding to the video frame group is determined.

[0062] As an optional implementation, the weights corresponding to the N prediction algorithm parameter sets are the same.

[0063] As an optional implementation manner, the processor is specifically configured to obtain the sample set in the following manner:

[0064] determining the type of editing performed by the user on the sample video frame using the editing software;

[0065] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0066] The sample set is determined according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0067] As an optional implementation, the processor is specifically configured to execute:

[0068] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0069] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0070] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the processor is specifically configured to determine the order of the multiple algorithm parameters in the following manner:

[0071] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0072] According to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0073] As an optional implementation, the processor is specifically configured to execute:

[0074] Counting the algorithm modules used in each operation step of the sample video frame;

[0075] In each operation step, according to the algorithm modules used by each sample video frame, determining the algorithm module most frequently used in the operation step;

[0076] According to the algorithm module with the highest frequency of use in each operation step, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0077] As an optional implementation, the processor is specifically configured to execute:

[0078] Determine a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step of each sample video frame;

[0079] For the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step;

[0080] According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0081] As an optional implementation, the processor is specifically configured to train the parameter prediction model in the following manner:

[0082] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function according to an output result and a sample parameter set corresponding to the sample video frame;

[0083] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0084] As an optional implementation,

[0085] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of an input video frame. The parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0086] As an optional implementation, the processor is further configured to execute:

[0087] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0088] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the processor is further configured to execute:

[0089] The multiple algorithm parameters are displayed in order;

[0090] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0091] As an optional implementation, the processor is further configured to execute:

[0092] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0093] In response to a second instruction from the user, the algorithm module and the corresponding algorithm parameters indicated by the second instruction are deleted.

[0094] As an optional implementation, the algorithm parameters and algorithm modules are displayed in sequence; the processor is further configured to execute:

[0095] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0096] In a third aspect, an embodiment of the present invention further provides a device for initializing algorithm parameters, the device comprising:

[0097] A prediction model determination unit is used to obtain a video frame and an editing type corresponding to the video frame, and determine a parameter prediction model corresponding to the editing type. The model parameters of the parameter prediction model corresponding to different editing types are different, and different editing types are used to implement different video editing functions;

[0098] an algorithm parameter prediction unit, configured to input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0099] The video editing processing unit is used to edit the video frame by using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0100] As an optional implementation manner, the acquired video frame includes a plurality of frames, and the prediction model determination unit is further specifically used for:

[0101] Grouping video frames containing scenes of the same type in a plurality of video frames into a video frame group;

[0102] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N≥1, N is an integer;

[0103] According to the N prediction algorithm parameter sets, an algorithm parameter set corresponding to the video frame group is determined.

[0104] As an optional implementation manner, the prediction model determination unit is specifically used for:

[0105] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summing is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0106] According to the algorithm parameters corresponding to the various prediction algorithm parameters, an algorithm parameter set corresponding to the video frame group is determined.

[0107] As an optional implementation, the weights corresponding to the N prediction algorithm parameter sets are the same.

[0108] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to obtain the sample set in the following manner:

[0109] determining the type of editing performed by the user on the sample video frame using the editing software;

[0110] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0111] The sample set is determined according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0112] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to:

[0113] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0114] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0115] As an optional implementation, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the algorithm parameter prediction unit is further used to determine the order of multiple algorithm parameters in the following manner:

[0116] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0117] According to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0118] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to:

[0119] Counting the algorithm modules used in each operation step of the sample video frame;

[0120] In each operation step, according to the algorithm modules used by each sample video frame, determining the algorithm module most frequently used in the operation step;

[0121] According to the algorithm module with the highest frequency of use in each operation step, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0122] As an optional implementation manner, the algorithm parameter prediction unit is specifically used to:

[0123] Determine a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step of each sample video frame;

[0124] For the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step;

[0125] According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0126] As an optional implementation, the algorithm parameter prediction unit is specifically used to train the parameter prediction model in the following manner:

[0127] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function according to an output result and a sample parameter set corresponding to the sample video frame;

[0128] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0129] As an optional implementation,

[0130] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of an input video frame. The parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0131] As an optional implementation, the display unit is further included to:

[0132] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0133] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the display unit is further specifically used for:

[0134] The multiple algorithm parameters are displayed in order;

[0135] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0136] As an optional implementation manner, the display unit is further used for:

[0137] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0138] In response to a second instruction from the user, the algorithm module and the corresponding algorithm parameters indicated by the second instruction are deleted.

[0139] As an optional implementation manner, the algorithm parameters and algorithm modules are displayed in sequence; and the display unit is further configured to:

[0140] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0141] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, it is used to implement the steps of the method described in the first aspect above.

[0142] These and other aspects of the present application will be more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0143] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0144] Figure 1 A specific implementation flow chart of a method for initializing algorithm parameters provided by an embodiment of the present invention;

[0145] Figure 2 An architectural diagram of a parameter prediction model provided by an embodiment of the present invention;

[0146] Figure 3 A detailed structural diagram of a parameter prediction model provided by an embodiment of the present invention;

[0147] Figure 4 A reasoning flow chart of a parameter prediction model provided by an embodiment of the present invention;

[0148] Figure 5 A schematic diagram of an application interface of editing software provided by an embodiment of the present invention;

[0149] Figure 6 A detailed display interface diagram of the editing software display content provided by an embodiment of the present invention;

[0150] Figure 7 A display effect diagram when a saturation algorithm is not used provided by an embodiment of the present invention;

[0151] Figure 8 A schematic diagram of a parameter optimization option selection provided by an embodiment of the present invention;

[0152] Fig. 9 A system flow chart of training and reasoning of a parameter prediction model provided by an embodiment of the present invention;

[0153] Fig.10 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0154] Fig.11 A schematic diagram of a device for initializing algorithm parameters provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0155] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0156] In the embodiments of the present invention, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0157] The application scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0158] Image / video editing software is software used to edit images and videos, and can achieve adjustments including but not limited to noise reduction, super-resolution, brightness, contrast, shadows, highlights, etc. The parameters of each algorithm module (such as noise reduction, brightness, contrast, etc.) are all parameter settings when the software leaves the factory. Users need to adjust the parameters of various algorithms from the beginning every time they use the software. For example, PS software, Da Vinci software, etc., generally require users to manually adjust the parameters of each algorithm module, and finally output the adjusted image / video. For example, the HDR function requires at least color gamut bt2020 conversion, bit depth 10bit conversion, frame rate 50 frame conversion, and super-resolution, noise reduction, brightness, contrast, hue, highlights, shadows, etc. Adjustment. Since the parameters of each algorithm module (noise reduction, brightness, contrast, etc.) are all parameter settings when the software leaves the factory, users need to adjust the various algorithm parameters from the beginning every time they use the software. When processing videos, even if the algorithm parameters of the same scene can be directly copied, once the scene is switched to a completely different scene, the user still needs to adjust the algorithm parameters from the beginning. Therefore, a lot of manpower is required to adjust the image.

[0159] The disclosed embodiment uses AI technology and a deep learning model to predict the algorithm parameters of each algorithm module for achieving the adjustment effect of the input video frame, and uses the predicted algorithm parameters as the initialization algorithm parameters of each algorithm module. Even when the scene is switched, the model can automatically predict the algorithm parameters based on the current input video frame to achieve dynamic algorithm parameter initialization. Through dynamic algorithm parameter initialization, it can be achieved that most scenes can present usable effects using automatically initialized algorithm parameters, and a small number of scenes only need to fine-tune the initialized algorithm parameters, which can greatly save labor costs.

[0160] like Figure 1 As shown, the specific implementation process of a method for initializing algorithm parameters provided in this embodiment is as follows:

[0161] Step 100: obtaining a video frame and an editing type corresponding to the video frame, and determining a parameter prediction model corresponding to the editing type. The parameter prediction model corresponding to different editing types has different model parameters, and different editing types are used to implement different video editing functions.

[0162] In practice, different editing types process video frames differently in the editing software, so the model parameters of the parameter prediction model corresponding to the video frames of the same editing type in this application are the same, while the model parameters of the parameter prediction model corresponding to the video frames of different editing types are different. For example, if the editing type is to implement HDR (High Dynamic Range) color adjustment, when HDR color adjustment is performed on both video frames, the parameter prediction model with the same model parameters is used to predict the algorithm parameters.

[0163] Optionally, the model structure of the parameter prediction model corresponding to different editing types is the same, but the model parameters are different. During implementation, the parameter prediction model is trained using sample sets corresponding to different editing types to obtain model parameters corresponding to different editing types. After determining the editing type of the input video frame, the corresponding model parameters can be called to use the parameter prediction model to predict (output) the algorithm parameters.

[0164] It is easy to understand that an editing type in this embodiment is used to represent an editing function of the editing software on the video frame.

[0165] Step 101: input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0166] During implementation, since the parameter prediction model is trained using sample sets of different editing types, and the sample sets of the same editing type contain sample video frames of different types of scenes, the parameter prediction model implemented in the present application can predict algorithm parameters for video frames of different editing types and video frames of different types of scenes, which is more targeted and improves the accuracy of prediction by distinguishing editing types and scenes.

[0167] In implementation, the sample parameter set includes one or more sample parameters, and one sample parameter corresponds to one algorithm module. Optionally, one algorithm parameter in this embodiment includes one algorithm parameter, and one sample parameter includes one sample parameter; or one algorithm parameter includes multiple algorithm parameters, and one sample parameter includes multiple sample parameters.

[0168] It should be noted that the scene in this embodiment refers to the content contained in the video frame, and the video frames of different types of scenes refer to the video frames whose content similarity is less than or equal to the threshold. The video frames of the same type of scenes include video frames with the same content and video frames with content similarity greater than the threshold.

[0169] In some embodiments, the present embodiment obtains a sample set through the following steps:

[0170] Step 1a, determining the editing type of the sample video frame edited by the user using editing software;

[0171] Optionally, determine the edit type as follows:

[0172] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0173] Optionally, the video parameters (original video parameters or required video parameters) in this embodiment include but are not limited to: video frame resolution, color space (standard), video frame bit depth, dynamic range, encapsulation format, codec format, pixel format, etc.

[0174] During implementation, taking the editing type of realizing 4K HDR as an example, the original video parameters of the input 2K SDR video frame include a resolution of 1920×1080, a color space of bt709, and a bit depth of 8 bits; the 2K SDR video frame needs to be edited to output a 4K HDR video frame, and the required video parameters of the output video frame include a resolution of 3840×2160, a color space of bt2020, a bit depth of 10 bits, and an HDR curve of HLG.

[0175] Optionally, the video parameters of the video frame currently input to the editing software and the output video frame set by the user are recorded.

[0176] Step 1b, recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0177] Optionally, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, wherein the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value. In implementation, one algorithm module corresponds to a set of algorithm parameter information.

[0178] During implementation, the algorithm modules used by users when editing video frames using editing software are recorded in sequence; a user behavior record database is constructed, and the database should at least contain algorithm parameter information, as shown below: original video parameters of input video frames, required video parameters of output video frames, according to different types of scenes, the names of algorithm modules are recorded in the order of use of algorithm modules for the same type of scenes, the algorithm parameter values ​​obtained after editing, the maximum algorithm parameter value, the minimum algorithm parameter value, and the normalized algorithm parameter value, as shown in Table 1 below.

[0179] Table 1 User behavior record database

[0180]

[0181]

[0182] Table 1 gives the editing record description of the video frames of the three scenes corresponding to the same editing type. The order of the algorithm modules used in scene 1 is: brightness, contrast, highlight, shadow, temporal noise reduction, and spatial noise reduction; the order of the algorithm modules used in scene 2 is: brightness, contrast, shadow, highlight, temporal noise reduction, and spatial noise reduction; the order of the algorithm modules used in scene 3 is: brightness, contrast, highlight, shadow, and temporal noise reduction.

[0183] Step 1c: Determine the sample set according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0184] In implementation, one sample set corresponds to one editing type; one sample set may include video frames of scenes of the same type, or may include video frames of scenes of different types.

[0185] This embodiment records the algorithm parameter adjustment work of all algorithm modules performed by the user using the editing software on the input video frame, wherein the recorded content mainly includes ① the algorithm module used in the editing process, ② the algorithm parameters corresponding to the used algorithm module, and the normalization of these algorithm parameters from 0 to 1, and ③ the order in which the algorithm modules are used. By recording the algorithm parameter adjustment work performed by different users using the editing software, a sample set is constructed for training the parameter prediction model.

[0186] Step 102: Edit the video frame using the algorithm module of the editing software and the algorithm parameters in the algorithm parameter set corresponding to the algorithm module.

[0187] In some embodiments, the algorithm parameter set includes multiple algorithm parameters, one algorithm parameter corresponds to one algorithm module; the order of the multiple algorithm parameters is determined by any of the following methods:

[0188] Method 1: Determine the order of multiple algorithm parameters according to a pre-set order;

[0189] In implementation, according to the algorithm modules in the editing software, the execution order of each algorithm module is set, and according to the set execution order of each algorithm module, the order of the corresponding algorithm parameters is determined. One algorithm module is used to determine one algorithm parameter.

[0190] Mode 2: determining the order of algorithm parameters corresponding to each algorithm module according to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set.

[0191] Optionally, the order of algorithm parameters corresponding to each algorithm module is determined by the following steps:

[0192] Step 2a) counting the algorithm modules used by the sample video frame in each operation step;

[0193] Step 2b) in each operation step, according to the algorithm modules used in each sample video frame, determining the algorithm module most frequently used in the operation step;

[0194] Step 2c) Determine the order of algorithm parameters corresponding to each algorithm module according to the algorithm module with the highest frequency of use in each operation step.

[0195] Optionally, the order of algorithm parameters is determined based on frequency by the following steps:

[0196] Step 3a) determining a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step for each sample video frame;

[0197] Step 3b) for the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step;

[0198] Step 3c) According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0199] In implementation, the order of algorithm module usage is recorded during use, and all algorithm modules are sorted according to their usage frequency. The sorting rule is: retrieve the algorithm module that is used most in the first operation step as the first algorithm module for parameter prediction, retrieve the algorithm module that is used most in the second operation step after removing the first algorithm module determined in the first operation step, and use it as the second algorithm module for parameter prediction, and so on, and finally complete the sorting of all algorithm modules. Finally, this order is used as the order of model parameters output by the parameter prediction model, and the manually adjusted model parameters are used as the true value data for parameter prediction model training.

[0200] For example, taking the editing type of realizing 4K HDR as an example, the original video parameters of the input 2K SDR video frame include a resolution of 1920×1080, a color space of bt709, and a bit depth of 8 bits; the 2K SDR video frame needs to be edited to output a 4K HDR video frame, and the required video parameters of the output video frame include a resolution of 3840×2160, a color space of bt2020, a bit depth of 10 bits, and an HDR curve of HLG. The recorded user behavior is shown in Table 1 above. When determining the order of algorithm parameters, first retrieve the algorithm modules used in the first operation step corresponding to all scenes, which are: brightness, brightness, brightness, respectively. Then, it is determined that under this editing type, the first algorithm module used by the parameter prediction module is brightness; after removing brightness, the algorithm modules used in the second operation step are retrieved, which are: contrast, contrast, contrast, respectively. Then, it is determined that under this editing type, the second algorithm module used by the parameter prediction module is contrast; after removing brightness and contrast, the algorithm modules used in the third operation step are retrieved, which are: highlights, shadows, highlights. According to the most frequently used algorithm modules, statistics are performed to determine that under this editing type, the parameter prediction module The third algorithm module used by the module is highlight; after removing brightness, contrast, and highlight, the algorithm modules used in the fourth operation step are retrieved as shadow, highlight, and shadow, respectively. Then, it is determined that the fourth algorithm module used by the parameter prediction module under this editing type is shadow; the algorithm modules used in the fifth operation step are retrieved as temporal noise reduction, temporal noise reduction, and temporal noise reduction, respectively. Then, it is determined that the fifth algorithm module used by the parameter prediction module under this editing type is temporal noise reduction; the algorithm nodes used in the sixth operation step are retrieved as spatial noise reduction, spatial noise reduction, and blank, respectively. Then, it is determined that the sixth algorithm module used by the parameter prediction module under this editing type is spatial noise reduction. After determining the order of all algorithm modules, the order is used as the order of algorithm parameters output by the final parameter prediction model. If a scene does not have the algorithm module, for example, scene 3 does not have spatial noise reduction, the parameter of spatial noise reduction can be set to "0" (here 0 does not necessarily mean the value 0, but a parameter that has no effect on the current algorithm module. Temporal noise reduction may mean that parameter 0 does not work, and contrast may mean that parameter 1 does not work).

[0201] It should be noted that the algorithm parameters output by the parameter prediction model in this embodiment are normalized algorithm parameters, that is, the output algorithm parameters are all between 0 and 1.

[0202] In some embodiments, the parameter prediction model is trained in the following manner:

[0203] The sample video frames in the sample set are input into the parameter prediction model, and the loss function is determined according to the output results and the sample parameter set corresponding to the sample video frames; the model parameters in the parameter prediction model are adjusted using the loss function, and when the loss function value meets the preset conditions or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0204] During implementation, a sample video frame is input and the corresponding algorithm parameter set is output. Each algorithm parameter in the output algorithm parameter set is compared with each sample parameter in the sample parameter set. The loss function is calculated using the output algorithm parameters and the corresponding sample parameters, and the model parameters are adjusted based on the loss function value to train the parameter prediction model.

[0205] Optionally, the parameter prediction model includes a feature extraction network and a parameter mapping network, the feature extraction network includes a convolutional layer and a residual network, and the feature extraction network is used to extract image features of the input video frame; the parameter mapping network includes a global average pooling network and a fully connected layer, and the parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0206] In implementation, the parameter prediction model is used to perform feature analysis on the input video frame (image) and output the predicted algorithm parameters of each algorithm module. Figure 2 As shown in FIG. 1 , this embodiment provides an architecture diagram of a parameter prediction model, which consists of two parts. The first part is a feature extraction network, which is used to extract and analyze features of input video frames (images); the second part is a parameter mapping network, which can map the extracted image features to the final output algorithm parameter sequence w1, w2, ..., wn. Figure 3 As shown, this embodiment provides a detailed structural diagram of a parameter prediction model, in which a feature extraction network is composed of a convolutional layer and a residual network, and a parameter mapping network is composed of a global average pooling network and a fully connected layer.

[0207] The function loss in this embodiment is used to calculate the difference between the algorithm parameters output by the parameter prediction model and the sample parameters (true value data) obtained by manual adjustment by the user in the sample set, and optimize the parameter prediction model based on the difference. The loss function in this embodiment includes but is not limited to L1 (mean absolute error) loss and MSE (mean square error) loss. The formula of the loss function is as follows:

[0208]

[0209]

[0210] In formula (1), Loss_l1 represents the value of the L1 loss function, n represents the number of algorithm parameters, and y irepresents the i-th sample parameter, f(x) i Represents the i-th algorithm parameter of the output.

[0211] In formula (2), Loss_mse represents the MSE loss function value, n represents the number of algorithm parameters, and y i represents the i-th sample parameter, f(x) i Represents the i-th algorithm parameter of the output.

[0212] In some embodiments, since the parameter prediction model in this embodiment can output different algorithm parameters according to different scenes, when the acquired video frames include multiple scenes, the scene can be segmented first, and the video frames of the same scene can be used as a group for parameter prediction. The specific implementation steps are as follows:

[0213] Step 4a) grouping video frames containing scenes of the same type among the multiple video frames into a video frame group;

[0214] In implementation, a video stream may be input, and the video frames in the video stream may be segmented according to the scene, and the video frames of the same type of scene may be divided into one video frame group, and the video stream may be segmented into multiple video frame groups, and the scenes of the video frames included in each video frame group may belong to the same type. In this embodiment, the video frames of the same type of scene refer to the video frames whose contents have a similarity greater than a threshold.

[0215] Step 4b) selects N video frames from the video frame group, inputs the N video frames into the parameter prediction model respectively, and outputs N prediction algorithm parameter sets; N≥1, N is an integer.

[0216] In implementation, N video frames can be randomly selected from the video frame group for parameter prediction. After completing the parameter prediction of the N video frames, the weighted average of each prediction algorithm parameter in the N prediction algorithm parameter sets is calculated to obtain the algorithm parameter set corresponding to the scene.

[0217] Step 4c) Determine the algorithm parameter set corresponding to the video frame group based on the N prediction algorithm parameter sets.

[0218] Optionally, a final algorithm parameter set is determined based on N prediction algorithm parameter sets in the following manner:

[0219] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted sum is performed on the prediction algorithm parameters using the weights corresponding to each of the N prediction algorithm parameter sets to obtain the algorithm parameters corresponding to the prediction algorithm parameters; and the algorithm parameter set corresponding to the video frame group is determined based on the algorithm parameters corresponding to the various prediction algorithm parameters.

[0220] Optionally, the weights corresponding to the N prediction algorithm parameter sets in this embodiment are the same, that is, the algorithm parameter set corresponding to the scenario is obtained by taking a weighted average of the prediction algorithm parameters in the N prediction algorithm parameter sets.

[0221] like Figure 4 As shown, this embodiment provides an inference process of a parameter prediction model, including the steps of scene segmentation, parameter estimation, weighted averaging, etc. In implementation, the video stream is input into the editing software, and the scene segmentation is first performed, and the video frames of the same type of scenes are divided into one video frame group, and the video stream is divided into multiple video frame groups, and the scenes of the video frames contained in each video frame group belong to the same type. Secondly, N video frames are randomly selected from the video frame group for parameter prediction, and the N video frames are respectively input into the parameter prediction model, and N prediction algorithm parameter sets are output. Finally, the weighted average of each prediction algorithm parameter in the N prediction algorithm parameter sets is calculated to obtain the algorithm parameter set corresponding to the scene. For example, 5 video frames are randomly selected, and the brightness parameters corresponding to the first algorithm module of the 5 video frames are 0.12, 0.13, 0.12, 0.14, and 0.11, respectively. Then, the brightness parameters are weighted averaged, and the final initialized brightness parameters corresponding to the 5 video frames of the same scene are w1=(0.12+0.13+0.12+0.14+0.11) / 5=0.124.

[0222] In some embodiments, this embodiment further provides a display method, which is specifically as follows:

[0223] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0224] like Figure 5 As shown, this embodiment provides a schematic diagram of the application interface of an editing software, including a file list area, a screen display area, a video scene segment selection area, an all-algorithm module parameter adjustment area, an oscilloscope display area, an initialized / used algorithm module processing flow area, and an initialized algorithm module list area. Among them, the initialized / used algorithm module processing flow area is a display area for all automatically initialized algorithm modules to process, and the initialized algorithm module list area is used to display the names of all initialized algorithm modules. A selection button is set in front of the name. A black dot indicates that the algorithm module is selected to be used, and no black dot indicates that the algorithm module is not used.

[0225] In some embodiments, the algorithm parameter set includes multiple algorithm parameters, one algorithm parameter corresponds to one algorithm module; the multiple algorithm parameters are displayed in sequence; and the algorithm modules corresponding to the multiple algorithm parameters are displayed in the sequence of the multiple algorithm parameters.

[0226] In implementation, algorithm parameters and algorithm module partitions are displayed. Figure 6As shown, this embodiment provides an editing software display content refinement display interface. Among them, the initialized / used algorithm module processing flow area is a display area for all automatically initialized algorithm modules to process, and the direction of the algorithm module processing flow is indicated by an arrow. The direction of the arrow indicates the processing order of the algorithm module. The algorithm module displayed in this area has been initialized for the algorithm parameters of the current input video frame according to the algorithm parameters output by the parameter prediction model. The initialized algorithm module list area is used to display the names of all initialized algorithm modules. A selection button is set in front of the name. A black dot indicates that the algorithm module is selected to be used, and no black dot indicates that the algorithm module is not used.

[0227] In some embodiments, after obtaining the initialized algorithm parameters using the parameter prediction model, the algorithm module and the corresponding algorithm parameters may be deleted according to actual needs by any one or more of the following methods:

[0228] (1) In response to a second instruction from the user, delete the algorithm module and corresponding algorithm parameters indicated by the second instruction.

[0229] by Figure 6 For example, if you do not want to use a certain algorithm module, you can directly delete this algorithm module. For example, the names of all initialized algorithm modules are displayed in the initialized algorithm module list area, and a selection button is set in front of the name. A black dot indicates that the algorithm module is selected, and no black dot indicates that the algorithm module is not used. The displayed algorithm module is used by default. Figure 7 As shown, this embodiment provides a display effect when the saturation algorithm is not used. The user can delete the saturation algorithm module and its corresponding parameters by clicking the button in front of the saturation algorithm (second instruction).

[0230] (2) in response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold;

[0231] Optionally, the initialized algorithm module list area also includes a parameter optimization option. The parameter optimization option is used to indicate whether to limit the numerical range of the algorithm parameters output by the parameter prediction model. For example, when the parameter optimization option is selected (the first instruction), the algorithm modules whose algorithm parameters are less than a threshold value can be deleted. If the algorithm parameters output by the parameter prediction model are less than the threshold value, it is considered that the algorithm module has not played a much improvement role. Optionally, the threshold value can be set at around 0.05, between 0 and 0.1. Not selecting the parameter optimization option means using all algorithm modules whose algorithm parameter outputs are not 0. Selecting the parameter optimization option means deleting the algorithm modules whose algorithm parameters are less than the threshold value. Figure 8 As shown, this embodiment provides a schematic diagram for selecting parameter optimization options (the two algorithm modules of hue and saturation are removed).

[0232] Optionally, the algorithm parameters and algorithm modules are displayed in order; according to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated. Figure 7 For example, when the user does not select saturation, delete the saturation algorithm module in the initialized / used algorithm module processing flow area, and redraw the connection arrows between the algorithm modules in the order after deletion; Figure 8 For example, after selecting parameter optimization, delete the two algorithm modules of hue and saturation in the initialized / used algorithm module processing flow area and the initialized algorithm module list area, redraw the connection arrows between the algorithm modules in the order after deletion, and update the display order of the algorithm modules.

[0233] It should be noted that the video frame in this embodiment is also understood as an image. The editing software in this embodiment includes but is not limited to image / video editing software. The parameter prediction model in this embodiment can be an application independent of the editing software, or it can be a functional module integrated in the editing software, and this embodiment does not make too many restrictions on this.

[0234] like Fig. 9 As shown, this embodiment provides a system flow for training and reasoning a parameter prediction model, inputs a sample video frame into the editing software, records the user's behavior of manually adjusting the algorithm parameters through the behavior recording module, obtains a sample set (including editing type, algorithm parameter information, scene type, use order of algorithm modules, etc.), inputs the sample set into the initial parameter prediction model to train the parameter prediction model, and adjusts the model parameters through loss function calculation to obtain a trained parameter prediction model. Input the video frame into the trained parameter prediction model, output the initialized algorithm parameters, the editing software obtains the initialized algorithm parameters, and the user can edit the input video frame based on the initialized algorithm parameters to obtain the corresponding effect.

[0235] The present embodiment provides a solution for automatically initializing the algorithm parameters of the algorithm modules of the image / video editing software. By designing a user behavior record database, matching input images / videos and the algorithm parameter values ​​of each algorithm module are obtained, and a parameter prediction model is established. The algorithm module to be used and the optimal algorithm parameters of the algorithm module can be automatically predicted based on the input image / video. Even if different materials are constantly changed, different algorithm parameters can still be initialized according to different materials, which greatly saves labor costs.

[0236] Example 2: Based on the same inventive concept, the embodiment of the present invention also provides an electronic device. Since the electronic device is the electronic device in the method in the embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0237] like Fig.10 As shown, the electronic device includes a processor 1000 and a memory 1001, wherein the memory 1001 is used to store a program executable by the processor 1000, and the processor 1000 is used to read the program in the memory 1001 and perform the following steps:

[0238] Acquire a video frame and an editing type corresponding to the video frame, determine a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions;

[0239] Input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0240] The video frame is edited using the algorithm module of the editing software and the algorithm parameters in the algorithm parameter set corresponding to the algorithm module.

[0241] As an optional implementation manner, the acquired video frame includes multiple frames, and the processor 1000 is specifically further configured to execute:

[0242] Grouping video frames containing scenes of the same type in a plurality of video frames into a video frame group;

[0243] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N≥1, N is an integer;

[0244] According to the N prediction algorithm parameter sets, an algorithm parameter set corresponding to the video frame group is determined.

[0245] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0246] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summing is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0247] According to the algorithm parameters corresponding to the various prediction algorithm parameters, an algorithm parameter set corresponding to the video frame group is determined.

[0248] As an optional implementation, the weights corresponding to the N prediction algorithm parameter sets are the same.

[0249] As an optional implementation manner, the processor 1000 is specifically configured to obtain the sample set in the following manner:

[0250] determining the type of editing performed by the user on the sample video frame using the editing software;

[0251] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0252] The sample set is determined according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0253] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0254] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0255] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0256] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the processor 1000 is specifically configured to determine the order of the multiple algorithm parameters in the following manner:

[0257] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0258] According to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0259] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0260] Counting the algorithm modules used in each operation step of the sample video frame;

[0261] In each operation step, according to the algorithm modules used by each sample video frame, determining the algorithm module most frequently used in the operation step;

[0262] According to the algorithm module with the highest frequency of use in each operation step, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0263] As an optional implementation manner, the processor 1000 is specifically configured to execute:

[0264] Determine a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step of each sample video frame;

[0265] For the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step;

[0266] According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0267] As an optional implementation, the processor 1000 is specifically configured to train the parameter prediction model in the following manner:

[0268] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function according to an output result and a sample parameter set corresponding to the sample video frame;

[0269] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0270] As an optional implementation,

[0271] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of an input video frame. The parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0272] As an optional implementation manner, the processor 1000 is further configured to execute:

[0273] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0274] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the processor 1000 is further configured to execute:

[0275] The multiple algorithm parameters are displayed in order;

[0276] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0277] As an optional implementation manner, the processor 1000 is further configured to execute:

[0278] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0279] In response to a second instruction from the user, the algorithm module and the corresponding algorithm parameters indicated by the second instruction are deleted.

[0280] As an optional implementation manner, the algorithm parameters and algorithm modules are displayed in sequence; the processor 1000 is further configured to execute:

[0281] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0282] Based on the same inventive concept, an embodiment of the present invention also provides a device for initializing algorithm parameters. Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0283] like Fig.11 As shown, the device comprises:

[0284] The prediction model determination unit 1100 is used to obtain a video frame and an editing type corresponding to the video frame, and determine a parameter prediction model corresponding to the editing type. The model parameters of the parameter prediction model corresponding to different editing types are different, and different editing types are used to implement different video editing functions.

[0285] The algorithm parameter prediction unit 1101 is used to input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames;

[0286] The video editing processing unit 1102 is used to edit the video frame by using the algorithm module of the editing software and the algorithm parameters corresponding to the algorithm module in the algorithm parameter set.

[0287] As an optional implementation manner, the acquired video frame includes multiple frames, and the prediction model determination unit 1100 is further specifically used for:

[0288] Grouping video frames containing scenes of the same type in a plurality of video frames into a video frame group;

[0289] Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N≥1, N is an integer;

[0290] According to the N prediction algorithm parameter sets, an algorithm parameter set corresponding to the video frame group is determined.

[0291] As an optional implementation manner, the prediction model determination unit 1100 is specifically used for:

[0292] For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summing is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain an algorithm parameter corresponding to the prediction algorithm parameter;

[0293] According to the algorithm parameters corresponding to the various prediction algorithm parameters, an algorithm parameter set corresponding to the video frame group is determined.

[0294] As an optional implementation, the weights corresponding to the N prediction algorithm parameter sets are the same.

[0295] As an optional implementation, the algorithm parameter prediction unit 1101 is specifically configured to obtain a sample set in the following manner:

[0296] determining the type of editing performed by the user on the sample video frame using the editing software;

[0297] Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module;

[0298] The sample set is determined according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

[0299] As an optional implementation manner, the algorithm parameter prediction unit 1101 is specifically used for:

[0300] The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

[0301] As an optional implementation, the algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

[0302] As an optional implementation, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the algorithm parameter prediction unit 1101 is further specifically used to determine the order of multiple algorithm parameters in the following manner:

[0303] Determine the order of multiple algorithm parameters in a pre-set order; or,

[0304] According to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0305] As an optional implementation manner, the algorithm parameter prediction unit 1101 is specifically used for:

[0306] Counting the algorithm modules used in each operation step of the sample video frame;

[0307] In each operation step, according to the algorithm modules used by each sample video frame, determining the algorithm module most frequently used in the operation step;

[0308] According to the algorithm module with the highest frequency of use in each operation step, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0309] As an optional implementation manner, the algorithm parameter prediction unit 1101 is specifically used for:

[0310] Determine a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step of each sample video frame;

[0311] For the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step;

[0312] According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

[0313] As an optional implementation, the algorithm parameter prediction unit 1101 is specifically used to train the parameter prediction model in the following manner:

[0314] Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function according to an output result and a sample parameter set corresponding to the sample video frame;

[0315] The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

[0316] As an optional implementation,

[0317] The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of an input video frame. The parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

[0318] As an optional implementation, the display unit is further included to:

[0319] The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

[0320] As an optional implementation manner, the algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the display unit is further specifically used for:

[0321] The multiple algorithm parameters are displayed in order;

[0322] The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

[0323] As an optional implementation manner, the display unit is further used for:

[0324] In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or,

[0325] In response to a second instruction from the user, the algorithm module and the corresponding algorithm parameters indicated by the second instruction are deleted.

[0326] As an optional implementation manner, the algorithm parameters and algorithm modules are displayed in sequence; and the display unit is further configured to:

[0327] According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

[0328] Based on the same inventive concept, an embodiment of the present disclosure provides a computer storage medium, the computer storage medium comprising: a computer program code, when the computer program code is executed on a computer, the computer executes any of the algorithm parameter initialization methods discussed above. Since the principle of solving the problem by the above-mentioned computer storage medium is similar to the method of initializing the algorithm parameters, the implementation of the above-mentioned computer storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.

[0329] In the specific implementation process, the computer storage medium may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other storage media that can store program codes.

[0330] Based on the same inventive concept, the embodiment of the present disclosure also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes any of the algorithm parameter initialization methods discussed above. Since the principle of solving the problem by the above computer program product is similar to the method of initializing the algorithm parameters, the implementation of the above computer program product can refer to the implementation of the method, and the repeated parts will not be repeated.

[0331] The computer program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0332] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0333] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that has the functions specified in one or more boxes.

[0334] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0335] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0336] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for initializing algorithm parameters, characterized in that: The method includes: Acquire a video frame and an editing type corresponding to the video frame, determine a parameter prediction model corresponding to the editing type, wherein different editing types correspond to different model parameters of the parameter prediction model, and different editing types are used to implement different video editing functions; Input the video frame into the parameter prediction model, and output an algorithm parameter set corresponding to the scene contained in the video frame, wherein the parameter prediction model is obtained by training with a training set, wherein the training set includes sample sets corresponding to different editing types, and the sample sets corresponding to the same editing type include sample video frames of different types of scenes and sample parameter sets corresponding to the sample video frames; The video frame is edited using the algorithm module of the editing software and the algorithm parameters in the algorithm parameter set corresponding to the algorithm module.

2. The method according to claim 1, characterized in that The acquired video frames include a plurality of frames, and the method further includes: Grouping video frames containing scenes of the same type in a plurality of video frames into a video frame group; Selecting N video frames from the video frame group, inputting the N video frames into the parameter prediction model respectively, and outputting N prediction algorithm parameter sets; N≥1, N is an integer; According to the N prediction algorithm parameter sets, an algorithm parameter set corresponding to the video frame group is determined.

3. The method according to claim 2, characterized in that The step of determining the algorithm parameter set corresponding to the video frame group according to the N prediction algorithm parameter sets includes: For each prediction algorithm parameter in the N prediction algorithm parameter sets, weighted summing is performed on the prediction algorithm parameters using the weights corresponding to the N prediction algorithm parameter sets to obtain an algorithm parameter corresponding to the prediction algorithm parameter; According to the algorithm parameters corresponding to the various prediction algorithm parameters, an algorithm parameter set corresponding to the video frame group is determined.

4. The method according to claim 3, characterized in that The N prediction algorithm parameter sets each have the same corresponding weight.

5. The method according to claim 1, characterized in that Get the sample set as follows: determining the type of editing performed by the user on the sample video frame using the editing software; Recording the algorithm module used by the user when editing the sample video frame using the editing software, and the algorithm parameter information obtained by editing the sample video frame using the algorithm module; The sample set is determined according to the editing type of the sample video frame, the type of scene included in the sample video frame, the algorithm module used, and the algorithm parameter information.

6. The method according to claim 5, characterized in that The determining the editing type of the sample video frame edited by the user using the editing software includes: The editing type corresponding to the sample video frame is determined according to the original video parameters of the sample video frame and the required video parameters of the sample video frame set by the user.

7. The method according to claim 5, characterized in that The algorithm parameter information corresponding to the algorithm module includes at least one of an algorithm parameter value, a maximum algorithm parameter value, a minimum algorithm parameter value, and a normalized algorithm parameter value, and the normalized algorithm parameter value is obtained by normalizing the algorithm parameter value.

8. The method according to claim 1, characterized in that The algorithm parameter set includes multiple algorithm parameters, one algorithm parameter corresponds to one algorithm module; the order of the multiple algorithm parameters is determined in the following manner: Determine the order of multiple algorithm parameters in a pre-set order; or, According to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set, the order of the algorithm parameters corresponding to each algorithm module is determined.

9. The method according to claim 8, characterized in that The step of determining the order of algorithm parameters corresponding to each algorithm module according to the usage frequency of each algorithm module corresponding to each sample video frame in the sample set includes: Counting the algorithm modules used in each operation step of the sample video frame; In each operation step, according to the algorithm modules used by each sample video frame, determining the algorithm module most frequently used in the operation step; According to the algorithm module with the highest frequency of use in each operation step, the order of the algorithm parameters corresponding to each algorithm module is determined.

10. The method according to claim 9, characterized in that The step of determining the order of algorithm parameters corresponding to each algorithm module according to the algorithm module with the highest frequency of use in each operation step includes: Determine a first algorithm module according to the algorithm module with the highest frequency of use in the first operation step of each sample video frame; For the second operation step and subsequent operation steps, remove the algorithm modules determined before the current operation step from the algorithm modules used by each sample video frame corresponding to the current operation step, and determine the current algorithm module from the remaining algorithm modules according to the algorithm module with the highest frequency of use in each sample video frame in the current operation step; According to the determined order of each algorithm module, the order of the algorithm parameters corresponding to each algorithm module is determined.

11. The method according to claim 1, characterized in that: The parameter prediction model is trained in the following manner: Inputting a sample video frame in the sample set into the parameter prediction model, and determining a loss function according to an output result and a sample parameter set corresponding to the sample video frame; The loss function is used to adjust the model parameters in the parameter prediction model. When the loss function value meets a preset condition or the number of training iterations reaches a threshold, it is determined that the parameter prediction model training is completed.

12. The method according to claim 1, characterized in that The parameter prediction model includes a feature extraction network and a parameter mapping network. The feature extraction network includes a convolutional layer and a residual network. The feature extraction network is used to extract image features of an input video frame. The parameter mapping network includes a global average pooling network and a fully connected layer. The parameter mapping network is used to map the extracted image features into output algorithm parameters.

13. The method according to any one of claims 1 to 12, characterized in that: The method further includes: The algorithm module corresponding to the video frame and the algorithm parameters corresponding to the algorithm module are displayed.

14. The method according to claim 13, characterized in that The algorithm parameter set includes multiple algorithm parameters, and one algorithm parameter corresponds to one algorithm module; the method further includes: The multiple algorithm parameters are displayed in order; The algorithm modules corresponding to the various algorithm parameters are displayed in the order of the various algorithm parameters.

15. The method according to claim 13, characterized in that The method further includes: In response to a first instruction from a user, deleting algorithm parameters and corresponding algorithm modules that are less than a threshold; and / or, In response to a second instruction from the user, the algorithm module and the corresponding algorithm parameters indicated by the second instruction are deleted.

16. The method according to claim 15, characterized in that The algorithm parameters and algorithm modules are displayed in order; further comprising: According to the deleted algorithm parameters, the display order of each algorithm parameter is updated, and according to the deleted algorithm modules, the display order of each algorithm module is updated.

17. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and execute the steps of any one of the methods of claims 1 to 16.

18. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images