Generation of sound special effects program file package and sound special effects generation method and device

Abstract

The embodiment of the present invention discloses a method and device for generating a sound special effect program file package and a sound special effect generation method, wherein the method for generating the sound special effect program file package includes: importing sound material; obtaining parameter values ​​of playback parameters of the sound material; and generating the sound special effect program file package according to the sound material and the parameter values ​​of the playback parameters. The embodiment of the present invention can generate a sound special effect program file executable by a rendering engine without manually writing a program file, and the operation is simple and the time required is short, which improves the overall efficiency of the sound special effect realization, avoids the errors that may occur in manually writing program files, and effectively ensures the accuracy of the special effect.

Description

Technical Field

[0001] The present invention relates to computer vision technology, in particular to a method and device for generating a sound special effect program file package and a sound special effect generating method. Background Art

[0002] Augmented Reality (AR) is a new technology that seamlessly integrates real-world information and virtual-world information. It simulates and superimposes virtual information on physical information that was originally within a certain time and space range in the real world, applies virtual information to the real world, and superimposes real-world characters, environments, and virtual objects on the same screen or space in real time, thereby achieving a sensory experience beyond reality. Summary of the invention

[0003] The embodiment of the present invention provides a technical solution for generating a sound special effect program file package and generating a sound special effect.

[0004] According to one aspect of an embodiment of the present invention, a method for generating a sound special effect program file package is provided, comprising:

[0005] Import sound materials;

[0006] Acquiring parameter values ​​of playback parameters of the sound material, and establishing a corresponding relationship between a display position of the sound material and at least one predetermined key point;

[0007] A sound special effect program file package is generated according to the sound material and the parameter values ​​of the playing parameters.

[0008] Optionally, in the above method embodiments of the present invention, the importing of sound material includes:

[0009] An import instruction input through the interactive interface of the operation bar is received, and the sound material in the material folder pointed to by the import instruction is imported.

[0010] Optionally, in the above-mentioned method embodiments of the present invention, the receiving of the import instruction input through the interactive interface of the operation bar, and importing the sound material in the material folder pointed to by the import instruction includes:

[0011] An import instruction sent through an interactive interface in the special effect parameter setting interface under the operation bar is received, and the sound material in the material folder pointed to by the import instruction is imported.

[0012] Optionally, in the above-mentioned method embodiments of the present invention, receiving an import instruction sent through an interactive interface, and importing the sound material in the material folder pointed to by the import instruction includes:

[0013] Receiving an import instruction sent through the interactive interface, and acquiring and displaying a material folder pointed to by the import instruction;

[0014] In response to receiving a selection operation for a sound material in the material folder, importing the sound material selected by the sound material selection operation; and / or

[0015] In response to not receiving an operation of selecting the sound material in the material folder, one or more sound materials in the material folder are selected according to a preset setting, and the sound material selected according to the preset setting is imported.

[0016] Optionally, in the above method embodiments of the present invention, obtaining the parameter value of the playback parameter of the sound material includes:

[0017] In response to receiving a parameter value of a playback parameter setting for the sound material sent through an interactive interface in the special effect parameter setting interface, using the set parameter value as the parameter value of the playback parameter of the sound material; and / or

[0018] In response to not receiving the parameter value for the playback parameter setting of the sound material sent through the interactive interface in the special effect parameter setting interface, using the preset parameter value as the parameter value of the playback parameter of the sound material.

[0019] Optionally, in the above method embodiments of the present invention, the playback parameters include any one or more of the following:

[0020] Playback control parameters: used to control whether the sound material is played;

[0021] Trigger mode parameter: used to indicate the trigger event that triggers the playing of the sound material;

[0022] Loop parameter: used to indicate the number of times the sound material is played in a loop;

[0023] Delay trigger parameter: used to indicate the time for delaying the playing of the sound material;

[0024] Trigger end parameter: used to indicate the trigger event for ending the playing of the sound material;

[0025] Playback frame number parameter: used to indicate the number of frames the sound material is played;

[0026] Beauty / makeup / face-changing effect parameters: used to indicate the beauty / makeup / face-changing effect displayed at a preset position when playing the sound material;

[0027] Deformation special effect parameter: used to indicate the deformation effect of generating a deformation area on the image when playing the sound material;

[0028] The sticker special effect parameter is used to indicate the special effect of generating a sub-material on the image when playing the sound material.

[0029] Optionally, in the above-mentioned method embodiments of the present invention, the trigger event includes any one or more of the following: no event trigger, eye movement, head movement, eyebrow movement, hand movement, mouth movement, shoulder movement, body movement, deformation special effects, sticker special effects, beauty / makeup / face-changing special effects.

[0030] Optionally, in the above-mentioned method embodiments of the present invention, the sound special effect program file package includes: a sound special effect program file package generated by a json program.

[0031] Optionally, in the above method embodiments of the present invention, the method further includes:

[0032] According to the received start-up instruction, an operation interface is displayed, wherein the operation interface includes: an operation bar, a content display bar and / or a program file bar.

[0033] Optionally, in the above method embodiments of the present invention, the operation interface includes three areas: left, middle and right;

[0034] The display operation interface includes:

[0035] The operation bar is displayed on the left side of the operation interface, the content display bar is displayed in the middle of the operation interface, and the program file bar is displayed on the right side of the operation interface.

[0036] Optionally, in the above-mentioned method embodiments of the present invention, after generating the sound special effect program file package, the method further includes:

[0037] The sound special effect program file package is saved at the location pointed to by the saving instruction according to the received saving instruction.

[0038] Optionally, in the above-mentioned method embodiments of the present invention, the step of saving the sound special effect program file package at the location pointed to by the saving instruction according to the received saving instruction includes:

[0039] In response to receiving the save instruction, displaying a save path selection interface and a compression interface;

[0040] Receiving the save location sent through the save path selection interface; and receiving the compression method sent based on the compression interface, and compressing the sound special effect program file package according to the compression method to generate a compressed file package;

[0041] The compressed file package is stored in the folder pointed to by the saving location.

[0042] Optionally, in the above-mentioned method embodiments of the present invention, during the process of compressing the sound special effect program file package, the size of the sound material in the sound special effect program file package remains unchanged at the size before being imported.

[0043] According to another aspect of an embodiment of the present invention, a method for generating sound effects is provided, comprising:

[0044] Obtaining the sound material in the sound special effect program file package and the parameter values ​​of the playback parameters of the sound material;

[0045] Based on the parameter values ​​of the playback parameters of the sound material, a sound effect of the sound material is generated on the image.

[0046] Optionally, in the above-mentioned embodiments of the method for generating sound special effects of the present invention, before obtaining the sound material in the sound special effects program file package and the parameter value of the playback parameter of the sound material, the method further includes:

[0047] Importing the sound special effects program file package;

[0048] The step of obtaining the sound material in the sound effect program file package and the parameter values ​​of the playback parameters of the sound material comprises: obtaining the sound material in the sound effect program file package and the parameter values ​​of the playback parameters of the sound material.

[0049] Optionally, in the above-mentioned embodiments of the sound special effects generation method of the present invention, the sound special effects program file package is generated using the method for generating the sound special effects program file package as described in any of the above-mentioned embodiments of the present application.

[0050] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the image includes any one or more of the following: a static image, an image in a video.

[0051] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameter includes: a playback control parameter, the playback control parameter is used to control whether the sound material is played;

[0052] The step of generating a sound effect of the sound material on an image based on the parameter value of the playback parameter of the sound material comprises:

[0053] In response to the parameter value of the playback control parameter being a parameter value for controlling the playback of the sound material, a sound effect of the sound material is generated on the image based on the parameter value of the playback parameter of the sound material.

[0054] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playing parameters include: a trigger mode parameter, the trigger mode parameter is used to indicate a trigger event for triggering the playing of the sound material;

[0055] The method further comprises: detecting whether a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image;

[0056] The step of generating a sound effect of the sound material on an image based on the parameter value of the playback parameter of the sound material comprises:

[0057] In response to detecting that a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image, a sound effect of the sound material is generated on the image based on the parameter value of the playback parameter of the sound material.

[0058] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameter includes: a delay trigger parameter, the delay trigger parameter is used to indicate the time for delaying the playback of the sound material;

[0059] The step of generating a sound effect of the sound material on an image based on the parameter value of the playback parameter of the sound material comprises:

[0060] In response to satisfying the playback conditions of the sound material, based on the parameter value of the playback parameter of the sound material, and in accordance with the delayed playback time corresponding to the parameter value of the delay trigger parameter, the sound effects of the sound material on the image in the video are delayed; the playback conditions of the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0061] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameter includes: a loop parameter, the loop parameter is used to indicate the number of loop playback times of the sound material;

[0062] The step of generating a sound effect of the sound material on an image based on the parameter value of the playback parameter of the sound material comprises:

[0063] In response to satisfying the playback conditions of the sound material, based on the parameter value of the playback parameter of the sound material, and according to the number of loops corresponding to the parameter value of the loop parameter, the sound material is looped on the image in the video to generate sound effects for the sound material; the playback conditions of the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0064] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameter includes: a playback frame number parameter, the playback frame number parameter is used to indicate how many frames the sound material is played;

[0065] The step of generating a sound effect of the sound material on an image based on the parameter value of the playback parameter of the sound material comprises:

[0066] In response to satisfying the playback conditions of the sound material, based on the parameter values ​​of the playback parameters of the sound material and according to the playback frames corresponding to the parameter value of the playback frame number parameter, a sound effect of the sound material is generated on an image corresponding to the playback frame number in the video; the playback conditions of the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0067] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameter includes: a trigger end parameter, the trigger end parameter is used to indicate a trigger event for ending the playback of the sound material;

[0068] The method further comprises:

[0069] Detecting whether a trigger event corresponding to the parameter value of the trigger end parameter appears in the image;

[0070] In response to detecting that a trigger event corresponding to the parameter value of the trigger end parameter appears in the image, the generation of the sound effect of the sound material on the image is stopped.

[0071] Optionally, in the above-mentioned embodiments of the sound special effects generation method of the present invention, the playback parameters include: beauty / makeup / face-changing effect parameters, and the beauty / makeup / face-changing effect parameters are used to indicate the beauty / makeup / face-changing effect displayed at a preset position when playing the sound material;

[0072] The method further comprises:

[0073] When generating the sound effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material, the beautification / makeup / face-changing effect is displayed at the preset part of the image according to the beautification / makeup / face-changing effect parameters.

[0074] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameters include: deformation special effect parameters, the deformation special effect parameters are used to indicate the deformation effect of generating a deformation area on the image when playing the sound material;

[0075] The method further comprises:

[0076] When generating the sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material, the deformation effect of the deformation area is generated in the image according to the deformation special effect parameter.

[0077] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the playback parameters include: sticker special effect parameters, the sticker special effect parameters are used to indicate the special effect of generating a sub-material on an image when playing the sound material;

[0078] The method further comprises:

[0079] When generating the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material, the special effects of the sub-material are generated in the image according to the sticker special effect parameters.

[0080] Optionally, in the above-mentioned embodiments of the sound special effect generation method of the present invention, the importing of the sound special effect program file package includes:

[0081] The sound special effect program file package is read into the memory by calling an interface function for reading the file package;

[0082] The sound special effect program file package is parsed to obtain the sound material and the sound special effect program file, wherein the sound special effect program file includes parameter values ​​of playback parameters of the sound material.

[0083] Optionally, in the above-mentioned embodiments of the method for generating sound special effects of the present invention, the step of obtaining the parameter value of the playback parameter of the sound material in the special effect program file package includes:

[0084] Create a sticker handle through the interface function used to create a sticker handle;

[0085] The sound material and the parameter values ​​of the playback parameters are read and stored in the sticker handle.

[0086] Optionally, in the above-mentioned sound special effect generation method embodiments of the present invention, generating the sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material includes:

[0087] By calling an interface function for rendering sound material, the parameter value of the playing parameter and the sound material to be played are read from the sticker handle;

[0088] Based on the parameter values ​​of the playback parameters of the sound material, a sound effect of the sound material is generated on the image.

[0089] Optionally, in the above-mentioned embodiments of the method for generating sound special effects of the present invention, when the image is an image in a video, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material further includes:

[0090] According to the parameter value of the playback parameter, the number of video frames required to generate the sound effect of the sound material is obtained, and images corresponding to the number of video frames are read from the video in advance.

[0091] Optionally, in the above-mentioned embodiments of the method for generating sound special effects of the present invention, the method further includes:

[0092] In response to the sound special effect program file package being played, the sticker handle is destroyed by calling an interface function for destroying the sticker handle.

[0093] Optionally, in the above-mentioned sound special effect generation method embodiments of the present invention, the step of generating the sound special effect of the sound material on the image includes:

[0094] By calling a built-in player or an external player of the electronic device, a sound effect of the sound material is generated on the image.

[0095] According to another aspect of an embodiment of the present invention, a device for generating a sound special effect program file package is provided, comprising:

[0096] The first import module is used to import sound materials;

[0097] A first acquisition module, used to acquire parameter values ​​of playback parameters of the sound material;

[0098] The first generating module is used to generate a sound special effect program file package according to the sound material and the parameter value of the playing parameter.

[0099] Optionally, in the above-mentioned device embodiments of the present invention, the first import module is specifically used to receive an import instruction input through the interactive interface of the operation bar, and import the sound material in the material folder pointed to by the import instruction.

[0100] Optionally, in the above-mentioned device embodiments of the present invention, the first import module is specifically used to receive an import instruction sent through the interactive interface in the special effect parameter setting interface under the operation bar, and import the sound material in the material folder pointed to by the import instruction.

[0101] Optionally, in the above-mentioned device embodiments of the present invention, the first import module is specifically used for:

[0102] Receiving an import instruction sent through the interactive interface, and acquiring and displaying a material folder pointed to by the import instruction;

[0103] In response to receiving a selection operation for a sound material in the material folder, importing the sound material selected by the sound material selection operation; and / or

[0104] In response to not receiving an operation of selecting the sound material in the material folder, one or more sound materials in the material folder are selected according to a preset setting, and the sound material selected according to the preset setting is imported.

[0105] Optionally, in the above-mentioned device embodiments of the present invention, the first acquisition module is specifically used to:

[0106] In response to receiving a parameter value of a playback parameter setting for the sound material sent through an interactive interface in the special effect parameter setting interface, using the set parameter value as the parameter value of the playback parameter of the sound material; and / or

[0107] In response to not receiving the parameter value for the playback parameter setting of the sound material sent through the interactive interface in the special effect parameter setting interface, using the preset parameter value as the parameter value of the playback parameter of the sound material.

[0108] Optionally, in the above-mentioned device embodiments of the present invention, the playback parameters include any one or more of the following:

[0109] Playback control parameters: used to control whether the sound material is played;

[0110] Trigger mode parameter: used to indicate the trigger event that triggers the playing of the sound material;

[0111] Loop parameter: used to indicate the number of times the sound material is played in a loop;

[0112] Delay trigger parameter: used to indicate the time for delaying the playing of the sound material;

[0113] Trigger end parameter: used to indicate the trigger event for ending the playing of the sound material;

[0114] Playback frame number parameter: used to indicate the number of frames the sound material is played;

[0115] Beauty / makeup / face-changing effect parameters: used to indicate the beauty / makeup / face-changing effect displayed at a preset position when playing the sound material;

[0116] Deformation special effect parameter: used to indicate the deformation effect of generating a deformation area on the image when playing the sound material;

[0117] The sticker special effect parameter is used to indicate the special effect of generating a sub-material on the image when playing the sound material.

[0118] Optionally, in the above-mentioned device embodiments of the present invention, the trigger event includes any one or more of the following: no event trigger, eye movement, head movement, eyebrow movement, hand movement, mouth movement, shoulder movement, body movement, deformation special effects, sticker special effects, beauty / makeup / face-changing special effects.

[0119] Optionally, in the above-mentioned device embodiments of the present invention, the sound special effect program file package includes: a sound special effect program file package generated by a json program.

[0120] Optionally, in the above-mentioned device embodiments of the present invention, the present invention further includes:

[0121] The operation interface includes: an operation bar, a content display bar and / or a program file bar.

[0122] Optionally, in the above-mentioned device embodiments of the present invention, the operation interface includes three areas: left, middle and right;

[0123] The operation bar is displayed on the left side of the operation interface, the content display bar is displayed in the middle of the operation interface, and the program file bar is displayed on the right side of the operation interface.

[0124] Optionally, in the above-mentioned device embodiments of the present invention, the present invention further includes:

[0125] The saving module is used to save the sound special effect program file package at the location pointed to by the saving instruction according to the received saving instruction.

[0126] Optionally, in the above-mentioned device embodiments of the present invention, the storage module is specifically used for:

[0127] In response to receiving the save instruction, displaying a save path selection interface and a compression interface;

[0128] Receiving the save location sent through the save path selection interface; and receiving the compression method sent based on the compression interface, and compressing the sound special effect program file package according to the compression method to generate a compressed file package;

[0129] The compressed file package is stored in the folder pointed to by the saving location.

[0130] Optionally, in the above-mentioned device embodiments of the present invention, during the process of compressing the sound special effect program file package, the size of the sound material in the sound special effect program file package remains unchanged at the size before being imported.

[0131] According to another aspect of an embodiment of the present invention, a sound special effect generating device is provided, comprising:

[0132] A second acquisition module is used to acquire the sound material in the sound special effect program file package and the parameter values ​​of the playback parameters of the sound material;

[0133] The second generating module is used to generate the sound effects of the sound material on the image based on the parameter values ​​of the playing parameters of the sound material.

[0134] Optionally, in each of the above-mentioned sound special effect generating device embodiments of the present invention, the present invention further includes:

[0135] A second import module, used for importing the sound special effect program file package;

[0136] The second acquisition module is specifically used to obtain the sound material in the sound special effect program file package and the parameter value of the playback parameter of the sound material.

[0137] Optionally, in each of the above-mentioned sound special effect generating device embodiments of the present invention, the sound special effect program file package is generated using the sound special effect program file package generation method or device described in any of the above-mentioned embodiments of the present invention.

[0138] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the image includes any one or more of the following: a static image, an image in a video.

[0139] Optionally, in each of the above-mentioned sound special effect generating device embodiments of the present invention, the playback parameter includes: a playback control parameter, the playback control parameter is used to control whether the sound material is played;

[0140] The second generating module is specifically used to generate a sound effect of the sound material on an image based on the parameter value of the playing parameter of the sound material in response to the parameter value of the playing control parameter being a parameter value for controlling the playing of the sound material.

[0141] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameters include: trigger mode parameters, the trigger mode parameters are used to indicate a triggering event for triggering the playing of the sound material;

[0142] The device further comprises: a first detection module, configured to detect whether a trigger event corresponding to a parameter value of the trigger mode parameter appears in the image;

[0143] The second generating module is specifically used to generate sound effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material in response to the first detecting module detecting a trigger event corresponding to the parameter value of the trigger mode parameter appearing in the image.

[0144] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameters include: a delay trigger parameter, the delay trigger parameter is used to indicate the time of delaying the playing of the sound material;

[0145] The second generating module is specifically used to delay the generation of the sound effects of the sound material on the image in the video in response to the playback condition of the sound material being met, based on the parameter value of the playback parameter of the sound material and according to the delayed playback time corresponding to the parameter value of the delay trigger parameter; the playback condition of the sound material being met includes: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0146] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameter includes: a loop parameter, the loop parameter is used to indicate the number of times the sound material is played in a loop;

[0147] The second generating module is specifically used to respond to the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material, and according to the number of loops corresponding to the parameter value of the loop parameter, to loop the sound material on the image in the video to generate the sound effect of the sound material; the playback condition of the sound material includes: the parameter value of the playback control parameter is a parameter value used to control the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0148] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameter includes: a playing frame number parameter, the playing frame number parameter is used to indicate how many frames the sound material is played;

[0149] The second generating module is specifically used to generate a sound effect of the sound material on an image corresponding to the playback frame number in the video in response to satisfying the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material and according to the playback frame number corresponding to the parameter value of the playback frame number parameter; the playback condition of the sound material includes: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0150] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameter includes: a trigger end parameter, the trigger end parameter is used to indicate a trigger event for ending the playing of the sound material;

[0151] The device further comprises: a second detection module, configured to detect whether a trigger event corresponding to the parameter value of the trigger end parameter appears in the image;

[0152] The second generating module is further configured to stop generating the sound effects of the sound material on the image in response to the second detecting module detecting the occurrence of a trigger event corresponding to the parameter value of the trigger end parameter in the image.

[0153] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playback parameters include: beautification / makeup / face-changing effect parameters, and the beautification / makeup / face-changing effect parameters are used to indicate the beautification / makeup / face-changing effect displayed at a preset position when playing the sound material;

[0154] The second generating module, when generating the sound effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material, is also used to display the beautification / makeup / face-changing effect at the preset part of the image according to the beautification / makeup / face-changing effect parameters.

[0155] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameters include: deformation special effect parameters, the deformation special effect parameters are used to indicate the deformation effect of generating a deformation area on the image when playing the sound material;

[0156] The second generating module, when generating the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material, is also used to generate the deformation effect of the deformation area in the image according to the deformation special effect parameters.

[0157] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the playing parameters include: sticker special effect parameters, the sticker special effect parameters are used to indicate the special effect of generating a sub-material on an image when playing the sound material;

[0158] The second generating module, when generating the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material, is also used to generate the special effects of the sub-material in the image according to the sticker special effect parameters.

[0159] Optionally, in the above-mentioned embodiments of the sound special effects generating device of the present invention, the second import module is specifically used to: read the sound special effects program file package into the memory by calling the interface function for reading the file package; and parse the sound special effects program file package to obtain the sound material and the sound special effects program file, wherein the sound special effects program file includes the parameter values ​​of the playback parameters of the sound material.

[0160] Optionally, in the above-mentioned embodiments of the sound special effects generating device of the present invention, when the second acquisition module obtains the parameter values ​​of the playback parameters of the sound material in the special effects program file package, it is specifically used to: create a sticker handle through an interface function for creating a sticker handle; and read the parameter values ​​of the sound material and the playback parameters, and store them in the sticker handle.

[0161] Optionally, in the above-mentioned embodiments of the sound special effects generating device of the present invention, the second generating module is specifically used to: read the parameter values ​​of the playback parameters and the sound material to be played from the sticker handle by calling an interface function for rendering the sound material; and generate the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material.

[0162] Optionally, in the above-mentioned embodiments of the sound special effects generating device of the present invention, when the image is an image in a video, the second generating module is specifically used to obtain the number of video frames required to generate the sound special effects of the sound material according to the parameter value of the playback parameter, and read the image corresponding to the number of video frames from the video in advance.

[0163] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the second acquisition module is also used to destroy the sticker handle by calling an interface function for destroying the sticker handle in response to the sound special effect program file package being played.

[0164] Optionally, in the above-mentioned embodiments of the sound special effect generating device of the present invention, the second generating module is specifically used to generate the sound special effect of the sound material on the image by calling the built-in player or external player of the electronic device.

[0165] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0166] Memory for storing computer programs;

[0167] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in any one of the above embodiments of the present invention is implemented.

[0168] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any of the above embodiments of the present invention is implemented.

[0169] According to another aspect of an embodiment of the present invention, a computer program is provided, including computer instructions, and when the computer instructions are executed in a processor of a device, the method described in any of the above embodiments of the present invention is implemented.

[0170] Based on the method and device for generating a sound special effects program file package provided by the above-mentioned embodiments of the present invention, the electronic device, program and medium, when generating the sound special effects program file package, the sound material is imported, the parameter values ​​of the playback parameters of the sound material are obtained, and the sound special effects program file package is generated according to the parameter values ​​of the sound material and its playback parameters, so that the image can be processed with sound special effects based on the sound special effects program file package to realize the sound special effects on the image. The embodiments of the present invention can generate a sound special effects program file executable by a rendering engine without manually writing a program file. The operation is simple and the time required is short, which improves the overall efficiency of the sound special effects implementation and avoids the errors that may occur in manually writing program files, effectively ensuring the accuracy of the sound special effects.

[0171] Based on the sound special effect generation method and device, electronic device, program and medium provided by the above-mentioned embodiments of the present invention, the parameter values ​​of the sound material and its playback parameters in the sound special effect program file package are obtained, and the sound special effect of the sound material is generated on the image based on the parameter values ​​of the playback parameters of the sound material. The embodiment of the present invention generates sound special effects on the image by using the parameter values ​​of the sound material and its playback parameters in the pre-generated sound special effect program file package, realizes the sound special effect playback of the image, enhances the entertainment, improves the immersion of the image playback, and improves the image playback effect.

[0172] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0173] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention.

[0174] The present invention may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0175] Figure 1 The present invention is a flowchart of an embodiment of a method for generating a sound special effect program file package.

[0176] Figure 2 This is an example diagram of an operation interface of a device for generating a sound special effect program file package in an embodiment of the present invention.

[0177] Figure 3 Schematic diagram of hand movements in an embodiment of the present invention.

[0178] Figure 4 The flowchart is another embodiment of the method for generating a sound special effect program file package of the present invention.

[0179] Figure 5 The figure is a flow chart of an embodiment of a method for generating sound effects according to the present invention.

[0180] Figure 6 The flowchart of another embodiment of the method for generating sound effects of the present invention.

[0181] Figure 7 The flowchart of another embodiment of the method for generating sound effects of the present invention.

[0182] Figure 8 The present invention is a schematic structural diagram of an embodiment of a device for generating a sound special effect program file package of the present invention.

[0183] Fig. 9 It is a structural schematic diagram of another embodiment of the device for generating a sound special effect program file package of the present invention.

[0184] Fig.10 The figure is a schematic structural diagram of an embodiment of a sound special effect generating device of the present invention.

[0185] Fig.11 It is a structural schematic diagram of another embodiment of the sound special effect generating device of the present invention.

[0186] Fig.12 The figure is a schematic structural diagram of an application embodiment of the electronic device of the present invention. DETAILED DESCRIPTION

[0187] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0188] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0189] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0190] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0191] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0192] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0193] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0194] Figure 1 Flow chart of one embodiment of the method for generating a sound special effect program file package of the present invention. The method for generating a sound special effect program file package of each embodiment of the present invention can be implemented by, for example, but not limited to, a device (the embodiment of the present invention is referred to as a device for generating a sound special effect program file package). Figure 1 As shown, the method for generating a sound special effect program file package in this embodiment includes:

[0195] 102. Import sound materials.

[0196] In each embodiment of the present invention, the sound material may be an audio file in various audio formats, for example, an audio file in the format of MP3, ARM, WAVE, APE, WMA, MPEG, etc.

[0197] 104. Obtain the parameter values ​​of the playback parameters of the above sound material.

[0198] 106. Generate a sound special effect program file package according to the above sound material and the parameter values ​​of its playback parameters.

[0199] In an embodiment of the present invention, the sound special effects program file package can be used to add sound functions to videos or images, and generate sound special effects in videos or on images. For example, background music can be added when playing images or videos, and special effects sounds can be played when certain actions appear in the video, thereby increasing the playback atmosphere and enhancing the sense of immersion.

[0200] Based on the method for generating a sound special effects program file package provided in the above-mentioned embodiment of the present invention, when generating the sound special effects program file package, the sound material is imported, the parameter values ​​of the playback parameters of the sound material are obtained, and the sound special effects program file package is generated according to the parameter values ​​of the sound material and its playback parameters, so that the image can be processed with sound special effects based on the sound special effects program file package to realize the sound special effects on the image. The embodiment of the present invention can generate a sound special effects program file executable by a rendering engine without manually writing a program file. The operation is simple and the time required is short, which improves the overall efficiency of the sound special effects implementation, avoids the errors that may occur in manually writing program files, and effectively ensures the accuracy of the sound special effects.

[0201] In one implementation of each embodiment of the present invention, the device for generating a sound special effect program file package may include a preset sound special effect program file, which may be, for example, a lightweight data exchange format (JavaScript Object Notiation, json) file based on the JavaScript language, or any other executable program file, and the generated sound special effect program file package may be a sound special effect program file package generated by a json program. The parameter values ​​of the playback parameters in the sound special effect program file may be blank or preset to default values. When the parameter values ​​for the playback parameters set for the sound material are received, the corresponding parameter values ​​in the sound special effect program file are automatically updated to the received parameter values. Optionally, the device for generating the above-mentioned sound special effect program file package may include an operation bar, in which at least one interactive interface is provided for importing sound materials and setting parameter values ​​for the playback parameters of the sound materials; in addition, the device for generating the sound special effect program file package may also include a program file display bar for displaying the sound special effect program file, which may include the playback parameters of the sound material and their parameter values. Figure 2 The figure shown is an example diagram of an operation interface of a device for generating a sound special effect program file package in an embodiment of the present invention. The operation interface of the device for generating the sound special effect program file package includes an operation bar and a program file display bar. After the device for generating the sound special effect program file package is started, corresponding to a playback parameter setting interface of a sound material in the operation bar, the program file display bar displays the sound special effect program file when the playback parameters of the sound material are vacant or preset to default values. When a parameter value for the playback parameter setting of a sound material is received through the interactive interface of the operation bar, the parameter value for the playback parameter setting of the sound material is updated to the most recently received parameter value, and the program file display bar displays the sound special effect program file after the parameter value is updated in real time. It should be noted that Figure 2The program files displayed in the program file display bar are only used to exemplify the operating interface presented by the device for generating the sound special effect program file package. The present invention does not need to pay attention to the expression form and specific content of the program files, nor does it limit the expression form and specific content of the program files.

[0202] In one implementation of the method for generating each special effect program file package of the present invention, operation 102 may include: receiving an import instruction sent through an interactive interface of the operation bar, and importing the sound material in the material folder pointed to by the import instruction.

[0203] like Figure 2 As shown in FIG. 1 , as an optional example of various embodiments of the present invention but not a limitation, the operation bar may include: at least one sound material playback parameter setting interface, each sound material playback parameter setting interface includes: an interactive interface 20 for importing sound materials and at least one interactive interface 21 for setting the parameter value of the playback parameter. Figure 2 In the example, lalala represents a sound special effect function, and lalala.mp3 is a sound material imported through the interactive interface 20. In an optional example of the above embodiment of the present invention, receiving an import instruction input through the interactive interface of the operation bar, and importing the sound material in the material folder pointed to by the import instruction may include: receiving an import instruction sent through the interactive interface 20 in the playback parameter setting interface under the operation bar, and importing the sound material in the material folder pointed to by the import instruction.

[0204] In addition, a sound material and its playback parameter setting interface can be added through the interactive interface of the operation bar. For example, a sound material adding instruction input through the interactive interface of the operation bar can be received to add a sound material and its playback parameter setting interface. For example, a sound material and its playback parameter setting interface can be added or copied by right-clicking the "add" or "copy" operation option in the control button such as the sound material "music" in the operation bar.

[0205] In another optional example of the above embodiment of the present invention, receiving an import instruction input through the interactive interface of the operation bar, and importing the sound material in the material folder pointed to by the import instruction may include:

[0206] Receiving an import instruction sent through the interactive interface 20, and acquiring and displaying a material folder pointed to by the import instruction;

[0207] In response to receiving an operation to select a sound material in a material folder, importing the sound material selected by the sound material selection operation; and / or, in response to not receiving an operation to select a sound material in a material folder, selecting one or more sound materials in the material folder according to preset settings, and importing the sound material selected according to the preset settings.

[0208] Each material folder may include multiple sound materials. In one implementation of each embodiment of the present invention, when importing sound materials, if the user does not select a sound material in the material folder, the sound material at a preset position or a preset sequence number in the material folder pointed to by the import instruction may be imported according to the preset setting. For example, when the user does not select a sound material, the first sound material or the last sound material in the material folder, or a randomly selected sound material, may be selected and imported by default.

[0209] In one implementation of various embodiments of the present invention, in operation 104, obtaining the parameter value of the playback parameter of the sound material may include:

[0210] In response to receiving a parameter value of a playback parameter setting for a sound material sent through an interactive interface in the playback parameter setting interface, using the parameter value of the setting as the parameter value of the playback parameter of the sound material; and / or

[0211] In response to not receiving the parameter value of the playback parameter setting for the sound material sent through the interactive interface in the playback parameter setting interface, the preset parameter value is used as the parameter value of the playback parameter of the sound material.

[0212] The embodiment of the present invention does not need to generate a rendering engine executable file by manually writing a program file. Based on the user's selection operation of the sound material in the operation bar and the setting operation of the parameter value of the playback parameter, the generation of the sound special effect program package can be realized. The operation is simple and the time required is short, which improves the overall efficiency of the sound special effect implementation, avoids the errors that may occur in manually writing program files, and effectively ensures the accuracy of the special effect.

[0213] In one implementation of each embodiment of the present invention, the playback parameters of the sound material may include, but are not limited to, any one or more of the following:

[0214] 1. Playback control parameter (Display): used to control whether the sound material is played. The parameter value includes two options: "Yes" and "No". When the parameter value is "Yes", it means that the corresponding sound material needs to be played, and when the parameter value is "No", it means that the corresponding sound material does not need to be played;

[0215] 2. Trigger mode parameter (TriggerType): used to indicate the trigger event that triggers the playback of the sound material, which refers to the trigger event that triggers the playback of the corresponding sound material. Its parameter value may include various trigger events, and the user may select at least one event from the preset event set as the trigger event. That is, during the video playback process, the corresponding sound material may be triggered to play when the corresponding trigger event is detected. For example, when the trigger event "opening the mouth" specified in the trigger mode parameter is detected in the video, a piece of advertising words, background music, etc. as the sound material may be played. The specific start display time, end display time, and display duration of the sound material may be determined based on the parameter values ​​of other playback parameters, such as the parameter values ​​of the delay trigger parameter, the trigger end parameter, and the loop parameter.

[0216] 3. TriggerLoop: It is used to indicate the number of times the sound material is played in a loop. You can set or select a specific value of the number of times the sound material is played in a loop as its parameter value, for example, 1, 5, etc. You can agree that the parameter value is set to 0 for infinite loop playback.

[0217] 4. Trigger Delay parameter: used to indicate the delay time of playing the sound material, that is, when a certain frame of image in the video is detected to trigger the event in the trigger mode parameter, the sound material is played after a certain delay of a certain number of frames. You can set or select the specific time or number of frames of delay to play the sound material as its parameter value;

[0218] 5. Trigger Stop parameter: used to indicate the trigger event for ending the playback of sound material, which refers to the event that ends the playback of sound material. Its parameter value includes various trigger events. The user can select at least one event from the preset event set as the trigger event for ending the playback of sound material. That is, during the video playback process, the corresponding sound material can be stopped when the trigger event specified by the trigger stop parameter is detected. For example, when the trigger event "open mouth" specified in the trigger mode parameter is detected in the video, a piece of background music starts to play. The parameter value in the trigger stop parameter can be set to "close mouth". Then, when the trigger event "close mouth" action is detected in the video, the background music stops playing.

[0219] 6. DisplayFrames: It is used to indicate the number of frames of the sound material. The number of frames can be selected by the user. For example, when the selected number of frames is set to 0, it means that the stiff material is played all the time.

[0220] 7. Beautification / makeup / face-changing (beautification and / or makeup and / or face-changing) effect parameters: used to indicate the beautification / makeup / face-changing effects displayed at the preset position when playing the sound material, such as red lips, colored pupils, blue eyes, etc.;

[0221] 8. Deformation effect parameters: used to indicate the deformation effect of generating a deformation area on the image when playing the sound material. The shape of the deformation area is not limited, for example, it can be a circle, an ellipse, a rectangle, a square, a triangle, or any other shape;

[0222] 9. Sticker special effect parameters are used to indicate the special effects of generating sub-materials on the image when playing sound materials. For example, when playing sound materials, the sub-material of a hat is played at the top of the head.

[0223] In one of the optional examples, the triggering event corresponding to the triggering mode parameter includes any one or more of the following:

[0224] No action trigger (NULL), that is, no action is required to display the sound material;

[0225] Eye movements, such as blinking, closing, and opening;

[0226] Head movements, such as shaking the head, nodding, tilting the head, turning the head, etc.;

[0227] Eyebrow movements, such as raising eyebrows;

[0228] Hand movements, such as love hand, support hand, palm, thumb up, fist to congratulate, one-hand heart, OK hand, scissors hand, pistol hand, index finger, etc. Figure 3 FIG. 1 is an exemplary schematic diagram of hand movements in an embodiment of the present invention;

[0229] Mouth movements, e.g., opening and closing the mouth;

[0230] Shoulder movements, such as shrugs;

[0231] Other actions.

[0232] In addition, see Figure 2 The device for generating a sound special effect program file package of the embodiment of the present invention may further include a content display bar, which is used to display reference images (such as average human faces, hands, limbs, etc.), beauty / makeup / face-changing effects, deformation special effects, sticker special effects, etc. In another embodiment of the method for generating a special effect program file package of the present invention, it also includes:

[0233] The device for generating the sound special effect program file package is started according to the received start-up instruction and displays an operation interface, which includes: an operation bar, a content display bar and a program file bar.

[0234] like Figure 2As shown, in one of the optional examples, the above operation interface includes three areas: left, middle and right. Accordingly, the above display operation interface may include: displaying an operation bar on the left side of the operation interface, displaying a content display bar in the middle of the operation interface, and displaying the program file bar on the right side of the operation interface.

[0235] Among them, the program file display bar on the right is used to display the contents of the sound special effect program file of the sound material with the current set parameter values ​​through the display area 22 therein, and the sound special effect program file package can be exported through the save instruction interface 23 in the program file display bar, that is, the sound special effect program file package is generated and saved.

[0236] Figure 4 FIG. 1 is a flow chart of another embodiment of the method for generating a special effect program file package of the present invention. Figure 4 As shown, Figure 1 Compared with the embodiment shown in the figure, the method for generating the sound special effect program file package of this embodiment, after generating the sound special effect program file package, further includes:

[0237] 302. According to the received saving instruction, save the sound special effect program file package at the location pointed by the saving instruction.

[0238] In one embodiment, according to the received saving instruction, saving the sound special effect program file package at the location pointed by the saving instruction may include:

[0239] In response to receiving the save instruction, displaying a save path selection interface and a compression interface;

[0240] Receiving the save location sent through the save path selection interface; and receiving the compression method sent based on the compression interface, and compressing the sound special effect program file package according to the compression method to generate a compressed file package;

[0241] Save the above compressed file package to the folder pointed to by the above save location.

[0242] When the size of the special effect program file package is large and is not suitable for running in a mobile phone terminal, the embodiment of the present invention can compress the sound special effect program file package and save it so that it can be imported into the mobile phone terminal to generate sound special effects. The embodiment of the present invention only compresses the size of the sound special effect program file package and does not change the size of the sound material in the sound special effect program file package, that is, the size of the sound material in the sound special effect program file package remains unchanged as the size before the sound material is imported.

[0243] After the special effect program file package is generated based on the above embodiments of the present invention, the special effect program file package can be imported into a terminal to generate special effects for a video played by the terminal.

[0244] Figure 5 FIG. 1 is a flow chart of an embodiment of a method for generating sound effects of the present invention. Figure 5 As shown, the sound special effect generation method of this embodiment includes:

[0245] 402, obtaining the sound material and the parameter values ​​of the playing parameters of the sound material in the sound special effect program file package.

[0246] 404 , generating a sound effect of the sound material on the image based on the parameter value of the playback parameter of the sound material.

[0247] Exemplarily, the image in each embodiment of the present invention may be a static image or an image in a video.

[0248] In some implementations of the embodiments of the present invention, the operation 404 may include: generating a sound effect of the sound material on the image by calling a built-in player or an external player of the electronic device.

[0249] Based on the sound special effect generation method provided by the above embodiment of the present invention, the parameter values ​​of the sound material and its playback parameters in the sound special effect program file package are obtained, and the sound special effect of the sound material is generated on the image based on the parameter values ​​of the playback parameters of the sound material. The embodiment of the present invention generates sound special effects on the image by using the parameter values ​​of the sound material and its playback parameters in the pre-generated sound special effect program file package, realizes the sound special effect playback of the image, enhances the entertainment, improves the immersion of the image playback, and improves the image playback effect.

[0250] Figure 6 FIG. 1 is a flow chart of another embodiment of the method for generating sound effects of the present invention. Figure 6 As shown, the sound special effect generation method of this embodiment includes:

[0251] 502, import the sound effect program file package.

[0252] In one embodiment, the operation 502 may include: reading the sound effect program file package into the memory by calling an interface function for reading the file package; parsing the sound effect program file package to obtain the sound material and the sound effect program file, wherein the sound effect program file includes the parameter values ​​of the playback parameters of the sound material.

[0253] In one of the optional examples, the special effect program file may include: a json program or a special effect program file of other executable programs.

[0254] Exemplarily, the sound special effect program file package in this embodiment can be generated by using the method for generating the sound special effect program file package described in any of the above embodiments of the present invention.

[0255] 504, obtaining the sound material in the sound special effect program file package and the parameter values ​​of the playing parameters of the sound material.

[0256] 506. Generate a sound effect of the sound material on the image based on the parameter value of the playback parameter of the sound material.

[0257] In some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters include: playback control parameters, and the playback control parameters are used to control whether the sound material is played. Accordingly, in this embodiment, in operation 404 or 506, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material may include: in response to the parameter value of the playback control parameter being a parameter value for controlling the playback of the sound material, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material.

[0258] In some implementations of the above-mentioned embodiments of the sound special effects generation method of the present invention, the above-mentioned playback parameters include: trigger mode parameters, and the trigger mode parameters are used to indicate a trigger event that triggers the playback of the sound material. Correspondingly, the sound special effects generation method of this embodiment may also include: 602, detecting whether a trigger event corresponding to the parameter value of the above-mentioned trigger mode parameter appears in the image. In this embodiment, in operation 404 or 506, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material may include: 604, in response to detecting a trigger event corresponding to the parameter value of the trigger mode parameter appearing in the image, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material. If in operation 602, a trigger event corresponding to the parameter value of the trigger mode parameter is not detected in the image, the subsequent process of this embodiment is not executed, that is: operation 604 is not executed. Figure 7 FIG. 1 is a flow chart of another embodiment of the method for generating sound special effects of the present invention.

[0259] In some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters include: a delay trigger parameter, and the delay trigger parameter is used to indicate the time of delaying the playback of the sound material. Accordingly, in this embodiment, in operation 404 or 506, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material may include: in response to satisfying the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material, according to the delayed playback time corresponding to the parameter value of the delay trigger parameter, delaying the generation of the sound special effect of the sound material on the image in the video.

[0260] Among them, the conditions for playing the sound material may include, but are not limited to: the parameter value of the playback control parameter is the parameter value used to control the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0261] In some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters include: a loop parameter, which is used to indicate the number of loop playback times of the sound material. Accordingly, in this embodiment, in operation 404 or 506, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material may include: in response to satisfying the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material, according to the number of loops corresponding to the parameter value of the loop parameter, looping the sound material on the image in the video to generate the sound special effect of the sound material.

[0262] Among them, the conditions for playing the sound material may include, but are not limited to: the parameter value of the playback control parameter is the parameter value used to control the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0263] In some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters include: playback frame number parameters, and the playback frame number parameters are used to indicate how many frames the sound material is played. Accordingly, in this embodiment, in operation 404 or 506, based on the parameter value of the playback parameter of the sound material, generating the sound special effect of the sound material on the image may include: in response to satisfying the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material, according to the playback frame number corresponding to the parameter value of the playback frame number parameter, generating the sound special effect of the sound material on the image corresponding to the playback frame number in the video.

[0264] Among them, the conditions for playing the sound material may include, but are not limited to: the parameter value of the playback control parameter is the parameter value used to control the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0265] In addition, in some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters may also include: a trigger end parameter, and the trigger end parameter is used to indicate a trigger event for ending the playback of the sound material. Figure 7The sound special effect generation method of this embodiment may further include: 606, detecting whether a trigger event corresponding to the parameter value of the trigger end parameter appears in the image; 608, in response to detecting the trigger event corresponding to the parameter value of the trigger end parameter appears in the image, stopping the generation of the sound special effect of the sound material on the image. If in operation 606, the trigger event corresponding to the parameter value of the trigger end parameter is not detected in the image, the subsequent process of this embodiment is not executed, that is, operation 608 is not executed.

[0266] In addition, in some implementations of the above-mentioned sound special effects generation method embodiments of the present invention, the above-mentioned playback parameters may include: beauty / makeup / face-changing effect parameters, and the beauty / makeup / face-changing effect parameters are used to indicate the beauty / makeup / face-changing effect displayed at a preset position when the sound material is played. Accordingly, in this embodiment, in operation 404 or 506, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material may include: generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material, and displaying the beauty / makeup / face-changing effect at a preset position in the image according to the beauty / makeup / face-changing effect parameters.

[0267] In addition, in some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters may include: deformation special effect parameters, and the deformation special effect parameters are used to indicate the deformation effect of generating a deformation area on the image when playing the sound material. Accordingly, in this embodiment, in operation 404 or 506, generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material may include: generating a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material, and generating a deformation effect of the deformation area in the image according to the deformation special effect parameters.

[0268] In addition, in some implementations of the above-mentioned sound special effect generation method embodiments of the present invention, the above-mentioned playback parameters may include: sticker special effect parameters, which are used to indicate the special effects of generating sub-materials on the image when playing the sound material. Accordingly, in this embodiment, in operation 404 or 506, generating the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material may include: generating the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material, and generating the special effects of the sub-material in the image according to the sticker special effect parameters.

[0269] In some implementations of the above-mentioned sound special effects generation method embodiments of the present invention, obtaining the parameter values ​​of the playback parameters of the sound material in the special effects program file package may include: creating a sticker handle through an interface function for creating a sticker handle; reading the parameter values ​​of the sound material and the playback parameters, and storing them in the sticker handle.

[0270] Accordingly, in some other embodiments, generating sound effects of sound materials on an image based on the parameter values ​​of the playback parameters of the sound materials may include: reading the parameter values ​​of the playback parameters and the sound materials to be played from the sticker handle by calling an interface function for rendering the sound materials; and generating sound effects of the sound materials on the image based on the parameter values ​​of the playback parameters of the sound materials.

[0271] In some optional examples of the above-mentioned implementation methods, when the above-mentioned image is an image in a video, sound effects of the sound material are generated on the image based on the parameter values ​​of the playback parameters of the sound material, and it also includes: according to the parameter values ​​of the above-mentioned playback parameters, obtaining the number of video frames required to generate the sound effects of the sound material, and pre-reading the image corresponding to the number of video frames from the video, so as to generate the sound effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material.

[0272] Furthermore, in some other implementations, the method may also include: in response to the sound effect program file package being played, destroying the sticker handle by calling an interface function for destroying the sticker handle.

[0273] The various sound effects generation method embodiments of the present invention can be used in various image or video playback scenarios, for example, in a video live broadcast scenario containing characters, to generate sound effects for the live video, and to overlay and play corresponding sound materials on the live video according to the sound effects program file package to increase the atmosphere effect, enhance customer entertainment, and improve the immersive feeling of use.

[0274] Any method for generating a sound special effect program file package and a method for generating a sound special effect provided in the embodiments of the present invention can be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any method for generating a sound special effect program file package and a method for generating a sound special effect provided in the embodiments of the present invention can be executed by a processor, such as the processor executing any method for generating a sound special effect program file package and a method for generating a sound special effect mentioned in the embodiments of the present invention by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0275] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0276] Figure 8FIG. 1 is a schematic diagram of a structure of an embodiment of a device for generating a sound special effect program file package of the present invention. The device for generating a sound special effect program file package of the embodiment can be used to implement the above-mentioned embodiments of the method for generating a sound special effect program file package of the present invention. Figure 8 As shown, the device for generating a sound special effect program file package in this embodiment includes: a first import module, a first acquisition module and a first generation module. Among them:

[0277] The first import module is used to import sound materials.

[0278] The first acquisition module is used to obtain parameter values ​​of playback parameters of the sound material.

[0279] The first generating module is used to generate a sound special effect program file package according to the sound material and the parameter values ​​of the playing parameters.

[0280] In some of the optional examples, the sound special effect program file package in the embodiments of the present invention may be a sound special effect program file package generated by a json program or any other executable program.

[0281] Based on the device for generating the sound special effects program file package provided by the above-mentioned embodiment of the present invention, when generating the sound special effects program file package, the sound material is imported, the parameter values ​​of the playback parameters of the sound material are obtained, and the sound special effects program file package is generated according to the parameter values ​​of the sound material and its playback parameters, so as to perform sound special effects processing on the image based on the sound special effects program file package and realize the sound special effects on the image. The embodiment of the present invention can generate a sound special effects program file executable by a rendering engine without manually writing a program file, and the operation is simple and the time required is short, which improves the overall efficiency of the sound special effects implementation, avoids the errors that may occur in manually writing program files, and effectively ensures the accuracy of the sound special effects.

[0282] In some implementation modes, the first import module is specifically configured to receive an import instruction input through an interactive interface of the operation bar, and import the sound material in the material folder pointed to by the import instruction.

[0283] In some of the optional examples, the first import module is specifically used to receive an import instruction sent through an interactive interface in the special effect parameter setting interface under the operation bar, and import the sound material in the material folder pointed to by the import instruction.

[0284] In some of the optional examples, the first import module is specifically used to: receive an import instruction sent through an interactive interface, obtain and display the material folder pointed to by the import instruction; in response to receiving a selection operation for sound materials in the material folder, import the sound materials selected by the sound material selection operation; and / or, in response to not receiving a selection operation for sound materials in the material folder, select one or more sound materials in the material folder according to preset settings, and import the sound materials selected according to the preset settings.

[0285] In some of the optional examples, the first import module is specifically used to: in response to receiving parameter values ​​for playback parameter settings for sound materials sent through an interactive interface in a special effect parameter setting interface, use the set parameter values ​​as parameter values ​​for playback parameters of the sound materials; and / or, in response to not receiving parameter values ​​for playback parameter settings for sound materials sent through an interactive interface in a special effect parameter setting interface, use preset parameter values ​​as parameter values ​​for playback parameters of the sound materials.

[0286] In some implementations, the playback parameters of the sound material may include, but are not limited to, any one or more of the following:

[0287] 1. Playback control parameter (Display): used to control whether the sound material is played. The parameter value includes two options: "Yes" and "No". When the parameter value is "Yes", it means that the corresponding sound material needs to be played, and when the parameter value is "No", it means that the corresponding sound material does not need to be played;

[0288] 2. Trigger mode parameter (TriggerType): used to indicate the trigger event that triggers the playback of the sound material, which refers to the trigger event that triggers the playback of the corresponding sound material. Its parameter value may include various trigger events, and the user may select at least one event from the preset event set as the trigger event. That is, during the video playback process, the corresponding sound material may be triggered to play when the corresponding trigger event is detected. For example, when the trigger event "opening the mouth" specified in the trigger mode parameter is detected in the video, a piece of advertising words, background music, etc. as the sound material may be played. The specific start display time, end display time, and display duration of the sound material may be determined based on the parameter values ​​of other playback parameters, such as the parameter values ​​of the delay trigger parameter, the trigger end parameter, and the loop parameter.

[0289] 3. TriggerLoop: It is used to indicate the number of times the sound material is played in a loop. You can set or select a specific value of the number of times the sound material is played in a loop as its parameter value, for example, 1, 5, etc. You can agree that the parameter value is set to 0 for infinite loop playback.

[0290] 4. Trigger Delay parameter: used to indicate the delay time of playing the sound material, that is, when a certain frame of image in the video is detected to trigger the event in the trigger mode parameter, the sound material is played after a certain delay of a certain number of frames. You can set or select the specific time or number of frames of delay to play the sound material as its parameter value;

[0291] 5. Trigger Stop parameter: used to indicate the trigger event for ending the playback of sound material, which refers to the event that ends the playback of sound material. Its parameter value includes various trigger events. The user can select at least one event from the preset event set as the trigger event for ending the playback of sound material. That is, during the video playback process, the corresponding sound material can be stopped when the trigger event specified by the trigger stop parameter is detected. For example, when the trigger event "open mouth" specified in the trigger mode parameter is detected in the video, a piece of background music starts to play. The parameter value in the trigger stop parameter can be set to "close mouth". Then, when the trigger event "close mouth" action is detected in the video, the background music stops playing.

[0292] 6. DisplayFrames: It is used to indicate the number of frames of the sound material. The number of frames can be selected by the user. For example, when the selected number of frames is set to 0, it means that the stiff material is played all the time.

[0293] 7. Beautification / makeup / face-changing (beautification and / or makeup and / or face-changing) effect parameters: used to indicate the beautification / makeup / face-changing effects displayed at the preset position when playing the sound material, such as red lips, colored pupils, blue eyes, etc.;

[0294] 8. Deformation effect parameters: used to indicate the deformation effect of generating a deformation area on the image when playing the sound material. The shape of the deformation area is not limited, for example, it can be a circle, an ellipse, a rectangle, a square, a triangle, or any other shape;

[0295] 9. Sticker special effect parameters are used to indicate the special effects of generating sub-materials on the image when playing sound materials. For example, when playing sound materials, the sub-material of a hat is played at the top of the head.

[0296] In one of the optional examples, the triggering event corresponding to the triggering mode parameter includes any one or more of the following:

[0297] No action trigger (NULL), that is, no action is required to display the sound material;

[0298] Eye movements, such as blinking, closing, and opening;

[0299] Head movements, such as shaking the head, nodding, tilting the head, turning the head, etc.;

[0300] Eyebrow movements, such as raising eyebrows;

[0301] Hand movements, such as love hand, support hand, palm, thumb up, fist to congratulate, one-hand heart, OK hand, scissors hand, pistol hand, index finger, etc. Figure 3 FIG. 1 is an exemplary schematic diagram of hand movements in an embodiment of the present invention;

[0302] Mouth movements, e.g., opening and closing the mouth;

[0303] Shoulder movements, such as shrugs;

[0304] Other actions.

[0305] Fig. 9 FIG. 2 is a schematic diagram of another embodiment of a device for generating a sound special effect program file package of the present invention. Fig. 9 As shown, in another embodiment of the device for generating a sound special effect program file package, it can also include an operation interface, and the operation interface includes: an operation bar, a content display bar and / or a program file bar.

[0306] like Figure 2 As shown, in some optional examples, the operation interface may include three areas: left, middle and right. An operation bar may be displayed on the left side of the operation interface, a content display bar may be displayed in the middle of the operation interface, and a program file bar may be displayed on the right side of the operation interface.

[0307] In addition, see Fig. 9 In another embodiment of the generating device, it may also include: a saving module, used to save the sound special effect program file package at the location pointed to by the saving instruction according to the received saving instruction.

[0308] In some of the embodiments, the saving module is specifically used to: display a saving path selection interface and a compression interface in response to receiving a saving instruction; receive a saving location sent through the saving path selection interface; and receive a compression method sent based on the compression interface, and compress the sound special effect program file package according to the compression method to generate a compressed file package; and store the compressed file package in the folder pointed to by the saving location.

[0309] In some optional examples, during the process of compressing the sound effect program file package, the size of the sound material in the sound effect program file package remains unchanged at the size before being imported.

[0310] Fig.10 FIG. 1 is a schematic diagram of the structure of an embodiment of a sound special effect generating device of the present invention. The sound special effect generating device of this embodiment can be used to implement the above-mentioned various sound special effect generating method embodiments of the present invention. Fig.10As shown, the device for generating the sound special effect program file package in this embodiment includes: a second acquisition module and a second generation module. Among them:

[0311] The second acquisition module is used to obtain the sound material in the sound effect program file package and the parameter values ​​of the playback parameters of the sound material.

[0312] The second generating module is used to generate the sound effects of the sound material on the image based on the parameter values ​​of the playing parameters of the sound material.

[0313] Based on the sound special effect generation device provided by the above embodiment of the present invention, the parameter values ​​of the sound material and its playback parameters in the sound special effect program file package are obtained, and the sound special effect of the sound material is generated on the image based on the parameter values ​​of the playback parameters of the sound material. The embodiment of the present invention generates sound special effects on the image by using the parameter values ​​of the sound material and its playback parameters in the pre-generated sound special effect program file package, realizes the sound special effect playback of the image, enhances the entertainment, improves the immersion of the image playback, and improves the image playback effect.

[0314] Fig.11 FIG. 2 is a schematic diagram of the structure of another embodiment of the sound special effect generating device of the present invention. Fig.11 As shown, Fig.10 Compared with the embodiment shown, the sound special effect generating device of this embodiment further includes: a second importing module for importing the sound special effect program file package. Accordingly, in this embodiment, the second obtaining module is specifically used to obtain the sound material in the sound special effect program file package and the parameter value of the playing parameter of the sound material.

[0315] Among them, the sound special effect program file package can be generated using the sound special effect program file package generation method or device of any of the above embodiments of the present application.

[0316] The image may include, but is not limited to, any one or more of the following: a static image, an image in a video.

[0317] In some implementations of the above-mentioned sound special effect generation devices, the playback parameter includes: a playback control parameter, which is used to control whether the sound material is played. Accordingly, in this embodiment, the second generation module is specifically used to generate a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material in response to the parameter value of the playback control parameter being a parameter value for controlling the playback of the sound material.

[0318] In some implementations of the above-mentioned sound special effect generating devices, the playing parameters include: trigger mode parameters, which are used to indicate a triggering event for triggering the playing of the sound material. Fig.11As shown, the sound special effect generating device of this embodiment may also include: a first detection module, which is used to detect whether a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image. In this embodiment, the second generation module is specifically used to generate a sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material in response to the first detection module detecting a trigger event corresponding to the parameter value of the trigger mode parameter appearing in the image.

[0319] In some implementations of the above-mentioned sound special effect generation devices, the playback parameters include: a delay trigger parameter, which is used to indicate the time for delaying the playback of the sound material. Accordingly, in this embodiment, the second generation module is specifically used to delay the generation of the sound special effect of the sound material on the image in the video in response to the satisfaction of the playback conditions of the sound material, based on the parameter value of the playback parameter of the sound material, and according to the delayed playback time corresponding to the parameter value of the delay trigger parameter; the satisfaction of the playback conditions of the sound material includes: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or, a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0320] In some implementations of the above-mentioned sound special effect generation devices, the playback parameters include: a loop parameter, which is used to indicate the number of times the sound material is played in a loop. Accordingly, in this embodiment, the second generation module is specifically used to respond to the conditions for playing the sound material, based on the parameter value of the playback parameter of the sound material, and according to the number of loops corresponding to the parameter value of the loop parameter, to loop the sound material on the image in the video to generate the sound special effect of the sound material; the conditions for playing the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0321] In some implementations of the above-mentioned sound special effect generation devices, the playback parameters include: a playback frame number parameter, which is used to indicate how many frames the sound material is played. Accordingly, in this embodiment, the second generation module is specifically used to generate a sound special effect of the sound material on an image corresponding to the playback frame number in the video based on the parameter value of the playback parameter of the sound material in response to satisfying the playback condition of the sound material, according to the playback frame number corresponding to the parameter value of the playback frame number parameter; satisfying the playback condition of the sound material includes: the parameter value of the playback control parameter is a parameter value used to control the playback of the sound material, and / or, a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

[0322] In some implementations of the above-mentioned sound special effect generating devices, the playing parameters include: a trigger end parameter, and the trigger end parameter is used to indicate a trigger event for ending the playing of the sound material. Fig.11As shown, the sound special effect generating device of this embodiment may also include: a second detection module, which is used to detect whether a trigger event corresponding to the parameter value of the trigger end parameter appears in the image. In this embodiment, the second generating module is also used to stop generating the sound special effect of the sound material on the image in response to the second detection module detecting the trigger event corresponding to the parameter value of the trigger end parameter appears in the image.

[0323] In some implementations of the above-mentioned sound special effect generation devices, the playback parameters include: beautification / makeup / face-changing effect parameters, which are used to indicate the beautification / makeup / face-changing effect displayed at a preset position when the sound material is played. Accordingly, in this embodiment, the second generation module, when generating the sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material, can also be used to display the beautification / makeup / face-changing effect at a preset position in the image according to the beautification / makeup / face-changing effect parameters.

[0324] In some implementations of the above-mentioned sound special effect generation devices, the playback parameters include: deformation special effect parameters, which are used to represent the deformation effect of generating a deformation area on the image when playing the sound material. Accordingly, in this embodiment, the second generation module, when generating the sound special effect of the sound material on the image based on the parameter value of the playback parameter of the sound material, can also be used to generate the deformation effect of the deformation area in the image according to the deformation special effect parameters.

[0325] In some embodiments of the above-mentioned sound special effect generation devices, the playback parameters include: sticker special effect parameters, which are used to indicate the special effects of the sub-material generated on the image when the sound material is played. Accordingly, in this embodiment, the second generation module, when generating the sound special effects of the sound material on the image based on the parameter value of the playback parameter of the sound material, can also be used to generate the special effects of the sub-material in the image according to the sticker special effect parameters.

[0326] In some embodiments of the above-mentioned sound effects generating devices, the second import module is specifically used to: read the sound effects program file package into the memory by calling the interface function for reading the file package; and parse the sound effects program file package to obtain the sound material and the sound effects program file, wherein the sound effects program file includes the parameter values ​​of the playback parameters of the sound material.

[0327] In some embodiments of the above-mentioned sound special effects generating devices, when the second acquisition module obtains the parameter values ​​of the playback parameters of the sound material in the special effects program file package, it is specifically used to: create a sticker handle through an interface function for creating a sticker handle; and read the parameter values ​​of the sound material and the playback parameters, and store them in the sticker handle.

[0328] In some embodiments of the above-mentioned sound special effects generating devices, the second generating module is specifically used to: read the parameter values ​​of the playback parameters and the sound material to be played from the sticker handle by calling the interface function for rendering the sound material; and generate the sound special effects of the sound material on the image based on the parameter values ​​of the playback parameters of the sound material.

[0329] In some embodiments of the above-mentioned sound special effects generating devices, when the image is an image in a video, the second generating module is specifically used to obtain the number of video frames required to generate the sound special effects of the sound material according to the parameter value of the playback parameter, and to read the image corresponding to the number of video frames from the video in advance.

[0330] In some implementations of the above-mentioned sound special effect generating devices, the second acquisition module can also be used to destroy the sticker handle by calling an interface function for destroying the sticker handle in response to the sound special effect program file package being played.

[0331] In some implementations of the above-mentioned sound special effect generating devices, the second generating module is specifically used to generate the sound special effect of the sound material on the image by calling the built-in player or the external player of the electronic device.

[0332] In addition, another electronic device provided by an embodiment of the present invention includes:

[0333] Memory for storing computer programs;

[0334] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method for generating a sound special effect program file package or the method for generating a sound special effect of any of the above embodiments of the present invention is implemented.

[0335] Fig.12 This is a schematic diagram of the structure of an application embodiment of the electronic device of the present invention. Fig.12 , which shows a schematic diagram of the structure of an electronic device suitable for implementing a terminal device or a server in an embodiment of the present application. Fig.12As shown, the electronic device includes one or more processors, a communication unit, etc., and the one or more processors are, for example, one or more central processing units (CPUs), and / or one or more image processing units (GPUs), etc. The processor can perform various appropriate actions and processes according to executable instructions stored in a read-only memory (ROM) or executable instructions loaded from a storage part to a random access memory (RAM). The communication unit may include, but is not limited to, a network card, and the network card may include, but is not limited to, an IB (Infiniband) network card. The processor can communicate with the read-only memory and / or the random access memory to execute executable instructions, and is connected to the communication unit through a bus, and communicates with other target devices through the communication unit, thereby completing the operation corresponding to any method provided in the embodiment of the present application, for example, importing sound material; obtaining the parameter value of the playback parameter of the sound material; generating a sound special effect program file package according to the parameter value of the sound material and the playback parameter. For another example, obtaining the parameter value of the sound material and the playback parameter of the sound material in the sound special effect program file package; based on the parameter value of the playback parameter of the sound material, generating the sound special effect of the sound material on the image.

[0336] In addition, various programs and data required for the operation of the device can also be stored in the RAM. The CPU, ROM and RAM are connected to each other via a bus. In the case of RAM, the ROM is an optional module. The RAM stores executable instructions, or writes executable instructions into the ROM at runtime, and the executable instructions enable the processor to perform operations corresponding to any of the above methods of the present invention. The input / output (I / O) interface is also connected to the bus. The communication unit can be integrated or can be configured to have multiple submodules (e.g., multiple IB network cards) and be linked on the bus.

[0337] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that a computer program read therefrom is installed into the storage section as needed.

[0338] If necessary, Fig.12 The architecture shown is only an optional implementation method. In the specific practice, the above can be modified according to actual needs. Fig.12The number and type of components can be selected, deleted, added or replaced; different functional components can be set up by separate settings or integrated settings, for example, the GPU and CPU can be set up separately or the GPU can be integrated on the CPU, the communication unit can be set up separately or integrated on the CPU or GPU, etc. These replaceable implementations all fall within the protection scope disclosed in the present invention.

[0339] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, the computer program includes a program code for executing the method shown in the flowchart, and the program code may include instructions corresponding to the steps of the face anti-counterfeiting detection method provided in the embodiment of the present application. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the CPU, the above functions defined in the method of the present application are executed.

[0340] In addition, an embodiment of the present invention further provides a computer program, including computer instructions, which, when executed in a processor of a device, implements a method for generating a special effect program file package or a special effect generating method of any of the above embodiments of the present invention.

[0341] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for generating a sound special effect program file package or a sound special effect generating method of any of the above embodiments of the present invention is implemented.

[0342] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0343] The method and apparatus of the present invention may be implemented in many ways. For example, the method and apparatus of the present invention may be implemented by software, hardware, firmware, or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0344] The description of the present invention is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the invention to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for particular uses.

Claims

1. A method for generating a sound special effect program file package, characterized in that: The method is executed by a device for generating a sound special effect program file package, wherein the device for generating the sound special effect program file package includes an executable sound special effect program file, wherein the sound special effect program file includes a playback parameter to be set; the method includes: Importing sound materials into the generating device of the sound special effect program file package; Obtaining parameter values ​​of playback parameters of the sound material; Generate a sound special effect program file package executable by a rendering engine according to the sound material and the parameter value of the playback parameter, wherein after the sound special effect program file package is imported into the electronic device, it is used to generate the sound special effect of the sound material when a trigger event occurs in the image played by the electronic device; The triggering event includes any one or more of the following: eye movement, head movement, eyebrow movement, hand movement, mouth movement, shoulder movement, and body movement.

2. The method according to claim 1, characterized in that The importing of sound material includes: An import instruction input through the interactive interface of the operation bar is received, and the sound material in the material folder pointed to by the import instruction is imported.

3. The method according to claim 2, characterized in that The step of receiving an import instruction input through an interactive interface of the operation bar and importing the sound material in the material folder pointed to by the import instruction includes: An import instruction sent through an interactive interface in the special effect parameter setting interface under the operation bar is received, and the sound material in the material folder pointed to by the import instruction is imported.

4. The method according to claim 2, characterized in that: Receiving an import instruction sent through an interactive interface, and importing the sound material in the material folder pointed to by the import instruction, including: Receiving an import instruction sent through the interactive interface, and acquiring and displaying a material folder pointed to by the import instruction; In response to receiving a selection operation for a sound material in the material folder, importing the sound material selected by the sound material selection operation; and / or In response to not receiving an operation of selecting the sound material in the material folder, one or more sound materials in the material folder are selected according to a preset setting, and the sound material selected according to the preset setting is imported.

5. The method according to claim 3, characterized in that: The step of obtaining the parameter value of the playback parameter of the sound material includes: In response to receiving a parameter value of a playback parameter setting for the sound material sent through an interactive interface in the special effect parameter setting interface, using the set parameter value as the parameter value of the playback parameter of the sound material; and / or In response to not receiving the parameter value for the playback parameter setting of the sound material sent through the interactive interface in the special effect parameter setting interface, using the preset parameter value as the parameter value of the playback parameter of the sound material.

6. The method according to any one of claims 1 to 5, characterized in that: The playback parameters include any one or more of the following: Playback control parameters: used to control whether the sound material is played; Trigger mode parameter: used to indicate the trigger event that triggers the playing of the sound material; Loop parameter: used to indicate the number of times the sound material is played in a loop; Delay trigger parameter: used to indicate the time for delaying the playing of the sound material; Trigger end parameter: used to indicate the trigger event for ending the playing of the sound material; Playback frame number parameter: used to indicate the number of frames the sound material is played; Beauty / makeup / face-changing effect parameters: used to indicate the beauty / makeup / face-changing effect displayed at a preset position when playing the sound material; Deformation special effect parameter: used to indicate the deformation effect of generating a deformation area on the image when playing the sound material; The sticker special effect parameter is used to indicate the special effect of generating a sub-material on the image when playing the sound material.

7. The method according to claim 6, characterized in that The triggering event also includes: no event triggering.

8. The method according to any one of claims 1 to 5, characterized in that: The sound special effect program file package includes: a sound special effect program file package generated by a json program.

9. The method according to any one of claims 1 to 5, characterized in that: Also includes: According to the received start-up instruction, an operation interface is displayed, wherein the operation interface includes: an operation bar, a content display bar and / or a program file bar.

10. The method according to claim 9, characterized in that The operation interface includes three areas: left, middle and right; The display operation interface includes: The operation bar is displayed on the left side of the operation interface, the content display bar is displayed in the middle of the operation interface, and the program file bar is displayed on the right side of the operation interface.

11. The method according to any one of claims 1 to 5, characterized in that: After generating the sound special effect program file package executable by the rendering engine, the method further includes: The sound special effect program file package is saved at the location pointed to by the saving instruction according to the received saving instruction.

12. The method according to claim 11, characterized in that The step of saving the sound special effect program file package at the location pointed to by the saving instruction according to the received saving instruction comprises: In response to receiving the save instruction, displaying a save path selection interface and a compression interface; Receiving the save location sent through the save path selection interface; and receiving the compression method sent based on the compression interface, and compressing the sound special effect program file package according to the compression method to generate a compressed file package; The compressed file package is stored in the folder pointed to by the saving location.

13. The method according to claim 12, characterized in that During the process of compressing the sound special effect program file package, the size of the sound material in the sound special effect program file package remains unchanged as the size before being imported.

14. A method for generating sound effects, characterized in that: include: Import pre-generated sound effects program file packages that can be executed by the rendering engine; Acquire the sound material in the sound special effect program file package and the parameter values ​​of the playback parameters of the sound material; Based on the parameter values ​​of the playback parameters of the sound material, generating a sound effect of the sound material when a triggering event occurs in the played image; The triggering event includes any one or more of the following: eye movement, head movement, eyebrow movement, hand movement, mouth movement, shoulder movement, and body movement.

15. The method according to claim 14, characterized in that The sound special effect program file package is generated using the method described in claim 1.

16. The method according to claim 14, characterized in that The image includes any one or more of the following: a static image, an image in a video.

17. The method according to any one of claims 14 to 16, characterized in that: The playback parameters include: playback control parameters, the playback control parameters are used to control whether the sound material is played; The generating of the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material comprises: In response to the parameter value of the playback control parameter being a parameter value for controlling the playback of the sound material, a sound effect of the sound material is generated on the played image based on the parameter value of the playback parameter of the sound material.

18. The method according to any one of claims 14 to 16, characterized in that: The playing parameters include: a trigger mode parameter, the trigger mode parameter is used to indicate a trigger event for triggering the playing of the sound material; The method further comprises: detecting whether a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image; The generating of the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material comprises: In response to detecting a trigger event corresponding to the parameter value of the trigger mode parameter appearing in the image, a sound effect of the sound material is generated on the played image based on the parameter value of the playback parameter of the sound material.

19. The method according to claim 17, characterized in that The playing parameters include: a delay trigger parameter, the delay trigger parameter is used to indicate the time for delaying the playing of the sound material; The generating of the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material comprises: In response to satisfying the playback conditions of the sound material, based on the parameter value of the playback parameter of the sound material, and in accordance with the delayed playback time corresponding to the parameter value of the delay trigger parameter, the sound effects of the sound material on the image in the video are delayed; the playback conditions of the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

20. The method according to any one of claims 14 to 16, characterized in that: The playback parameters include: a loop parameter, which is used to indicate the number of loop playback times of the sound material; The generating of the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material comprises: In response to satisfying the playback conditions of the sound material, based on the parameter value of the playback parameter of the sound material and according to the number of loops corresponding to the parameter value of the loop parameter, the sound material is looped on the image in the video to generate sound effects for the sound material; the playback conditions of the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

21. The method according to any one of claims 14 to 16, characterized in that: The playback parameters include: a playback frame number parameter, the playback frame number parameter is used to indicate how many frames the sound material is played; The generating of the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material comprises: In response to satisfying the playback conditions of the sound material, based on the parameter values ​​of the playback parameters of the sound material and according to the playback frames corresponding to the parameter value of the playback frame number parameter, a sound effect of the sound material is generated on an image corresponding to the playback frame number in the video; the playback conditions of the sound material include: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

22. The method according to any one of claims 14 to 16, characterized in that: The playback parameters include: a trigger end parameter, the trigger end parameter is used to indicate a trigger event for ending the playback of the sound material; The method further comprises: Detecting whether a trigger event corresponding to the parameter value of the trigger end parameter appears in the image; In response to detecting that a trigger event corresponding to the parameter value of the trigger end parameter appears in the image, the generation of the sound effect of the sound material on the image is stopped.

23. The method according to any one of claims 14 to 16, characterized in that: The playback parameters include: beautification / makeup / face-changing effect parameters, and the beautification / makeup / face-changing effect parameters are used to indicate the beautification / makeup / face-changing effect displayed at a preset position when the sound material is played; The method further comprises: When generating the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material, the beauty / makeup / face-changing effect is displayed at the preset part of the image according to the beauty / makeup / face-changing effect parameters.

24. The method according to any one of claims 14 to 16, characterized in that: The playing parameters include: deformation special effect parameters, the deformation special effect parameters are used to represent the deformation effect of generating a deformation area on the image when playing the sound material; The method further comprises: When generating the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material, the deformation effect of the deformation area is generated in the image according to the deformation effect parameters.

25. The method according to any one of claims 14 to 16, characterized in that: The playback parameters include: sticker special effect parameters, which are used to indicate the special effects of generating sub-materials on the image when playing the sound material; The method further comprises: When generating the sound special effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material, the special effects of the sub-material are generated in the played image according to the sticker special effect parameters.

26. The method according to any one of claims 14 to 16, characterized in that: The sound special effect program file package executable by the imported rendering engine includes: The sound special effect program file package is read into the memory by calling an interface function for reading the file package; The sound special effect program file package is parsed to obtain the sound material and the sound special effect program file, wherein the sound special effect program file includes parameter values ​​of playback parameters of the sound material.

27. The method according to claim 26, characterized in that The step of obtaining the parameter value of the playback parameter of the sound material in the special effect program file package includes: Create a sticker handle through the interface function used to create a sticker handle; The sound material and the parameter values ​​of the playback parameters are read and stored in the sticker handle.

28. The method according to claim 27, characterized in that Generating a sound effect of the sound material on the played image based on the parameter value of the playback parameter of the sound material includes: By calling an interface function for rendering sound material, the parameter value of the playing parameter and the sound material to be played are read from the sticker handle; Based on the parameter values ​​of the playback parameters of the sound material, a sound effect of the sound material is generated on the played image.

29. The method according to claim 28, characterized in that When the image is an image in a video, generating a sound effect of the sound material on the played image based on the parameter value of the playback parameter of the sound material, further comprising: According to the parameter value of the playback parameter, the number of video frames required to generate the sound effect of the sound material is obtained, and images corresponding to the number of video frames are read from the video in advance.

30. The method according to claim 27, characterized in that Also includes: In response to the sound special effect program file package being played, the sticker handle is destroyed by calling an interface function for destroying the sticker handle.

31. The method according to any one of claims 14 to 16, characterized in that: Generating the sound effects of the sound material on the played image includes: By calling a built-in player or an external player of the electronic device, a sound effect of the sound material is generated on the played image.

32. A device for generating a sound special effect program file package, characterized in that: The device for generating the sound special effect program file package includes an executable sound special effect program file, and the sound special effect program file includes playback parameters to be set; the device includes: A first import module, used for importing sound materials into the generating device of the sound special effect program file package; A first acquisition module, used to acquire parameter values ​​of playback parameters of the sound material; A first generating module, for generating a sound special effect program file package executable by a rendering engine according to the sound material and the parameter value of the playing parameter, wherein after the sound special effect program file package is imported into the electronic device, it is used to generate the sound special effect of the sound material when a triggering event occurs in the image; The triggering event includes any one or more of the following: eye movement, head movement, eyebrow movement, hand movement, mouth movement, shoulder movement, and body movement.

33. The device according to claim 32, characterized in that The first import module is specifically configured to receive an import instruction input through an interactive interface of the operation bar, and import the sound material in the material folder pointed to by the import instruction.

34. The device according to claim 33, characterized in that The first import module is specifically used to receive an import instruction sent through the interactive interface in the special effect parameter setting interface under the operation bar, and import the sound material in the material folder pointed to by the import instruction.

35. The device according to claim 33, characterized in that The first import module is specifically used for: Receiving an import instruction sent through the interactive interface, and acquiring and displaying a material folder pointed to by the import instruction; In response to receiving a selection operation for a sound material in the material folder, importing the sound material selected by the sound material selection operation; and / or In response to not receiving an operation of selecting the sound material in the material folder, one or more sound materials in the material folder are selected according to a preset setting, and the sound material selected according to the preset setting is imported.

36. The device according to claim 34, characterized in that The first acquisition module is specifically used to: In response to receiving the parameter value of the playback parameter setting for the sound material sent through the interactive interface in the special effect parameter setting interface, using the set parameter value as the parameter value of the playback parameter of the sound material; and / or In response to not receiving the parameter value for the playback parameter setting of the sound material sent through the interactive interface in the special effect parameter setting interface, using the preset parameter value as the parameter value of the playback parameter of the sound material.

37. The device according to any one of claims 32 to 36, characterized in that: The playback parameters include any one or more of the following: Playback control parameters: used to control whether the sound material is played; Trigger mode parameter: used to indicate the trigger event for triggering the playing of the sound material; Loop parameter: used to indicate the number of times the sound material is played in a loop; Delay trigger parameter: used to indicate the time for delaying the playing of the sound material; Trigger end parameter: used to indicate the trigger event for ending the playing of the sound material; Playback frame number parameter: used to indicate the number of frames the sound material is played; Beauty / makeup / face-changing effect parameters: used to indicate the beauty / makeup / face-changing effect displayed at a preset position when playing the sound material; Deformation special effect parameter: used to indicate the deformation effect of generating a deformation area on the image when playing the sound material; The sticker special effect parameter is used to indicate the special effect of generating a sub-material on the image when playing the sound material.

38. The device according to claim 37, characterized in that The triggering event also includes: no event triggering.

39. The device according to any one of claims 32 to 36, characterized in that: The sound special effect program file package includes: a sound special effect program file package generated by a json program.

40. The device according to any one of claims 32 to 36, characterized in that: Also includes: The operation interface includes: an operation bar, a content display bar and / or a program file bar.

41. The device according to claim 40, characterized in that The operation interface includes three areas: left, middle and right; The operation bar is displayed on the left side of the operation interface, the content display bar is displayed in the middle of the operation interface, and the program file bar is displayed on the right side of the operation interface.

42. The device according to any one of claims 32 to 36, characterized in that: Also includes: The saving module is used to save the sound special effect program file package at the location pointed to by the saving instruction according to the received saving instruction.

43. The device according to claim 42, characterized in that The storage module is specifically used for: In response to receiving the save instruction, displaying a save path selection interface and a compression interface; Receiving the save location sent through the save path selection interface; and receiving the compression method sent based on the compression interface, and compressing the sound special effect program file package according to the compression method to generate a compressed file package; The compressed file package is stored in the folder pointed to by the saving location.

44. The device according to claim 43, characterized in that During the process of compressing the sound special effect program file package, the size of the sound material in the sound special effect program file package remains unchanged as the size before being imported.

45. A sound special effect generating device, characterized in that: include: The second import module is used to import a pre-generated sound special effect program file package executable by the rendering engine; A second acquisition module is used to acquire the sound material in the sound special effect program file package and the parameter value of the playback parameter of the sound material; A second generating module, configured to generate a sound effect of the sound material when a triggering event occurs in the played image based on a parameter value of a playback parameter of the sound material; The triggering event includes any one or more of the following: eye movement, head movement, eyebrow movement, hand movement, mouth movement, shoulder movement, and body movement.

46. ​​The device according to claim 45, characterized in that The sound special effect program file package is generated by the method described in claim 1 or the device described in claim 32.

47. The device according to claim 45, characterized in that The image includes any one or more of the following: a static image, an image in a video.

48. The device according to any one of claims 45 to 47, characterized in that The playback parameters include: playback control parameters, the playback control parameters are used to control whether the sound material is played; The second generating module is specifically used to generate a sound effect of the sound material on the played image based on the parameter value of the playback parameter of the sound material in response to the parameter value of the playback control parameter being a parameter value for controlling the playback of the sound material.

49. The device according to any one of claims 45 to 47, characterized in that The playing parameters include: a trigger mode parameter, the trigger mode parameter is used to indicate a trigger event for triggering the playing of the sound material; The device further comprises: a first detection module, configured to detect whether a trigger event corresponding to a parameter value of the trigger mode parameter appears in the image; The second generating module is specifically used to generate sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material in response to the first detecting module detecting a trigger event corresponding to the parameter value of the trigger mode parameter appearing in the image.

50. The device according to claim 48, characterized in that The playing parameters include: a delay trigger parameter, the delay trigger parameter is used to indicate the time for delaying the playing of the sound material; The second generating module is specifically used to delay the generation of the sound effects of the sound material on the image in the video in response to the playback condition of the sound material being met, based on the parameter value of the playback parameter of the sound material and according to the delayed playback time corresponding to the parameter value of the delay trigger parameter; the playback condition of the sound material being met includes: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

51. The device according to any one of claims 45 to 47, characterized in that: The playback parameters include: a loop parameter, which is used to indicate the number of loop playback times of the sound material; The second generating module is specifically used to respond to the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material, and according to the number of loops corresponding to the parameter value of the loop parameter, to loop the sound material on the image in the video to generate the sound effect of the sound material; the playback condition of the sound material includes: the parameter value of the playback control parameter is the parameter value used to control the playback of the sound material, and / or, a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

52. The device according to any one of claims 45 to 47, characterized in that: The playback parameters include: a playback frame number parameter, the playback frame number parameter is used to indicate how many frames the sound material is played; The second generating module is specifically used to generate a sound effect of the sound material on an image corresponding to the playback frame number in the video in response to satisfying the playback condition of the sound material, based on the parameter value of the playback parameter of the sound material and according to the playback frame number corresponding to the parameter value of the playback frame number parameter; the playback condition of the sound material includes: the parameter value of the playback control parameter is a parameter value for controlling the playback of the sound material, and / or a trigger event corresponding to the parameter value of the trigger mode parameter appears in the image.

53. The device according to any one of claims 45 to 47, characterized in that: The playback parameters include: a trigger end parameter, the trigger end parameter is used to indicate a trigger event for ending the playback of the sound material; The device further comprises: a second detection module, configured to detect whether a trigger event corresponding to the parameter value of the trigger end parameter appears in the image; The second generating module is further configured to stop generating the sound effects of the sound material on the image in response to the second detecting module detecting the occurrence of a trigger event corresponding to the parameter value of the trigger end parameter in the image.

54. The device according to any one of claims 45 to 47, characterized in that: The playback parameters include: beautification / makeup / face-changing effect parameters, and the beautification / makeup / face-changing effect parameters are used to indicate the beautification / makeup / face-changing effect displayed at a preset position when the sound material is played; The second generating module, when generating the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material, is also used to display the beauty / makeup / face-changing effect at the preset part of the image according to the beauty / makeup / face-changing effect parameters.

55. The device according to any one of claims 45 to 47, characterized in that The playing parameters include: deformation special effect parameters, the deformation special effect parameters are used to represent the deformation effect of generating a deformation area on the image when playing the sound material; The second generating module, when generating the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material, is also used to generate the deformation effect of the deformation area in the image according to the deformation special effect parameters.

56. The device according to any one of claims 45 to 47, characterized in that: The playback parameters include: sticker special effect parameters, which are used to indicate the special effects of generating sub-materials on the image when playing the sound material; The second generating module, when generating the sound special effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material, is also used to generate the special effects of the sub-material in the image according to the sticker special effect parameters.

57. The device according to any one of claims 45 to 47, characterized in that The second import module is specifically used to: read the sound special effect program file package into the memory by calling the interface function for reading the file package; and parse the sound special effect program file package to obtain the sound material and the sound special effect program file, wherein the sound special effect program file includes the parameter values ​​of the playback parameters of the sound material.

58. The device according to claim 57, characterized in that When the second acquisition module obtains the parameter value of the playback parameter of the sound material in the special effect program file package, it is specifically used to: create a sticker handle through an interface function for creating a sticker handle; and read the parameter values ​​of the sound material and the playback parameter, and store them in the sticker handle.

59. The device according to claim 58, characterized in that The second generation module is specifically used to: read the parameter values ​​of the playback parameters and the sound material to be played from the sticker handle by calling the interface function for rendering the sound material; and generate the sound effects of the sound material on the played image based on the parameter values ​​of the playback parameters of the sound material.

60. The device according to claim 59, characterized in that When the image is an image in a video, the second generating module is specifically used to obtain the number of video frames required to generate the sound effects of the sound material according to the parameter value of the playback parameter, and to read the image corresponding to the number of video frames from the video in advance.

61. The device according to claim 58, characterized in that The second acquisition module is further used to destroy the sticker handle by calling an interface function for destroying the sticker handle in response to the sound special effect program file package being played.

62. The device according to any one of claims 45 to 47, characterized in that The second generating module is specifically used to generate the sound effects of the sound material on the played image by calling the built-in player or the external player of the electronic device.

63. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute a computer program stored in the memory, and wherein when the computer program is executed, the method described in any one of claims 1 to 31 is implemented.

64. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 31 is implemented.

65. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed in a processor of a device, the method described in any one of claims 1 to 31 is implemented.

Images