Method, device, storage medium and electronic device for processing audio file

Abstract

The application discloses a processing method and device of an audio file, a storage medium and an electronic device, and relates to the technical field of smart homes. The processing method of the audio file comprises the following steps: acquiring an audio file to be processed, and obtaining lyric data of the audio file and audio data of the audio file; acquiring a target material group corresponding to the lyric data from a preset material library; and generating a target video corresponding to the audio file according to the lyric data, the target material group and the audio data. By using the technical solution, the technical problem of how to generate the target video corresponding to the audio file is solved.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a method, apparatus, storage medium, and electronic device for processing audio files. Background Technology

[0002] Currently, as smart home technology continues to improve, the functions of smart home appliances are becoming more and more diversified, and users' requirements for smart home appliances are also getting higher and higher. For example, when users listen to music on devices with screens, the devices can only display music information, resulting in a poor user experience.

[0003] To improve user experience, common solutions involve manually configuring videos to accompany the music. However, this method is labor-intensive, requiring significant manpower and is inefficient. Therefore, a technical challenge exists in generating target videos that correspond to the audio files.

[0004] No effective solution has yet been proposed for the technical problem of how to generate a target video corresponding to an audio file in related technologies. Summary of the Invention

[0005] This application provides a method, apparatus, storage medium, and electronic device for processing audio files, to at least solve the technical problem in the related art of how to generate a target video corresponding to an audio file.

[0006] According to one embodiment of this application, an audio file processing method is provided, comprising: acquiring an audio file to be processed, and obtaining lyrics data and audio data of the audio file; acquiring a target material group corresponding to the lyrics data from a preset material library; and generating a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data.

[0007] In an exemplary embodiment, obtaining the target material group corresponding to the lyrics data from a preset material library includes: using the Jieba word segmentation tool to segment the lyrics in the lyrics data to obtain multiple sets of word segmentation results corresponding to the lyrics data, wherein each set of word segmentation results corresponds to a line of lyrics; obtaining a set of materials corresponding to each set of word segmentation results from the preset material library to obtain multiple sets of materials; and combining the multiple sets of materials according to the order of the lyrics corresponding to each set of word segmentation results to obtain the target material group.

[0008] In an exemplary embodiment, during the process of segmenting lyrics in the lyrics data using the Jieba segmentation tool, the method further includes: searching for a first word in a preset dictionary that is consistent with the first segmentation result in the multi-group segmentation results, wherein the preset dictionary includes different words and dictionary information of the different words; determining the segmentation priority of the first segmentation result based on the dictionary information of the first word; and setting the number of materials corresponding to the first segmentation result based on the segmentation priority of the first segmentation result.

[0009] In an exemplary embodiment, during the process of segmenting lyrics in the lyrics data using the Jieba segmentation tool, the method further includes: obtaining a second word from a preset stop word library, wherein the preset stop word library is used to verify the multi-group segmentation results; if it is determined that the second segmentation result in the multi-group segmentation results is consistent with the second word, confirming that the second segmentation result has not passed the verification and discarding the second segmentation result; if it is determined that the second segmentation result is inconsistent with the second word, confirming that the second segmentation result has passed the verification and obtaining the material corresponding to the second segmentation result from the preset material library.

[0010] In an exemplary embodiment, obtaining materials corresponding to each group of word segmentation results from the preset material library to obtain multiple sets of materials includes: for each word segment in each group of word segmentation results, obtaining material tags matching each word segment from the preset material library; determining multiple materials corresponding to the material tags as a material set corresponding to each word segment; obtaining multiple material sets corresponding to each group of word segmentation results based on a material set corresponding to each word segment in each group of word segmentation results; and determining the multiple material sets as a set of materials corresponding to each group of word segmentation results to obtain multiple sets of materials corresponding to the multiple groups of word segmentation results.

[0011] In an exemplary embodiment, in the process of determining multiple materials corresponding to the material tag as a material set corresponding to each word segmentation, the method further includes: obtaining the matching degree between each material in the multiple materials and the material tag; determining the recommendation priority of each material in the material set based on the comparison results of multiple matching degrees; and sorting each material in the material set according to the recommendation priority.

[0012] In an exemplary embodiment, during the process of generating a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data, the method further includes: generating multiple target videos corresponding to the audio file based on the lyrics data, multiple sets of materials within the target material group, and the audio data; obtaining a filtering result of the target object filtering the multiple target videos; if the filtering result is satisfactory, obtaining the target video that meets the expected requirements indicated by the filtering result; if the filtering result is unsatisfactory, generating a target video corresponding to the audio file based on the modified target material group and the audio data by the target object, according to the lyrics data.

[0013] According to another embodiment of the present application, an audio file processing apparatus is also provided, comprising: a first acquisition module, configured to acquire an audio file to be processed, and obtain lyrics data and audio data of the audio file; a second acquisition module, configured to acquire a target material group corresponding to the lyrics data from a preset material library; and a generation module, configured to generate a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described audio file processing method when it is run.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described audio file processing method through the computer program.

[0016] In this embodiment, an audio file to be processed is obtained, and lyrics data and audio data of the audio file are obtained; a target material group corresponding to the lyrics data is obtained from a preset material library; based on the lyrics data, the target material group and the audio data, a target video corresponding to the audio file is generated; by adopting the above technical solution, the technical problem of how to generate a target video corresponding to an audio file is solved, thereby achieving the technical effect of automatically generating a video corresponding to an audio file. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the hardware environment for an audio file processing method according to an embodiment of this application.

[0020] Figure 2 This is a flowchart of an audio file processing method according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the data composition according to an embodiment of this application;

[0022] Figure 4 This is a flowchart (a) illustrating an audio file processing method according to an embodiment of this application;

[0023] Figure 5 This is a flowchart (II) illustrating the audio file processing method according to an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of a material library according to an embodiment of this application;

[0025] Figure 7 This is a structural block diagram of an audio file processing apparatus according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to one aspect of the embodiments of this application, a method for processing audio files is provided. This audio file processing method is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned audio file processing method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0029] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0030] This embodiment provides a method for processing audio files, applied to the aforementioned computer terminal. Figure 2 This is a flowchart of an audio file processing method according to an embodiment of this application, which includes the following steps:

[0031] Step S202: Obtain the audio file to be processed, and obtain the lyrics data and audio data of the audio file;

[0032] Step S204: Obtain the target material group corresponding to the lyrics data from the preset material library;

[0033] It should be noted that the aforementioned preset material library (which can be referred to as the material library) may include an unstructured file storage system (images, etc.) and a tag library. The tag library includes tags corresponding to the images. Tags can be used to transform incomprehensible materials into systematic, multi-dimensional, and categorized materials that the program can understand. In other words, obtaining materials from the preset material library can be understood as the process of obtaining materials corresponding to tags.

[0034] Step S206: Based on the lyrics data, the target material group, and the audio data, generate the target video corresponding to the audio file.

[0035] Through the above steps, an audio file to be processed is obtained, along with lyrics data and audio data. A target material group corresponding to the lyrics data is retrieved from a preset material library. Based on the lyrics data, the target material group, and the audio data, a target video corresponding to the audio file is generated. This solves the technical problem of how to generate a target video corresponding to an audio file in related technologies, thereby achieving the technical effect of automatically generating a video corresponding to an audio file.

[0036] It should be noted that the target video playback device may include, but is not limited to, smart TVs, tablets, and other screen-equipped devices.

[0037] In an exemplary embodiment, to better understand the process of obtaining the target material group corresponding to the lyrics data from the preset material library in step S204 above, the following technical solution is proposed, the specific steps of which include: using the Jieba word segmentation tool to segment the lyrics in the lyrics data to obtain multiple sets of word segmentation results corresponding to the lyrics data, wherein each set of word segmentation results corresponds to a line of lyrics; obtaining a set of materials corresponding to each set of word segmentation results from the preset material library to obtain multiple sets of materials; and combining the multiple sets of materials according to the order of the lyrics corresponding to each set of word segmentation results to obtain the target material group.

[0038] It should be noted that the Jieba segmentation tool in this embodiment can be the jieba segmentation framework under the Python language system. The jieba segmentation framework supports four segmentation modes: precise mode, full mode, search engine mode, and Paddle. Specifically: Precise mode attempts to segment sentences as accurately as possible, suitable for text analysis. Full mode scans all words that can form words in a sentence, which is very fast but cannot resolve ambiguity. Search engine mode, based on precise mode, further segments long words to improve recall, suitable for search engine segmentation. Paddle mode utilizes the Paddle deep learning framework to train a sequence labeling network model for segmentation. In precise mode, the following preparations can also be made: Custom dictionary: customize priorities (words, word frequency, part of speech). Custom stop word library: remove some characters and words, such as function words, modal particles, special punctuation marks, etc. Set the default segmentation result to take the first 1 / 3.

[0039] In an exemplary embodiment, further, during the process of segmenting the lyrics in the lyrics data using the Jieba segmentation tool, the following steps can also be implemented, specifically including: searching for a first word in a preset dictionary that is consistent with the first segmentation result in the multi-group segmentation results, wherein the preset dictionary includes different words and dictionary information of the different words; determining the segmentation priority of the first segmentation result based on the dictionary information of the first word, and setting the number of materials corresponding to the first segmentation result based on the segmentation priority of the first segmentation result.

[0040] It should be noted that the dictionary information mentioned above may include word frequency, word part of speech, word priority, etc., and this application does not impose any restrictions on this.

[0041] In the process of setting the number of materials corresponding to the first segmentation result according to the segmentation priority of the segmentation result, the first segmentation result with a higher segmentation priority corresponds to a larger number of materials.

[0042] In an exemplary embodiment, other technical solutions are proposed in the process of segmenting lyrics in the lyrics data using the Jieba word segmentation tool. Specific steps include: obtaining a second word from a preset stop word library, wherein the preset stop word library is used to verify the multi-group segmentation results; if the second segmentation result in the multi-group segmentation results is consistent with the second word, confirming that the second segmentation result has not passed the verification and discarding the second segmentation result; if the second segmentation result is inconsistent with the second word, confirming that the second segmentation result has passed the verification and obtaining the material corresponding to the second segmentation result from the preset material library.

[0043] The aforementioned preset stop word list can include, but is not limited to, function words, modal particles, and special punctuation marks. For example, if the second word segmentation result is "very" or "extremely", and it is determined that the second word segmentation result is consistent with the function words "very" or "extremely" in the preset stop word list, then the second word segmentation result is discarded.

[0044] In an exemplary embodiment, the technical solution for obtaining multiple sets of materials by acquiring the materials corresponding to each group of word segmentation results from the preset material library is further described by the following steps: For each word segment in each group of word segmentation results, a material tag matching each word segment is acquired from the preset material library; multiple materials corresponding to the material tag are determined as a material set corresponding to each word segment; multiple material sets corresponding to each group of word segmentation results are obtained based on a material set corresponding to each word segment in each group of word segmentation results; and the multiple material sets are determined as a set of materials corresponding to each group of word segmentation results, thereby obtaining multiple sets of materials corresponding to the multiple groups of word segmentation results.

[0045] In an exemplary embodiment, an implementation scheme is proposed in the process of determining multiple materials corresponding to the material tag as a material set corresponding to each word segmentation. The specific steps include: obtaining the matching degree between each material in the multiple materials and the material tag; determining the recommendation priority of each material in the material set according to the comparison results of multiple matching degrees; and sorting each material in the material set according to the recommendation priority.

[0046] In an exemplary embodiment, to better understand the technical solution in step S206 above of generating a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data, the following technical solution is further proposed, with specific steps including: generating multiple target videos corresponding to the audio file based on the lyrics data, multiple sets of materials in the target material group, and the audio data; obtaining the filtering results of the target object filtering the multiple target videos; if the filtering results are satisfactory, obtaining the target video that meets the expected requirements indicated by the filtering results; if the filtering results are unsatisfactory, generating the target video corresponding to the audio file based on the modified target material group and the audio data of the target object, according to the lyrics data.

[0047] Through the above embodiments, by generating multiple videos at once for selection, the accuracy and matching degree of the target video can be greatly improved. Furthermore, the video can be regenerated based on the results of manual modification, forming a positive feedback loop and improving the satisfaction of the target user (i.e., the target object) with the video.

[0048] Furthermore, in one embodiment, the process of combining the multiple sets of materials according to the lyric order corresponding to each set of word results to obtain the target material group may further include the following steps: determining the order value of the lyric order corresponding to each set of word results; calculating the matching degree of the set of materials according to a preset matching algorithm; obtaining the sum of the weight ratio corresponding to the order value and the weight ratio corresponding to the matching degree, and determining the sum as the weight value of the set of materials corresponding to each set of word results; combining the multiple sets of materials according to the weight value to obtain the target material group.

[0049] In an optional embodiment, a technical solution is also proposed to generate a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data. Specifically, this includes: obtaining the singer information and composition information of the audio file; generating cover information of the audio file based on the singer information, composition information, and the cover template of the audio file; and generating a target video corresponding to the audio file based on the lyrics data, the target material group, the audio data, and the cover information.

[0050] To better understand the process of the above-mentioned audio file processing method, the implementation flow of the above-mentioned audio file processing method will be described below in conjunction with optional embodiments, but it is not intended to limit the technical solution of the embodiments of this application.

[0051] This embodiment provides a method for processing audio files. Figure 3 This is a schematic diagram of the data composition according to an embodiment of this application, such as... Figure 3 As shown, a video file consists of an audio stream, a video stream, a subtitle stream, and other components. An audio file (equivalent to the audio data mentioned above) includes audio, subtitles (equivalent to the lyrics data mentioned above), and other components. In the process of generating a video file from an audio file, the required audio and subtitle streams can be directly obtained from the music file. As for the video stream, since it is essentially a continuous playback of images and materials at different frame rates, suitable materials (i.e., the target material group mentioned above) can be matched based on the audio and lyrics, and then a video stream can be generated based on these materials.

[0052] Next, in one embodiment, in combination Figures 4-6 The audio file processing flow is explained. Specifically:

[0053] Step 1: Determine the basic configuration parameters, which are the default values ​​for globally common configuration parameters used when processing data. For example, video-related configuration parameters may include: output format, video frame rate, video bitrate, frame rate, video width and height, aspect ratio, resolution, etc. These configuration parameters have default values, but the target object can also modify these parameters.

[0054] Step 2: Analyze the music: This involves processing the audio file. The music can be broken down into three parts: audio, subtitles, and cover art (i.e., other information). Then, an audio stream is generated based on the audio, a subtitle stream is generated based on the subtitles, and a video cover art is generated based on the cover art and author information. If no cover art is available, it can be manually added.

[0055] Step 3: Matching materials: Match the materials in the material library according to the word segmentation results of the subtitles (i.e., the word segmentation results mentioned above). A single query can return n sets of material tuples (i.e., the multiple sets of materials mentioned above). Generate a video stream based on the returned n sets of material tuples and their corresponding weights, where n is a positive integer.

[0056] It should be noted that, as Figure 6 As shown, the media library (i.e., the aforementioned media library) includes an unstructured data storage system (such as images, transition effects, and video clips) and a tag library. Images in the media library need to be automatically tagged upon upload, or tags can be manually added and the tag library managed. The media library also includes an Elasticsearch (ES) search engine and an OSS object storage service. OSS converts all files into URLs and stores them in ES. ES's reverse indexing capability enables rapid matching based on word segmentation and provides a media maintenance and management page for easy maintenance and management by operations personnel.

[0057] When matching materials, the built-in "Markov chain algorithm" can be used to statistically analyze the probability of each word segmentation match and material selection, intelligently predict the matching materials, generate a priority ranking, and then match materials according to the priority, thereby achieving increasingly higher matching accuracy.

[0058] In one embodiment, if no matching material is found in the material library, a default material is returned. The default material represents general-purpose material suitable for various scenarios; for example, it may include looping images or short videos of sunlight, beaches, or flowing water. Additionally, the material library can be integrated with web crawler systems to automatically expand the library by fetching data from vertical industry websites.

[0059] Optionally, in other embodiments, a linear regression algorithm can be used to statistically analyze the probability of each word segmentation match and material selection. Using a linear regression algorithm, only one material can be matched based on one word segmentation. For example, if image A is selected, image B cannot be selected. Therefore, after setting constraints, a unique image can be determined by linear regression.

[0060] In one embodiment, along with returning n sets of source material tuples, the system can also return video configuration parameters (transition effects, subtitle effects, etc.). After returning the n sets of source material tuples, the tuples are manually corrected. The correction process can include selecting, deleting, and adjusting elements, such as adjusting subtitle effects and transition effects. Then, n sets of videos are generated based on the corrected source material tuples.

[0061] Step 4: Generate Videos: Generate n videos based on the obtained video stream, audio stream, and subtitle stream.

[0062] Specifically, the process of converting images to videos can be achieved through the following procedure:

[0063] $ffmpeg -r 25 -i img / img%03d.jpg -vcodec libx264-pix_fmtyuv420pvideo.mp4.

[0064] The meaning is as follows:

[0065] -r: rate, used to set the video frame rate. Video frame rate is the number of frames displayed per second, commonly 30FPS, 25FPS, or 24FPS. This setting is 25FPS, meaning 25 images per second.

[0066] -i: input, which means input the source file.

[0067] %03d is actually 001, 002, 003...100.

[0068] -vcodec: video codec, which is the video encoding format. Common examples include H.264, or libx264.

[0069] -pix_fmt: pixel formats, i.e., yuv420p is one of the YUV formats mentioned above.

[0070] video.mp4: The final output filename.

[0071] Furthermore, the process of adding audio can also be achieved through the following procedure:

[0072] $ffmpeg-i input.mp4-i input.mp3-c:a aac-c:v libx264-shortest audio_recode_short.mp4.

[0073] Furthermore, the process of generating and adding subtitles can also be achieved through the following procedure:

[0074] For example, SRT format subtitles:

[0075] $ffmpeg -i input.mp4 -vf subtitles=input.srt video_with_srt.mp4.

[0076] Step 5, Manual Review: Generate multiple videos at once for manual screening.

[0077] Step 6: Regenerate: If multiple videos do not meet the requirements, you can manually modify some configuration parameters and then choose to regenerate. Alternatively, you can edit the footage of a video segment and regenerate multiple videos.

[0078] In the above embodiments, word segmentation is performed based on mature frameworks and technologies in the field of natural language processing, which can obtain word segmentation results of lyrics data. The word segmentation results are matched with materials in the material library, and videos are generated based on the matched materials, which can greatly improve the efficiency of generating videos corresponding to audio.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0080] Figure 7 This is a structural block diagram of an audio file processing apparatus according to an embodiment of this application; as shown below. Figure 7 As shown, it includes:

[0081] The first acquisition module 62 is used to acquire the audio file to be processed and obtain the lyrics data and audio data of the audio file;

[0082] The second acquisition module 64 is used to acquire the target material group corresponding to the lyrics data from the preset material library;

[0083] It should be noted that the aforementioned preset material library may include an unstructured file storage system (images, etc.) and a tag library. The tag library includes tags corresponding to the images. Tags can be used to transform incomprehensible materials into systematic, multi-dimensional, and categorized materials that the program can understand. In other words, obtaining materials from the preset material library can be understood as the process of obtaining materials corresponding to tags.

[0084] The generation module 66 is used to generate a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data.

[0085] The above-described device acquires an audio file to be processed, and obtains the lyrics data and audio data of the audio file; retrieves a target material group corresponding to the lyrics data from a preset material library; and generates a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data. This solves the technical problem of how to generate a target video corresponding to an audio file in related technologies, thereby achieving the technical effect of automatically generating a video corresponding to an audio file.

[0086] It should be noted that the target video playback device may include, but is not limited to, smart TVs, tablets, and other screen-equipped devices.

[0087] In an exemplary embodiment, the second acquisition module 64 is further configured to: use the Jieba word segmentation tool to segment the lyrics in the lyrics data to obtain multiple sets of word segmentation results corresponding to the lyrics data, wherein each set of word segmentation results corresponds to a line of lyrics; acquire a set of materials corresponding to each set of word segmentation results from the preset material library to obtain multiple sets of materials; and combine the multiple sets of materials according to the order of the lyrics corresponding to each set of word segmentation results to obtain the target material set.

[0088] It should be noted that the Jieba segmentation tool in this embodiment can be the jieba segmentation framework under the Python language system. The jieba segmentation framework supports four segmentation modes: precise mode, full mode, search engine mode, and Paddle. Specifically: Precise mode attempts to segment sentences as accurately as possible, suitable for text analysis. Full mode scans all words that can form words in a sentence, which is very fast but cannot resolve ambiguity. Search engine mode, based on precise mode, further segments long words to improve recall, suitable for search engine segmentation. Paddle mode utilizes the Paddle deep learning framework to train a sequence labeling network model for segmentation. In precise mode, the following preparations can also be made: Custom dictionary: customize priorities (words, word frequency, part of speech). Custom stop word library: remove some characters and words, such as function words, modal particles, special punctuation marks, etc. Set the default segmentation result to take the first 1 / 3.

[0089] In an exemplary embodiment, the second acquisition module 64 is further configured to: search for a first word in a preset dictionary that is consistent with the first word segmentation result in the multi-group word segmentation results, wherein the preset dictionary includes different words and dictionary information of the different words; determine the word segmentation priority of the first word segmentation result according to the dictionary information of the first word, and set the number of materials corresponding to the first word segmentation result according to the word segmentation priority of the word segmentation result.

[0090] It should be noted that the dictionary information mentioned above may include word frequency, word part of speech, word priority, etc., and this application does not impose any restrictions on this.

[0091] In the process of setting the number of materials corresponding to the first segmentation result according to the segmentation priority of the segmentation result, the first segmentation result with a higher segmentation priority corresponds to a larger number of materials.

[0092] In an exemplary embodiment, the second acquisition module 64 is further configured to: acquire a second word from a preset stop word library, wherein the preset stop word library is used to verify the multi-group word segmentation results; if it is determined that the second word segmentation result in the multi-group word segmentation results is consistent with the second word, confirm that the second word segmentation result has not passed the verification and discard the second word segmentation result; if it is determined that the second word segmentation result is inconsistent with the second word, confirm that the second word segmentation result has passed the verification and acquire the material corresponding to the second word segmentation result from the preset material library.

[0093] The aforementioned preset stop word list can include, but is not limited to, function words, modal particles, and special punctuation marks. For example, if the second word segmentation result is "very" or "extremely", and it is determined that the second word segmentation result is consistent with the function words "very" or "extremely" in the preset stop word list, then the second word segmentation result is discarded.

[0094] In an exemplary embodiment, the second acquisition module 64 is further configured to: for each word segment in each group of word segmentation results, acquire a material tag matching each word segmentation from the preset material library; determine multiple materials corresponding to the material tag as a material set corresponding to each word segmentation; obtain multiple material sets corresponding to each group of word segmentation results based on a material set corresponding to each word segmentation result; determine the multiple material sets as a set of materials corresponding to each group of word segmentation results, thereby obtaining multiple sets of materials corresponding to the multiple groups of word segmentation results.

[0095] In an exemplary embodiment, the second acquisition module 64 is further configured to: acquire the matching degree between each material in the plurality of materials and the material tag; determine the recommendation priority of each material in the material set based on the comparison results of the plurality of matching degrees; and sort each material in the material set according to the recommendation priority.

[0096] In an exemplary embodiment, the generation module 66 is further configured to: generate multiple target videos corresponding to the audio file based on the lyrics data, multiple sets of materials in the target material group, and the audio data; obtain the filtering results of the target object filtering the multiple target videos; if the filtering results are satisfactory, obtain the target video that meets the expected requirements indicated by the filtering results; if the filtering results are unsatisfactory, generate the target video corresponding to the audio file based on the modified target material group and the audio data of the target object, according to the lyrics data.

[0097] Furthermore, in one embodiment, the second acquisition module 64 is further configured to: determine the order value of the lyrics order corresponding to each group of word results; calculate the matching degree of the group of materials according to a preset matching algorithm; obtain the sum of the weight ratio corresponding to the order value and the weight ratio corresponding to the matching degree, and determine the sum as the weight value of the group of materials corresponding to each group of word results; and combine the multiple groups of materials according to the weight value to obtain the target material group.

[0098] In an optional embodiment, the generation module 66 is further configured to: obtain the singer information and composition information of the audio file; generate the cover information of the audio file based on the singer information, composition information and cover template of the audio file; and generate the target video corresponding to the audio file based on the lyrics data, the target material group, the audio data and the cover information.

[0099] Embodiments of this application also provide a storage medium including a stored program, wherein the program executes any of the methods described above when it is run.

[0100] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0101] S1, Obtain the audio file to be processed, and obtain the lyrics data and audio data of the audio file;

[0102] S2, retrieve the target material group corresponding to the lyrics data from the preset material library;

[0103] S3, Based on the lyrics data, the target material group, and the audio data, generate the target video corresponding to the audio file.

[0104] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0105] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0106] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0107] S1, Obtain the audio file to be processed, and obtain the lyrics data and audio data of the audio file;

[0108] S2, retrieve the target material group corresponding to the lyrics data from the preset material library;

[0109] S3, Based on the lyrics data, the target material group, and the audio data, generate the target video corresponding to the audio file.

[0110] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0111] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0112] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0113] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing audio files, characterized in that, include: Obtain the audio file to be processed, and obtain the lyrics data and audio data of the audio file; Retrieve the target material group corresponding to the lyrics data from the preset material library; Based on the lyrics data, the target material group, and the audio data, a target video corresponding to the audio file is generated; Obtaining the target material group corresponding to the lyrics data from the preset material library includes: using the Jieba word segmentation tool to segment the lyrics in the lyrics data to obtain multiple sets of word segmentation results corresponding to the lyrics data, wherein each set of word segmentation results corresponds to a line of lyrics; obtaining a set of materials corresponding to each set of word segmentation results from the preset material library to obtain multiple sets of materials; and combining the multiple sets of materials according to the order of the lyrics corresponding to each set of word segmentation results to obtain the target material group. In the process of segmenting lyrics in the lyrics data using the Jieba word segmentation tool, the method further includes: searching for a first word in a preset dictionary that is consistent with the first word segmentation result in the multi-group word segmentation results, wherein the preset dictionary includes different words and dictionary information of the different words; determining the word segmentation priority of the first word segmentation result based on the dictionary information of the first word, and setting the number of materials corresponding to the first word segmentation result based on the word segmentation priority of the first word segmentation result.

2. The method for processing audio files according to claim 1, characterized in that, In the process of segmenting the lyrics in the lyrics data using the Jieba segmentation tool, the method further includes: Obtain the second word from a preset stop word list, wherein the preset stop word list is used to verify the multi-group word results; If it is determined that the second word segmentation result in the multi-group word segmentation results is consistent with the second word, and the second word segmentation result fails the test, the second word segmentation result is discarded. If it is determined that the second word segmentation result is inconsistent with the second word, the second word segmentation result is confirmed to pass the test, and the material corresponding to the second word segmentation result is obtained from the preset material library.

3. The method for processing audio files according to claim 1, characterized in that, The materials corresponding to each group of word results are obtained from the preset material library to obtain multiple groups of materials, including: For each word segment in each group of segmentation results, obtain the material tag that matches each word segmentation from the preset material library; The multiple materials corresponding to the material tags are determined as a material set corresponding to each word segmentation; Based on a set of materials corresponding to each word in each group of word segmentation results, multiple sets of materials corresponding to each group of word segmentation results are obtained. The multiple sets of materials are determined as a set of materials corresponding to each set of word results, thus obtaining multiple sets of materials corresponding to the multiple sets of word results.

4. The method for processing audio files according to claim 3, characterized in that, In the process of determining multiple materials corresponding to the material tags as a material set corresponding to each word segmentation, the method further includes: Obtain the matching degree between each of the multiple materials and the material tag; The recommendation priority of each material in the material set is determined based on the comparison results of multiple matching degrees; Each material in the material set is sorted according to the recommended priority.

5. The method for processing audio files according to claim 1, characterized in that, In the process of generating the target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data, the method further includes: Based on the lyrics data, multiple sets of materials within the target material group and the audio data are used to generate multiple target videos corresponding to the audio file; Obtain the filtering results of the target object on the multiple target videos; If the filtering results are deemed satisfactory, the target video that meets the expected requirements as indicated by the filtering results is obtained. If the filtering result is deemed unsatisfactory, the target video corresponding to the audio file is generated based on the lyrics data, the modified target material group of the target object, and the audio data.

6. An audio file processing apparatus, characterized in that, include: The first acquisition module is used to acquire the audio file to be processed and obtain the lyrics data and audio data of the audio file; The second acquisition module is used to acquire the target material group corresponding to the lyrics data from the preset material library; The generation module is used to generate a target video corresponding to the audio file based on the lyrics data, the target material group, and the audio data; The second acquisition module is further configured to use the Jieba word segmentation tool to segment the lyrics in the lyrics data to obtain multiple sets of word segmentation results corresponding to the lyrics data, wherein each set of word segmentation results corresponds to a line of lyrics; acquire a set of materials corresponding to each set of word segmentation results from the preset material library to obtain multiple sets of materials; combine the multiple sets of materials according to the order of the lyrics corresponding to each set of word segmentation results to obtain the target material set; The second acquisition module is further configured to search for a first word that is consistent with the first word segmentation result in the multi-group word segmentation results from a preset dictionary, wherein the preset dictionary includes different words and dictionary information of the different words; determine the word segmentation priority of the first word segmentation result according to the dictionary information of the first word, and set the number of materials corresponding to the first word segmentation result according to the word segmentation priority of the word segmentation result.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 5.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 5 through the computer program.

Images