Song work generation method and device, computer equipment and readable storage medium

Abstract

The application relates to a song work generation method and device, computer equipment and a readable storage medium, and relates to the technical field of audio processing. The method comprises the following steps: in response to a song work generation instruction, calling an audio synthesizer of an operating system to synthesize accompaniment audio and human voice audio recorded based on the accompaniment audio, so as to obtain target audio of a song work; in the case that a calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, using the human voice audio to perform integrity detection on the target audio, and in the integrity detection process of the target audio, a generation progress of the song work is displayed on a song work recording interface; in the case that the target audio passes the integrity detection, a work preview interface of the song work is displayed; the work preview interface is used for guiding to save or publish the song work. The method can improve the accuracy of an audio synthesis result and a song work generation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio processing, and in particular to a song work generation method and device, a computer device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] In a song work production scenario, audio needs to be synthesized first, and then the synthesized audio is used to produce a song work. A program software for audio processing can be installed in a device to complete audio synthesis and other tasks. When processing audio synthesis and other tasks, a third-party synthesis tool such as FFmpeg (Fast Forward Mpeg, an open source program tool for recording or converting digital audio or video) can be used for audio synthesis. Although this method is not limited by the black box feature, it will significantly increase the storage resources occupied by the application program, and at the same time, the hardware acceleration and internal optimization functions brought by the operating system of the device cannot be used, resulting in a significant decrease in the speed of synthesis.

[0003] If the third-party synthesis tool is abandoned and the audio synthesizer function provided by the operating system of the device is used to synthesize audio, when the system fluctuates or is abnormal, the callback result of the operating system may show that the synthesis is successful, but the actual synthesized audio is abnormal when played. Since the audio synthesizer of the operating system is a black box implementation, the problem cannot be solved by modifying its implementation, thereby reducing the accuracy of the audio synthesis result and further reducing the accuracy of the song work generation. SUMMARY

[0004] Therefore, it is necessary to provide a song work generation method and device, a computer device, a computer readable storage medium, and a computer program product capable of improving the accuracy of audio synthesis results and song work generation results.

[0005] In a first aspect, an embodiment of the present application provides a song work generation method. The method comprises:

[0006] In response to a song work generation instruction, calling an audio synthesizer provided by an operating system to synthesize accompaniment audio and human voice audio recorded based on the accompaniment audio to obtain target audio of a song work;

[0007] In a case where a calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, performing integrity detection on the target audio using the human voice audio, and during the integrity detection of the target audio, displaying a generation progress of the song work on a song work recording interface;

[0008] In the case that the target audio passes the integrity detection, a work preview interface of the song work is displayed; the work preview interface is used to guide saving or publishing the song work.

[0009] In one of the embodiments, the integrity detection of the target audio by using the human voice audio comprises:

[0010] Determining at least one content to be detected in the target audio;

[0011] According to the audio metadata of the target audio, determining a content position of the content to be detected; the content position represents a relative address of the content to be detected in an audio file of the target audio;

[0012] Obtaining first audio data corresponding to the content position from the audio file of the target audio, and obtaining second audio data corresponding to the content position from the audio file of the human voice audio;

[0013] In the case that the audio fingerprints of the first audio data and the second audio data are consistent, determining that the target audio passes the integrity detection.

[0014] In one of the embodiments, before the step of determining the content position of the content to be detected according to the audio metadata of the target audio, the method further comprises:

[0015] Obtaining the audio metadata of the target audio, and detecting whether each audio parameter recorded in the audio metadata is normal;

[0016] In the case that the each audio parameter is detected to be normal, performing the step of determining the content position of the content to be detected according to the audio metadata of the target audio.

[0017] In one of the embodiments, the content to be detected comprises: a start content of the target audio, an end content of the target audio, and at least one content of the target audio other than the start content and the end content.

[0018] In one of the embodiments, the audio file of the target audio comprises a plurality of data packets arranged in sequence, and the audio metadata comprises a sampling rate of the target audio and an audio frame number included in the data packet; the step of determining the content position of the content to be detected according to the audio metadata of the target audio comprises:

[0019] Obtaining a start time and an end time corresponding to the content to be detected in the target audio;

[0020] determining a relative address of the start data packet in an audio file of the target audio according to the start time, the sampling rate, and a number of audio frames included in the data packet; the start data packet being a data packet in which the to-be-detected content corresponding to the start time is located;

[0021] determining a relative address of the end data packet in an audio file of the target audio according to the end time, the sampling rate, and a number of audio frames included in the data packet; the end data packet being a data packet in which the to-be-detected content corresponding to the end time is located;

[0022] wherein the relative address of the start data packet in the audio file of the target audio and the relative address of the end data packet in the audio file of the target audio are content positions of the to-be-detected content.

[0023] In one of the embodiments, the audio file of the target audio includes a header information region and a streaming media data region; and the plurality of data packets are arranged in the streaming media data region in sequence.

[0024] The determining of the relative address of the start data packet in the audio file of the target audio according to the start time, the sampling rate, and the number of audio frames included in the data packet includes:

[0025] obtaining a sequence position of the start data packet in the audio file according to the start time, the sampling rate, and the number of audio frames included in the data packet; mapping the sequence position of the start data packet in the audio file to a relative address of the start data packet in the streaming media data region by using an offset position function; and obtaining the relative address of the start data packet in the audio file of the target audio according to the relative address of the streaming media data region in the audio file of the target audio and the relative address of the start data packet in the streaming media data region.

[0026] The determining of the relative address of the end data packet in the audio file of the target audio according to the end time, the sampling rate, and the number of audio frames included in the data packet includes:

[0027] obtaining a sequence position of the end data packet in the audio file according to the end time, the sampling rate, and the number of audio frames included in the data packet; mapping the sequence position of the end data packet in the audio file to a relative address of the end data packet in the streaming media data region by using an offset position function; and obtaining the relative address of the end data packet in the audio file of the target audio according to the relative address of the streaming media data region in the audio file of the target audio and the relative address of the end data packet in the streaming media data region.

[0028] In one of the embodiments, the method further includes:

[0029] In a case where the calling result indicates that the audio synthesizer fails to synthesize or the target audio fails to pass the integrity detection, a step of returning to the calling operating system to call an audio synthesizer to synthesize the accompaniment audio and the human voice audio based on the accompaniment audio to obtain the target audio of the song work is performed.

[0030] In one of the embodiments, the method further comprises:

[0031] recording a number of times of unsuccessfully calling the audio synthesizer to synthesize the target audio;

[0032] In a case where the number of times meets a preset number threshold, generating a device restart prompt to instruct the device to restart.

[0033] In one of the embodiments, the method further comprises:

[0034] In response to a draft saving operation for the song work, saving the target audio;

[0035] In a case where the operating system is idle, taking all audio data in the saved target audio as third audio data and taking all audio data in the human voice audio as fourth audio data;

[0036] In a case where audio fingerprints between the third audio data and the fourth audio data are consistent, determining that the target audio passes the integrity detection.

[0037] In a second aspect, the application further provides a song work generation device. The device comprises:

[0038] a target audio synthesizing module configured to, in response to a song work generation instruction, call an audio synthesizer to synthesize the accompaniment audio and the human voice audio based on the accompaniment audio to obtain the target audio of the song work;

[0039] an integrity verifying module configured to, in a case where a calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, use the human voice audio to perform integrity detection on the target audio, and in the integrity detection process of the target audio, display a generation progress of the song work on a song work recording interface;

[0040] a work preview interface displaying module configured to, in a case where the target audio passes the integrity detection, display a work preview interface of the song work; the work preview interface is configured to guide saving or publishing the song work.

[0041] In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0042] In response to the song work generation instruction, calling an audio synthesizer of an operating system to synthesize the accompaniment audio and the human voice audio recorded based on the accompaniment audio, to obtain target audio of the song work;

[0043] In a case where the calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, performing integrity detection on the target audio using the human voice audio, and during the integrity detection of the target audio, displaying a generation progress of the song work on a song work recording interface;

[0044] In a case where the target audio passes the integrity detection, displaying a work preview interface of the song work; the work preview interface is used to guide saving or publishing the song work.

[0045] In a fourth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0046] In response to the song work generation instruction, calling an audio synthesizer of an operating system to synthesize the accompaniment audio and the human voice audio recorded based on the accompaniment audio, to obtain target audio of the song work;

[0047] In a case where the calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, performing integrity detection on the target audio using the human voice audio, and during the integrity detection of the target audio, displaying a generation progress of the song work on a song work recording interface;

[0048] In a case where the target audio passes the integrity detection, displaying a work preview interface of the song work; the work preview interface is used to guide saving or publishing the song work.

[0049] In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program, and the computer program is executed by a processor to implement the following steps:

[0050] In response to the song work generation instruction, calling an audio synthesizer of an operating system to synthesize the accompaniment audio and the human voice audio recorded based on the accompaniment audio, to obtain target audio of the song work;

[0051] In a case where the calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, integrity detection is performed on the target audio using the human voice audio, and during the integrity detection of the target audio, a generation progress of the song work is displayed on a song work recording interface;

[0052] In a case where the target audio passes the integrity detection, a work preview interface of the song work is displayed; the work preview interface is used to guide saving or publishing the song work.

[0053] The song work generation method, device, computer device, storage medium and computer program product have the following advantages. First, the system calls a self-provided audio synthesizer to synthesize the accompaniment audio and the recorded human voice audio to form a target audio; when the synthesis is successful, integrity detection is performed on the target audio using the human voice audio; during the integrity detection, the system updates a generation progress of the song work in real time and displays the generation progress on a recording interface, so that a user can intuitively understand the progress; if the target audio passes the integrity detection, the system automatically displays a work preview interface to guide the user to perform a saving or publishing operation. The technical solution effectively improves the efficiency of song creation and improves the user experience. The integrity detection compensates for the deficiency of the self-provided audio synthesis function of the system, improves the success rate of the song work based on the system audio synthesis, guarantees the integrity and usability of the synthesized song work, effectively avoids the problem that the user has a poor experience due to the abnormal song work that cannot be normally played, and improves the accuracy of the audio synthesis result. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art descriptions. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0055] Figure 1 An application environment diagram of the song work generation method in an embodiment;

[0056] Figure 2 A flowchart of the song work generation method in an embodiment;

[0057] Figure 3 A flowchart of the song work generation method in another embodiment;

[0058] Figure 4 A whole flowchart of the song work generation method in an embodiment;

[0059] Figure 5Fig. 1 is a schematic diagram of a work audio file format of a song work generation method in an embodiment;

[0060] Figure 6 Fig. 2 is a schematic diagram of a need to detect the position of content in a file of a song work generation method in an embodiment;

[0061] Figure 7 Fig. 3 is a structural block diagram of a song work generation device in an embodiment;

[0062] Figure 8 Fig. 4 is an internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0063] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0065] In an embodiment, as shown in Figure 1 Fig. 1, a song work generation method is provided, and the embodiment is exemplified by the method applied to a terminal. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be realized through the interaction of the terminal and the server. In the embodiment, the method includes the following steps:

[0066] S201, in response to a song work generation instruction, calling an audio synthesizer built-in an operating system to synthesize accompaniment audio and human voice audio recorded based on the accompaniment audio, to obtain target audio of a song work.

[0067] The song work refers to a multimedia work related to audio, video or lyrics of a song, for example, a user's human voice and accompaniment can be synthesized, and then synchronized video and lyrics subtitles are added to obtain an audio and video work. Correspondingly, the song work generation instruction is a program instruction for generating a song work, for example, a user triggers the instruction by clicking a "work generation" button displayed in the terminal.

[0068] The system refers to an operating system of a device on which a program software for audio processing is installed, that is, an operating system of a device implementing the method, and the audio synthesizer refers to an audio synthesizing function of the system, such as a K-song software, which is installed in a terminal, and the operating system of the terminal is an IOS operating system, which has an audio synthesizing function of the audio synthesizer, can perform the function of audio synthesis in response to the call of the program software, and feedback the call result after synthesis.

[0069] The accompaniment audio refers to accompaniment of a certain music work, the vocal audio refers to audio of a user recorded by using the terminal in a scene of synchronous playing of the accompaniment audio, and the target audio is audio synthesized by the vocal and the accompaniment. Specifically, the music accompaniment and the vocal audio of the user recorded following the music accompaniment are synthesized to obtain the target audio after synthesis. At this time, the target audio can be audio of the user singing a song with accompaniment.

[0070] S202, in a case where the call result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, integrity detection is performed on the target audio using the vocal audio, and during the integrity detection of the target audio, a generation progress of the song work is displayed on the song work recording interface.

[0071] The successful synthesis of the target audio refers to a call result of the audio synthesis task being audio synthesis success. Specifically, after the audio synthesizer completes the audio synthesis task, the synthesis result is fed back to the calling party as a callback result.

[0072] The integrity detection can include checking various contents of the target audio, such as parameters of the audio and contents of the audio. Generally, the process of integrity detection is related to the generation progress of the song work, however, the generation progress of the song work actually includes various progress, such as audio synthesis progress, integrity verification progress, re-synthesis progress, and progress of generating the song work from the audio and the video.

[0073] In order to improve the experience of the user in the creation of the song work, the embodiments of the present application integrate various progress into one progress, that is, the generation progress of the song work. The advantage of this setting is that the user does not need to understand many technical details such as whether the audio synthesis is called, whether the audio synthesis is successful, whether the audio detection is passed, and the like, but only needs to focus on whether the song work is finally generated successfully, that is, the technical implementation process of audio synthesis, detection, and re-synthesis is imperceptible to the user. When the progress is completed, it indicates that the song work is successfully and completely produced, rather than only the audio synthesis is successful or the audio detection is successful, which greatly improves the user experience in the production of the song work.

[0074] S203, in the case where the target audio passes the integrity detection, displaying a work preview interface of the song work.

[0075] The work preview interface is used to guide saving or publishing the song work. Generally, the generated song work can be previewed in the work preview interface, for example, the finally generated audio and video work can be played. In addition, the work preview interface also includes a control for saving the song work and a control for publishing the song work, which are used to guide the user to perform a triggering operation to select whether to publish and save the generated song work. Illustratively, after the user previews the work and confirms that the work is generated correctly, the user can click the draft saving or work publishing button.

[0076] The above song work generation method, device, computer device, storage medium and computer program product first call the built-in audio synthesizer to synthesize the accompaniment audio and the recorded human voice audio to form a target audio. When the synthesis is successful, the human voice audio is used to perform integrity detection on the target audio. In this process, the system updates the generation progress of the song work in real time and displays it in the recording interface, so that the user can intuitively understand the progress. If the target audio passes the integrity detection, the system will automatically display a work preview interface to guide the user to perform a saving or publishing operation. This technical solution effectively improves the efficiency of song creation and improves the user experience. The integrity detection compensates for the shortcomings of the system's built-in audio synthesis function, improves the success rate of song work based on system audio synthesis, ensures the integrity and usability of the synthesized work, effectively avoids the problem of poor user experience caused by abnormal work that cannot be played normally, and improves the accuracy of the audio synthesis result.

[0077] In one embodiment, the integrity detection of the target audio using the human voice audio includes: determining at least one to-be-detected content in the target audio; determining a content position of the to-be-detected content according to audio metadata of the target audio; the content position represents a relative address of the to-be-detected content in an audio file of the target audio; obtaining first audio data corresponding to the content position from the audio file of the target audio, and obtaining second audio data corresponding to the content position from an audio file of the human voice audio; in the case where the audio fingerprints between the first audio data and the second audio data are consistent, it is determined that the target audio passes the integrity detection.

[0078] The number of the to-be-detected content in the target audio is at least one, and generally, a plurality of to-be-detected contents can be set. The to-be-detected content can be represented by time, for example, the to-be-detected content is the content from the first second to the fifth second, the content from the twentieth second to the twenty-fifth second, and the like. Correspondingly, the content position is the relative address of the to-be-detected content in the audio file of the target audio, specifically, the address relative to the start address of the audio file of the target audio. Since it is necessary to perform consistency detection on the target audio and the human voice audio used to record the target audio, the content position is used for reading the audio data from the audio file, including reading the audio data of the to-be-detected content from the audio file of the target audio, and also including reading the audio data of the to-be-detected content from the audio file of the human voice audio.

[0079] The audio metadata is non-audio information related to the audio file, and represents the structure and attributes of the audio file. In a specific implementation manner, the metadata includes duration, channel data, code rate, and sampling rate, and the like, which can be used to calculate the content position of the to-be-detected content.

[0080] The first audio data is the audio data corresponding to the target audio at the content position; and the second audio data is the audio data corresponding to the human voice audio at the content position.

[0081] Exemplarily, the relative address represented by the content position is used to read the first audio data and the second audio data, and then the audio fingerprints of the first audio data and the second audio data are extracted, and then it is judged whether the audio fingerprints of the first audio data and the second audio data are consistent. If consistent, it indicates that the features of the human voice in the synthesized target audio are consistent with the features of the human voice audio before the synthesis, it is explained that the feature information of the human voice is maintained in the synthesis process, and the synthesis result can retain the timbre and expressiveness of the original singer, so that it is considered that the audio synthesis is successful. It can be understood that, the scheme sets the flag indicating that the target audio synthesis is correct, that is, the human voice features of the first audio data and the second audio data are consistent.

[0082] In this embodiment, by determining the to-be-detected content in the target audio and the content position thereof, the first audio data and the second audio data are extracted, and the consistency comparison is performed by using the audio fingerprints, so as to verify the success of the audio synthesis. The scheme ensures that the features of the synthesized human voice audio are consistent with the original audio, guarantees the sound quality and expressiveness, so that the final generated audio work retains the timbre of the singer, guarantees the accuracy of the integrity verification of the audio, and further improves the reliability of the audio synthesis.

[0083] In one embodiment, before the step of determining the content position of the content to be detected according to the audio metadata of the target audio, the method further comprises: obtaining the audio metadata of the target audio, and detecting whether each audio parameter recorded in the audio metadata is normal; and in the case that each audio parameter is detected to be normal, performing the step of determining the content position of the content to be detected according to the audio metadata of the target audio.

[0084] The audio parameters recorded in the audio metadata include duration, channel data, code rate and sampling rate, etc. The duration refers to the total time of audio playing, the channel data indicates the audio configuration such as mono or stereo, the code rate reflects the sound quality and compression degree, and the sampling rate is the number of samples per second. Verifying the accuracy of the audio metadata can ensure that the quality of the audio file meets the expectation, thereby improving the playing effect and user experience.

[0085] Exemplarily, when detecting the integrity of the target audio, first, each audio parameter recorded in the audio metadata is detected to ensure that the parameters are complete and compliant, which facilitates the subsequent calculation of the content position. If each audio parameter recorded in the audio metadata is incomplete or incorrect, the integrity verification fails, and the audio synthesis needs to be performed again.

[0086] In this embodiment, first, the audio metadata of the target audio is obtained, and each parameter such as the duration, channel data, code rate and sampling rate is detected to ensure that it is normal and compliant. After confirming that the parameters are correct, the position of the content to be detected is determined according to the audio metadata. This effectively ensures the quality of the audio file, improves the accuracy of the subsequent integrity detection, and facilitates the accuracy of the subsequent calculation of the content position and the audio data detection process.

[0087] In one embodiment, the content to be detected includes: the beginning content of the target audio, the end content of the target audio, and at least one content in the target audio other than the beginning content and the end content.

[0088] The content to be detected includes fixed detection content and random detection content. The fixed detection content includes the beginning content and the end content, and the random detection content can be determined in other content other than the fixed detection content by randomly determining a content segment. The at least one content in the target audio other than the beginning content and the end content refers to the random detection content.

[0089] Exemplarily, the detection of the audio content is mainly divided into fixed content detection and random content detection. The fixed part is the audio start position and end position which are most prone to problems, and the middle part is detected by random sampling.

[0090] In this embodiment, by determining the fixed detection content and the random detection content in sequence, the stable detection of the position prone to problems is ensured, and the data amount actually detected is reduced in a random sampling manner, so as to improve the efficiency of the integrity detection on the basis of considering the detection accuracy.

[0091] In one embodiment, according to the audio metadata of the target audio, the content position of the to-be-detected content is determined, including: obtaining the start time and the end time of the to-be-detected content corresponding in the target audio; determining the relative address of the start data packet in the audio file of the target audio according to the start time, the sampling rate and the audio frame number included in the data packet; determining the relative address of the end data packet in the audio file of the target audio according to the end time, the sampling rate and the audio frame number included in the data packet.

[0092] In one embodiment, according to the audio metadata of the target audio, the content position of the to-be-detected content is determined, including: obtaining the start time and the end time of the to-be-detected content corresponding in the target audio; determining the relative address of the start data packet in the audio file of the target audio according to the start time, the sampling rate and the audio frame number included in the data packet; determining the relative address of the end data packet in the audio file of the target audio according to the end time, the sampling rate and the audio frame number included in the data packet.

[0093] In one embodiment, according to the audio metadata of the target audio, the content position of the to-be-detected content is determined, including: obtaining the start time and the end time of the to-be-detected content corresponding in the target audio; determining the relative address of the start data packet in the audio file of the target audio according to the start time, the sampling rate and the audio frame number included in the data packet; determining the relative address of the end data packet in the audio file of the target audio according to the end time, the sampling rate and the audio frame number included in the data packet.

[0094] It should be noted that the start time and the end time of the to-be-detected content corresponding in the target audio refer to the relative time in the audio, for example, the total length of the audio of a song can be 3 minutes, and the to-be-detected content can be the audio content corresponding to the 10th second to the 20th second, in which case the start time can be the 10th second, and the content duration can be 10 seconds, so that the end time of the to-be-detected content corresponding in the target audio is determined as the 20th second.

[0095] In an embodiment, the start time and the end time of the content to be detected can be directly obtained. In another embodiment, the end time of the content to be detected in the target audio is determined according to the start time and the content duration of the content to be detected in the target audio. Exemplarily, the start time of the content to be detected is known as startTime, and the detection duration is known as duration, and the end time of the content to be detected can be calculated as endTime = startTime + duration. Then, the relative addresses of the data packets corresponding to startTime and endTime in the audio file of the target audio (including the relative address of the start data packet in the audio file of the target audio startFilePos and the relative address of the end data packet in the audio file of the target audio endTime) are calculated respectively by using the sampling rate sample and the number of frames framesPerPacket in the data packet in the metadata, which are the content positions of the content to be detected.

[0096] In the embodiment, the start time and the end time of the content to be detected in the target audio are first obtained, and the relative addresses of the data packets corresponding to the start time and the end time in the audio file of the target audio are calculated respectively, which are used as the content positions, thereby improving the calculation efficiency of the content positions.

[0097] In an embodiment, the relative address of the start data packet in the audio file of the target audio is determined according to the start time, the sampling rate and the number of audio frames included in the data packet, including: obtaining the sorting position of the start data packet in the audio file according to the start time, the sampling rate and the number of audio frames included in the data packet; mapping the sorting position of the start data packet in the audio file to the relative address of the start data packet in the streaming media data region by using the offset position function; and obtaining the relative address of the start data packet in the audio file of the target audio according to the relative address of the streaming media data region in the audio file of the target audio and the relative address of the start data packet in the streaming media data region.

[0098] In an embodiment, the relative address of the end data packet in the audio file of the target audio is determined according to the end time, the sampling rate and the number of audio frames included in the data packet, including: obtaining the sorting position of the end data packet in the audio file according to the end time, the sampling rate and the number of audio frames included in the data packet; mapping the sorting position of the end data packet in the audio file to the relative address of the end data packet in the streaming media data region by using the offset position function; and obtaining the relative address of the end data packet in the audio file of the target audio according to the relative address of the streaming media data region in the audio file of the target audio and the relative address of the end data packet in the streaming media data region.

[0099] The audio file of the target audio includes a header information region and a streaming media data region, and the plurality of data packets are arranged in the streaming media data region in sequence.

[0100] Exemplarily, the start time is startTime, the end time is endTime, the sampling rate is sample, and the number of audio frames included in a data packet is framesPerPacket; the calculation of the ordering position of the start data packet and the end data packet in the audio file can be as follows:

[0101] startPacket = startTime * sample / framesPerPacket;

[0102] endPacket = endTime * sample / framesPerPacket;

[0103] Wherein, startPacket is the ordering position of the start data packet in the audio file, and endPacket is the ordering position of the end data packet in the audio file.

[0104] Wherein, the offset position function refers to a function for obtaining the offset position of the data packet of the content to be detected in the streaming media data region (also referred to as a streaming media data segment), which can be denoted as outDataByteOffset(), and the parameter of the offset position function is startPacket or endPacket. That is, startPacket and endPacket are respectively transmitted as parameters into the offset position function to obtain the relative address startSegment of the start data packet in the streaming media data region and the relative address endSegment of the end data packet in the streaming media data region, which are specifically as follows:

[0105] startSegment = outDataByteOffset(startPacket);

[0106] endSegment = outDataByteOffset(endPacket)。

[0107] After startSegment and endSegment are calculated, the relative address startFilePos of the start data packet in the audio file of the target audio and the relative address endFilePos of the end data packet in the audio file of the target audio can be calculated in combination with the relative address of the streaming media data region in the audio file of the target audio. The specific calculation is as follows:

[0108] startFilePos = mdatOffset + startSegment;

[0109] endFilePos = mdatOffset + endSegment.

[0110] wherein the relative address of the stream media data region in the audio file of the target audio, also referred to as the offset value of the stream media data segment in the audio file of the target audio, can be denoted as mdatOffset. Specifically, the mdatOffset is a parameter recorded in the audio metadata, and the mdatOffset can be obtained by parsing the audio metadata. It should be noted that the content position of the to-be-detected content is determined by the startFilePos and the endFilePos. Specifically, the specific meaning of the content position is the position between the relative address corresponding to the startFilePos and the relative address corresponding to the endFilePos.

[0111] In this embodiment, the start packet and the end packet are calculated according to the start time, the end time, the sampling rate and the number of audio frames included in the data packet, and the sorting positions of the start packet and the end packet in the audio file are obtained. Then, the sorting positions are mapped to the relative addresses of the data packets in the stream media data region by using the offset position function. Finally, the relative address of the start packet in the audio file of the target audio is obtained according to the relative address of the stream media data region in the audio file of the target audio. Through the mapping from the sorting position to the relative address, the accurate positioning of the to-be-detected content is ensured, and the accuracy of the integrity detection is ensured.

[0112] In one embodiment, the method further comprises the step of returning to call the audio synthesizer provided by the operating system to synthesize the accompaniment audio and the human voice audio recorded based on the accompaniment audio, to obtain the target audio required by the song work, in the case that the calling result indicates that the synthesis of the audio synthesizer fails or the target audio fails to pass the integrity detection.

[0113] In this embodiment, whether the synthesis of the audio synthesizer fails or the target audio fails to pass the integrity detection indicates that the synthesized audio is abnormal. According to the data statistics on the synthesis abnormality, 95% of the synthesis abnormality can be overcome by re-synthesis. Therefore, when the synthesized audio is abnormal, the re-synthesis is preferentially used to repair the abnormality.

[0114] Specifically, in the case that the synthesized audio is abnormal in the first or first few times, the audio synthesis is preferentially performed again in silence. The meaning of silence is that the synthesis process does not need to be perceived by the user, and the user can only see that the progress bar does not reach 100% and the song work is not successfully generated.

[0115] In the embodiment, when it is detected that the audio synthesizer fails to synthesize or the target audio fails to pass the integrity detection, the accompaniment audio and the human voice audio recorded based on the accompaniment are re-synthesized by calling the audio synthesizer of the operating system; in this process, the audio synthesis is preferentially selected in a mute mode according to the statistical result of the previous synthesis exception, so as to ensure that the user is not disturbed in the synthesis process and the synthesis success rate can be effectively improved.

[0116] In one embodiment, the method further includes: recording the number of times of unsuccessfully calling the audio synthesizer to synthesize the target audio; and generating a device restart prompt information to instruct the device to restart when the number of times meets a preset number threshold.

[0117] The unsuccessfully calling the audio synthesizer can mean that the audio synthesizer has a callback result of synthesis failure after synthesis, and the number of times is the number of times of synthesis failure. Generally, the preset number threshold can be one or two times.

[0118] For example, when the audio synthesizer fails to synthesize the target audio for the first time, it is identified as a system synthesis failure, and the mute retry synthesis is performed. If the synthesis fails again for the second time, the user is guided to restart the device, and the operation log is reported to the background for the developer to locate the problem.

[0119] In the embodiment, the number of times of unsuccessfully calling the audio synthesizer to synthesize the target audio is recorded, and the preset number threshold is set to monitor the number of unsuccessful calls. When the condition is met, the device is instructed to restart, which avoids wasting computing resources in the scenario of repeated synthesis without success, and restarts the device to overcome system exceptions and improve the success rate of audio synthesis.

[0120] In one embodiment, the method further includes: saving the target audio in response to a draft saving operation for a song work; in the case that the operating system is idle, taking all audio data in the saved target audio as third audio data, and taking all audio data in the human voice audio as fourth audio data; and determining that the target audio passes the integrity detection in the case that the audio fingerprints between the third audio data and the fourth audio data are consistent.

[0121] The third audio data is all audio data in the target audio, and the fourth audio data is all audio data in the recorded audio.

[0122] For example, to further improve the accuracy of the detection result, on the basis of the current function, if the user selects to save a draft after recording the audio, all audio data can be detected asynchronously when the system is idle.

[0123] In the embodiment, the draft saving operation of the user is identified, and all audio data is compared when the system is idle, which ensures the accuracy of audio data detection.

[0124] In another embodiment, as shown in Figure 3 a song work generation method is provided, comprising the following steps:

[0125] S301, in response to a song work generation instruction, calling an audio synthesizer provided by an operating system to synthesize accompaniment audio and human voice audio recorded based on the accompaniment audio, to obtain target audio of the song work;

[0126] S302, in the case that the calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, determining at least one to-be-detected content in the target audio;

[0127] S303, obtaining audio metadata of the target audio, and detecting whether each audio parameter recorded in the audio metadata is normal;

[0128] S304, in the case that each audio parameter is detected to be normal, determining a content position of the to-be-detected content according to the audio metadata of the target audio;

[0129] S305, obtaining first audio data corresponding to the content position from an audio file of the target audio, and obtaining second audio data corresponding to the content position from an audio file of the human voice audio; in the case that audio fingerprints between the first audio data and the second audio data are consistent, determining that the target audio passes the integrity detection;

[0130] S306, in the integrity detection process of the target audio, displaying a generation progress of the song work on a song work recording interface;

[0131] S307, in the case that the target audio passes the integrity detection, displaying a work preview interface of the song work;

[0132] S308, in the case that the calling result indicates that the audio synthesizer fails to synthesize, or the target audio fails to pass the integrity detection, returning to the step of calling the audio synthesizer provided by the operating system to synthesize the accompaniment audio and the human voice audio recorded based on the accompaniment audio, to obtain the target audio required by the song work.

[0133] It should be noted that the specific definition of the above steps can refer to the specific definition of the song work generation method described above, and will not be repeated here.

[0134] In the prior art, under the operating system (such as the iOS system) of a certain terminal, the program software (such as the K song software) for audio processing can use the audio synthesizer provided by the system to synthesize a human voice and accompaniment into an audio file when recording a song and synthesizing a work. However, when the system fluctuates or is abnormal, it may cause the actual synthesized work to be abnormal although the system callback result is successful synthesis.

[0135] According to the data analysis result for the above problem, the probability of the above problem is 0.06%. Since the system audio synthesizer is a black box implementation, it cannot be solved by modifying its implementation.

[0136] If a third-party synthesis tool (such as FFmpeg, Fast Forward Mpeg, an open source program tool for recording or converting digital audio or video) is used, it is not limited by the black box, but the application program installation package size will be significantly increased, and the speed of audio synthesis will be significantly reduced due to the inability to use the system internal hardware acceleration and optimization.

[0137] According to the data analysis result, there are mainly three cases of system callback synthesis success but work abnormality: (1) the work audio metadata is lost, causing the work to be unable to play; (2) the work audio metadata is correct, but the data segment data corresponding to the audio file is lost, and the duration can be displayed, but cannot be played; (3) the work audio metadata is correct, but the data segment data corresponding to the audio file is abnormal, and the played sound is abnormal.

[0138] Among them, recording a song refers to the ability to support a user to record a song while listening to the accompaniment sound of the song; recording and synthesizing a song refers to synthesizing the raw human voice recorded by the user and the accompaniment data into a complete audio work; audio metadata refers to various parameter information of the audio file, including audio code rate, sampling rate, duration, channel, and other information.

[0139] Based on this, the embodiment of the present application provides a song work generation method, and the following Figures 4 to 6 The song work generation method is described in detail with a specific embodiment. It should be understood that the following description is only exemplary and is not a specific limitation of the application.

[0140] The embodiment of the present application provides a song work generation method, also known as a complete integrity detection scheme for synthesizing a song work based on an iOS system. In the technical scheme provided by the song work generation method, when the system synthesis callback is successful, a work integrity detection is performed again, so that the synthesized work is complete and available. First, the audio metadata of the work is detected to be normal (work duration, sampling rate, etc.), and then the data content detection is performed. The detection methods include fixed and random content detection. The beginning and end of the audio are detected fixedly, and the middle part of the audio is detected randomly due to the differences in the recorded songs and the generated sound data. The detection method is to compare the audio fingerprints of the partial sound data of the work and the mixed data of the original voice and accompaniment, to determine whether the sound before and after the synthesis is consistent. Only when all the detections pass, the work is recognized to be complete and available. If any process fails, the system synthesis is recognized to fail, and the silent retry synthesis is performed. Currently, more than 95% of the abnormalities can be solved, and if the synthesis fails again, the user is guided to restart the device and report the operation log to the background, so that the developer can locate the problem.

[0141] On the product side, the song work generation method provided by the embodiment of the present application can be applied to the preview and release page of the recording function of a certain K song program software. When the user selects to save or release the audio file, when the system callback work synthesis is successful, the integrity detection is performed again, so that the work is complete and effective.

[0142] The overall flowchart of the song work generation method provided by the embodiment of the present application is shown in Figure 4 The embodiment of the present application provides a song work generation method, also known as a complete integrity detection scheme for synthesizing a song work based on an iOS system. In the technical scheme provided by the song work generation method, when the system synthesis callback is successful, a work integrity detection is performed again, so that the synthesized work is complete and available. First, the audio metadata of the work is detected to be normal (work duration, sampling rate, etc.), and then the data content detection is performed. The detection methods include fixed and random content detection. The beginning and end of the audio are detected fixedly, and the middle part of the audio is detected randomly due to the differences in the recorded songs and the generated sound data. The detection method is to compare the audio fingerprints of the partial sound data of the work and the mixed data of the original voice and accompaniment, to determine whether the sound before and after the synthesis is consistent. Only when all the detections pass, the work is recognized to be complete and available. If any process fails, the system synthesis is recognized to fail, and the silent retry synthesis is performed. Currently, more than 95% of the abnormalities can be solved, and if the synthesis fails again, the user is guided to restart the device and report the operation log to the background, so that the developer can locate the problem.

[0143] a. The user synthesizes the accompaniment audio and the human voice audio recorded based on the accompaniment audio, and when the system callback synthesis is successful, a complete detection is performed. The specific complete detection process is described in detail in steps b, c, d and e.

[0144] b. First, the audio metadata in the synthesized target audio is detected to be normal. By decoding the audio metadata, the obtained parameters (such as duration, channel data, code rate, sampling rate, etc.) are confirmed again, to ensure that the parameter quantity, data format and data value range of each parameter are consistent with the expectation.

[0145] c. After confirming that the audio metadata detection is normal, the audio content can be detected. The detection mainly includes fixed content detection and random content detection. The fixed content detection part includes the audio start position and end position which are most likely to have problems. The random content detection part is also called the middle part of the audio, that is, random sampling detection needs to be performed on the middle part.

[0146] d. In the process of detecting the audio content, the detection method is to use the audio fingerprint comparison technology commonly used in the industry. By obtaining the pre-synthesis human voice audio and the corresponding position of the target audio after synthesis, audio fingerprint comparison is performed to detect whether the voiceprint is consistent.

[0147] e. If any of the above steps fails the test, it is preferred to be silent and resynthesized once. The meaning of silence is that the synthesis process does not need to be perceived by the user. For example, when resynthesis is needed, the user can only see that the progress bar does not reach 100%, and the song work is not successfully generated. At the same time, the task of resynthesis is being performed synchronously.

[0148] It should be noted that the online statistical data analysis shows that 95% of users can use the resynthesis method to solve the exception when the exception occurs. If the synthesis fails during the resynthesis process, the current operating system is generally abnormal; At this time, the user needs to be guided to restart the system to recover.

[0149] In step b above, the detailed implementation principle of audio metadata detection includes: through data analysis results, it is found that audio metadata abnormalities account for a large proportion of work playback abnormalities. By detecting the audio header data, a large part of the problem can be found.

[0150] Work audio file format Figure 5 As shown, specifically, the audio file of the target audio includes two parts of header information and streaming media data segment, wherein the header information is the audio metadata of the target audio, and the streaming media data segment is the specific content data of the target audio. More specifically, the audio metadata synthesized by the iOS system is located at the front of the file, and its size will not exceed 5% of the total size of the file. Therefore, the audio metadata (including duration, sampling rate, channel data, bit depth, etc.) can be parsed very efficiently, and compared with the expected value to confirm whether it is consistent with the expected value.

[0151] In step c above, the detailed implementation principle of the detection content acquisition and detection method includes:

[0152] Through the audio metadata of the work, the sampling rate sampleRate of the audio, the number of audio frames included in the audio packet framesPerPacket, the start time startTime of the selected detection range, and the detection duration duration, the audio packet position startPacket corresponding to the detection start time can be calculated, and the formula is as follows:

[0153] startPacket = startTime * sample / framesPerPacket ;

[0154] The start position startFilePos of the content position corresponding to the content to be detected in the file can be determined by combining the stream media data segment offset value mdatOffset in the audio metadata and the audio stream media acquisition function outDataByteOffset() (the function naming is related to the platform, and here is only a placeholder) that specifies the offset position of the audio packet in the data segment.

[0155] The stream media data segment (also referred to as a stream media data region) refers to a logical block for storing media data in a digital audio file, and is a measurement unit for representing the segmentation characteristics of the audio file.

[0156] The determination process of the content position corresponding to the content to be detected in the file can be expressed by the following formula:

[0157] startFilePos = mdatOffset + outDataByteOffset(startPacket);

[0158] Similarly, the end position of the content position corresponding to the content to be detected in the file is determined by the following formula:

[0159] endPacket = (startTime+duration)* sample / framesPerPacket;

[0160] endFilePos = mdatOffset + outDataByteOffset(endPacket);

[0161] The above calculations are as follows:

[0162] The content position of the content to be detected can be represented as [startFilePos, endFilePos].

[0163] Through the above formula, a list of position information of all content to be detected in the file (including fixed detection content and random detection content) can be obtained. The position of the content to be detected in the file is as shown in Figure 6 Figure 6 ​In the embodiment, the position of the start fixed detection point can be represented as [startFilePos1, endFilePos1], the position of the end fixed detection point can be represented as [startFilePos2, endFilePos2], the position of the random detection point 1 can be represented as [startFilePos3, endFilePos3], the position of the random detection point 2 can be represented as [startFilePos4, endFilePos4], and the position of the random detection point N can be represented as [startFilePosM, endFilePosM].

[0164] After obtaining the position information of all the data segments to be detected, the audio data in the synthesized work is obtained through the position information, and whether the sound at the corresponding positions before and after the synthesis is consistent is detected through the audio fingerprint extraction and comparison technology commonly used in the industry.

[0165] In step e, the process of detecting the abnormality processing includes:

[0166] If the target audio in the song work is detected to be incorrect, automatic recovery needs to be attempted to reduce the impact on the user as much as possible. Through online data analysis of the K-song program software, the synthesis abnormality is generally identified as successful due to system fluctuations, and 95% of the users can successfully synthesize the correct work when retrying. The other 5% of the users need to be guided to restart the device to recover due to system abnormalities. Therefore, in the technical solution of the present application, when the synthesized work is detected to be abnormal, the system will first silently retry the synthesis once, and when the retry fails to synthesize the correct work, the user will be guided to restart the device, and the abnormal log will be uploaded to the background server to provide the developer for analysis and optimization.

[0167] The song work generation method provided by the embodiment of the present application improves the success rate of audio synthesis based on the iOS system, ensures that the synthesized work is complete and usable, effectively avoids the problem of poor user experience caused by the abnormal work being unable to play, and simultaneously does not need to introduce a third-party synthesis tool, thereby avoiding the increase in the size of the application program and the slowing down of the synthesis speed of the work.

[0168] The song work generation method provided by the embodiment of the present application, after being put online in the K-song program software, causes the number of reports of playing abnormities due to incomplete work synthesis to decrease by 95%. The technical solution of the present application is based on audio metadata checking and content sampling checking, the detection speed is very fast, and the user experience is greatly improved.

[0169] In some other embodiments, the scheme of the present application is based on sampling inspection in the random content detection part, in order to further improve the accuracy of the detection result, the following optimizations can be made on the basis of the current function: (1) if the user selects to directly upload the audio file for publishing the work after synthesis, no inspection is performed; (2) if the user selects to save the draft, all detection can be performed asynchronously when the system is idle, and if there is a problem in the detection, silent re-synthesis is performed.

[0170] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0171] Based on the same inventive concept, the present application also provides a song work generation device for implementing the song work generation method as described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more song work generation device embodiments provided below can refer to the limitations of the song work generation method in the above text, and will not be repeated here.

[0172] In one embodiment, as shown in Figure 7 A song work generation device is provided, comprising: a target audio synthesis module 701, an integrity verification module 702, and a work preview interface display module 703.

[0173] The target audio synthesis module 701 is configured to, in response to a song work generation instruction, call an audio synthesizer provided by an operating system to synthesize a accompaniment audio and a human voice audio recorded based on the accompaniment audio, to obtain a target audio of a song work.

[0174] The integrity verification module 702 is configured to, in a case where a calling result returned by the operating system indicates that the audio synthesizer successfully synthesizes the target audio, perform integrity detection on the target audio using the human voice audio, and during the integrity detection of the target audio, display a generation progress of the song work on a song work recording interface.

[0175] The work preview interface display module 703 is configured to display a work preview interface of the song work if the target audio passes the integrity detection. The work preview interface is configured to guide saving or publishing the song work.

[0176] In one embodiment, the device is configured to: determine at least one to-be-detected content in the target audio; determine a content position of the to-be-detected content according to audio metadata of the target audio; the content position represents a relative address of the to-be-detected content in an audio file of the target audio; acquire first audio data corresponding to the content position from the audio file of the target audio, and acquire second audio data corresponding to the content position from the audio file of the human voice audio; and if audio fingerprints between the first audio data and the second audio data are consistent, determine that the target audio passes the integrity detection.

[0177] In one embodiment, the device is configured to: acquire audio metadata of the target audio, and detect whether each audio parameter recorded in the audio metadata is normal; if each audio parameter is detected to be normal, perform the step of determining a content position of the to-be-detected content according to the audio metadata of the target audio.

[0178] In one embodiment, the device is configured to: acquire a start time and an end time corresponding to the to-be-detected content in the target audio; determine a relative address of a start data packet in an audio file of the target audio according to the start time, the sampling rate, and a number of audio frames included in the data packet; the start data packet is a data packet in which the to-be-detected content corresponding to the start time is located; determine a relative address of an end data packet in the audio file of the target audio according to the end time, the sampling rate, and the number of audio frames included in the data packet; the end data packet is a data packet in which the to-be-detected content corresponding to the end time is located.

[0179] In one embodiment, the apparatus is configured to: obtain the ordering position of the start data packet in the audio file according to the start time, the sampling rate and the number of audio frames included in the data packet; map the ordering position of the start data packet in the audio file to the relative address of the start data packet in the streaming media data region using an offset position function; obtain the relative address of the start data packet in the audio file of the target audio according to the relative address of the streaming media data region in the audio file of the target audio and the relative address of the start data packet in the streaming media data region; obtain the ordering position of the end data packet in the audio file according to the end time, the sampling rate and the number of audio frames included in the data packet; map the ordering position of the end data packet in the audio file to the relative address of the end data packet in the streaming media data region using the offset position function; and obtain the relative address of the end data packet in the audio file of the target audio according to the relative address of the streaming media data region in the audio file of the target audio and the relative address of the end data packet in the streaming media data region.

[0180] In one embodiment, the apparatus is configured to: in the case that the calling result indicates that the audio synthesizer fails to synthesize or the target audio fails to pass the integrity detection, return a step of calling an audio synthesizer provided by the operating system to synthesize the accompaniment audio and the human voice audio based on the accompaniment audio to obtain the target audio required by the song work.

[0181] In one embodiment, the apparatus is configured to: record the number of times of unsuccessfully calling the audio synthesizer to synthesize the target audio; and in the case that the number of times meets a preset number threshold, generate a device restart prompt to instruct the device to restart.

[0182] In one embodiment, the apparatus is configured to: save the target audio in response to a draft saving operation for the song work; in the case that the operating system is idle, take all audio data in the saved target audio as third audio data and take all audio data in the human voice audio as fourth audio data; and in the case that the audio fingerprints between the third audio data and the fourth audio data are consistent, determine that the target audio passes the integrity detection.

[0183] The above modules in the song work generation apparatus can be implemented by software, hardware or a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0184] In one embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 8The computer device shown in the figure includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize a song work generation method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0185] Those skilled in the art can understand that, Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0186] In one embodiment, a computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in each of the above method embodiments.

[0187] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to realize the steps in each of the above method embodiments.

[0188] In one embodiment, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to realize the steps in each of the above method embodiments.

[0189] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory, ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0190] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0191] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for generating a song, characterized in that, The method includes: In response to the song creation command, the operating system's built-in audio synthesizer is invoked to synthesize the accompaniment audio with the human voice audio recorded based on the accompaniment audio, thus obtaining the target audio of the song. If the operating system returns a call result indicating that the audio synthesizer has successfully synthesized the target audio, then the target audio is subjected to integrity detection using the human voice audio; this further includes: determining at least one content to be detected in the target audio; determining the content location of the content to be detected based on the audio metadata of the target audio; the content location representing the relative address of the content to be detected in the audio file of the target audio; obtaining first audio data corresponding to the content location from the audio file of the target audio, and obtaining second audio data corresponding to the content location from the audio file of the human voice audio; if the audio fingerprints between the first audio data and the second audio data are consistent, then the target audio is determined to have passed the integrity detection; During the integrity detection process of the target audio, the generation progress of the song is displayed on the song recording interface; If the target audio passes the integrity check, a preview interface for the song is displayed; the preview interface is used to guide users to save or publish the song.

2. The method according to claim 1, characterized in that, Prior to the step of determining the content location of the content to be detected based on the audio metadata of the target audio, the method further includes: Obtain the audio metadata of the target audio and check whether the various audio parameters recorded in the audio metadata are normal; If all audio parameters are detected as normal, the step of determining the content location of the content to be detected based on the audio metadata of the target audio is executed.

3. The method according to claim 1, characterized in that, The content to be detected includes: the start content of the target audio, the end content of the target audio, and at least one other content in the target audio besides the start content and the end content.

4. The method according to claim 1, characterized in that, The target audio file comprises multiple data packets arranged sequentially, and the audio metadata includes the sampling rate of the target audio and the number of audio frames included in the data packets; determining the content location of the content to be detected based on the audio metadata of the target audio includes: Obtain the start and end times of the content to be detected in the target audio. Based on the start time, the sampling rate, and the number of audio frames included in the data packet, the relative address of the start data packet in the audio file of the target audio is determined; the start data packet is the data packet containing the content to be detected corresponding to the start time. Based on the end time, the sampling rate, and the number of audio frames included in the data packet, the relative address of the end data packet in the audio file of the target audio is determined; the end data packet is the data packet containing the content to be detected corresponding to the end time. The relative addresses of the start data packet and the end data packet in the target audio file represent the content locations of the content to be detected.

5. The method according to claim 4, characterized in that, The target audio file includes a header information area and a streaming media data area; the multiple data packets are arranged sequentially in the streaming media data area; Determining the relative address of the starting data packet in the audio file of the target audio based on the start time, the sampling rate, and the number of audio frames included in the data packet includes: Based on the start time, the sampling rate, and the number of audio frames included in the data packet, the sorting position of the starting data packet in the audio file is obtained; the sorting position of the starting data packet in the audio file is mapped to the relative address of the starting data packet in the streaming media data region using an offset position function; based on the relative address of the streaming media data region in the target audio file and the relative address of the starting data packet in the streaming media data region, the relative address of the starting data packet in the target audio file is obtained. Determining the relative address of the end data packet in the audio file of the target audio based on the end time, the sampling rate, and the number of audio frames included in the data packet includes: Based on the end time, the sampling rate, and the number of audio frames included in the data packet, the sorting position of the ending data packet in the audio file is obtained; the sorting position of the ending data packet in the audio file is mapped to the relative address of the ending data packet in the streaming media data region using an offset position function; based on the relative address of the streaming media data region in the target audio file and the relative address of the ending data packet in the streaming media data region, the relative address of the ending data packet in the target audio file is obtained.

6. The method according to claim 1, characterized in that, The method further includes: If the call result indicates that the audio synthesizer has failed to synthesize, or the target audio has not passed the integrity check, the process returns to the step of calling the operating system's built-in audio synthesizer to synthesize the accompaniment audio with the vocal audio recorded based on the accompaniment audio to obtain the target audio required for the song.

7. The method according to claim 6, characterized in that, The method further includes: Record the number of times the target audio was not successfully synthesized using the audio synthesizer; If the number of attempts meets a preset threshold, a device restart prompt message is generated to instruct the device to restart.

8. The method according to claim 1, characterized in that, The method further includes: In response to a draft saving operation for the song, the target audio is saved; When the operating system is idle, all audio data in the saved target audio is used as the third audio data, and all audio data in the human voice audio is used as the fourth audio data. If the audio fingerprints between the third and fourth audio data match, the target audio is determined to have passed the integrity check.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images