Method, device, equipment and storage medium for determining advertisement insertion position

Abstract

The present invention discloses a method, device, equipment and storage medium for determining an advertisement insertion position. The method for determining an advertisement insertion position includes: obtaining the similarity between an audio frame of a candidate audio and an audio frame of an advertisement audio; determining a weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function characterizes the corresponding relationship between the advertisement insertion position and the degree of memory forgetting; determining a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value; and determining an insertion position of an advertisement word corresponding to the advertisement audio according to the target similarity. In this way, by determining the advertisement insertion position according to the target similarity between the candidate audio and the advertisement audio, the rationality of determining the advertisement insertion position can be improved.

Description

Technical Field

[0001] The present invention relates to the field of advertising technology, and in particular to a method, device, equipment and computer-readable storage medium for determining an advertisement insertion position. Background Art

[0002] With the development of Internet technology, advertising has deeply penetrated into all walks of life. For example, in the music industry, advertisements can be inserted when playing songs.

[0003] However, currently, the method of inserting advertisements in audio is usually to directly insert advertisements at a set time point during the audio playback process, which may result in unreasonable determination of the advertisement insertion position. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and computer-readable storage medium for determining an advertisement insertion position, aiming to improve the rationality of determining the advertisement insertion position.

[0005] To achieve the above object, the present invention provides a method for determining an advertisement insertion position, the method comprising the following steps:

[0006] Obtaining similarity between an audio frame of the candidate audio and an audio frame of the advertisement audio;

[0007] Determining a weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function represents a corresponding relationship between an advertisement insertion position and a memory forgetting degree;

[0008] Determine a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value;

[0009] The insertion position of the advertisement words corresponding to the advertisement audio is determined according to the target similarity.

[0010] Optionally, the step of determining the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity includes:

[0011] Determine the target candidate audio corresponding to the maximum target similarity;

[0012] The advertisement insertion position is determined according to the relevant information of the target candidate audio.

[0013] Optionally, the step of obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio includes:

[0014] Obtaining a first mel-frequency cepstrum matrix formed by mel-frequency cepstrum coefficients of the audio frame of the candidate audio, and a second mel-frequency cepstrum matrix formed by mel-frequency cepstrum coefficients of the audio frame of the advertisement audio;

[0015] Cosine similarity calculation is performed on the first Mel-frequency cepstrum matrix and the second Mel-frequency cepstrum matrix to obtain similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio.

[0016] Optionally, the step of determining the weight value corresponding to each audio frame according to the memory forgetting function includes:

[0017] Converting the memory forgetting function to the first quadrant to obtain a converted forgetting function;

[0018] Normalize the converted forgetting function to obtain a normalized forgetting function;

[0019] The weight value is obtained according to the normalized forgetting function.

[0020] Optionally, before the step of obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio, the step further includes:

[0021] Obtain the first syllable number of the audio to be selected and the second syllable number of the advertisement audio;

[0022] determining a target syllable number range according to the first syllable number and the second syllable number;

[0023] The audio to be selected whose first syllable number is within the target syllable number range is determined as the candidate audio.

[0024] Optionally, the step of determining a target syllable number range according to the first syllable number and the second syllable number includes:

[0025] Obtaining the maximum number of syllables and the minimum number of syllables in the first number of syllables;

[0026] When the second number of syllables is less than or equal to the minimum number of syllables, determining an upper limit of the target syllable number range according to the second number of syllables and the minimum number of syllables, and using the second number of syllables as a lower limit of the target syllable number range;

[0027] When the second number of syllables is greater than or equal to the maximum number of syllables, determining a lower limit of the target syllable number range according to the maximum number of syllables and the second number of syllables, and using the second number of syllables as an upper limit of the target syllable number range;

[0028] When the second number of syllables is greater than the minimum number of syllables and less than the maximum number of syllables, the lower limit of the target syllable number range is determined according to the minimum number of syllables and the second number of syllables, and the upper limit of the target syllable number range is determined according to the maximum number of syllables and the second number of syllables.

[0029] Optionally, the step of obtaining the first number of syllables of the audio to be selected and the second number of syllables of the advertisement audio includes:

[0030] The audio to be selected is read aloud with the audio sound in the audio to be selected to obtain a read audio, and the number of syllables of the read audio is obtained as the first number of syllables of the audio to be selected;

[0031] The advertisement words corresponding to the advertisement audio are read aloud with the audio sound in the selected audio to obtain the advertisement reading audio, and the number of syllables of the advertisement reading audio is obtained as the second number of syllables of the advertisement audio.

[0032] In addition, to achieve the above-mentioned purpose, the present invention also provides a device for determining an advertisement insertion position, wherein the device for determining an advertisement insertion position includes a memory, a processor, and a program for determining an advertisement insertion position stored on the processor and executable on the processor, and when the processor executes the program for determining an advertisement insertion position, the steps of the method for determining an advertisement insertion position as described above are implemented.

[0033] In addition, to achieve the above-mentioned purpose, the present invention further provides a device for determining an advertisement insertion position, the device comprising: an acquisition module, a first determination module, a second determination module and a third determination module, wherein:

[0034] Acquisition module: used to acquire the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio;

[0035] A first determination module: used to determine a weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function represents a corresponding relationship between an advertisement insertion position and a memory forgetting degree;

[0036] A second determination module: used to determine a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value;

[0037] The third determination module is used to determine the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a program for determining an advertisement insertion position is stored. When the program for determining an advertisement insertion position is executed by a processor, the steps of the method for determining an advertisement insertion position as described above are implemented.

[0039] In an embodiment of the present invention, by obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio, and determining the weight value corresponding to each audio frame according to the memory forgetting function that can characterize the correspondence between the advertisement insertion position and the degree of memory forgetting, and then determining the target similarity between the candidate audio and the advertisement audio according to the obtained similarity and the determined weight value, and then determining the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity, the advertisement audio can be associated with the candidate audio to determine the advertisement insertion position, and then the unreasonable advertisement insertion position caused by directly inserting the advertisement according to the preset time point can be avoided. That is, by determining the advertisement insertion position according to the target similarity between the candidate audio and the advertisement audio, the rationality of the advertisement insertion position determination can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a structural schematic diagram of a device for determining an advertisement insertion position involved in an embodiment of the present invention;

[0041] Figure 2 It is a flow chart of a first embodiment of a method for determining an advertisement insertion position of the present invention;

[0042] Figure 3 is a schematic diagram of a normalized forgetting function in an exemplary embodiment;

[0043] Figure 4 It is a flow chart of a second embodiment of a method for determining an advertisement insertion position according to the present invention;

[0044] Figure 5 The figure is a schematic diagram of the structure of a device for determining an advertisement insertion position in one embodiment of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0047] The main solution of the present invention is: obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio; determining the weight value corresponding to each audio frame according to the memory forgetting function, and the memory forgetting function characterizes the correspondence between the advertisement insertion position and the memory forgetting degree; determining the target similarity between the candidate audio and the advertising audio according to the similarity and the weight value; determining the insertion position of the advertising words corresponding to the advertising audio according to the target similarity.

[0048] Since currently, when inserting advertisements into audio, the advertisements are usually inserted at a set time, this will result in an unreasonable advertisement insertion position and affect the advertisement insertion effect. Therefore, the above solution proposed by the present invention aims to improve the rationality of determining the advertisement insertion position to improve the advertisement insertion effect.

[0049] Reference Figure 1 , Figure 1 It is a schematic diagram of the structure of a device for determining an advertisement insertion position in a hardware operating environment according to an embodiment of the present invention.

[0050] like Figure 1 As shown, the device for determining the advertisement insertion position may include: a communication bus 1002, a processor 1001, such as a CPU, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The structure of the device for determining the advertisement insertion position shown in the figure does not constitute a limitation on the device for determining the advertisement insertion position, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] exist Figure 1In the device for determining the advertisement insertion position shown, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the client (user end) and perform data communication with the client; and the processor 1001 can be used to call the advertisement insertion position determination program stored in the memory 1005, and execute the relevant steps of each embodiment of the following advertisement insertion position determination method.

[0053] It should be noted that the execution subject of the method for determining the advertisement insertion position proposed in each of the following embodiments may be an advertisement insertion position determination device or an advertisement insertion position determination device, for example, it may be a server, or a terminal device with an audio output function (such as a television or a mobile phone, etc.), or a control device of a terminal device with an audio output function (such as a television remote control), etc., which is not specifically limited here. Optionally, the advertisement insertion position determination device may be set in the advertisement insertion position determination device or be set independently of the advertisement insertion position determination device. Among them, when the advertisement insertion position determination device and the advertisement insertion position determination device are set independently, the advertisement insertion position determination device and the advertisement insertion position determination device may be communicatively connected.

[0054] Reference Figure 2 , Figure 2 This is a flow chart of a first embodiment of a method for determining an advertisement insertion position of the present invention. In this embodiment, the method for determining an advertisement insertion position includes the following steps:

[0055] Step S10: obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio;

[0056] It should be noted that the applicable scenario of this embodiment can be an application scenario with audio output, such as music playback or video playback. Among them, the candidate audio refers to the audio sentence that can be used to insert advertisements, and each audio sentence contains at least one audio frame; the advertisement audio refers to the audio of the advertisement words that need to be inserted in the audio, such as the audio of the advertisement words, and the advertisement audio contains at least one audio frame;

[0057] Optionally, the candidate audio may be an audio sentence pre-stored in a preset database, or an audio sentence determined in real time from a network resource based on at least one of copyright information, popularity, number of syllables, number of audio text words, and number of audio frames. For example, an audio sentence whose number of audio text words (such as the number of lyrics words) is greater than or equal to the number of words in the advertising audio may be used as a candidate audio; or, an audio sentence whose number of syllables is greater than or equal to the number of syllables of the advertising audio may be used as a candidate audio; or, an audio sentence whose number of syllables is greater than or equal to the number of syllables of the advertising audio may be used as a candidate audio; or, an audio sentence whose number of audio frames is greater than or equal to the number of audio frames of the advertising audio may be used as a candidate audio, etc., and no specific limitation is made here. In order to improve the integrity of advertising insertion and avoid the need for sentence insertion, etc., it may be preferred to use an audio sentence whose number of audio text words is equal to the number of words in the advertising audio and / or whose number of syllables is equal to the number of syllables of the advertising audio and / or whose number of audio frames is equal to the number of audio frames of the advertising audio as a candidate audio.

[0058] Optionally, after determining the candidate audio and the advertisement audio, in order to obtain the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio, the audio frame features of the candidate audio (such as short-time energy and spectral entropy, etc.) may be compared with the audio frame features of the advertisement audio to determine the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio. For example, the closer the short-time energy and / or spectral entropy, the higher the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio.

[0059] In one embodiment, considering the connection between the output audio and the auditory characteristics of humans, the mel-frequency cepstral coefficients (recorded as the first mel-frequency cepstral coefficients) corresponding to the audio frame of the candidate audio and the mel-frequency cepstral coefficients (recorded as the second mel-frequency cepstral coefficients) corresponding to the audio frame of the advertisement audio can be obtained, so that the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio can be determined based on the first mel-frequency cepstral coefficients and the second mel-frequency cepstral coefficients, so as to fully consider the auditory characteristics of humans and facilitate more accurate calculation of the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio. Specifically, the first mel-frequency cepstral matrix composed of the mel-frequency cepstral coefficients of the audio frame of the candidate audio and the second mel-frequency cepstral matrix composed of the mel-frequency cepstral coefficients of the audio frame of the advertisement audio can be first obtained, and then the cosine similarity calculation is performed on the first mel-frequency cepstral matrix and the second mel-frequency cepstral matrix to obtain the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio.

[0060] Among them, the first Mel-frequency cepstral matrix is ​​a matrix composed of the first Mel-frequency cepstral coefficients as rows and the number of audio frames of the candidate audio as columns, that is, the first row in the first Mel-frequency cepstral matrix is ​​the Mel-frequency cepstral coefficients of the first audio frame in the candidate audio, the second row is the Mel-frequency cepstral coefficients of the second audio frame in the candidate audio, and so on; the second Mel-frequency cepstral matrix is ​​a matrix composed of the second Mel-frequency cepstral matrix as rows and the number of audio frames of the advertising audio as columns, that is, the first row in the second Mel-frequency cepstral matrix is ​​the Mel-frequency cepstral coefficients of the first audio frame in the advertising audio, the second row is the Mel-frequency cepstral coefficients of the second audio frame in the advertising audio, and so on.

[0061] Then, the cosine similarity between each audio frame of the candidate audio and each audio frame of the advertisement audio can be calculated by performing cosine similarity calculation on the first mel-frequency cepstrum matrix and the second mel-frequency cepstrum matrix row by row. The specific calculation formula is:

[0062]

[0063] That is, the first row of the first Mel-frequency cepstrum matrix can be used as vector A, and the first row of the second Mel-frequency cepstrum matrix can be used as vector B to calculate the cosine similarity between the first video frame of the candidate audio and the first video frame of the advertisement audio; then the second row of the first Mel-frequency cepstrum matrix can be used as vector A, and the second row of the second Mel-frequency cepstrum matrix can be used as vector B to calculate the cosine similarity between the second video frame of the candidate audio and the second video frame of the advertisement audio; and so on, until the last row of the first Mel-frequency cepstrum matrix or the last row of the second Mel-frequency cepstrum matrix is ​​calculated.

[0064] Step S20: determining a weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function represents a corresponding relationship between an advertisement insertion position and a memory forgetting degree;

[0065] Taking into account that different audio frames have different audio characteristics, after obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio, the weight value corresponding to each audio frame can also be determined, so that when the target similarity between the candidate audio and the advertising audio is calculated based on the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio and the corresponding weight value, the accuracy of the subsequent determination of the similarity between the candidate audio and the advertising audio can be improved.

[0066] Optionally, the steps of obtaining the first Mel-frequency cepstral coefficient, the second Mel-frequency cepstral coefficient and determining the weight value corresponding to each audio frame may be performed simultaneously or sequentially in a preset order, which is not specifically limited here.

[0067] Optionally, in order to facilitate the insertion of advertisements to leave a deeper impression on users and to improve the effect of advertisement insertion, a corresponding memory forgetting function can be selected according to the memory forgetting characteristics to determine the weight value corresponding to each audio frame according to the memory forgetting function; in order to comprehensively consider the different audio characteristics of different audio frames, the weight value corresponding to the audio frame of the candidate audio can also be determined according to the audio frame characteristics of the candidate audio (such as the number of syllables, the number of words in the audio text, the audio frequency band, etc.); of course, other methods can also be used, which are not specifically limited here. In this embodiment, the weight value corresponding to each audio frame can be preferably determined according to the memory forgetting function; wherein the memory forgetting function is used to characterize the corresponding relationship between the advertisement insertion position and the degree of memory forgetting, that is, the degree of memory forgetting corresponding to different audio frames in the candidate audio as the advertisement insertion position. Among them, the higher the degree of memory forgetting, the smaller the corresponding weight value, and the lower the degree of memory forgetting, the larger the corresponding weight value.

[0068] Optionally, in order to ensure that the determined weight value is a positive number, the memory forgetting function can be transformed to convert the memory forgetting function to the first quadrant of the coordinate system to obtain a converted forgetting function; then, the converted forgetting function is normalized to obtain a normalized forgetting function; and the weight value corresponding to each audio frame is obtained according to the normalized forgetting function.

[0069] For example, in a specific application example, the following function can be used as the memory forgetting function based on the memory forgetting characteristics that the first and last parts are well remembered and the middle part is forgotten more:

[0070]

[0071] Optionally, in the above memory forgetting function, μ=(X / 2), σ=(X / 6).

[0072] In order to better reflect the similarity between the audio frames of the candidate audio and the audio frames of the advertisement audio, the memory forgetting function can be negated to obtain the following negated forgetting function:

[0073]

[0074] In this way, the easier it is to forget an audio frame, the lower the corresponding weight value, and the degree of forgetting of an audio frame is negatively correlated with the weight value. Among them, μ = (X / 2), which indicates that the degree of forgetting in the middle is the highest. According to the normal distribution 3σ principle, the probability of the value being distributed in (μ-3σ, μ+3σ) is 0.9974. To ensure that all frames are covered within 3σ, μ-3σ = 0, x = 0 is the first frame, and σ = (X / 6).

[0075] Since the weight value currently corresponding to the forgetting function may be negative, in order to ensure that the weight value is positive, the negated forgetting function can be shifted upward By translating the negated forgetting function to the first quadrant of the coordinate axis, we get the following translated forgetting function:

[0076]

[0077] The translated forgetting function is normalized to obtain a normalized forgetting function. Thus, the weight value q(a)=f(a-1) / s corresponding to each audio frame can be obtained according to the normalized forgetting function, where s=Σf(a-1), a=1, 2, ..., x.

[0078] Optionally, when X=50, μ=25, σ=9, the corresponding normalized forgetting function curve is as follows: Figure 3 shown.

[0079] Step S30: determining a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value;

[0080] After obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio and determining the weight value corresponding to each audio frame, the target similarity between the candidate audio and the advertisement audio can be determined based on the obtained similarity and weight value.

[0081] Optionally, the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio can be weighted and summed with the weight value corresponding to each audio frame, and the sum obtained by the weighted summation is used as the target similarity between the candidate audio and the advertising audio. Wherein, when the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio is weighted and summed with the weight value corresponding to each audio frame, the similarity between the first audio frame of the candidate audio and the first audio frame of the advertising audio is multiplied by the weight value corresponding to the first audio frame, and the similarity between the second video frame of the candidate audio and the second video frame of the advertising audio is multiplied by the weight value corresponding to the second audio frame, and so on, until the last audio frame of the candidate audio and / or the last audio frame of the advertising audio is calculated; then, all the calculated products are added to obtain the weighted sum value. Optionally, when there are multiple candidate audios, the target similarity between the multiple candidate audios and the advertising audio can be calculated. Optionally, an audio sentence with the same number of audio frames as the number of audio frames of the advertising audio can be preferably used as the candidate audio.

[0082] Of course, in some other embodiments, the similarity between the audio frames of the candidate audio and the audio frames of the advertising audio may be weightedly summed with the weight value corresponding to each audio frame, and the quotient obtained by dividing the sum obtained by the weighted sum by the total number of audio frames of the candidate audio may be used as the target similarity between the candidate audio and the advertising audio, etc., and no specific limitation is made here.

[0083] Step S40: determining the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity;

[0084] When determining the advertisement insertion position according to the target similarity, the insertion position of the advertisement words corresponding to the advertisement audio can be determined according to the candidate audio corresponding to the largest target similarity (hereinafter referred to as the advertisement insertion position). Specifically, the target candidate audio corresponding to the largest target similarity can be first determined from the candidate audios, and then the advertisement insertion position can be determined according to the relevant information of the target candidate audio. For example, the position or playback time node of the target candidate audio can be used as the advertisement insertion position.

[0085] Optionally, the relevant information of the target candidate audio may include feature information corresponding to the target candidate audio, such as the number of audio text words, syllables, and audio frames corresponding to the target candidate audio, etc.; it may also include attribute information corresponding to the target candidate audio, such as copyright attributes (such as song name and singer name, etc.) and playback attributes such as the start time node and end time node of the target candidate audio during audio playback, the number of playbacks of the target candidate audio, and the playback time node, etc. Further, the advertisement insertion position may be determined according to the feature information and attribute information of the target candidate audio.

[0086] For example, if it is determined based on the characteristic information of the target candidate audio that the number of words in the audio text of the target candidate audio is equal to the number of words in the target advertising words, then the advertisement can be inserted based on the attribute information using the start time node of the target candidate audio as the starting position of the advertisement insertion; if it is determined based on the characteristic information of the target candidate audio that the number of words in the audio text of the target candidate audio is greater than the number of words in the advertising words of the target advertising audio, then the advertisement insertion time point can be determined based on the difference between the number of words in the audio text of the target candidate audio and the number of words in the advertising words of the advertising audio. For example, when the target candidate audio contains 8 words and the target advertising words contain 5 words, the advertisement insertion position should be before the third word in the target candidate audio is played when the target candidate audio is played. This is only an example and is not specifically limited. In a specific application example, the advertisement insertion position can be characterized by the singer name, song name and playback time of the target candidate audio corresponding to the target candidate audio.

[0087] Optionally, when determining the advertisement insertion position according to the target similarity, it is also possible to use the candidate audio whose similarity with the advertisement audio exceeds a preset similarity as the target candidate audio. In this case, the target candidate audio may be one or two or more. Correspondingly, advertisements may be inserted at multiple positions in a candidate audio, so that advertisements may be played at multiple different positions in a candidate audio.

[0088] Optionally, after determining the advertisement insertion position, when inserting advertisements, the pre-recorded advertisement audio is usually directly inserted into the corresponding advertisement insertion position in the currently playing audio for playback. However, when this method is adopted, since the pre-recorded advertisement audio is not associated with the currently playing audio, the advertisement insertion will be very abrupt due to the unreasonable advertisement insertion method, which will cause user disgust; and when inserting advertisements, the currently playing audio will usually be interrupted or the volume of the currently playing audio will be reduced as background sound, etc., thereby affecting the playback effect of the currently playing audio. Therefore, in order to improve the rationality of the advertisement insertion method and to improve the audio playback effect during advertisement insertion, the corresponding audio text in the currently playing audio can be replaced with the target advertisement words corresponding to the advertisement audio on the basis of maintaining the audio melody of the original playing audio, so as to reduce the impact of advertisement insertion on the original audio playback, so as to improve the advertisement playback effect on the basis of improving the original audio playback effect.

[0089] Specifically, after the audio corresponding to the target candidate audio is detected, when the advertisement insertion position is played, the audio text corresponding to the advertisement insertion position is first replaced with the target advertisement words, and then the target advertisement words are used as the audio text corresponding to the advertisement insertion position for playback without changing the playback of the original audio. Optionally, since it takes a certain amount of processing time to replace the audio text corresponding to the advertisement insertion position with the target advertisement words, if the audio text replacement is performed only when the advertisement insertion position is detected, it may cause audio playback delays, etc. Therefore, in order to improve the continuity of audio playback, the audio text corresponding to the advertisement insertion position may also be replaced with the target advertisement words in advance, so that when the audio corresponding to the target candidate audio is played, the advertisement playback can be directly realized without the need for real-time audio text replacement processing; or, the audio segment corresponding to the advertisement insertion position may be extracted in advance, the audio text corresponding to the audio segment may be replaced with the target advertisement words, and then the audio segment after the target advertisement words are replaced may be replaced with the audio segment in the audio corresponding to the target candidate audio, so that when the audio corresponding to the target candidate audio is played, the advertisement playback can be directly realized.

[0090] This embodiment obtains the similarity between the audio frame of the candidate audio and the audio frame of the advertising audio, and determines the weight value corresponding to each audio frame according to the memory forgetting function, wherein the memory forgetting function characterizes the correspondence between the advertisement insertion position and the memory forgetting degree, and then determines the target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value, and determines the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity, so that the advertisement audio can be associated with the candidate audio to determine the advertisement insertion position, so as to improve the rationality of determining the advertisement insertion position and avoid directly inserting the advertisement at a set time point, which may cause user disgust.

[0091] Based on the above embodiments, a second embodiment of the method for determining an advertisement insertion position of the present invention is proposed. Figure 4 In this embodiment, before step S10, the following steps are also included:

[0092] Step S01: Obtain the first syllable number of the audio to be selected and the second syllable number of the advertisement audio;

[0093] Step S02: determining a target syllable number range according to the first syllable number and the second syllable number;

[0094] Step S03: Determine the audio to be selected whose first syllable number is within the target syllable number range as the candidate audio.

[0095] Before obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio, the audio sentences in the candidate audio can be screened to select suitable audio sentences as candidate audio, and then the target candidate audio can be screened from the candidate audio to avoid directly using all the audio sentences in the candidate audio as candidate audio, which is not conducive to improving the efficiency of determining the target similarity, and further not conducive to improving the efficiency of determining the advertisement insertion position. The candidate audio refers to the audio sentences that can be obtained by the device for determining the advertisement insertion position, such as the audio sentences in the copyrighted songs.

[0096] Specifically, the audio sentences in the audio to be selected can be screened according to the number of syllables of the audio to be selected (recorded as the first number of syllables) and the number of syllables of the advertising audio (recorded as the second number of syllables), so as to screen out audio sentences with the same number of syllables as the advertising audio as candidate audio, so as to avoid the situation where the number of syllables of the candidate audio and the number of syllables of the advertising audio differ greatly, which is not conducive to the adaptation of the candidate audio and the advertising audio, resulting in the final determined advertising insertion position being unreasonable. For example, an audio sentence in which the difference between the first number of syllables and the second number of syllables is within a preset number of syllables range can be determined as a candidate audio; wherein the preset number of syllables range can be set according to actual conditions and is not specifically limited here.

[0097] In this embodiment, the target syllable number range can be determined based on the first syllable number and the second syllable number, and then the audio sentences whose first syllable number is within the target syllable number range are determined as candidate audio. For example, in order to screen out an appropriate number of audio sentences as candidate audio, the target syllable number range can be determined based on the distribution of the first syllable number and the size of the second syllable number. When the first syllable number close to the second syllable number is densely distributed, the target syllable number range can be appropriately narrowed; when the first syllable number close to the second syllable number is sparsely distributed, the target syllable number range can be appropriately increased.

[0098] In one embodiment, in order to determine the target syllable number range based on the first number of syllables and the second number of syllables, the maximum number of syllables and the minimum number of syllables in the first number of syllables may be determined first, and then the second number of syllables may be compared with the maximum number of syllables and the minimum number of syllables, and the target syllable number range may be determined based on the comparison result.

[0099] Among them, it can be divided into the following three situations according to the comparison results:

[0100] (1) When the second number of syllables is less than or equal to the minimum number of syllables, the upper limit of the number of syllables may be determined according to the second number of syllables and the minimum number of syllables. For example, the maximum N (N is a positive integer) quantiles of the range of the number of syllables formed by the second number of syllables and the minimum number of syllables may be taken as the lower limit of the number of syllables. The second number of syllables or the minimum number of syllables may be taken as the upper limit of the number of syllables. In this way, the audio sentences whose first number of syllables is between the upper limit of the number of syllables and the lower limit of the number of syllables may be screened out as candidate audios.

[0101] (2) When the second number of syllables is greater than or equal to the maximum number of syllables, the upper limit of the number of syllables may be determined according to the maximum number of syllables and the second number of syllables. For example, the minimum N (N is a positive integer) quantiles of the range of syllables formed by the maximum number of syllables and the second number of syllables may be taken as the lower limit of the number of syllables; the second number of syllables or the maximum number of syllables may be taken as the lower limit of the number of syllables; in this way, audio sentences having a first number of syllables between the lower limit of the number of syllables and the second number of syllables may be screened out as candidate audios;

[0102] (3) When the second number of syllables is greater than the minimum number of syllables and less than the maximum number of syllables, the lower limit of the number of syllables can be determined based on the minimum number of syllables and the second number of syllables. For example, the minimum N (N is a positive integer) quantile of the syllable number range formed by the minimum number of syllables and the second number of syllables can be taken as the lower limit of the number of syllables; the upper limit of the number of syllables can be determined based on the maximum number of syllables and the second number of syllables. For example, the maximum N (N is a positive integer) quantile of the syllable number range formed by the second number of syllables and the maximum number of syllables can be taken as the upper limit of the number of syllables; in this way, the audio sentence with the first number of syllables between the lower limit of the number of syllables and the upper limit of the number of syllables can be taken as the candidate audio.

[0103] Optionally, N may be optimized to 4.

[0104] Since audio playback is usually accompanied by background noise and other noises, and advertising words are usually presented in the form of reading aloud, in order to improve the accuracy of the adaptation of candidate audio and advertising audio, in one embodiment, before obtaining the first syllable number of the selected audio and the second syllable number of the advertising audio, the selected audio can be converted into a reading audio, so as to reduce the influence on the acquisition of the number of syllables while reducing noise interference. Specifically, the audio sound in the selected audio can be extracted first, and then the selected audio can be read aloud with the extracted audio sound to obtain the reading audio, and then the syllable number of the reading audio can be obtained as the first syllable number of the selected audio. Correspondingly, the target advertising words can also be read aloud with the audio sound in the candidate audio to obtain the advertising reading audio, and then the advertising reading audio can be used as the advertising audio. In this way, the audio sentence and the target advertising word in the selected audio can be read aloud by the original audio sound in the selected audio at the same time, so that the consistency of the reading sound source of the audio sentence and the advertising audio can be maintained, which is more conducive to matching the audio sentence with the advertising audio.

[0105] This embodiment obtains the first syllable number of the audio to be selected and the second syllable number of the advertising audio, and determines the target syllable number range according to the first syllable number and the second syllable number, and determines the audio to be selected with the first syllable number within the target syllable number range as the candidate audio. This can avoid directly treating all the audio to be selected as candidate audio, which results in a large amount of time spent on similarity calculation and target similarity calculation, is not conducive to improving the efficiency of target similarity determination and further is not conducive to improving the efficiency of determining the advertisement insertion position.

[0106] In a specific application scenario, advertisements can be inserted during music playback. The specific process of inserting advertisements is as follows:

[0107] 1. Use the copyrighted songs in the database as the audio to be selected, and extract the singer's voice V1, V2, ..., Vn in the copyrighted songs in the database.

[0108] 2. Convert the audio sentences in all songs of the mth singer into audio readings by the singer's voice Vm, denoted as N11...Nnn (Nkk represents the audio reading of the Kth sentence of the Kth song of singer m, m≤n).

[0109] 3. The target advertising words are read aloud by the singer's voice Vk (k=1.....n) to convert the target advertising words into advertising reading audio read aloud by the singer's voice Vk, and obtain the advertising audio corresponding to the singer's voice Vk.

[0110] 4. Extract the number of syllables of the advertisement audio, recorded as J0; then extract each reading audio N11...Nnn, and the corresponding number of syllables are recorded as J11.....Jnn respectively.

[0111] 5. Sort J11....Jnn and J0 from small to large to get the order of Jmin.....J0......Jmax.

[0112] 6. When J0 is between Jmin and Jmax, the maximum quartile J75% of the syllable number range formed by Jmin and J0 can be taken; and the minimum quartile J25% of the syllable number range formed by J0 and Jmax can be taken. Then, the audio sentences with syllable numbers between J75% and J25% are taken as candidate audios. Subsequent processing operations are performed on the candidate audios that meet the conditions.

[0113] 7. Extract the Mel-frequency cepstral coefficients (second Mel-frequency cepstral coefficients) of the advertising audio and obtain the coefficient matrix T (T is a matrix with X rows and Y columns, X: the number of frames of the speech segment).

[0114] 8. Extract the Mel-frequency cepstral coefficient (first Mel-frequency cepstral coefficient) of Nkk corresponding to the audio sentence to obtain the coefficient matrix Tkk (Tkk is a matrix with X rows and Y columns, X represents the number of frames of the speech segment, and the first Mel-frequency cepstral coefficient is obtained by sampling according to the number of frames of T).

[0115] 9. Perform cosine similarity calculation on each row of Tkk and T to obtain a vector P with one column and X rows. The vector P can represent the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio.

[0116] 10. According to the theory of cognitive psychology, proactive inhibition and retroactive inhibition are the interference of the first-learned material on the recognition and recall of the later-learned material. Retroactive inhibition is the interference of the later-learned material on the retention and recall of the first-learned material. The first and last parts are well remembered, and the middle part is forgotten more. The memory forgetting curve satisfies the following normal distribution function (also known as the preset forgetting function):

[0117]

[0118] Among them, μ=(X / 2), σ=(X / 6);

[0119] It should be noted that μ = (X / 2), which expresses that the middle part of the entire memory is the most forgotten part. According to the 3σ principle of normal distribution, the probability that the value is distributed in (μ-3σ, μ+3σ) is 0.9974. To ensure that all frames are covered within 3σ, μ-3σ = 0, x = 0 is the first frame, then σ = (X / 6).

[0120] 11. Each advertisement word or lyrics reading is composed of multiple frames. The corresponding similarity of the corresponding frames is calculated as above. In order to better express the similarity between the advertisement audio and the audio sentence, the more forgotten, the lower the weight, showing a negative correlation. Therefore, the inverse function of the above normal distribution function is taken to obtain the following inverse forgetting function:

[0121]

[0122] 12. The current value of the weight function is negative. In order to ensure that the weight is positive, the negated forgetting function is uniformly weighted to shift the function upward. The function becomes the following translated forgetting function:

[0123]

[0124] 13. Obtain the weight value corresponding to each audio frame according to the function value, and normalize it to obtain the final weight of each frame; then, take out the value f(a-1) corresponding to a=1.....X, s=Σf(a-1). The weight of each frame is q(a)=f(a-1) / s.

[0125] 14. Similarity tkk = P(a)*q(a), where P(a) is the similarity value of each frame calculated in step 9.

[0126] 15. According to the above method, calculate all similarities tkk in turn.

[0127] 16. If there are m singers in total, all tkk values ​​of each copyrighted singer can be calculated according to the above calculation method. According to the largest tkk value, we can find which singer's song and which sentence's lyrics should be replaced with the target advertising words.

[0128] 17. Convert the target advertising words into audio sentences corresponding to the singer's voice and insert them into the song.

[0129] In addition, the present application also provides a device for determining an advertisement insertion position, the device for determining an advertisement insertion position comprising a memory, a processor, and a program for determining an advertisement insertion position stored in the memory and running on the processor, and the processor implements the steps of the method for determining the advertisement insertion position as described above when executing the program for determining the advertisement insertion position.

[0130] In addition, the present application also provides a device for determining an advertisement insertion position. Optionally, the device for determining an advertisement insertion position may include the device for determining an advertisement insertion position or be externally connected to the device for determining an advertisement insertion position.

[0131] In one embodiment, reference Figure 5 The device 100 for determining an advertisement insertion position may include: an acquisition module 10, a first determination module 20, a second determination module 30 and a third determination module 40, wherein:

[0132] Acquisition module 10: used to acquire the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio;

[0133] The first determination module 20 is used to determine the weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function represents the corresponding relationship between the advertisement insertion position and the memory forgetting degree;

[0134] A second determination module 30: configured to determine a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value;

[0135] The third determination module 40 is used to determine the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity.

[0136] It should be noted that the various embodiments of the device for determining an advertisement insertion position 100 are substantially the same as the various embodiments of the method for determining an advertisement insertion position described above, and will not be described in detail herein.

[0137] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a program for determining an advertisement insertion position is stored. When the program for determining an advertisement insertion position is executed by a processor, the steps of the method for determining an advertisement insertion position as described above are implemented.

[0138] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0139] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0140] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, TV, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0141] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for determining an advertisement insertion position, characterized in that: The method for determining the advertisement insertion position comprises the following steps: Obtaining similarity between an audio frame of the candidate audio and an audio frame of the advertisement audio; Determining a weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function represents a corresponding relationship between an advertisement insertion position and a memory forgetting degree; Determine a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value; Determining the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity; Before the step of obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio, the method further includes: Obtain the first syllable number of the audio to be selected and the second syllable number of the advertisement audio; determining a target syllable number range according to the first syllable number and the second syllable number; The audio to be selected whose first syllable number is within the target syllable number range is determined as the candidate audio.

2. The method for determining an advertisement insertion position according to claim 1, wherein: The step of determining the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity comprises: Determine the target candidate audio corresponding to the maximum target similarity; The insertion position of the advertisement words corresponding to the advertisement audio is determined according to the relevant information of the target candidate audio.

3. The method for determining an advertisement insertion position according to claim 1, wherein: The step of obtaining the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio comprises: Obtaining a first mel-frequency cepstrum matrix formed by mel-frequency cepstrum coefficients of the audio frame of the candidate audio, and a second mel-frequency cepstrum matrix formed by mel-frequency cepstrum coefficients of the audio frame of the advertisement audio; Cosine similarity calculation is performed on the first Mel-frequency cepstrum matrix and the second Mel-frequency cepstrum matrix to obtain similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio.

4. The method for determining an advertisement insertion position according to claim 1, wherein: The step of determining the weight value corresponding to each audio frame according to the memory forgetting function comprises: Converting the memory forgetting function to the first quadrant to obtain a converted forgetting function; Normalize the converted forgetting function to obtain a normalized forgetting function; The weight value is obtained according to the normalized forgetting function.

5. The method for determining an advertisement insertion position according to claim 1, wherein: The step of determining a target syllable number range according to the first syllable number and the second syllable number comprises: Obtaining the maximum number of syllables and the minimum number of syllables in the first number of syllables; When the second number of syllables is less than or equal to the minimum number of syllables, determining a lower limit of the target syllable number range according to the second number of syllables and the minimum number of syllables, and using the second number of syllables as an upper limit of the target syllable number range; When the second number of syllables is greater than or equal to the maximum number of syllables, determining a lower limit of the target syllable number range according to the maximum number of syllables and the second number of syllables, and using the second number of syllables as an upper limit of the target syllable number range; When the second number of syllables is greater than the minimum number of syllables and less than the maximum number of syllables, the lower limit of the target syllable number range is determined according to the minimum number of syllables and the second number of syllables, and the upper limit of the target syllable number range is determined according to the maximum number of syllables and the second number of syllables.

6. The method for determining an advertisement insertion position according to claim 1, wherein: The step of obtaining the first syllable number of the selected audio and the second syllable number of the advertisement audio comprises: The audio to be selected is read aloud with the audio sound in the audio to be selected to obtain a read audio, and the number of syllables of the read audio is obtained as the first number of syllables of the audio to be selected; The advertisement words corresponding to the advertisement audio are read aloud with the audio sound in the selected audio to obtain the advertisement reading audio, and the number of syllables of the advertisement reading audio is obtained as the second number of syllables of the advertisement audio.

7. A device for determining an advertisement insertion position, characterized in that: The device for determining the advertisement insertion position includes a memory, a processor, and an advertisement insertion position determination program stored in the memory and executable on the processor. When the processor executes the advertisement insertion position determination program, the steps of the advertisement insertion position determination method described in any one of claims 1 to 6 are implemented.

8. A device for determining an advertisement insertion position, characterized in that: The device for determining the advertisement insertion position comprises: an acquisition module, a first determination module, a second determination module and a third determination module, wherein: Acquisition module: used to acquire the similarity between the audio frame of the candidate audio and the audio frame of the advertisement audio; A first determination module: used to determine a weight value corresponding to each audio frame according to a memory forgetting function, wherein the memory forgetting function represents a corresponding relationship between an advertisement insertion position and a memory forgetting degree; A second determination module: used to determine a target similarity between the candidate audio and the advertisement audio according to the similarity and the weight value; A third determination module is used to determine the insertion position of the advertisement words corresponding to the advertisement audio according to the target similarity; The acquisition module is also used to obtain a first syllable number of the audio to be selected and a second syllable number of the advertising audio; determine a target syllable number range according to the first syllable number and the second syllable number; and determine the audio to be selected whose first syllable number is within the target syllable number range as the candidate audio.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an advertisement insertion position determination program, and when the advertisement insertion position determination program is executed by a processor, the steps of the advertisement insertion position determination method according to any one of claims 1 to 6 are implemented.

Images