Intelligent advertisement identification method and system based on audio and video data feature analysis

Abstract

The invention discloses an intelligent advertisement recognition method and system based on audio and video data feature analysis, and relates to the technical field of audio and video processing, and the method comprises the steps: carrying out the voice recognition of an advertisement audio, and obtaining an audio text of the advertisement audio; obtaining a first audio feature through the audio text; based on short-time Fourier transform, obtaining a spectrogram of the advertisement audio according to the sound signal; determining a plurality of key point positions of the spectrogram according to the frequency peak value of the spectrogram; extracting a second audio feature of the spectrogram based on the plurality of key point positions; determining first feature information according to the first audio feature and the second audio feature; determining second feature information through the advertisement video; generating an advertisement label according to the first feature information and the second feature information, and determining key information of an advertisement; based on the key information and the second feature information, an advertisement regulation and control strategy is determined, and the advertisement regulation and control strategy comprises a code rate regulation and control strategy. The method can effectively improve the experience feeling of watching the advertisement video by the user.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio and video processing technology, and in particular, to a method and system for intelligently identifying advertisements based on feature analysis of audio and video data. Background Art

[0002] With the rapid development of audio and video technology, the form of advertising has changed from single text or pictures to multimedia content containing rich visual and auditory elements. This change not only enhances the attractiveness of advertising, but also puts higher requirements on advertising communication strategies, especially in terms of network transmission and playback stability. Bit rate, as a key indicator to measure the transmission rate of audio and video data, directly determines the smoothness, clarity and user experience of advertising playback. Therefore, how to reasonably adjust the bit rate of advertising to meet the needs of different users has become an urgent problem to be solved in the field of advertising communication.

[0003] Traditional advertising bitrate control strategies mainly rely on fixed bitrate settings. However, traditional advertising bitrate control strategies often ignore the characteristics of the advertising content itself, such as audio complexity, video dynamic range, etc. For example, for advertisements with high audio complexity, more bitrates need to be allocated to ensure sound quality; while for advertisements with large video dynamic range, higher bitrates are required to maintain the clarity and smoothness of the picture, and fixed bitrate settings cannot make reasonable bitrate allocations based on the content characteristics of the advertisement. Due to the above reasons, in a high bitrate environment, if the advertising content itself is not complex, excessive bitrate allocation will cause a waste of bandwidth resources; while in a low bitrate environment, if the advertising content is highly complex, too little bitrate allocation will lead to a decline in playback effects, such as audio distortion, video freeze, etc. Therefore, for the traditional advertising bitrate control strategy with fixed bitrate settings, it will cause poor user experience and seriously affect the user's acceptance of advertisements. Summary of the invention

[0004] The embodiments of the present application provide an intelligent advertisement recognition method and system based on audio and video data feature analysis, which are used to improve the user experience of watching advertisement videos.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a method for intelligently identifying advertisements based on feature analysis of audio and video data is provided, comprising:

[0007] Obtain the advertising video and corresponding advertising audio of the advertisement to be tested;

[0008] Performing speech recognition on the advertisement audio to obtain an audio text of the advertisement audio;

[0009] Obtaining a first audio feature through the audio text;

[0010] Acquire an acoustic signal of the advertisement audio, and obtain a spectrogram of the advertisement audio according to the acoustic signal based on a short-time Fourier transform;

[0011] Determining multiple key point positions of the spectrogram according to the frequency peaks of the spectrogram;

[0012] Extracting a second audio feature of the spectrogram based on the plurality of key point positions;

[0013] Determining first feature information according to the first audio feature and the second audio feature;

[0014] Based on the key point positions, determining second feature information of the advertising video;

[0015] Extracting a first text content of the first feature information and a second text content of the second feature information, and calculating a similarity between the first text content and the second text content;

[0016] If the similarity is greater than a preset threshold, generating an advertisement tag according to the first feature information and the second feature information;

[0017] Determining key information of the advertisement according to the advertisement tag;

[0018] Based on the key information and the second characteristic information, an advertisement regulation strategy is determined, the advertisement regulation strategy including a bit rate regulation strategy, wherein the second characteristic information and the key information are used to characterize the advertisement complexity of the advertisement to be tested.

[0019] In a possible implementation manner of the first aspect, obtaining the first audio feature through the audio text includes:

[0020] Obtaining the domain type of the advertisement to be tested;

[0021] Dividing the audio text according to preset word categories to obtain multiple audio words;

[0022] Determining audio keywords from the plurality of audio words according to the field type;

[0023] According to the audio keyword, determining a corresponding preset database;

[0024] Determining the number of times the audio keyword appears in the audio text;

[0025] Obtaining a first repetition degree of the audio keyword according to the number of times the audio keyword appears in the audio text;

[0026] Determining the number of times the audio keyword appears in a preset database, wherein the preset database stores historical advertisement text keywords;

[0027] Obtaining a second repetition degree of the audio keyword according to the number of times the audio keyword appears in a preset database;

[0028] A weight value of the first repetition degree and the second repetition degree is calculated, and if the weight value is greater than a preset threshold, a feature keyword is determined, where the feature keyword is used to characterize the first audio feature.

[0029] In a possible implementation manner of the first aspect, extracting the feature of the spectrogram based on the multiple key point positions to obtain the second audio feature includes:

[0030] Dividing the spectrogram according to the key point positions to obtain a plurality of spectrogram segments;

[0031] determining the frequency, amplitude and intensity of each of said spectrogram segments;

[0032] Based on the frequency, amplitude and intensity of each of the spectrogram segments, basic feature information of the speech segment corresponding to each of the spectrogram segments is obtained;

[0033] A preset portrait generation algorithm is used to construct a corresponding voice portrait through each of the basic feature information, wherein the voice portrait includes an audio tag, and the audio tag is used to characterize the second audio feature.

[0034] In a possible implementation of the first aspect, the basic feature information of the speech segment corresponding to each of the spectrogram segments includes age features, gender features and emotional features, wherein the age feature is determined based on the frequency and intensity of the spectrogram segment, the gender feature is determined based on the frequency and amplitude of the spectrogram segment, and the emotional feature is determined based on the intensity and amplitude of the spectrogram segment.

[0035] In a possible implementation manner of the first aspect, determining first feature information according to the first audio feature and the second audio feature includes:

[0036] The semantic similarity between the first audio feature and the second audio feature is calculated, and a keyword whose semantic similarity is greater than a preset similarity is used as the first feature information.

[0037] In a possible implementation manner of the first aspect, determining the second feature information based on the key point position includes:

[0038] Divide the advertisement video based on the key point positions to obtain multiple video slices;

[0039] By using a preset recognition model, feature extraction is performed on each of the video slices to obtain video features, wherein the video features are used to characterize the second feature information.

[0040] In a possible implementation manner of the first aspect, the key information includes a brand, the video features include video resolution, clarity, color, contrast, duration, loading time, and inter-frame difference, and determining the advertising regulation strategy based on the key information and the second feature information includes:

[0041] Clustering the key information to obtain the user's preference characteristics for the advertisement to be tested;

[0042] According to the preference characteristics, a corresponding search algorithm is retrieved in a preset database;

[0043] Using the retrieval algorithm to retrieve a target advertisement video in a user terminal of the user, and determining a target audience category of the user according to the target advertisement video;

[0044] Based on a preset quality scoring algorithm, determining the advertisement quality score of the advertisement to be tested according to the video features of the advertisement to be tested;

[0045] The bit rate regulation strategy is determined according to the advertisement quality score, the brand and the target audience category.

[0046] In a possible implementation manner of the first aspect, determining the bit rate control strategy according to the advertisement quality score, the brand, and the target audience includes:

[0047] Searching the brand in a preset database to obtain the corresponding brand value;

[0048] When any two of the following conditions are met: the target audience category is a first preset type, the advertisement quality score is greater than a first preset threshold, and the brand value is greater than a second preset threshold, the bit rate control strategy is determined to be a first strategy, and the first strategy is used to increase the bit rate;

[0049] When any two of the following conditions are met: the target audience category is a second preset type, the advertisement quality score is not greater than the first preset threshold, and the brand value is not greater than the second preset threshold, the bit rate control strategy is determined to be a second strategy, and the second strategy is used to lower the bit rate.

[0050] In a second aspect, the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned method for intelligent advertisement identification based on feature analysis of audio and video data.

[0051] In a third aspect, the present application provides an advertisement intelligent recognition system based on audio and video data feature analysis, comprising:

[0052] a memory configured to store instructions; and

[0053] The processor is configured to call the instructions from the memory and implement the above-mentioned advertisement intelligent identification method based on audio and video data feature analysis when executing the instructions.

[0054] Through the above technical scheme, by performing speech recognition on the advertising audio, obtaining the audio text, extracting keywords from the audio text, and determining the first audio feature, it can help obtain the theme and core content of the advertising audio, saving time and energy. By obtaining the acoustic signal of the advertising audio and obtaining the spectrogram of the advertising audio, the spectrum distribution of the advertising audio can be intuitively displayed, which is helpful for further analysis of the characteristics of the advertising audio and determining the second audio feature of the advertising audio. Determining the first feature information through the first audio feature and the second audio feature helps to more accurately determine the feature information of the advertising audio and ensure the accuracy of the advertising audio analysis. By performing feature extraction on the advertising video and obtaining the second feature information of the advertising video, the information of the advertisement can be accurately grasped through the feature information. By calculating the similarity of the first feature information and the second feature information, and generating an advertising label based on the feature information whose similarity is greater than a preset threshold, the key information of the advertisement to be tested is determined, which can better clarify the key information of the advertisement and help improve the advertising effect. According to the key information and the second characteristic information of the advertisement, the advertisement regulation strategy is determined and the bit rate of the advertisement is regulated, which can ensure that the advertisement video is played at a reasonable bit rate, which can not only reduce the waste of bandwidth resources caused by excessive bit rate allocation, but also ensure that the advertisement can be presented to the user in the best state under different network environments and device performance, thereby improving the user's acceptance and satisfaction with the advertisement. In summary, this technical solution can dynamically adjust the advertisement video bit rate according to the complexity of the advertisement, effectively improving the user experience and acceptance of the advertisement.

[0055] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the structure of an advertisement intelligent recognition method based on audio and video data feature analysis provided in an embodiment of the present application;

[0057] Figure 2 This is a diagram showing one of the interfaces of a management platform for an intelligent advertising identification method based on audio and video data feature analysis provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0059] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0060] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0061] Figure 1 The following schematically shows a flow chart of an advertisement intelligent recognition method based on audio and video data feature analysis according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides an advertisement intelligent recognition method based on audio and video data feature analysis, and the method may include the following steps.

[0062] S101, obtaining an advertisement video and corresponding advertisement audio of an advertisement to be tested;

[0063] S102, performing voice recognition on the advertisement audio to obtain an audio text of the advertisement audio;

[0064] S103, obtaining a first audio feature through the audio text;

[0065] S104, obtaining an acoustic signal of the advertisement audio, and obtaining a spectrogram of the advertisement audio according to the acoustic signal based on a short-time Fourier transform;

[0066] S105, determining multiple key point positions of the spectrogram according to the frequency peaks of the spectrogram;

[0067] S106, extracting a second audio feature of the spectrogram based on the multiple key point positions;

[0068] S107, determining first feature information according to the first audio feature and the second audio feature;

[0069] S108, determining second characteristic information of the advertisement video based on the key point position;

[0070] S109, extracting the first text content of the first feature information and the second text content of the second feature information, and calculating the similarity between the first text content and the second text content;

[0071] S110: if the similarity is greater than a preset threshold, generate an advertisement tag according to the first feature information and the second feature information;

[0072] S111. Determine key information of the advertisement according to the advertisement tag;

[0073] S112. Determine an advertisement control strategy based on the key information and the second characteristic information, the advertisement control strategy including a bit rate control strategy, wherein the second characteristic information and the key information are used to characterize the advertisement complexity of the advertisement to be tested.

[0074] Figure 2 The following is an interface display diagram of a management platform for an advertisement intelligent identification method based on audio and video data feature analysis provided by this embodiment. Figure 2 As shown in the interface display diagram, it can be seen that the spectrogram of the advertising audio and the video slices of the advertising video are analyzed to determine information such as the name, brand and category of the advertisement.

[0075] First, obtain the advertising video of the advertisement to be tested and the audio corresponding to the advertisement, and perform speech recognition on the advertising audio to obtain the audio text of the advertising audio. The advertisement to be tested can be an advertisement of any field type, such as automobile field advertisement, home appliance field advertisement, etc., and this application does not limit this. It should be noted that in this embodiment, the advertisement to be tested is an advertisement that contains both video and audio. Specifically, in the advertisement to be tested, the speech content in the advertising audio can be converted into text form through a speech recognition algorithm. The speech recognition algorithm can be an algorithm based on dynamic time regularization. The algorithm based on dynamic time regularization is a method based on template matching, which performs recognition by calculating the similarity between the input speech and the template.

[0076] In one embodiment, the first audio feature can be obtained through the audio text. Specifically, the field type of the advertisement to be tested can be determined through the audio text. For example, through the audio text of the advertisement, it is determined that one or more keywords of the advertisement are juice, and the field type of the advertisement to be tested is the beverage field. After determining the field type of the advertisement to be tested, the audio text can be divided according to the preset word category. In this embodiment, the preset word category can be determined according to the actual situation. For example, the preset word category can be a two-character word or a four-character word. The word category can be set according to the field of the advertisement. For example, if the field of the advertisement is the agricultural field, the two-character word can be set to words such as fertilizer, high yield, and corn. After the audio text is divided according to the preset word category, the audio words are extracted from the audio text according to the preset two-character word or four-character word to obtain multiple audio words. After obtaining multiple audio words, the audio keywords are determined in the multiple audio words according to the field type. In this embodiment, the audio keywords can be determined according to the field type of the advertisement. The number of times the audio keywords appear in the audio text can be first determined, and the first repetition degree of the audio keywords can be obtained by the number of times the audio keywords appear in the audio text. In addition, the number of times the audio keywords appear in the preset database can be counted. The preset database in this embodiment stores historical advertisement text keywords. The second repetition degree of the audio keyword can be obtained by the number of times the audio keyword appears in the preset database. After obtaining the first repetition degree and the second repetition degree, the weight value of the first repetition degree and the second repetition degree can be calculated. If the weight value is greater than the preset threshold, the characteristic keyword is determined, and the characteristic keyword is used to characterize the first audio feature. In this embodiment, the first audio feature refers to the audio feature obtained through the keywords of the advertising audio. For example, if the keywords of the advertising audio are mobile phone, high-definition sound quality, ultra-long battery life and young people, the number of times "mobile phone" appears in the preset data is determined. The second repetition degree of "mobile phone" can be calculated through the formula of the second repetition degree.

[0077] In another embodiment, the audio feature can also be obtained through the acoustic signal of the advertising audio. In this embodiment, the acoustic signal of the advertising audio can be converted into a spectrogram of the advertising audio through short-time Fourier transform, wherein the short-time Fourier transform is a mathematical transformation related to the Fourier transform, which is used to determine the frequency and phase of the sine wave of the local area of ​​the time-varying signal, and the spectrogram refers to a speech spectrum diagram, which can be obtained by processing the received time domain signal. Since the technology of obtaining the spectrogram by short-time Fourier transform is widely used, this application will not repeat it.

[0078] Through the spectrogram of the advertising audio, the position of the frequency peak in the spectrogram is determined, and the position of the frequency peak in the spectrogram is used as the multiple key point positions of the spectrogram. Based on the multiple key point positions, the second audio feature of the spectrogram is extracted. In this embodiment, the second audio feature refers to the characteristic information of the advertising audio obtained by analyzing the spectrogram. In a specific implementation, the spectrogram can be divided according to the key point positions to obtain multiple spectrogram segments, and the frequency, amplitude and intensity of each spectrogram segment are determined. Through the frequency, amplitude and intensity of each spectrogram segment, the basic feature information of the speech segment corresponding to each spectrogram segment is obtained, wherein the basic feature information of the speech segment corresponding to each spectrogram segment may include age features, gender features and emotional features, the age feature is determined based on the frequency and intensity of the spectrogram segment, the gender feature is determined based on the frequency and amplitude of the spectrogram segment, and the emotional feature is determined based on the intensity and amplitude of the spectrogram segment.

[0079] After obtaining the first audio feature and the second audio feature, the semantic similarity between the first audio feature and the second audio feature is calculated. In this embodiment, the semantic similarity between the first audio feature and the second audio feature can be calculated by a word embedding model, such as Word2Vec and GloVe, by mapping words to a high-dimensional vector space and calculating the cosine similarity between words to measure the semantic similarity. After obtaining the semantic similarity, the keywords with a semantic similarity greater than a preset similarity are used as the first feature information. In this embodiment, the preset similarity can be determined according to the actual situation. The first feature information refers to feature information whose semantic similarity exceeds a preset threshold in the first audio feature and the second audio feature.

[0080] Based on the key point position, the second feature information of the advertising video is determined. In this embodiment, the second feature information refers to the feature information obtained after feature extraction of the advertising video, and the key point position refers to the position of the frequency peak of the spectrogram. First, the position of the frequency peak in the spectrogram is determined, and the advertising video is divided based on the position of the frequency peak to obtain multiple video slices. After obtaining the video slices, feature extraction is performed on each video slice through a preset recognition model. In this embodiment, the preset recognition model can be a 3D convolutional neural network model. The 3D convolutional neural network model is a spatiotemporal extension of the 2D convolutional neural network model. It not only slides the convolution kernel in the spatial domain, but also slides in the temporal domain. It can simultaneously mine the spatial information and temporal information of the video frame, and mine the features in the advertising video to obtain video features, wherein the video features are used to characterize the second feature information. In this embodiment, the video features include video resolution, clarity, color, contrast, duration, loading time, and frame differences.

[0081] After determining the first characteristic information of the advertising audio and the second characteristic information of the advertising video, the first text content of the first characteristic information and the second text content of the second characteristic information can be extracted, and the similarity between the first text content and the second text content can be calculated to facilitate the subsequent determination of the advertising label. In this embodiment, the similarity between the first text content and the second text content can be calculated through word embedding models, such as Word2Vec and GloVe, by mapping words to high-dimensional vector space and calculating the cosine similarity between words to measure the semantic similarity. If the similarity is greater than a preset threshold, an advertising label is generated based on the first characteristic information and the second characteristic information. In this embodiment, the advertising label may include a type label, a style label, etc. of the advertisement. After obtaining the advertising label, the key information of the advertisement is determined based on the advertising label. In this embodiment, the key information includes brand, advertising category, advertising theme, etc.

[0082] The advertising control strategy is determined by the key information and the second characteristic information. In this embodiment, the advertising control strategy includes a bit rate control strategy and a scheme control strategy. The second characteristic information and the key information are used to characterize the advertising complexity of the advertisement to be tested. In this embodiment, the advertising complexity refers to the complexity of the content of the advertisement video screen. Specifically, first, the key information is clustered to obtain the user's preference characteristics for the advertisement to be tested. In this embodiment, the user's preference characteristics for the advertisement to be tested can be the user's preference for the type of advertisement to be watched, the theme of the advertisement, and the style of the advertisement. After obtaining the user's preference characteristics, the corresponding retrieval algorithm is retrieved in the preset database, and the target advertisement video is retrieved in the user terminal of the user using the retrieval algorithm. The user's target audience category is determined by the target advertisement video. In this embodiment, the target audience category can be young people, middle-aged people, and the elderly. After determining the target audience category, the advertisement quality score of the advertisement to be tested is determined by the preset quality scoring algorithm and the video characteristics of the advertisement to be tested, and the bit rate control strategy is determined according to the advertisement quality score, brand, and target audience category.

[0083] Through the above technical scheme, by performing speech recognition on the advertising audio, obtaining the audio text, extracting keywords from the audio text, and determining the first audio feature, it can help obtain the theme and core content of the advertising audio, saving time and energy. The acoustic signal of the advertising audio is obtained, and the spectrogram of the advertising audio is obtained. The spectrum distribution of the advertising audio can be intuitively displayed through the spectrogram of the advertising audio, which is helpful for further analysis of the characteristics of the advertising audio and determination of the second audio feature of the advertising audio. Determining the first feature information through the first audio feature and the second audio feature helps to more accurately determine the feature information of the advertising audio and ensure the accuracy of the advertising audio analysis. By extracting features from the advertising video and obtaining the second feature information of the advertising video, the information of the advertisement can be accurately grasped through the feature information. By calculating the similarity of the first feature information and the second feature information, and generating an advertising label based on the feature information whose similarity is greater than a preset threshold, the key information of the advertisement to be tested is determined, which can better clarify the key information of the advertisement and help improve the advertising effect. According to the key information and second characteristic information of the advertisement, the advertisement regulation strategy is determined and the bit rate of the advertisement is adjusted to ensure that the advertisement video is played at a reasonable bit rate. This can not only reduce the waste of bandwidth resources caused by excessive bit rate allocation, but also ensure that the advertisement can be presented to users in the best state under different network environments and device performance, thereby improving user acceptance and satisfaction with the advertisement.

[0084] In one implementation of this embodiment, obtaining the first audio feature through the audio text includes:

[0085] S210, obtaining the domain type of the advertisement to be tested;

[0086] S220, dividing the audio text according to preset word categories to obtain multiple audio words;

[0087] S230, determining an audio keyword from a plurality of audio words according to the field type;

[0088] S240, determining a corresponding preset database according to the audio keyword;

[0089] S250, determining the number of times the audio keyword appears in the audio text;

[0090] S260, obtaining a first repetition degree of the audio keyword according to the number of times the audio keyword appears in the audio text;

[0091] S270, determining the number of times the audio keyword appears in a preset database, where historical advertisement text keywords are stored;

[0092] S280, obtaining a second repetition degree of the audio keyword according to the number of times the audio keyword appears in a preset database;

[0093] S290: Calculate weight values ​​of the first repetition degree and the second repetition degree. If the weight values ​​are greater than a preset threshold, determine a feature keyword, where the feature keyword is used to characterize the first audio feature.

[0094] First, the field type of the advertisement to be tested is obtained. In this embodiment, the field type of the advertisement to be tested can be obtained through the audio text of the advertisement. For example, through the audio text of the advertisement, the keywords of the advertisement are determined to be plants and fertilizers, and the field type of the advertisement to be tested is determined to be the agricultural field. After determining the field type of the advertisement to be tested, the audio text is divided according to the preset word categories. In this embodiment, the preset word categories can be determined according to actual conditions. For example, the preset word categories can be two-character words or four-character words. The word categories can be set according to the field of the advertisement. For example, if the field of the advertisement is the agricultural field, the preset two-character words can be set to words such as fertilizer, high yield, and corn, and the preset four-character words can be set to four-character words such as farmland propaganda and green advertisement. The audio text is divided according to the preset word categories to obtain multiple audio words. That is to say, after obtaining the audio text, the audio text is divided according to the preset word type, such as two-character words or four-character words. For example, the audio text of the advertisement is that health is the cornerstone of life, and our health products are your right-hand man to protect your health. With selected natural ingredients and scientific proportions, our health products will create a healthy new life for you. Take care of yourself and start by choosing our health products, so that health is no longer a luxury. According to the field type of the audio text, the preset word categories can be health, health products, etc. The audio text is divided according to the preset word categories, and the audio words obtained can be health, life, health products, protection, and scientific proportions.

[0095] According to the determined audio words and field types, audio keywords are determined among the multiple audio words. Specifically, audio words with semantic repetition are determined among the audio words, and the audio words with semantic repetition are determined as audio keywords. For example, health care products and health appear repeatedly in the audio words, therefore, health care products and health are the audio keywords among the multiple audio words.

[0096] In order to more accurately determine the first audio feature of the advertisement, after determining the audio keyword, the corresponding preset database can be determined by the audio keyword. In this embodiment, the preset database can be determined according to the field type of the audio keyword. For example, if the audio keyword is medicine, the preset database can be determined as a text database in the health field. After obtaining the corresponding preset database, the number of times the audio keyword appears in the audio text is determined. The more times the audio keyword appears in the audio text, the more important the audio keyword is. According to the number of times the audio keyword appears in the audio text, the first repetition degree of the audio keyword is determined. The formula for determining the first repetition degree of the audio keyword is as follows:

[0097]

[0098] Among them, n i，j Indicates the number of times the audio keyword appears in the audio text, TF i，j Indicates the first repetition degree of the audio keyword;

[0099] After determining the first repetition degree of the audio keyword, the number of times the audio keyword appears in a preset database is determined. The preset database stores historical advertisement text keywords. That is, the number of times the audio keyword appears in the preset database is determined in the preset database, and the second repetition degree of the audio keyword is obtained by the number of times the audio keyword appears in the preset database. The formula for determining the second repetition degree of the audio keyword is as follows:

[0100]

[0101] Where N represents the number of historical advertising text keywords stored in the preset database, n i Indicates the number of times the audio keyword appears in the preset database with the historical advertising text keyword. IDF i，j Indicates the second repetition degree of the audio keyword;

[0102] After obtaining the first repetition degree and the second repetition degree of the audio keyword, the weight value of the first repetition degree and the second repetition degree is calculated, and the weight value of the audio keyword is obtained by multiplying the first repetition degree and the second repetition degree. The formula is:

[0103] W i，j =TF i，j ×IDF i，j

[0104] Among them, W i，j Indicates the weight value of the audio keyword;

[0105] When the weight value of the audio keyword is greater than a preset threshold, a feature keyword can be determined. The feature keyword is used to characterize the first audio feature. In this embodiment, the preset threshold can be determined according to actual conditions. The feature keyword is determined by the audio keyword, and the audio keyword greater than the preset threshold is used as the feature keyword.

[0106] By determining the audio keywords of the audio text and then determining the first audio feature, the keyword features of the advertising audio can be accurately obtained, which helps to accurately grasp the information of the advertising audio, and can more accurately target the target users, ensuring that the advertising information can better meet the needs of users and improve the user experience.

[0107] In one implementation of this embodiment, based on multiple key point positions, extracting features of the spectrogram to obtain the second audio feature includes:

[0108] S310, dividing the spectrogram according to the key point positions to obtain a plurality of spectrogram segments;

[0109] S320, determining the frequency, amplitude and intensity of each spectrogram segment;

[0110] S330, based on the frequency, amplitude and intensity of each spectrogram segment, obtaining basic feature information of the speech segment corresponding to each spectrogram segment;

[0111] S340. Using a preset portrait generation algorithm, construct a corresponding voice portrait through each basic feature information, wherein the voice portrait includes an audio tag, and the audio tag is used to characterize the second audio feature.

[0112] Based on multiple key point positions, the features of the spectrogram are extracted to obtain the second audio feature. Specifically, first, the spectrogram is divided according to the key point positions. The spectrogram refers to a speech spectrum graph, which is a spectrum graph obtained by processing the received time domain signal. In this embodiment, the multiple key point positions refer to the positions of frequency peaks in the spectrogram. The spectrogram is divided according to the positions of the frequency peaks to obtain multiple spectrogram segments.

[0113] After obtaining multiple spectrogram segments, determine the frequency, amplitude and intensity in each spectrogram segment. In the spectrogram, the frequency is usually represented on the horizontal axis to show the existence of different frequency components in the speech signal. Amplitude refers to the amplitude of the signal at different frequencies, reflecting the strength of the signal. Intensity is closely related to amplitude and is usually used to describe the energy or power of the signal. In the spectrogram, the intensity can be intuitively displayed by the amplitude. The larger the amplitude, the higher the signal intensity at that frequency.

[0114] The basic characteristic information of the speech segment corresponding to each spectrogram segment is obtained by analyzing the frequency, amplitude and intensity of each spectrogram segment, that is, the frequency, amplitude and intensity of each spectrogram segment can be determined by analyzing the spectrogram segment, and the basic characteristic information is determined according to the frequency, amplitude and intensity. In this embodiment, the basic characteristic information includes age characteristics, gender characteristics and emotional characteristics. The age characteristics are determined by analyzing the frequency and intensity of the spectrogram segment. For example, the vocal cords of young people are tight, the pitch of the sound emitted is high, and higher frequency sound waves can be heard, while the vocal cords of the elderly are loose, the pitch of the sound emitted is low, and usually only lower frequency sound waves can be heard. Therefore, the greater the frequency of the spectrogram segment, the younger the age may be. Generally speaking, with the increase of age, the voice may become hoarse and the intensity may decrease. Therefore, the greater the intensity of the spectrogram segment, the younger the age is. The gender characteristics determined by the frequency and amplitude of the spectrogram segment, for example, the frequency of female voices is usually higher, while the frequency of male voices is lower. This is due to the difference in the structure of male and female vocal cords. Female vocal cords are shorter, thinner, and have a higher vibration frequency; male vocal cords are longer, wider, thicker, and have a lower vibration frequency. Amplitude represents the strength of the signal. Generally speaking, the voice intensity of men is greater than that of women. Therefore, spectrogram segments with higher frequencies and lower intensity are female, while spectrogram segments with lower frequencies and higher intensity are male. The emotional characteristics are determined by the intensity and amplitude of the spectrogram segments. In other words, intensity usually reflects the energy of the speech signal and is related to the speaker's emotional expression. High intensity may indicate strong emotions, such as anger or excitement, while low intensity may indicate calmness or sadness. Amplitude represents the vibration amplitude of the signal and is related to the loudness and clarity of the speech. Different amplitudes may correspond to different emotional states. For example, a high tone may indicate surprise or happiness, while a low tone may indicate sadness or dissatisfaction. Therefore, the emotional characteristics are determined by the intensity and amplitude of the spectrogram segments.

[0115] After obtaining the basic feature information of the speech segment corresponding to each spectrogram segment, a preset portrait generation algorithm is used to construct a corresponding speech portrait through each basic feature information. The preset portrait generation algorithm can be a generative adversarial network algorithm. The generative adversarial network algorithm is composed of a generator and a discriminator. Through adversarial training, the generator gradually learns to generate high-quality images. By using the generative adversarial network algorithm, a corresponding speech portrait is constructed according to each basic feature information. By using artificial intelligence technology, by analyzing the individual's speech features, a variety of information about the speaker can be identified, such as emotions, social status, growth experience, age, weight, height and facial features. By constructing a speech portrait, wherein the speech portrait includes an audio label, the label of each spectrogram segment is determined, thereby determining the second audio feature. In this embodiment, the second audio feature refers to the feature information of the advertising audio obtained by analyzing the spectrogram.

[0116] By dividing the key point positions of the spectrogram, we obtain spectrogram segments, and determine the age characteristics, gender characteristics and emotional characteristics of each spectrogram segment. Based on the basic feature information, we generate a voice portrait to obtain the second audio feature, which can accurately determine the information of the advertising audio and help to more accurately determine the audio tag of the advertisement.

[0117] In one implementation manner of this embodiment, determining the first feature information according to the first audio feature and the second audio feature includes:

[0118] S410: Calculate the semantic similarity between the first audio feature and the second audio feature, and use the keyword with a semantic similarity greater than a preset similarity as the first feature information.

[0119] The first feature information is determined according to the first audio feature and the second audio feature. Specifically, by calculating the semantic similarity between the first audio feature and the second audio feature, the keywords with semantic similarity greater than the preset similarity are used as the first feature information. In this embodiment, the first audio feature refers to the audio feature obtained through the keywords of the advertising audio; the second audio feature refers to the feature information of the advertising audio obtained by analyzing the spectrogram; the first feature information refers to the feature information whose semantic similarity in the first audio feature and the second audio feature exceeds the preset threshold; the preset similarity can be determined according to the actual situation. The keywords with semantic similarity greater than the preset similarity in the first audio feature and the second audio feature are retained as the first feature information. The semantic similarity between the first audio feature and the second audio feature can be calculated by a word embedding model, such as Word2Vec and GloVe, by mapping words to a high-dimensional vector space and calculating the cosine similarity between words to measure the semantic similarity.

[0120] By calculating the semantic similarity between the first audio feature and the second audio feature, and taking the keyword with a semantic similarity greater than a preset similarity as the first feature information, the feature information of the advertising audio can be accurately obtained, which helps to better grasp the information of the advertisement, improve user acceptance and satisfaction with the advertisement, and optimize the user experience.

[0121] In one implementation of this embodiment, determining the second feature information based on the key point position includes:

[0122] S510, dividing the advertisement video based on the key point positions to obtain multiple video slices;

[0123] S520: extract features from each video slice using a preset recognition model to obtain video features, where the video features are used to represent the second feature information.

[0124] Based on the key point positions, the second feature information is determined. Specifically, first, the advertising video is divided based on the key point positions to obtain multiple video slices. In this embodiment, the key point positions are the positions of the frequency peaks in the spectrogram. According to the positions of the frequency peaks in the spectrogram, the advertising video is divided to obtain multiple video slices. Video slices are to divide the complete video file into multiple small segments, which can be processed, transmitted and played independently.

[0125] After obtaining multiple video slices, feature extraction is performed on each video slice through a preset recognition model. In this embodiment, the preset recognition model is a 3D convolutional neural network model. The 3D convolutional neural network model is a spatiotemporal extension of the 2D convolutional neural network model. It not only slides the convolution kernel in the spatial domain, but also slides in the temporal domain. It can simultaneously mine the spatial information and temporal information of the video frame, and mine the features in the advertising video to obtain video features. The 3D convolutional neural network model is used to continuously mine the features of the advertising video and extract video features to determine the video features of each video slice. In this embodiment, the video features include video resolution, clarity, color, contrast, duration, loading time, and inter-frame difference. That is to say, the resolution, clarity, color, contrast, duration, loading time, and inter-frame difference in each video slice are respectively confirmed. The determined video features are used to characterize the second feature information. In this embodiment, the second feature information refers to the feature information obtained after feature extraction of the advertising video.

[0126] By dividing the advertising video, video slices are obtained, and feature extraction is performed on each video slice to obtain video features. By analyzing the advertising video, the second feature information of the advertisement can be better determined, which is helpful for subsequent analysis of the advertisement and improves the processing efficiency of advertising video analysis.

[0127] In one implementation of this embodiment, the key information includes the brand, and the video features include video resolution, clarity, color, contrast, duration, loading time, and inter-frame difference. Based on the key information and the second feature information, an advertising regulation strategy is determined, including:

[0128] S610, clustering the key information to obtain the user's preference characteristics for the advertisement to be tested;

[0129] S620, searching a preset database for a corresponding search algorithm according to the preference feature;

[0130] S630, using a retrieval algorithm to retrieve a target advertisement video in a user terminal of the user, and determining a target audience category of the user according to the target advertisement video;

[0131] S640, determining an advertisement quality score of the advertisement to be tested according to video features of the advertisement to be tested based on a preset quality scoring algorithm;

[0132] S650: Determine a bit rate control strategy based on the advertisement quality score, brand, and target audience category.

[0133] Based on the key information and the second characteristic information, the advertising regulation strategy is determined. In this embodiment, the key information may include brand, advertising category, advertising theme, etc. In this embodiment, the advertising regulation strategy includes bit rate regulation strategy and advertising scheme regulation strategy. The bit rate regulation strategy refers to a strategy for adjusting the data traffic used by the advertising video in a unit time. The scheme regulation strategy refers to an in-depth understanding of the age, gender, geographical location, interests and hobbies of the target audience to formulate a more effective advertising strategy. First, the key information is clustered to obtain the user's preference characteristics for the tested advertisement. In this embodiment, the user's preference characteristics for the tested advertisement are mainly reflected in content preference, personal characteristics, advertising effect and scene association. Cluster analysis is an analysis of grouping a collection of physical or abstract objects into multiple classes composed of similar objects. The key information is clustered to obtain the user's preference characteristics for the tested advertisement. For example, the key information of the advertising video is middle-aged and elderly, health products, and health. It can be determined that the user's preference characteristics for the tested advertisement are inclined to watch advertising videos about health products for middle-aged and elderly people, and advertising videos that help protect physical health.

[0134] After determining the user's preference characteristics for the tested advertisement, the corresponding retrieval algorithm is retrieved from the preset database according to the preference characteristics. Since users have different requirements for advertisements when watching advertisements, for example, the elderly tend to watch advertisements related to physical health, and do not have too high requirements for the clarity of advertisements. Young people tend to watch advertisements with higher quality, have relatively high requirements for the clarity of advertisements, and tend to watch high-definition advertisements. High-definition advertisements may increase young people's desire to buy the goods in the advertisements, and increase the sales of goods. Therefore, when watching advertisements, users have inconsistent requirements for advertisements. According to the user's preference characteristics, the corresponding retrieval algorithm is retrieved from the preset database, and the user's target advertisement video is determined through the user's exclusive retrieval algorithm. For example, young people are more concerned about the clarity of the advertisement, the type of goods in the advertisement, and the duration of the advertisement. Therefore, in the preset database, the retrieval algorithm for the clarity of the advertisement, the type of goods in the advertisement, and the duration of the advertisement is determined, and the user retrieves the target advertisement video in the user terminal through the exclusive algorithm.

[0135] The target advertising video is retrieved in the user terminal by using a retrieval algorithm, and the target audience category of the user is determined based on the target advertising video. That is, the target advertising video is retrieved in the user terminal by using a user-specific algorithm. For example, the user's preference is the type of goods in the advertising video. According to the user-specific retrieval algorithm, the advertising video of the brand of goods preferred by the user is determined in the user terminal and played for the user. After the user's target advertising video is determined, the target audience category of the user is determined based on the target advertising video. That is, through the user's target advertising video, for example, if the user's target advertising video is an advertising video of fast-moving consumer goods, the user's target audience category can be determined to be fast-moving consumer goods category users, indicating that the user prefers to watch and purchase fast-moving consumer goods products.

[0136] Based on the preset quality scoring algorithm, the advertisement quality score of the advertisement to be tested is determined by the video features of the advertisement to be tested. In this embodiment, the quality scoring algorithm is preset and used to evaluate the advertisement quality score, which can reflect the user's satisfaction with the advertisement video. The judging criteria for judging the advertisement video in the preset quality scoring algorithm can be determined according to the actual situation, and the quality score of the advertisement to be tested is determined by the preset quality scoring algorithm.

[0137] After determining the quality score of the advertisement to be tested, the bit rate control strategy is determined according to the advertisement quality score, brand and target audience category. In this embodiment, key information includes brand, advertisement category, advertisement theme, etc. The bit rate control strategy of the advertisement video is determined by the quality score of the advertisement to be tested, the value of the brand and the target audience category. For example, young people in the target audience category have relatively high requirements for the bit rate of the advertisement video, and the brand value is also relatively high for high-end luxury goods. When the advertisement quality score is relatively high, the bit rate of the advertisement video needs to be increased to ensure the viewing effect of the user.

[0138] By obtaining the user's preference characteristics for the tested ads, we can accurately grasp the user's preferred advertising videos. Based on the preference characteristics, we can retrieve the corresponding retrieval algorithm in the preset database to accurately obtain the user's target advertising video. According to the advertising quality score, brand and target audience category, we can determine the bit rate control strategy, which can accurately determine the value of the user's viewing advertising video bit rate adjustment, and ensure the user's experience and satisfaction when watching the advertising video.

[0139] In one implementation of this embodiment, a bit rate control strategy is determined according to the advertisement quality score, brand, and target audience, including:

[0140] S710, searching the brand in a preset database to obtain the corresponding brand value;

[0141] S720: When the target audience category is a first preset type, the advertisement quality score is greater than a first preset threshold, and the brand value is greater than a second preset threshold, any two of the following conditions are met: determining that the bitrate control strategy is a first strategy, and the first strategy is used to increase the bitrate;

[0142] S730: When any two of the following conditions are met: the target audience category is the second preset type, the advertisement quality score is not greater than the first preset threshold, and the brand value is not greater than the second preset threshold, the bit rate control strategy is determined to be the second strategy, and the second strategy is used to lower the bit rate.

[0143] The bit rate control strategy is determined based on the advertising quality score, brand and target audience. Specifically, the brand is searched in the preset database to obtain the corresponding brand value. The brand value refers to the comprehensive evaluation and recognition of the brand value by consumers, reflecting the comprehensive image and appeal of the brand in the minds of consumers.

[0144] After obtaining the brand value, first, determine the target audience category. When the target audience category is the first preset type, if any two of the following conditions are met: the advertisement quality score is greater than the first preset threshold and the brand value is greater than the second preset threshold, determine the bitrate control strategy as the first strategy, which is used to increase the bitrate.

[0145] In this embodiment, the first preset type can be young people or middle-aged people, and the first preset threshold and the second preset threshold can be determined according to actual conditions. For example, when it is determined that the target audience category is 20-year-old young people, the advertising quality score is greater than the first preset threshold and the brand value is greater than the second preset threshold, the first strategy is executed to increase the bit rate, improve the user's perception of the advertisement, and maximize the user's interest in the advertisement. In this embodiment, the first strategy is to increase the bit rate of the advertising video. The bit rate of the advertising video can be a fixed bit rate. For example, if the current bit rate is 5Mbps, if the target audience category is the first preset type, the advertising quality score is greater than the first preset threshold and the brand value is greater than the second preset threshold, the current bit rate is increased to 15Mbps. At this time, 15Mbps is a fixed bit rate, which is suitable for high-quality video playback scenarios. Similarly, if the current bit rate is 15Mbps, and the target audience category is the first preset type, the advertising quality score is greater than the first preset threshold, and the brand value is greater than the second preset threshold, the first strategy is still to increase the current bit rate to 15Mbps, but because the current bit rate is the same as the fixed bit rate, in actual implementation, the first strategy may not be implemented.

[0146] It should be noted that for the following three conditions, only at least two of them need to be met, namely: (1) the target audience category is the first preset type; (2) the advertising quality score is greater than the first preset threshold; (3) the brand value is greater than the second preset threshold.

[0147] In another embodiment, the bit rate of the advertisement video can be dynamically adjusted according to the above three conditions. The population corresponding to the target audience category can be divided into m age intervals according to age, the advertisement quality score can be divided into n intervals, and the brand value can be divided into p intervals. Taking the following Table 1 as an example, it can be seen that the advertisement video bit rate can be dynamically adjusted according to the target audience category age interval, the advertisement quality score, and the brand value. Table 1 is as follows:

[0148] Target audience category age range Ad quality score>75 Brand value>100 Bitrate setting Youth [18-30] 88 120 15Mbps Youth [30-44] 80 150 20Mbps Middle age [45-50] 86 200 15Mbps Middle age [50-59] 95 120 10Mbps

[0149] It should be noted that Table 1 is only a partial illustrative description and is not intended to limit the present application.

[0150] In a specific implementation, the bit rate corresponding to the target audience category age range, advertisement quality score and brand value can be queried in the preset bit rate database.

[0151] Secondly, when the target audience category is the second preset type, if any two of the following conditions are met: the advertising quality score is not greater than the first preset threshold and the brand value is not greater than the second preset threshold, the bit rate control strategy is determined to be the second strategy, which is used to lower the bit rate.

[0152] In this embodiment, the second preset type can be elderly people over 59 years old, and the first preset threshold and the second preset threshold can be determined according to the actual situation. For example, when it is determined that the target audience category is a middle-aged person of 60 years old, and the advertising quality score is less than the first preset threshold and the brand value is less than the second preset threshold, at this time, the second strategy is executed to lower the bit rate, and optimize the transmission efficiency of the advertising video and the various storage requirements of the user to store the advertising video under the premise of ensuring the quality of the advertising video. In this embodiment, the second strategy is used to lower the bit rate, and the bit rate of the advertising video can be a fixed bit rate. For example, if the current bit rate is 15Mbps, if the target audience category is the second preset type, the advertising quality score is less than the first preset threshold and the brand value is less than the second preset threshold, the current bit rate is lowered to 10Mbps. At this time, 10Mbps is a fixed bit rate, which is suitable for playback scenarios with limited network speed. Similarly, if the current bit rate is 10Mbps, and the target audience category is the second preset type, the advertising quality score is less than the first preset threshold, and the brand value is less than the second preset threshold, the second strategy is still to lower the current bit rate to 10Mbps, but because the current bit rate is the same as the fixed bit rate, in actual implementation, the second strategy may not be implemented.

[0153] It should be noted that for the following three conditions, only at least two of them need to be met, namely: (1) the target audience category is the second preset type; (2) the advertising quality score is less than the first preset threshold; (3) the brand value is less than the second preset threshold.

[0154] In another embodiment, the bit rate of the advertisement video can be dynamically adjusted according to the above three conditions. The population corresponding to the target audience category can be divided into m age intervals according to age, the advertisement quality score can be divided into n intervals, and the brand value can be divided into p intervals. Taking the following Table 2 as an example, it can be seen that the advertisement video bit rate can be dynamically adjusted according to the target audience category age interval, the advertisement quality score, and the brand value. Table 2 is as follows:

[0155] Target audience category age range Ad quality score <75 Brand value <100 Bitrate setting Old age [60-64] 70 99 10Mbps Old age [65-70] 72 80 15Mbps Old age [71-75] 69 97 13Mbps Old age [76-80] 74 95 18Mbps

[0156] It should be noted that Table 2 is only a partial illustrative description and is not intended to limit the present application.

[0157] In a specific implementation, the bit rate corresponding to the target audience category age range, advertisement quality score and brand value can be queried in the preset bit rate database.

[0158] Determining the adjustment bit rate of advertisements through brand value, target audience type and advertisement quality rating can not only ensure the quality of advertisement videos, but also save storage space and bandwidth costs, improve advertisement delivery efficiency, enhance user viewing experience, adapt to different scene requirements and reduce costs, which is of great significance.

[0159] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned advertisement intelligent identification method based on audio and video data feature analysis.

[0160] The present application also provides an electronic device, including:

[0161] a memory configured to store instructions; and

[0162] The processor is configured to call instructions from the memory and implement the above-mentioned advertisement intelligent recognition method based on audio and video data feature analysis when executing the instructions.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0164] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0169] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0170] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0171] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An intelligent advertisement recognition method based on audio and video data feature analysis, characterized in that: include: Obtain the advertising video and corresponding advertising audio of the advertisement to be tested; Performing speech recognition on the advertisement audio to obtain an audio text of the advertisement audio; Obtaining a first audio feature through the audio text; Acquire an acoustic signal of the advertisement audio, and obtain a spectrogram of the advertisement audio according to the acoustic signal based on a short-time Fourier transform; Determining multiple key point positions of the spectrogram according to the frequency peaks of the spectrogram; Extracting a second audio feature of the spectrogram based on the plurality of key point positions; Determining first feature information according to the first audio feature and the second audio feature; Based on the key point positions, determining second feature information of the advertising video; Extracting a first text content of the first feature information and a second text content of the second feature information, and calculating a similarity between the first text content and the second text content; If the similarity is greater than a preset threshold, generating an advertisement tag according to the first feature information and the second feature information; Determining key information of the advertisement according to the advertisement tag; Based on the key information and the second characteristic information, an advertisement regulation strategy is determined, the advertisement regulation strategy including a bit rate regulation strategy, wherein the second characteristic information and the key information are used to characterize the advertisement complexity of the advertisement to be tested.

2. The method according to claim 1, characterized in that: The step of obtaining a first audio feature through the audio text includes: Obtaining the domain type of the advertisement to be tested; Dividing the audio text according to preset word categories to obtain multiple audio words; Determining audio keywords from the plurality of audio words according to the field type; According to the audio keyword, determining a corresponding preset database; Determining the number of times the audio keyword appears in the audio text; Obtaining a first repetition degree of the audio keyword according to the number of times the audio keyword appears in the audio text; Determining the number of times the audio keyword appears in a preset database, wherein the preset database stores historical advertisement text keywords; Obtaining a second repetition degree of the audio keyword according to the number of times the audio keyword appears in a preset database; A weight value of the first repetition degree and the second repetition degree is calculated, and if the weight value is greater than a preset threshold, a feature keyword is determined, where the feature keyword is used to characterize the first audio feature.

3. The method according to claim 2, characterized in that The extracting the feature of the spectrogram based on the multiple key point positions to obtain the second audio feature includes: Dividing the spectrogram according to the key point positions to obtain a plurality of spectrogram segments; determining the frequency, amplitude and intensity of each of said spectrogram segments; Based on the frequency, amplitude and intensity of each of the spectrogram segments, basic feature information of the speech segment corresponding to each of the spectrogram segments is obtained; A preset portrait generation algorithm is used to construct a corresponding voice portrait through each of the basic feature information, wherein the voice portrait includes an audio tag, and the audio tag is used to characterize the second audio feature.

4. The method according to claim 3, characterized in that The basic feature information of the speech segment corresponding to each spectrogram segment includes age feature, gender feature and emotion feature, wherein the age feature is determined based on the frequency and intensity of the spectrogram segment, the gender feature is determined based on the frequency and amplitude of the spectrogram segment, and the emotion feature is determined based on the intensity and amplitude of the spectrogram segment.

5. The method according to claim 3, characterized in that: The determining first feature information according to the first audio feature and the second audio feature includes: The semantic similarity between the first audio feature and the second audio feature is calculated, and a keyword whose semantic similarity is greater than a preset similarity is used as the first feature information.

6. The method according to claim 3, characterized in that: The determining the second feature information based on the key point position includes: Divide the advertisement video based on the key point positions to obtain multiple video slices; By using a preset recognition model, feature extraction is performed on each of the video slices to obtain video features, wherein the video features are used to characterize the second feature information.

7. The method according to claim 6, characterized in that The key information includes brand, the video features include video resolution, clarity, color, contrast, duration, loading time and inter-frame difference, and the determining of the advertising regulation strategy based on the key information and the second feature information includes: Clustering the key information to obtain the user's preference characteristics for the advertisement to be tested; According to the preference characteristics, a corresponding search algorithm is retrieved in a preset database; Using the retrieval algorithm to retrieve a target advertisement video in a user terminal of the user, and determining a target audience category of the user according to the target advertisement video; Based on a preset quality scoring algorithm, determining the advertisement quality score of the advertisement to be tested according to the video features of the advertisement to be tested; The bit rate regulation strategy is determined according to the advertisement quality score, the brand and the target audience category.

8. The method according to claim 7, characterized in that The determining the bit rate control strategy according to the advertisement quality score, the brand and the target audience includes: Searching the brand in a preset database to obtain the corresponding brand value; When any two of the following conditions are met: the target audience category is a first preset type, the advertisement quality score is greater than a first preset threshold, and the brand value is greater than a second preset threshold, the bit rate control strategy is determined to be a first strategy, and the first strategy is used to increase the bit rate; When any two of the following conditions are met: the target audience category is a second preset type, the advertisement quality score is not greater than the first preset threshold, and the brand value is not greater than the second preset threshold, the bit rate control strategy is determined to be a second strategy, and the second strategy is used to lower the bit rate.

9. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for enabling a machine to execute the advertisement intelligent identification method based on audio and video data feature analysis according to any one of claims 1 to 8.

10. An advertisement intelligent recognition system based on audio and video data feature analysis, characterized in that: include: a memory configured to store instructions; as well as The processor is configured to call the instruction from the memory and implement the advertisement intelligent identification method based on audio and video data feature analysis according to any one of claims 1 to 8 when executing the instruction.

Images