Advertisement video rhythm sensation analysis system

Abstract

The invention provides an advertisement video rhythm sensation analysis system, which initially splits a fixed section of advertisement video into visual image information and auditory audio information, and then uses an image analysis module to carry out editing point identification operation on the visual image information to form a normalized video salient frame difference sequence; the audio analysis module is used in parallel to carry out time-phased pitch editing operation on auditory audio information to form normalized significant pitch difference sequences, the two sequences are input into the comprehensive analysis module, a two-dimensional curved surface is obtained through a comprehensive analysis function, the most ideal frame difference threshold value and the most ideal pitch difference threshold value are set based on the feedback of the two-dimensional curved surface, and the optimal frame difference threshold value and the optimal pitch difference threshold value are obtained. Therefore, the maximum rhythm sensation of mutual cooperation of the video image and the audio image is obtained. In the system, the audio information and the video information are processed separately and then go together, the best rhythm of the video file is effectively obtained, manual intervention is avoided to the maximum extent, and the rhythm of the advertisement video is controlled in an automatic and intelligent mode.

Description

Technical Field

[0001] The present invention relates to the field of advertising video analysis, and in particular to an advertising video rhythm analysis system. Background Art

[0002] In modern advertising communication, video advertising is an important communication medium, and its sense of rhythm is crucial to attracting audience attention and improving the effectiveness of information communication. Traditional advertising analysis often focuses on the content, picture quality and creativity of the video, but ignores the importance of rhythm for advertising videos.

[0003] However, the rhythm of advertising videos is not just a technical detail, but also a core element of effective communication. Through reasonable rhythm arrangement, advertisements can better attract audiences, improve information transmission efficiency, enhance brand image, and ultimately promote sales and increase brand loyalty. Therefore, when designing videos, advertising creators must pay attention to the grasp and application of rhythm.

[0004] In order to improve the effectiveness of advertising, there is an urgent need for an efficient analysis method that can objectively analyze and evaluate the rhythm of the video. Summary of the invention

[0005] The present invention provides a method to effectively solve the above-mentioned problems in the prior art.

[0006] Specifically, the present invention provides an advertising video rhythm analysis system for analyzing an advertising video containing a sequence of N frames of video images, where N is an integer and N>3. The system comprises an audiovisual modality separation module, an image analysis module, an auditory analysis module, and a comprehensive analysis module, wherein the audiovisual modality separation module separates the advertising video into visual image information and auditory audio information, the visual image information is input into the image analysis module, and the image analysis module forms a normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' based on the N frame video image sequence according to a set frame difference threshold VP. M The auditory audio information is input into the auditory analysis module, in which the duration of the auditory audio information is evenly divided into N time periods, and a normalized significant pitch difference sequence Δh'1, Δh'2, ..., Δh' is formed based on the N time periods according to the set pitch difference threshold VH. m , normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' M and the normalized audio significant pitch difference sequence Δh'1, Δh'2, ..., Δh' mAll are input into the comprehensive analysis module, and the comprehensive analysis function is f(Δp, Δh, VP, VH), where Δp represents the normalized video significant frame difference sequence, Δh represents the normalized audio significant pitch difference sequence, VP represents the frame difference threshold as an independent variable, VH represents the pitch difference threshold as an independent variable. The extreme points VP0 and VH0 of the comprehensive analysis function f(Δp, Δh, VP, VH) are taken. The frame difference threshold VP0 is input into the image analysis module to obtain the optimal normalized video significant frame difference sequence Δp0. The taken pitch difference threshold VH0 is input into the auditory analysis module, and the optimal normalized audio significant pitch difference sequence Δh0 can be obtained.

[0007] Preferably, the image analysis module calculates the normalized significant frame difference sequences Δp’1, Δp’2, …, Δp’ through the following steps M : The image analysis module sets the image information of any k-th frame in the video image sequence from the 1st frame to the N-th frame as P k , where k is an integer and 1 ≤ k ≤ N - 1, and calculates the frame difference ΔP between the image information of the (k + 1)-th frame and the k-th frame k :

[0008] ΔP k = ||P k+1 - P k ||

[0009] As the value of k ranges from 1 to N - 1, N - 1 frame differences ΔP1, ΔP2, …, ΔP are formed N-1 , and all the frame differences greater than the frame difference threshold VP among the N - 1 frame differences are extracted to form M significant frame differences, that is, ΔP’1, ΔP’2, …, ΔP’ M , where M < N - 1. Each item in the M significant frame differences is divided by the frame difference threshold VP to form the normalized significant frame difference sequences Δp’1, Δp’2, …, Δp’ M .

[0010] Preferably, in the auditory analysis module, the duration of the auditory audio information is evenly divided into N time periods, thereby forming a pitch sequence containing N pitches. The pitch of any j-th time period from the 1st time period to the N-th time period is H j , where j is an integer and 1 ≤ j ≤ N - 1, and calculates the pitch difference ΔH between the pitch H j of the (j + 1)-th time period and the pitch H j+1 of the j-th time period j : ΔH j = ||H j+1 - H j ||. As the value of j ranges from 1 to N - 1, N - 1 pitch differences ΔH1, ΔH2, …, ΔH are formed N-1 , and the N - 1 pitch differences ΔH1, ΔH2, …, ΔH are extractedN-1 All pitch differences greater than the pitch difference threshold VH in [the relevant content] form m significant pitch differences ΔH’1, ΔH’2, …, ΔH’ m , where m < N - 1. Divide each of the m significant pitch differences ΔH’1, ΔH’2, …, ΔH’ m by the pitch difference threshold VH to form a normalized significant pitch difference sequence Δh’1, Δh’2, …, Δh’ m .

[0011] Preferably, the comprehensive analysis function f(Δp, Δh, VP, VH) forms a two - dimensional surface in the x - y - z three - dimensional space. In the x - y - z three - dimensional space, the x - axis represents the value of the frame difference threshold VP, the y - axis represents the value of the pitch difference threshold VH, and the z - axis represents the function value of the comprehensive analysis function.

[0012] Preferably, the comprehensive analysis function f(Δp, Δh, VP, VH) is defined as:

[0013]

[0014] In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp’ i ; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh’ n . Additionally, the s function in the summation formula above is defined as:

[0015] s(Δp′ i ,VP)=(Δp′ i - VP)×sigmoid(Δp′ i - VP);

[0016] s(Δh′ n ,VH)=(Δh' n - VH)×sigmoid(Δh' n - VH).

[0017] Preferably, the comprehensive analysis function f(Δp, Δh, VP, VH) is defined as:

[0018]

[0019] In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp’ i; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh' n .

[0020] Preferably, the comprehensive analysis function f(Δp, Δh, VP, VH) is defined as:

[0021]

[0022] In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp' i ; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh' n .

[0023] In summary, the present invention provides an advertising video rhythm analysis system, which initially splits a fixed advertising video into visual image information and auditory audio information, then uses an image analysis module to perform a clipping point recognition operation on the visual image information, thereby forming a normalized video significant frame difference sequence, and uses an audio analysis module to perform a time period pitch clipping operation on the auditory audio information in parallel, thereby forming a normalized significant pitch difference sequence, and both sequences are input into a comprehensive analysis module, and a two-dimensional surface is obtained through a comprehensive analysis function, and the most ideal frame difference threshold and pitch difference threshold are set based on the feedback of the two-dimensional surface, thereby obtaining the maximum rhythm of the video image and the audio image. In the system provided by the present invention, the audio information and the video information are processed separately and then processed in parallel, thereby effectively achieving the best rhythm acquisition of the video file, avoiding manual intervention to the greatest extent, and controlling the rhythm of the advertising video in an automated and intelligent form. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be discussed below. Obviously, the technical solutions described in conjunction with the drawings are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments and their drawings can be obtained based on the embodiments shown in these drawings without paying creative work.

[0025] Figure 1 The system operation flow chart of the advertising video rhythm analysis system provided by the present invention is shown.

[0026] Figure 2 A two-dimensional surface example of a function run by a comprehensive analysis module in the advertising video rhythm analysis system provided by the present invention is shown. DETAILED DESCRIPTION

[0027] The technical solutions of various embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments described in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The specific structure and operation of the distributed crawler scheduling system based on server performance monitoring provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0029] Figure 1 The system operation flow chart of the advertising video rhythm analysis system provided by the present invention is shown.

[0030] like Figure 1 As shown in FIG. 1 , a video advertisement is first introduced into the audiovisual modality separation module for audiovisual modality separation, thereby separating the video advertisement into visual image information and auditory audio information. In the subsequent system modules, there are both separate analyses of the visual image information and the auditory audio information, and comprehensive adjustments of the two types of information.

[0031] The visual image information is input into the image analysis module for clip point recognition. Clip point recognition can be implemented using an inter-frame difference method. Specifically, the inter-frame recognition method is used to perform a difference operation on two adjacent frames in a video image sequence formed by the visual graphic information.

[0032] For example, a video image sequence can be divided into N frames (N>3, in practice, N is often a large integer value, such as N≥10000, or even N≥100000 is normal), and the image information of any kth frame in the video image sequence from the 1st frame to the Nth frame is summarized as P k , where k is an integer and 1≤k≤N-1.

[0033] Thus, the frame difference ΔP between the k+1th frame image information and the kth frame image information can be calculated. k :

[0034] ΔP k =||P k+1 -P k ||

[0035] As the value of k increases from 1 to N-1, N-1 frame differences ΔP1, ΔP2, ..., ΔP N-1 . Set the frame difference threshold VP>0.

[0036] Extract N-1 frame differences ΔP1, ΔP2, ..., ΔPN-1 All frame differences greater than the frame difference threshold VP form M significant frame differences, namely, ΔP'1, ΔP'2, ..., ΔP' M , where M <N-1。

[0037] This process is intended to screen the frame differences, so as to select significant frame differences with larger values ​​from N-1 frame differences based on the frame difference threshold.

[0038] The M significant frame differences ΔP'1, ΔP'2, ..., ΔP' M Each item in is divided by the frame difference threshold VP to form a normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' M , where Δp'1=ΔP'1 / VP, Δp'2=ΔP'2 / VP, and so on, until Δp' M =ΔP' M / VP.

[0039] Thus, the image analysis module outputs a normalized video significant frame difference sequence Δp'1, Δp'2, ..., Δp' M .

[0040] In parallel, the auditory audio information is input into the auditory analysis module for rhythm point recognition. The auditory audio information forms a pitch sequence, in other words, the auditory audio information forms different degrees of pitch at each time point, and these pitches are arranged into a pitch sequence according to time sequence.

[0041] The duration of the auditory audio information can also be evenly divided into N time periods, each of which has a pitch, thereby forming a pitch sequence containing N pitches. The pitch of any j-th time period from the 1st time period to the Nth time period is H j , where j is an integer and 1≤j≤N-1.

[0042] The pitch H of the j+1th period is calculated from this j and H in period j j+1 The pitch difference ΔH j :

[0043] ΔH j =||H j+1 -H j ||

[0044] As the value of j increases from 1 to N-1, N-1 pitch differences ΔH1, ΔH2, ..., ΔH are formed. N-1 . Set the pitch difference threshold VH>0.

[0045] Extract N-1 pitch differences ΔH1, ΔH2, ..., ΔH N-1All pitch differences greater than the pitch difference threshold VH form m significant pitch differences, i.e., ΔH'1, ΔH'2, ..., ΔH' m , where m <N-1。

[0046] This process aims to screen the pitch differences, so as to select significant pitch differences with larger values ​​from N-1 pitch differences based on the pitch difference threshold.

[0047] The m significant pitch differences ΔH'1, ΔH'2, ..., ΔH' m Each item in is divided by the pitch difference threshold VH to form a normalized significant pitch difference sequence Δh'1, Δh'2, ..., Δh' m , where Δp'1=ΔH'1 / VH, Δh'2=ΔH'2 / HP, and so on, until Δh' m =ΔH' m / VH.

[0048] Thus, the audio analysis module outputs a normalized audio significant pitch difference sequence Δh'1, Δh'2, ..., Δh' m .

[0049] Next, normalize the video significant frame difference sequence Δp'1, Δp'2, ..., Δp' M and the normalized audio significant pitch difference sequence Δh'1, Δh'2, ..., Δh' m All are input into the comprehensive analysis module.

[0050] A comprehensive analysis function f(Δp, Δh, VP, VH) is set in the comprehensive analysis module, where Δp represents the normalized video significant frame difference sequence, Δh represents the normalized audio significant pitch difference sequence, and VP and VH have been mentioned above, VP represents the frame difference threshold, and VH represents the pitch difference threshold.

[0051] As mentioned above, when the original input advertisement video remains unchanged, the normalized video significant frame difference sequence will change with the different values ​​of the frame difference threshold VP. On the other hand, the normalized audio significant pitch difference sequence will also change with the different values ​​of the pitch difference threshold VH. Therefore, in the comprehensive analysis function f(Δp, Δh, VP, VH), the frame difference threshold VP and the pitch difference threshold VH constitute the two independent variables of the function.

[0052] Therefore, as the values ​​of these two independent variables are different, the comprehensive analysis function f(Δp, Δh, VP, VH) forms a two-dimensional surface in the xyz three-dimensional space, where the x-axis represents the value of the frame difference threshold VP, the y-axis represents the value of the pitch difference threshold VH, and the z-axis represents the function value of the comprehensive analysis function.

[0053] In this two-dimensional surface, there are extreme points VP0 and VH0 of the function values ​​of the frame difference threshold VP and the pitch difference threshold VH. The audio sequence and image sequence formed at the extreme points indicate that the image sequence and the pitch sequence have achieved the best fit. At this time, the frame difference threshold VP takes the value of VP0, and the pitch difference threshold VH takes the value of VH0. By inputting the taken value of the frame difference threshold VP0 into the image analysis module, the best normalized video significant frame difference sequence Δp0 can be obtained. By inputting the taken value of the pitch difference threshold VH0 into the auditory analysis module, the best normalized audio significant pitch difference sequence Δh0 can be obtained.

[0054] By observing the best normalized video significant frame difference sequence Δp0 and the best normalized audio significant pitch difference sequence Δh0, the user can obtain the best rhythm of the video image in the audio environment, and can also obtain the best rhythm of the auditory audio in the current video image environment.

[0055] The comprehensive analysis function can be limited according to specific circumstances. For example, the comprehensive analysis function f(Δp, Δh, VP, VH) can be defined as follows:

[0056]

[0057] In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp' i ; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh' n .

[0058] In the above formula, each term in the previous and next summation is an s function, where:

[0059] s(Δp′ i ,VP)=(Δp′ i -VP)×sigmoid(Δp′ i -VP);

[0060] s(Δh' n ,VH)=(Δh' n -VH)×sigmoid(Δh′ n -VH).

[0061] The above function forms a two-dimensional surface in the xyz three-dimensional space about VP and VH, where the X-axis represents the value of the frame difference threshold VP, the Y-axis represents the value of the pitch difference threshold VH, and the Z-axis represents the function value of the comprehensive analysis function. The shape of the two-dimensional surface is as follows: Figure 2 shown.

[0062] As mentioned above, the comprehensive analysis function is not limited to the above form. For example, it can also be defined as follows:

[0063]

[0064] Compared with the previous formula, this formula adds absolute value signs on both sides of each summation term. The definitions of specific parameters are exactly the same, but the shape of the two-dimensional surface in the three-dimensional space is changed.

[0065] Optionally, the above comprehensive analysis function can also be transformed into the following form:

[0066]

[0067] Compared with the previous formula, this formula is to take the square of each summation term. The definition of specific parameters is exactly the same, but the shape of the two-dimensional surface in the three-dimensional space is changed.

[0068] The technical solution of the present invention has been basically introduced here. In summary, the present invention provides an advertising video rhythm analysis system, which initially splits a fixed advertising video into visual image information and auditory audio information, then uses the image analysis module to perform clipping point recognition operations on the visual image information, thereby forming a normalized video significant frame difference sequence, and uses the audio analysis module to perform time period pitch clipping operations on the auditory audio information in parallel, thereby forming a normalized significant pitch difference sequence, and both sequences are input into the comprehensive analysis module, and a two-dimensional surface is obtained through a comprehensive analysis function, and the most ideal frame difference threshold and pitch difference threshold are set based on the feedback of the two-dimensional surface, thereby obtaining the maximum rhythm of the video image and the audio image. In the system provided by the present invention, the audio information and the video information are processed separately and then processed in parallel, thereby effectively achieving the best rhythm acquisition of the video file, avoiding manual intervention to the greatest extent, and controlling the rhythm of the advertising video in an automated and intelligent form.

[0069] The above description is only an exemplary embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An advertising video rhythm analysis system, used for analyzing an advertising video containing a sequence of N video frames, where N is an integer and N>3, characterized in that: The system includes an audio-visual modality separation module, an image analysis module, an auditory analysis module, and a comprehensive analysis module, among which: The audio-visual modality separation module separates the advertisement video into visual image information and auditory audio information. The visual image information is input into the image analysis module, and the image analysis module forms a normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' based on the N-frame video image sequence according to the set frame difference threshold VP. M , The auditory audio information is input into the auditory analysis module, in which the duration of the auditory audio information is evenly divided into N time periods, and a normalized significant pitch difference sequence Δh'1, Δh'2, ..., Δh' is formed based on the N time periods according to the set pitch difference threshold VH. m , Normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' M and the normalized audio significant pitch difference sequence Δh'1, Δh'2, ..., Δh' m All are input into the comprehensive analysis module, the comprehensive analysis function f(Δp, Δh, VP, VH), where Δp represents the normalized video significant frame difference sequence, Δh represents the normalized audio significant pitch difference sequence, VP represents the frame difference threshold as an independent variable, and VH represents the pitch difference threshold as an independent variable. The extreme points VP0 and VH0 of the comprehensive analysis function f(Δp, Δh, VP, VH) are taken. The frame difference threshold VP0 is input into the image analysis module to obtain the best normalized video significant frame difference sequence Δp0, and the pitch difference threshold VH0 is input into the auditory analysis module to obtain the best normalized audio significant pitch difference sequence Δh0.

2. The system according to claim 1, characterized in that The image analysis module calculates the normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' through the following steps: M : The image analysis module sets any k-th frame image information in the video image sequence from the 1st frame to the Nth frame as P k , where k is an integer and 1≤k≤N-1, calculate the frame difference ΔP between the k+1th frame image information and the kth frame image information k : ΔP k =||P k+1 -P k || As the value of k increases from 1 to N-1, N-1 frame differences ΔP1, ΔP2, ..., ΔP N-1 , extract all frame differences greater than the frame difference threshold VP from the N-1 frame differences to form M significant frame differences, i.e., ΔP'1, ΔP'2, ..., ΔP' M , where M <N-1, Each of the M significant frame differences is divided by the frame difference threshold VP to form a normalized significant frame difference sequence Δp'1, Δp'2, ..., Δp' M .

3. The system according to claim 1, characterized in that In the auditory analysis module, the duration of the auditory audio information is evenly divided into N time periods, thereby forming a pitch sequence containing N pitches, and the pitch of any j-th time period from the 1st time period to the Nth time period is H j , where j is an integer and 1≤j≤N-1, calculate the pitch H of the j+1th time period j and H in period j j+1 The pitch difference ΔH j :ΔH j =||H j+1 -H j ||, as the value of j changes from 1 to N-1, N-1 pitch differences ΔH1, ΔH2, ..., ΔH are formed. N-1 , Extract N-1 pitch differences ΔH1, ΔH2, ..., ΔH N-1 All pitch differences greater than the pitch difference threshold VH form m significant pitch differences ΔH'1, ΔH'2, ..., ΔH' m , where m <N-1, The m significant pitch differences ΔH'1, ΔH'2, ..., ΔH' m Each item in is divided by the pitch difference threshold VH to form a normalized significant pitch difference sequence Δh'1, Δh'2, ..., Δh' m .

4. The system according to claim 1, characterized in that The comprehensive analysis function f(Δp, Δh, VP, VH) forms a two-dimensional surface in the xyz three-dimensional space, where the x-axis represents the value of the frame difference threshold VP, the y-axis represents the value of the pitch difference threshold VH, and the z-axis represents the function value of the comprehensive analysis function.

5. The system according to claim 4, characterized in that The comprehensive analysis function f(Δp, Δh, VP, VH) is defined as: In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp' i ; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh' n , In addition, the s function in the summation of the above formula is defined as: s(Δp’ i ,VP)=(Δp’ i -VP)×sigmoid(Δp’ i -VP); s(Δh' n ,VH)=(Δh' n -VH)×sigmoid(Δh' n -VH)。 6. The system according to claim 4, characterized in that The comprehensive analysis function f(Δp, Δh, VP, VH) is defined as: In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp' i ; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh' n , In addition, the s function in the summation of the above formula is defined as: s(Δp’ i ,VP)=(Δp’ i -VP)×sigmoid(Δp’ i -VP); s(Δh' n ,VH)=(Δh' n -VH)×sigmoid(Δh' n -VH)。 7. The system according to claim 4, characterized in that The comprehensive analysis function f(Δp, Δh, VP, VH) is defined as: In the above formula, i is the number of each item in the normalized video significant frame difference sequence, i ranges from 1 to M, and each item in the normalized video significant frame difference sequence is expressed as Δp' i ; n is the number of each item in the normalized audio significant pitch difference sequence, n ranges from 1 to m, and each item in the normalized audio significant pitch difference sequence is expressed as Δh' n , In addition, the s function in the summation of the above formula is defined as: s(Δp’ i ,VP)=(Δp’ i -VP)×sigmoid(Δp’ i -VP); s(Δh' n ,VH)=(Δh′ n -VH)×sigmoid(Δh' n -VH)。

Images