Marking method, device, equipment and medium for card point label

Abstract

The application discloses a marking method, device and equipment for a card point label and a medium, and belongs to the field of audio processing. The method comprises the following steps: firstly, determining a beat point and a rhythm point in music, obtaining a first beat point set, then, cutting the music into a first music segment and a second music segment, determining a second beat point set by setting a time window on the rhythm point and combining the rhythm change amplitudes of different music segments on the rhythm point, finally, determining a third beat point set by combining the time intervals between the beat points on different music segments, and marking the card point label of the music on the third beat point set. The above method ensures that the determined card point label not only meets the hearing characteristics of the human ear, but also meets the rhythm characteristics of the music, and the screened card points are all located at the heavy rhythm points of the music.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio processing, and in particular to a marker method, device and equipment for marker of a card point label and a medium. BACKGROUND

[0002] The basic principle of video splicing is to splice multiple videos according to preselected music, that is, to splice multiple videos at special time points of music.

[0003] In related technologies, all beat points are extracted based on the rhythm of music, and all beat points are used as special time points of the music.

[0004] Since the related technologies do not screen the beat points, all beat points are directly used as time points for video splicing. However, invalid points often exist in the beat points, such as beat points positioned at a mute point of music, which are not suitable for video splicing. SUMMARY

[0005] The present application provides a marker method, device and equipment for marker of a card point label and a medium, which can screen beat points of heavy rhythm. The technical solution is as follows:

[0006] According to one aspect of the present application, a marker method for marker of a card point label is provided, which comprises:

[0007] determining beat points and rhythm points in music, the beat points being audio signal points of music positioned based on a rhythm rule, and the rhythm points being audio signal points of music positioned based on a volume peak;

[0008] determining target beat points of the music based on the beat points and the rhythm points, the target beat points being a subset of the beat points;

[0009] marking a card point label of the music at the target beat points, the card point label being used to identify audio signal points of the music for multimedia splicing.

[0010] According to one aspect of the present application, a marker device for marker of a card point label is provided, which comprises:

[0011] a determination module configured to determine beat points and rhythm points in music, the beat points being audio signal points of music positioned based on a rhythm rule, and the rhythm points being audio signal points of music positioned based on a volume peak;

[0012] a screening module configured to determine target beat points of the music based on the beat points and the rhythm points, the target beat points being a subset of the beat points;

[0013] a marking module configured to mark a card point label of the music at the target beat points, the card point label being used to identify audio signal points of the music for multimedia splicing.

[0014] According to an aspect of the present application, a computer device is provided, comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the marking method of the card point label as described above.

[0015] According to another aspect of the present application, a computer readable storage medium is provided, the storage medium storing a computer program, the computer program being loaded and executed by a processor to implement the marking method of the card point label as described above.

[0016] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the marking method of the card point label as described above.

[0017] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0018] By extracting the audio signal point where the volume peak of the music is located, and setting a screening condition on the audio signal point, the screening of the beat points in the music is realized, so that the screened beat points can all be used for video splicing. The above method ensures that the screened beat points not only meet the auditory characteristics of the human ear, but also meet the rhythm characteristics of the music, and the screened beat points are all at the heavy rhythm points of the music. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is a schematic diagram of a target beat point generation system of music provided by an exemplary embodiment of the present application;

[0021] Figure 2 is a flowchart of a marking method of a card point label provided by an exemplary embodiment of the present application;

[0022] Figure 3 is a schematic diagram of an audio signal provided by an exemplary embodiment of the present application;

[0023] Figure 4is a schematic diagram of an audio signal provided by another example embodiment of the present application;

[0024] Figure 5 is a schematic diagram of setting a first time window at a beat point provided by an example embodiment of the present application;

[0025] Figure 6 is a flow chart of a marking method of a beat point label provided by another example embodiment of the present application;

[0026] Figure 7 is a flow chart of a marking method of a beat point label provided by another example embodiment of the present application;

[0027] Figure 8 is a structural block diagram of a marking device of a beat point label provided by an example embodiment of the present application;

[0028] Figure 9 shows a structural block diagram of a computer device provided by an example embodiment of the present application. DETAILED DESCRIPTION

[0029] For the purpose, technical solutions and advantages of the present application to be clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0030] First, the terms involved in the embodiments of the present application are briefly introduced:

[0031] Beat point: beat is an organizational form in a piece of music indicating a fixed unit of time value and a law of strength. Also known as tempo. Beat is a unit for measuring rhythm. In music, a series of tempos with certain strength appear repeatedly at every certain time. For example, 2 / 4, 4 / 4, 3 / 4 tempo, etc. The music of 2 / 4 tempo indicates that the current measure is composed of one strong beat and one weak beat; the music of 4 / 4 tempo indicates that the current measure is composed of a strong beat, a weak beat, a secondary strong beat and a weak beat; the music of 3 / 4 tempo indicates that the current measure is composed of one strong beat and two weak beats. As can be seen, each measure in music of various tempos has a clear strong beat, and the strong beat is always the first beat of the measure. Therefore, as long as the audio signal point corresponding to the strong beat note is detected, the audio signal point can be taken as a beat point.

[0032] That is, the beat point is an audio signal point positioned by music based on the beat law.

[0033] Rhythm point: refers to a wave peak point of an audio signal, i.e. a wave peak of an audio signal waveform. That is, a volume peak value in a cycle of an audio signal waveform.

[0034] Beat point label: refers to a point in an audio signal of music used for multimedia splicing, in an embodiment, the point in the audio signal of music used for multimedia splicing is determined by the rhythm and beat of the music obtained through signal analysis processing. In an embodiment, the point in the audio signal of music used for multimedia splicing is determined by manually analyzing the beat of the music.

[0035] STFT (Short-Time Fourier Transform): STFT is a commonly used transform method in signal processing. The specific transform process is to first frame the signal, then window the frame, then perform Fourier transform on each frame, and finally stack the results of each frame. In an embodiment, the spectrogram of a sound signal can be obtained from the sound signal by STFT.

[0036] Mel (Mel Bank Features): The frequency interval of the spectrogram obtained by STFT is often too large, in order to obtain a suitable frequency interval, the spectrogram is often transformed into a Mel spectrum by a Mel scale filter bank.

[0037] In fact, the frequency range that the human ear can hear is 20-20000 Hz, but the human ear does not perceive the scale unit of hertz in a linear relationship. For example, if the frequency of a sound is 1000 Hz, the human ear will not perceive that the frequency has doubled if the frequency is increased to 2000 Hz. Therefore, in order to realize the linear perception of the human ear to the frequency, the sound spectrum is converted into a Mel spectrum, at which time the scale of the Mel spectrum is linearly related to the frequency perception of the human ear.

[0038] Time domain feature zero-crossing rate: refers to the number of sign changes of a signal per unit time, and a sign change refers to a change from positive to negative.

[0039] K-means (a clustering algorithm): a clustering algorithm, which is a representative of a typical prototype-based objective function clustering method, which uses the distance of data points to prototypes as the optimization objective function, and uses the function extreme value method to obtain the adjustment rule of iterative operation.

[0040] The K-means algorithm process is as follows:

[0041] (1) Select k initial center points, for example, c[0]=data[0], …, c[k-1]=data[k-1];

[0042] (2) For data[0]…data[n], compare with c[0]…c[k-1] respectively, and assume that the current data has the least difference with c[i], then mark the current data as the ith family;

[0043] (3) For all data marked as the ith group, recalculate c[i], c[i] = (sum of all data[j] marked as the ith group) / (number of data marked as the ith group);

[0044] (4) Repeat (2) (3) until all c[i] values change less than a given threshold.

[0045] Based on the above, the detailed introduction of the nouns involved in the present application has been completed, and the implementation environment of the embodiments of the present application will be introduced next.

[0046] Figure 1 is a schematic diagram of a music target beat point generation system provided by an exemplary embodiment of the present application, Figure 1 The music target beat point generation system 100 is shown in the figure, which includes a first-level beat point generation system 101, a music segment cutting system 102, a second-level beat point generation system 103, and a third-level beat point generation system 104.

[0047] The first-level beat point generation system 101 is used to generate first-level beat points. In an embodiment, the terminal determines the beat points and rhythm points in the music, and outputs the beat points obtained at this time as a first-level beat point set.

[0048] The music cutting system 102 is used to cut the input music into a first music segment and a second music segment. Optionally, the frequency mean of the first music segment is greater than the frequency mean of the second music segment.

[0049] In an embodiment, first, the music cutting system 102 extracts the audio basic features based on the short-time Fourier transform (STFT), the audio basic features based on Mel, and the time-domain feature zero-crossing rate of the music; then, the music cutting system 102 adopts a clustering algorithm to perform two-class clustering on the audio basic features based on STFT, the audio basic features based on Mel, and the time-domain feature zero-crossing rate, to obtain a feature label sequence of the music, the feature label sequence containing frequency information of the music; next, the music cutting system 102 calculates a boundary line of the feature label sequence; finally, based on the boundary line, the music cutting system 102 cuts the music into the first music segment and the second music segment.

[0050] The second-level beat point generation system 103 outputs a second-level beat point set.

[0051] In one embodiment, first, the secondary beat point generation system 103 sets a first time window at the first rhythm point, and then, in the case that the first rhythm point is in the first music segment, the rhythm change amplitude is not less than a second amplitude threshold, and there is the first beat point in the first time window, the secondary beat point generation system 103 outputs the first beat point; in the case that the first rhythm point is in the second music segment, the rhythm change amplitude is not less than a third amplitude threshold, and there is the first beat point in the first time window, the secondary beat point generation system 103 outputs the first beat point. The secondary beat point set is a set of beat points in the primary beat point set that are output by the secondary beat point generation system 103.

[0052] The first rhythm point is any one of the rhythm points, and the first time window is a time window set with the first rhythm point as the center. In one embodiment, first, the secondary beat point generation system 103 calculates the energy difference ratio between the waveform peak where the first rhythm point is located and the waveform valley immediately before the waveform peak, and then the secondary beat point generation system 103 takes the energy difference ratio as the rhythm change amplitude.

[0053] The tertiary beat point generation system 104 outputs a tertiary beat point set. In one embodiment, the above method is applied to n video splicing modes, where n is a positive integer.

[0054] In one embodiment, when the secondary beat point set is input to the tertiary beat point generation system 104, in response to the time distance between the first beat point and the second beat point on the first music segment being greater than a first time length, the tertiary beat point generation system 104 determines the first beat point as a tertiary beat point; or, in response to the time distance between the first beat point and the third beat point on the second music segment being greater than a second time length, the tertiary beat point generation system 104 determines the first beat point as a tertiary beat point. The tertiary beat point set is a set of beat points in the secondary beat point set that are output by the tertiary beat point generation system 104.

[0055] The music target beat point generation system 100 can be applied to a terminal or a server, or both.

[0056] Those skilled in the art can know that the number of terminals and servers can be more or less. For example, the terminal can be only one, or the terminal can be dozens or hundreds, or more. The server can be only one, or the server can be dozens or hundreds, or more. The number of terminals and the type of equipment, the number of servers are not limited in the embodiments of the present application.

[0057] The following embodiments take the music target beat point generation system 100 applied to a terminal as an example for explanation and description.

[0058] To screen out the beat points of heavy rhythm, Figure 2 is a flowchart of a marking method of a beat point label provided by an exemplary embodiment of the present application, to Figure 2 The method shown in Figure 1 The marking system of the beat point label shown in Figure 2 The method shown in

[0059] Step 220, determining the beat points and rhythm points in the music;

[0060] The beat point is an audio signal point of the music positioned based on the beat rule, and the rhythm point is an audio signal point of the music positioned based on the volume peak.

[0061] In an embodiment, each measure of music of various rhythms has a clear heavy beat, and the heavy beat is the first beat of the measure. Therefore, as long as the audio signal point corresponding to the heavy beat note is detected, the audio signal point can be taken as the beat point.

[0062] In an embodiment, the terminal extracts the beat points of the music by using librosa (an open source algorithm).

[0063] In an embodiment, the rhythm point refers to the peak point of the audio signal, that is, a peak of the audio signal waveform. That is, the volume peak in a cycle of the audio signal waveform.

[0064] Schematically, Figure 3 An exemplary diagram of an audio signal provided by an exemplary embodiment of the present application is shown; wherein the rhythm point 301 is a rhythm point of the audio signal. The rhythm point 301 is located at a peak of the audio signal.

[0065] In an embodiment, the terminal first sets an energy window on the Mel spectrum of the music; then, the terminal determines the audio signal point with sudden change in energy growth rate based on the energy window as the rhythm point;

[0066] Schematically, Figure 4 An exemplary diagram of positioning the rhythm point based on the energy query box is shown, wherein, Figure 4 The upper half of the middle part is the sound spectrum of the music, Figure 4 The lower half of the middle part is the Mel spectrum of the music, Figure 4 The two rectangular windows adjacent to the lower half are energy windows, and the right window in each two adjacent rectangular windows in the lower half represents the energy obtained by querying at the rhythm point of a frame of audio signal, and the left window represents the energy obtained by querying at the previous trough of the rhythm point of a frame of audio signal. The terminal takes the current audio signal point as the rhythm point by querying the energy growth rate of the current audio signal point to have sudden change.

[0067] Step 240, determining target beat points of the music based on the beat points and the rhythm points;

[0068] The target beat points are a subset of the beat points.

[0069] In one embodiment, the time interval between the beat points is much larger than the time interval between the rhythm points, and the target beat points are determined by setting a time window on the rhythm points. Optionally, the time interval between the beat points is 0.5s, and the time interval between the rhythm points is 120ms.

[0070] In one embodiment, the rhythm points are determined by the rhythm change range of the rhythm points.

[0071] In one embodiment, the target beat points are determined by the time interval between the beat points.

[0072] Step 260, marking the beat point tags of the music at the target beat points.

[0073] The beat point tags are used to identify the audio signal points of the music for multimedia splicing. In one embodiment, based on the beat point tags, the terminal can splice at least one form of multimedia. Optionally, based on the beat point tags, the terminal can splice at least two videos; optionally, based on the beat point tags, the terminal can splice at least two audios; in this application, the beat point tags are exemplified by video splicing.

[0074] In one embodiment, the terminal marks the beat point tags of the music at the target beat points obtained based on step 240. Based on the obtained beat point tags, the user can splice the videos.

[0075] In summary, by extracting the audio signal points where the volume peaks of the music are located, and setting a screening condition on the audio signal points, the beat points of the music are screened, so that the screened beat points can be used for video splicing. The above method ensures that the screened beat points not only meet the auditory characteristics of the human ear, but also meet the rhythm characteristics of the music, and the screened beat points are all at the heavy rhythm points of the music.

[0076] To set the above screening condition, based on Figure 2 In the optional embodiment shown in FIG. 8, step 240 further includes the following steps:

[0077] Step 241, setting a first time window at a first rhythm point;

[0078] The first rhythm point is any one of the rhythm points, and the first time window is a time window set with the first rhythm point as the center.

[0079] Illustratively, Figure 5A schematic diagram showing the positional relationship between the first tempo point and the first beat point is shown, wherein the first time window 503 is a time window centered on the first tempo point 501, and optionally, the width of the time window is adjustable; optionally, the width of the time window is adjusted according to the input music.

[0080] Optionally, the width of the first time window is 30 ms; optionally, the width of the first time window is 50 ms.

[0081] Step 242, determining the first beat point as the target beat point of the music.

[0082] Optionally, the first beat point is a beat point existing in the first time window.

[0083] In combination with reference Figure 5 , if the first beat point 502 exists in the first time window 503, the first beat point 502 is determined as the target beat point of the music.

[0084] In one embodiment, in the case that the first beat point exists in the first time window, the following steps are further included:

[0085] S1: determining the rhythm change amplitude of the music at the first tempo point.

[0086] The rhythm change amplitude refers to the energy change amplitude of the music, and the rhythm change amplitude of the first tempo point refers to the amplitude of the energy change of the music at the first tempo point. The greater the energy change amplitude, i.e. the greater the rhythm change amplitude, the more suitable the current tempo point is for the shot transition in video splicing.

[0087] In one embodiment, the rhythm change amplitude is represented by an energy difference ratio. Optionally, the terminal calculates the energy difference ratio between the waveform peak at which the first tempo point is located and the waveform valley next to it; and the terminal takes the energy difference ratio as the rhythm change amplitude.

[0088] In combination with reference Figure 4 , Figure 4 The two rectangular windows adjacent to the lower half part are energy windows, the right window in every two adjacent rectangular windows represents the energy E1 obtained by querying at the tempo point of a frame of audio signal, and the left window represents the energy E2 obtained by querying at the valley next to the tempo point of a frame of audio signal. The energy difference ratio P is calculated as (E1-E2) / E2, and the terminal takes the energy difference ratio P as the rhythm change amplitude.

[0089] S2: in the case that the rhythm change amplitude is not less than a first amplitude threshold, determining the first beat point as the target beat point of the music.

[0090] Optionally, the first amplitude threshold is a value preset by the staff; optionally, the first amplitude threshold is adjusted according to the input music.

[0091] For example, if the rhythm change amplitude obtained in step S1 is 0.3, the first amplitude threshold is 0.2, and there is a first beat point in the first time window, the terminal determines the first beat point as the target beat point of the music.

[0092] For example, if the rhythm change amplitude obtained in step S1 is 0.3, the first amplitude threshold is 0.35, and there is a first beat point in the first time window, the terminal eliminates the first beat point.

[0093] In summary, by extracting and screening the audio signal points where the volume peaks of the music are located, and setting a time threshold frame at the audio signal points, the beat points in the music are screened, so that the screened beat points can be used for video splicing. The above method ensures that the screened beat points not only meet the auditory characteristics of the human ear, but also meet the rhythm characteristics of the music, and the screened beat points are all at the heavy rhythm points of the music.

[0094] To generate target beat points of different rhythm segments, Figure 6 A flowchart of a marking method of a card point label of an example embodiment of the present application is shown.

[0095] Step 610: determining beat points and rhythm points in the music;

[0096] The beat point is an audio signal point of the music positioned based on a beat rule, and the rhythm point is an audio signal point of the music positioned based on a volume peak.

[0097] Step 620: cutting the music into a first music segment and a second music segment;

[0098] The frequency mean of the first music segment is greater than the frequency mean of the second music segment.

[0099] In an embodiment, the frequency mean of the first music segment is greater than the frequency mean of the second music segment, i.e., the rhythm of the first music segment is faster than that of the second music segment.

[0100] In an embodiment, based on the audio basic features of the music, the terminal cuts the music into a first music segment and a second music segment.

[0101] In an embodiment, the specific steps of cutting the music into a first music segment and a second music segment include the following steps:

[0102] Step 621: extracting audio basic features obtained based on short-time Fourier transform (STFT), audio basic features obtained based on Mel, and time-domain feature zero-crossing rate of the music;

[0103] In one embodiment, the terminal extracts the STFT spectrogram, the Mel spectrogram and the time-domain feature zero-crossing rate from the audio signal of the music.

[0104] Step 622, using a clustering algorithm to perform two-class clustering on the audio basic features based on STFT, the audio basic features based on Mel and the time-domain feature zero-crossing rate, to obtain a label sequence of the music.

[0105] The label sequence contains the frequency information of the music.

[0106] In one embodiment, the clustering algorithm includes at least one of K-means, mean shift clustering, density-based clustering algorithm, expectation maximization clustering based on Gaussian mixture model, agglomerative hierarchical clustering and graph community detection clustering. In this application, K-means is taken as an example for illustration.

[0107] In one embodiment, first, the terminal extracts the timbre spectrum, the STFT energy spectrum and the STFT amplitude spectrum from the STFT spectrogram of the audio signal; then, the terminal performs high-pass filtering on the STFT energy spectrum, the STFT amplitude spectrum and the Mel spectrogram, and discards the energy in the low-frequency band of the three spectra that cannot show obvious rhythm changes; then, the terminal splices the filtered STFT energy spectrum and the time-domain feature zero-crossing rate to obtain audio basic feature A, and splices the filtered STFT amplitude spectrum and the timbre spectrum to obtain audio basic feature B; then, the terminal performs two-class clustering on the audio basic feature A, the audio basic feature B and the audio basic feature C (obtained by extracting the Mel spectrogram) using K-means, to obtain two feature label sequences (01 sequences).

[0108] Step 623, calculating the demarcation line of the feature label sequence.

[0109] In one embodiment, the terminal first sets a sliding window to remove outliers in the feature label sequence, then calculates the length of each feature label sequence, the distance between two feature label sequences and the multiple of the distance according to the feature label sequence; then, the terminal sets a distance threshold according to the distance and the multiple, and sets a smoothing window on the distance threshold to perform smoothing operation on the label, wherein the smoothed feature label sequence is a complete 01 sequence with few outliers; finally, the terminal determines the demarcation line of the 01 sequence.

[0110] Step 624, cutting the music into a first music segment and a second music segment based on the demarcation line.

[0111] In one embodiment, after the terminal finds the demarcation line of the 01 sequence, the terminal performs slicing operation on the audio, and then determines the first music segment and the second music segment according to the audio position.

[0112] Step 630, in the case that the first rhythm point is in the first music segment and the rhythm change amplitude is not less than the second amplitude threshold, the first rhythm point is determined as the target rhythm point of the music.

[0113] The second amplitude threshold and the third amplitude threshold are set similarly to the first amplitude threshold, which has been described above and will not be repeated here.

[0114] In an embodiment, the marking method of the card point label can be applied to n video splicing modes, where n is a positive integer. Optionally, the n video splicing modes include at least one of video file-based splicing and picture-based splicing.

[0115] In an embodiment, when the marking method of the card point label is applied to the i-th video splicing mode, in response to the time distance between the first rhythm point and the second rhythm point on the first music segment being greater than the first time length, and the first rhythm point being in the first music segment and the rhythm change amplitude being not less than the second amplitude threshold, the first rhythm point is determined as the target rhythm point of the music.

[0116] The first time length is the time length of the shot transition in the first music segment in the i-th splicing mode, and i is a positive integer not greater than n.

[0117] Optionally, the i-th video splicing mode is video file-based splicing, and the terminal is set to have a shot transition time of the first time length in the region of the first music segment. Optionally, the first time length is 2s. If the interval between the first rhythm point and the second rhythm point in the first music segment is greater than the first time length, the first rhythm point is determined as the target rhythm point of the music.

[0118] Optionally, the i-th video splicing mode is picture-based splicing, and the terminal is set to have a shot transition time of the first time length in the region of the first music segment. Optionally, the first time length is 1.5s. If the interval between the first rhythm point and the second rhythm point in the first music segment is greater than the first time length, the first rhythm point is determined as the target rhythm point of the music.

[0119] Step 640, in the case that the first rhythm point is in the second music segment and the rhythm change amplitude is not less than the third amplitude threshold, the first rhythm point is determined as the target rhythm point of the music.

[0120] In an embodiment, the marking method of the card point label can be applied to n video splicing modes, where n is a positive integer. Optionally, the n video splicing modes include at least one of video file-based splicing and picture-based splicing.

[0121] In one embodiment, when the marking method of the card point label is applied to the ith video splicing mode, in response to the time distance between the first beat point and the third beat point on the second music segment being greater than the second time length, and the first rhythm point being in the second music segment, and the rhythm change amplitude being not less than the third amplitude threshold, the first beat point is determined as the target beat point of the music.

[0122] The second time length is the time length of the shot transition in the second music segment in the ith splicing mode, and i is a positive integer not greater than n.

[0123] Optionally, the ith splicing mode is video file-based splicing, and the terminal is set to have a shot transition time of the second time length in the region of the second music segment. Optionally, the second time length is 1 s. If the interval between the first beat point and the third beat point in the second music segment is greater than the second time length, the first beat point is determined as the target beat point of the music.

[0124] Optionally, the ith splicing mode is picture-based splicing, and the terminal is set to have a shot transition time of the second time length in the region of the second music segment. Optionally, the second time length is 0.4 s. If the interval between the first beat point and the third beat point in the second music segment is greater than the second time length, the first beat point is determined as the target beat point of the music.

[0125] It is worth noting that the steps 630 and 640 are the execution steps of the terminal when the first rhythm point is in different music segments. When the first rhythm point is in the first music segment, the terminal executes the step 630, and then the terminal executes the step 650. When the first rhythm point is in the second music segment, the terminal executes the step 640, and then the terminal executes the step 650.

[0126] The step 650 marks the card point label of the music at the target beat point.

[0127] The card point label is used to identify the audio signal point of the music for multimedia splicing. In one embodiment, based on the card point label, the terminal can implement splicing of at least one form of file. Optionally, based on the card point label, the terminal can implement splicing of at least two videos; optionally, based on the card point label, the terminal can implement splicing of at least two audios; in this application, the card point label is exemplified for video splicing.

[0128] In one embodiment, the terminal outputs the target beat point obtained based on the step 630 or the step 640 as the card point of the music. Based on the obtained card point, the user can splice the video.

[0129] In summary, the terminal first cuts the music into a first music segment and a second music segment, then sets a rhythm change amplitude threshold on the first music segment and the second music segment, then screens rhythm points with a rhythm change amplitude greater than the amplitude threshold, and further screens beat points based on the screened rhythm points. The above method makes different music segments have their own screening rules, facilitates user selection of points when performing video splicing, and realizes diversified generation of music card points, i.e., different types of music can generate points suitable for video splicing.

[0130] The above method further sets an interval threshold on the first music segment and the second music segment, and when the time distance between beat points on the music segment is greater than the set interval threshold, the terminal screens the beat point as a target beat point. The above method enables the terminal to set a customized music card point label marking method according to different video splicing modes, and determines card point labels suitable for multiple different video splicing modes.

[0131] In one embodiment, Figure 7 A flowchart of a card point label marking method of one exemplary embodiment of the present application is shown.

[0132] Illustratively, the output of the card point label marking method includes a first beat point set 710, an initial second beat point set 720, a first set of fast and slow rhythm segment second beat points 730, a second set of fast and slow rhythm segment second beat points 740, and a fast and slow rhythm segment third beat point set 750.

[0133] In one embodiment, the card point label marking method includes the following steps:

[0134] Q1: The first beat point set 710 is obtained by the terminal first acquiring an audio feature Mel spectrum, and then calculated from the Mel spectrum. The beat point is a beat point that meets the human ear hearing characteristics, and the interval of the point in the time domain has certain regularity. The first beat point set is the basic beat point for determining the second beat point and the third beat point.

[0135] The terminal calculates the rhythm point through the Mel spectrum, wherein the rhythm point is a position that can locate the rhythm conversion drum point in the music.

[0136] The beat points and rhythm points obtained above are particularly dense points, especially in music with strong rhythm, the interval of the rhythm point is only more than one hundred milliseconds, therefore, the rhythm point is not suitable for video splicing.

[0137] Q2: The initial second-level beat point set 720 is determined by the terminal by setting a time threshold window on the rhythm points to the first-level beat point set 710. The terminal sets a time window with a certain time threshold around each rhythm point. If a beat point appears within the window, it not only meets certain human auditory characteristics, but also can be accurately positioned to the rhythm point position. Finally, the terminal obtains the card point position which is relatively sparse and combines the advantages of the two point positions.

[0138] The terminal automates the rhythm detection segmentation function. The scheme calculates multiple audio basic features (Mel spectrum, time domain feature zero-crossing rate, STFT), uses the K-means method to perform two-class clustering on multiple features, then merges the clustering results, removes outliers and merges short-time segments to obtain the fast and slow rhythm dividing lines. Finally, the terminal cuts the entire music into segments according to these dividing lines to obtain fast and slow rhythm segments 760.

[0139] Q3: The first set of fast and slow rhythm segment second-level beat points 730 is obtained by the terminal by acquiring the second-level beat points within the fast and slow rhythm segments 730. Illustratively, the first set of fast and slow rhythm segment second-level beat points 730 includes fast rhythm segment second-level beat point A1, slow rhythm segment second-level beat point set B1, fast rhythm segment second-level beat point A2, slow rhythm segment second-level beat point set B2, fast rhythm segment second-level beat point A3, and slow rhythm segment second-level beat point set B3.

[0140] To determine the second set of fast and slow rhythm segment second-level beat points 740, the terminal assigns a rhythm change amplitude to the rhythm points in each fast and slow rhythm segment by feature energy, and then uses an amplitude threshold to determine the must-hit points in each fast and slow rhythm segment.

[0141] (1) Rhythm change amplitude calculation: The rhythm change amplitude of the second-level beat point is calculated based on the frequency band energy of the Mel spectrum. In one embodiment, the terminal calculates the difference ratio of the feature energy between the wave peak where the rhythm point is located and the previous wave trough of the waveform. The difference ratio is the rhythm change amplitude of the rhythm point. The greater the rhythm change amplitude, the greater the rhythm fluctuation amplitude of the music at the point, which is more suitable for video splicing.

[0142] (2) Amplitude threshold setting: The terminal sets different amplitude thresholds for fast and slow rhythm segments. Based on this, the first amplitude threshold is used to determine the rhythm points for fast rhythm segments, and the second amplitude threshold is used to determine the rhythm points for slow rhythm segments, thereby determining the second set of fast and slow rhythm segment second-level beat points 740.

[0143] Q4: The second set of secondary beat points 740 of the fast and slow rhythm segments is obtained by the terminal from the secondary beat points corresponding to the rhythm points meeting the rhythm change range. Illustratively, the second set of secondary beat points 740 of the fast and slow rhythm segments includes fast rhythm segment secondary beat point A1, fast rhythm segment secondary beat point A2, slow rhythm segment secondary beat point B2, and slow rhythm segment secondary beat point B3.

[0144] To determine the set of tertiary beat points 750 of the fast and slow rhythm segments, the secondary beat points are determined for different splicing modes.

[0145] In an embodiment, the terminal sets two determination modes for two splicing modes, i.e., a video file-based splicing mode and a picture-based splicing mode. For the video file-based splicing mode, the terminal sets the shot transition time in the area of the slow rhythm segment as a first duration, which is optionally 2s, and the shot transition time in the area of the fast rhythm segment as a second duration, which is optionally 1s. If the interval between the secondary beat points in the slow rhythm segment is greater than the first duration, the secondary beat point is determined as a tertiary beat point; if the interval between the secondary beat points in the fast rhythm segment is greater than the second duration, the secondary beat point is determined as a tertiary beat point.

[0146] For the picture-based splicing mode, the terminal sets the shot transition time in the area of the slow rhythm segment as a first duration, which is optionally 1.5s, and the shot transition time in the area of the fast rhythm segment as a second duration, which is optionally 0.4s. If the interval between the secondary beat points in the slow rhythm segment is greater than the first duration, the secondary beat point is determined as a tertiary beat point; if the interval between the secondary beat points in the fast rhythm segment is greater than the second duration, the secondary beat point is determined as a tertiary beat point.

[0147] Q5: The set of tertiary beat points 750 of the fast and slow rhythm segments is obtained by the terminal from the beat points meeting the interval threshold, and the specific process is described in Q4 above. Illustratively, the set of tertiary beat points 750 of the fast and slow rhythm segments includes fast rhythm segment tertiary beat point A1 and fast rhythm segment tertiary beat point A2.

[0148] In summary, the above method extracts and selects the audio signal points where the volume peaks of the music are located, and sets a time threshold frame at the audio signal points, thereby realizing the selection of beat points in the music, so that the selected beat points can be used for video splicing. The above method ensures that the selected beat points not only meet the auditory characteristics of the human ear, but also meet the rhythm characteristics of the music, and the selected beat points are all at the heavy rhythm points of the music.

[0149] The method also enables different music segments to be provided with respective filtering rules, thereby facilitating the user to select a point when performing video splicing, and the method enables diversified generation of music cut points, i.e., different types of music can generate points suitable for video splicing.

[0150] The method also sets an interval threshold on the first music segment and the second music segment, and when a time distance between beat points on the music segment is greater than the set interval threshold, the terminal filters the beat points as target beat points. The method enables the terminal to set a customized marking method of music cut point labels according to different video splicing modes, and the filtered cut point labels are suitable for multiple different video splicing modes.

[0151] In an optional embodiment, Figure 8 A structural block diagram of a cut point label marking device of an example embodiment of the application is shown, which comprises Figure 8 The device comprises:

[0152] The determining module 801 is configured to determine beat points and rhythm points in the music, the beat points being audio signal points in the music positioned based on a beat rule, and the rhythm points being audio signal points in the music positioned based on a volume peak.

[0153] The filtering module 802 is configured to determine target beat points of the music based on the beat points and the rhythm points, the target beat points being a subset of the beat points.

[0154] The marking module 803 is configured to mark cut point labels of the music on the target beat points, the cut point labels being used to identify audio signal points in the music for multimedia splicing.

[0155] In an optional embodiment, the filtering module 802 is further configured to set a first time window on a first rhythm point, the first rhythm point being any one of the rhythm points, and the first time window being a time window set with the first rhythm point as a center.

[0156] In an optional embodiment, the filtering module 802 is further configured to determine the first beat point as a target beat point of the music, the first beat point being a beat point existing in the first time window.

[0157] In an optional embodiment, the filtering module 802 is further configured to determine a rhythm variation amplitude of the music at the first rhythm point.

[0158] In an optional embodiment, the filtering module 802 is further configured to determine the first beat point as a target beat point of the music when the rhythm variation amplitude is not less than a first amplitude threshold and the first beat point exists in the first time window.

[0159] In an optional embodiment, the rhythm variation amplitude is represented by an energy difference ratio.

[0160] In an optional embodiment, the screening module 802 is further configured to calculate a ratio of energy difference between a wave peak where the first beat point is located and a wave valley before the wave peak.

[0161] In an optional embodiment, the determining module 801 is further configured to set an energy window on a Mel of the music.

[0162] In an optional embodiment, the determining module 801 is further configured to determine an audio signal point where the energy growth rate suddenly changes as the beat point based on the energy window.

[0163] In an optional embodiment, the screening module 802 is further configured to cut the music into a first music segment and a second music segment.

[0164] The frequency average of the first music segment is greater than the frequency average of the second music segment.

[0165] In an optional embodiment, the screening module 802 is further configured to extract an audio basis feature based on a short-time Fourier transform (STFT), an audio basis feature based on a Mel, and a zero-crossing rate of a time domain feature.

[0166] In an optional embodiment, the screening module 802 is further configured to perform two-class clustering on the audio basis feature based on the STFT, the audio basis feature based on the Mel, and the zero-crossing rate of the time domain feature by using a clustering algorithm to obtain a label sequence of the music, the label sequence containing frequency information of the music.

[0167] In an optional embodiment, the screening module 802 is further configured to calculate a boundary line of the label sequence.

[0168] In an optional embodiment, the screening module 802 is further configured to cut the music into the first music segment and the second music segment based on the boundary line.

[0169] In an optional embodiment, the screening module 802 is further configured to determine the first beat point as a target beat point of the music when the first beat point is located in the first music segment and the beat change amplitude is not less than a second amplitude threshold.

[0170] In an optional embodiment, the screening module 802 is further configured to determine the first beat point as the target beat point of the music when the first beat point is located in the second music segment and the beat change amplitude is not less than a third amplitude threshold.

[0171] In an optional embodiment, the marking device of the card point label is applied to n splicing modes, n is a positive integer, and i is a positive integer not greater than n.

[0172] In an optional embodiment, the screening module 802 is further configured to, when the device is applied to the ith video splicing mode, determine the first beat point as the target beat point of the music in response to the time distance between the first beat point and the second beat point on the first music segment being greater than the first time length, the first rhythm point being located in the first music segment, and the rhythm change amplitude being not less than the second amplitude threshold.

[0173] The first time length is the time length of the shot transition in the first music segment in the ith splicing mode.

[0174] In an optional embodiment, the screening module 802 is further configured to, when the device is applied to the ith video splicing mode, determine the first beat point as the target beat point of the music in response to the time distance between the first beat point and the third beat point on the second music segment being greater than the second time length, the first rhythm point being located in the second music segment, and the rhythm change amplitude being not less than the third amplitude threshold.

[0175] The second time length is the time length of the shot transition in the second music segment in the ith splicing mode.

[0176] In summary, the device described above determines the beat points in the music by extracting and determining the audio signal points where the volume peaks of the music are located, and setting a time threshold frame at the audio signal points, so that the determined beat points can be used for video splicing. The device described above ensures that the determined beat points not only meet the auditory characteristics of the human ear, but also meet the rhythm characteristics of the music, and the determined beat points are all located at the heavy rhythm points of the music.

[0177] The device described above also allows different music segments to have their own determination rules, which facilitates the selection of points by the user when performing video splicing. In addition, the device described above realizes the diversification of the generation of music card points, i.e., different types of music can generate points suitable for video splicing.

[0178] The device described above also sets an interval threshold on the first music segment and the second music segment, and when the time distance between the beat points on the music segment is greater than the set interval threshold, the terminal determines the beat point as the target beat point. The device described above allows the terminal to set customized marking rules for the music card point labels according to different video splicing modes, and the determined card point labels are suitable for multiple different video splicing modes.

[0179] Figure 9A structural block diagram of a computer device 900 provided by an example embodiment of the present application is shown. The computer device 900 can be a terminal, such as a smart phone, a tablet computer, an MP3 player, an MP4 player, a notebook computer, a desktop computer, a smart television, and the like. The computer device 900 can also be a server.

[0180] Generally, the computer device 900 includes a processor 901 and a memory 902.

[0181] The processor 901 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 901 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 901 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 901 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content to be displayed by a display screen. In some embodiments, the processor 901 can further include an AI (Artificial Intelligence) processor for processing machine learning related computing operations.

[0182] The memory 902 can include one or more computer-readable storage media, which can be non-transitory. The memory 902 can also include a high-speed random access memory, and a non-volatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction for being executed by the processor 901 to implement a card point label marking method provided by a method embodiment of the present application.

[0183] In some embodiments, the computer device 900 may optionally include a peripheral device interface 903 and at least one peripheral device. The processor 901, memory 902, and peripheral device interface 903 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 908.

[0184] Peripheral device interface 903 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 901 and memory 902. In some embodiments, processor 901, memory 902 and peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 901, memory 902 and peripheral device interface 903 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0185] The radio frequency (RF) circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 904 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 904 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0186] The display screen 905 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 is further configured to capture touch signals on or above the surface of the display screen 905. The touch signals can be input to the processor 901 as control signals for processing. In this case, the display screen 905 can also be configured to provide virtual buttons and / or virtual keyboard, also known as soft buttons and / or soft keyboard. In some embodiments, the display screen 905 can be one, disposed on the front panel of the computer device 900; in other embodiments, the display screen 905 can be at least two, respectively disposed on different surfaces of the computer device 900 or in a folding design; in other embodiments, the display screen 905 can be a flexible display screen, disposed on a curved surface or a folding surface of the computer device 900. Even, the display screen 905 can also be disposed in an irregular shape, i.e., a special-shaped screen. The display screen 905 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0187] The camera assembly 906 is configured to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a long-focus camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 906 can further include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0188] The audio circuit 907 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 901 for processing, or input to the radio frequency circuit 904 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the computer device 900. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker can be a conventional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can it convert electrical signals into sound waves that humans can hear, but it can also convert electrical signals into sound waves that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 907 can also include a headphone jack.

[0189] The power supply 908 is used to power the various components in the computer device 900. The power supply 908 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 908 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery that is charged through a wired line, and the wireless charging battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0190] In some embodiments, the computer device 900 further includes one or more sensors 909. The one or more sensors 909 include, but are not limited to, an acceleration sensor 910, a gyroscope sensor 911, a pressure sensor 912, an optical sensor 913, and a proximity sensor 914.

[0191] The acceleration sensor 910 can detect the acceleration in three coordinate axes of the coordinate system established by the computer device 900. For example, the acceleration sensor 910 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 910. The acceleration sensor 910 can also be used for game or user motion data collection.

[0192] The gyroscope sensor 911 can detect the orientation and rotation angle of the computer device 900. The gyroscope sensor 911 can work with the acceleration sensor 910 to collect 3D motion of the user on the computer device 900. The processor 901 can realize the following functions according to the data collected by the gyroscope sensor 911: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0193] A pressure sensor 912 is disposed on the side bezel of the computer device 900 and / or on the lower layer of the display screen 905. When the pressure sensor 912 is disposed on the side bezel of the computer device 900, it can detect the user's grip signal on the computer device 900, and the processor 901 performs left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 912. When the pressure sensor 912 is disposed on the lower layer of the display screen 905, the processor 901 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 905. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0194] An optical sensor 913 is used to collect ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 based on the ambient light intensity collected by the optical sensor 913. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is decreased. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity collected by the optical sensor 913.

[0195] A proximity sensor 914, also known as a distance sensor, is typically located on the front panel of a computer device 900. The proximity sensor 914 is used to detect the distance between the user and the front of the computer device 900. In one embodiment, when the proximity sensor 914 detects that the distance between the user and the front of the computer device 900 is gradually decreasing, the processor 901 controls the display screen 905 to switch from a screen-on state to a screen-off state; when the proximity sensor 914 detects that the distance between the user and the front of the computer device 900 is gradually increasing, the processor 901 controls the display screen 905 to switch from a screen-off state to a screen-on state.

[0196] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the computer device 900, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0197] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the marking method for checkpoint tags provided in the above method embodiments.

[0198] The application provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the marking method of the card point label provided in the method embodiment.

[0199] The above-mentioned sequence numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0200] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0201] The above-mentioned is only the optional embodiment of the application, and does not limit the application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for marking checkpoint labels, characterized in that, The method includes: Determine the beat points and rhythm points in the music, wherein the beat points are audio signal points located in the music based on the beat pattern, and the rhythm points are audio signal points located in the music based on the volume peak. A first time window is set at a first rhythm point, where the first rhythm point is any one of the rhythm points, and the first time window is a time window set with the first rhythm point as the center; the rhythm change amplitude of the music at the first rhythm point is determined; if the rhythm change amplitude is not less than a first amplitude threshold, the first beat point is determined as the target beat point of the music, the time distance between the first beat point and other beat points is greater than an interval threshold, the first beat point is a beat point existing within the first time window, and the target beat point is a subset of the beat points; Mark the target beat point with a beat label, which is used to identify the audio signal point in the music used for multimedia splicing.

2. The method according to claim 1, characterized in that, The amplitude of the rhythm change is represented by the energy difference ratio; Determining the amplitude of the rhythmic change of the music at the first rhythm point includes: Calculate the ratio of the energy difference between the peak of the waveform at the first rhythm point and the previous trough of the waveform.

3. The method according to claim 1 or 2, characterized in that, Determining the rhythm points in the music includes: An energy window is set on the Mel spectrum of the music; The audio signal point where the energy growth rate changes abruptly, as queried based on the energy window, is determined as the rhythm point.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The music is cut into a first music segment and a second music segment, wherein the average frequency of the first music segment is greater than the average frequency of the second music segment; The step of determining the first beat point as the target beat point of the music when the amplitude of the rhythm change is not less than the first amplitude threshold includes: When the first rhythm point is located in the first music segment and the rhythm change amplitude is not less than the second amplitude threshold, the first beat point is determined as the target beat point of the music. When the first rhythm point is located in the second music segment and the rhythm change amplitude is not less than the third amplitude threshold, the first beat point is determined as the target beat point of the music.

5. The method according to claim 4, characterized in that, The step of cutting the music into a first music segment and a second music segment includes: Extract the basic audio features of the music based on Short Time Fourier Transform (STFT), the basic audio features based on Mel, and the zero-crossing rate of the time-domain feature; A clustering algorithm is used to perform binary classification clustering on the basic audio features obtained based on Short Time Fourier Transform (STFT), the basic audio features obtained based on Mel, and the zero-crossing rate of the time-domain features to obtain the tag sequence of the music, which contains the frequency information of the music. The boundary line of the tag sequence is calculated; Based on the dividing line, the music is cut into a first music segment and a second music segment.

6. The method according to claim 4, characterized in that, The method is applied to n video stitching methods, where n is a positive integer; The step of determining the first beat point as the target beat point of the music when the first rhythm point is within the first music segment and the rhythm change amplitude is not less than the second amplitude threshold includes: When the method is applied to the i-th video splicing method, in response to the time distance between the first beat point and the second beat point on the first music segment being greater than the first duration, and the first rhythm point being located in the first music segment, and the rhythm change amplitude being not less than the second amplitude threshold, the first beat point is determined as the target beat point of the music. Wherein, the first duration is the duration of the shot transition on the first music segment in the i-th video splicing method, and i is a positive integer not greater than n.

7. The method according to claim 4, characterized in that, The method is applied to n video stitching methods, where n is a positive integer; The step of determining the first beat point as the target beat point of the music when the first rhythm point is located in the second music segment and the rhythm change amplitude is not less than the third amplitude threshold includes: When the method is applied to the i-th video splicing method, in response to the time distance between the first beat point and the third beat point on the second music segment being greater than the second duration, and the first rhythm point being located in the second music segment, and the rhythm change amplitude being not less than the third amplitude threshold, the first beat point is determined as the target beat point of the music. Wherein, the second duration is the duration of the shot transition on the second music segment in the i-th video splicing method, and i is a positive integer not greater than n.

8. A marking device for a checkpoint label, characterized in that, The device includes: The determination module is used to determine the beat points and rhythm points in the music. The beat points are audio signal points located by the music based on the beat pattern, and the rhythm points are audio signal points located by the music based on the volume peak. A filtering module is used to set a first time window at a first rhythm point, where the first rhythm point is any one of the rhythm points, and the first time window is a time window set with the first rhythm point as the center; determine the rhythm change amplitude of the music at the first rhythm point; and, if the rhythm change amplitude is not less than a first amplitude threshold, determine the first beat point as the target beat point of the music, where the time distance between the first beat point and other beat points is greater than an interval threshold, the first beat point is a beat point existing within the first time window, and the target beat point is a subset of the beat points; A marking module is used to mark beat labels of the music at the target beat points, the beat labels being used to identify audio signal points in the music used for multimedia splicing.

9. The apparatus according to claim 8, characterized in that, The amplitude of the rhythm change is represented by the energy difference ratio; The filtering module is also used to calculate the energy difference ratio between the waveform peak where the first rhythm point is located and the previous trough of the waveform.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the marking method for checkpoint tags as described in any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the marking method for checkpoint tags as described in any one of claims 1 to 7.

12. A computer program product, characterized in that, The computer program product includes computer instructions that are loaded and executed by a processor to implement the marking method for checkpoint tags as described in any one of claims 1 to 7.

Images