Animation material data acquisition method and system based on artificial intelligence

Abstract

The invention relates to the technical field of data processing, in particular to a cartoon material data acquisition method and system based on artificial intelligence, and the method comprises the steps: obtaining the image similarity of each frame and other frames according to the difference of color channels of each frame and other frames in an original cartoon video and the difference of image information; obtaining a plurality of key frames according to the picture similarity; obtaining the audio weight of each key frame according to the change condition of the sound frequencies of each key frame and a plurality of frames near each key frame; in combination with the picture similarity and the audio weight, obtaining fragment continuity of each key frame and other frames; and cutting the original animation video according to the fragment continuity to obtain a plurality of animation material fragments, thereby realizing animation material data acquisition based on artificial intelligence. According to the method, the picture similarity and the audio continuity are combined, the fragment continuity of each key frame and other frames is calculated, the animation material fragments are obtained according to the fragment continuity, and the situation that continuous plot switching shots are cut off is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an artificial intelligence-based animation material data collection method and system. Background Art

[0002] Animation materials provide the necessary original materials for animation production and content creators. These materials can be used to provide reference or inspiration for creators to create new animation works or animation-related products, or to carry out secondary creation, thereby promoting the development and diversification of animation and animation-related industries. Animation clips are often used in the secondary creation of animation videos. Building an animation clip material library can facilitate creators to retrieve and use materials and improve creation efficiency.

[0003] Constructing an animation clip library requires segmenting the animation video. Usually, when using artificial intelligence to perform picture recognition to segment the video, the recognition results obtained may be inaccurate, and only the pictures are considered for segmentation. Continuous plot switching shots may be interrupted, resulting in unreasonable segmentation results, which in turn leads to low quality of the constructed animation clip library. Summary of the invention

[0004] The present invention provides an artificial intelligence-based animation material data collection method and system to solve the problem of unreasonable segmentation results of existing animation clips.

[0005] The present invention provides an artificial intelligence-based animation material data collection method and system using the following technical solutions:

[0006] An embodiment of the present invention provides a method for collecting animation material data based on artificial intelligence, the method comprising the following steps:

[0007] Obtain an original animation video, the original animation video includes a plurality of frames, and obtain a sound frequency corresponding to each frame of the original animation video;

[0008] According to the changes in each color channel and the difference in picture information between each frame and other frames in the original animation video, the picture similarity between each frame and other frames is obtained; according to the picture similarity between each frame and the previous adjacent frame, several key frames are obtained;

[0009] According to the sound frequency of each frame and the change of the sound frequencies of several frames near each key frame, the audio forward relevance and the audio backward relevance of each key frame are obtained; according to the audio forward relevance and the audio backward relevance of each key frame, the audio weight of each key frame is obtained;

[0010] According to the picture similarity between each key frame and every other frame, the picture similarity cardinality between each key frame and every other frame is obtained; according to the audio weight of each key frame and the picture similarity cardinality of several nearby frames, as well as the degree of continuity of the audio near each key frame, the fragment continuity between each key frame and every other frame is obtained; according to the fragment continuity between each key frame and every other frame, cutting is performed to obtain several animation material fragments, thereby realizing animation material data collection based on artificial intelligence.

[0011] Furthermore, obtaining the picture similarity between each frame and other frames includes:

[0012] The similarity G between the i-th frame and the j-th frame except the i-th frame i,j The calculation method is:

[0013]

[0014] Where: n i.j represents the differential weight between the i-th frame and the j-th frame, d i,j represents the Euclidean distance between the semantic feature vectors of the i-th frame and the j-th frame, g i,r represents the average value of all pixels in the i-th frame in the r-th channel, g j,r represents the average value of the values ​​of all pixels in the jth frame in the rth channel, R represents the number of channels; | | represents the absolute value function, and exp() represents the exponential function with a natural constant as the base.

[0015] Furthermore, the specific method for obtaining the differential weight and the semantic feature vector is:

[0016] A pre-trained artificial intelligence model for picture semantic recognition is used to perform semantic recognition on each frame of the original animation video to obtain the semantic feature vector of each frame in the original animation video. The binary difference image between the two frames is obtained through inter-frame difference technology, and the number of white area pixels in the binary difference image is recorded as the difference weight.

[0017] Furthermore, the obtaining of a plurality of key frames includes:

[0018] A feature threshold is preset, and all frames whose picture similarity with the previous adjacent frame is less than the feature threshold are recorded as key frames.

[0019] Further, the obtaining of the audio forward relevance and the audio backward relevance of each key frame includes:

[0020] A frequency threshold and an interval upper limit are preset, and all frames whose sound frequency is greater than the frequency threshold are recorded as high-frequency frames. The frames with the interval upper limit number forward from each key frame are recorded as the forward interval frames of each key frame;

[0021] The audio forward relevance S of the t-th key frame t The calculation method is:

[0022]

[0023] Where: v t represents the number of high-frequency frames in the forward interval frames of the t-th key frame, f t Represents the variance of the sound frequency of all forward interval frames of the t-th key frame;

[0024] The audio backward relevance of the t-th key frame is obtained, and the calculation method of the audio backward relevance of the t-th key frame is the same as the calculation method of the audio forward relevance of the t-th key frame.

[0025] Further, obtaining the audio weight of each key frame includes:

[0026] The audio weight L of the tth keyframe t The calculation method is:

[0027] L t =Sigmoid(S t ×S′ t ×|S t ―S′ t |)

[0028] Where: S t represents the audio forward relevance of the t-th keyframe, S′ t represents the backward correlation of the audio of the t-th key frame; | | represents the absolute value function, and Sigmoid() represents the Sigmoid function.

[0029] Furthermore, obtaining the picture similarity cardinality between each key frame and each other frame includes:

[0030] The similarity cardinality Z of the image from the tth key frame to the mth frame t,m The calculation method is:

[0031]

[0032] Where: G t,k represents the picture similarity between the t-th key frame and the k-th frame before the t-th key frame, m represents the m-th frame before the t-th key frame, k represents the frame with an interval of k from the t-th key frame before, and Sigmoid[] represents the Sigmoid function.

[0033] Furthermore, obtaining the segment continuity between each key frame and each other frame includes:

[0034] The segment continuity F from the tth key frame to the mth framet,m The calculation method is:

[0035]

[0036] Where: L t represents the audio weight of the tth key frame, S t represents the audio forward relevance of the t-th keyframe, f t,m Indicates the sound frequency of the mth frame forward from the tth key frame, It represents the mean value of the sound frequency of all frames from the mth frame to the tth key frame, Z t,m represents the similarity cardinality of the image from the t-th key frame to the m-th frame forward, || represents the absolute value function, and Sigmoid[] represents the Sigmoid function.

[0037] Furthermore, the obtaining of a plurality of animation material clips includes:

[0038] A continuity threshold is preset, and each key frame is taken as an initial segment. When the segment continuity of the first frame from the t-th key frame is greater than or equal to the continuity threshold, the first frame from the t-th key frame is added to the initial segment where the t-th key frame is located. When the segment continuity of the second frame from the t-th key frame is greater than or equal to the continuity threshold, the second frame from the t-th key frame is added to the initial segment where the t-th key frame is located. When the segment continuity of the third frame from the t-th key frame is greater than or equal to the continuity threshold, the third frame from the t-th key frame is added to the initial segment where the t-th key frame is located. This is deduced by analogy until a constant c appears, such that the segment continuity of the c-th frame from the t-th key frame is less than the continuity threshold or the c-th frame from the t-th key frame is a video boundary, and the forward extension of the t-th key frame is stopped, and all frames from the c-th frame from the t-th key frame to the first frame from the t-th key frame are recorded as the forward segmentation result of the t-th key frame.

[0039] According to the method for obtaining the forward segmentation result of the t-th key frame, the backward segmentation result of the t-th key frame is obtained, and the forward segmentation result and the backward segmentation result of the t-th key frame and the video clip composed of the t-th frame are recorded as the segmentation result of the t-th key frame; the segmentation results of all key frames are obtained, and when the segmentation results of several key frames have an intersection, the several segmentation results with an intersection are merged to obtain all animation material clips.

[0040] An artificial intelligence-based animation material data collection system comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the artificial intelligence-based animation material data collection method are implemented.

[0041] The beneficial effects of the technical solution of the present invention are as follows: according to the changes in each color channel between each frame and other frames in the original animation video and the difference in picture information, the picture similarity between each frame and other frames is obtained, the picture similarity between each two frames is calculated by combining the semantic feature vector and the difference in each channel between each two frames, and the picture similarity result obtained by analyzing from multiple picture parameters is more reliable; according to the changes in the sound frequency of each frame and the sound frequency of several frames near each key frame, the audio forward correlation and audio backward correlation of each key frame are obtained, several frames before and after each key frame are taken to analyze the forward and backward correlation of the audio of each key frame, and the audio weight of each key frame is obtained according to the forward and backward correlation of the audio of each key frame, and the audio weight is used to reflect the audio weight of each key frame. The possibility of continuous plot switching in the dialogue is avoided near each key frame, so as to avoid the situation where the continuous plot switching shots are cut off; according to the picture similarity between each key frame and each other frame, the picture similarity cardinality between each key frame and each other frame is obtained, and the picture similarity cardinality is obtained by weighted accumulation of the picture similarities of all frames between each frame and its corresponding key frame, reflecting the overall picture similarity of all frames between each frame and its corresponding key frame; according to the audio weight of each key frame and the picture similarity cardinality of several nearby frames, as well as the continuity of the audio near each key frame, the segment continuity between each key frame and each other frame is obtained, and the initial segment where each key frame is located is extended according to the segment continuity, and the segments containing repeated frames are merged to obtain more accurate animation material segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 The present invention is a flowchart of the steps of an animation material data collection method based on artificial intelligence. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of an animation material data collection method and system based on artificial intelligence proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0045] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0046] The following is a detailed description of a method and system for collecting animation material data based on artificial intelligence provided by the present invention in conjunction with the accompanying drawings.

[0047] First, see Figure 1 , which shows a flowchart of a method for collecting animation material data based on artificial intelligence provided by an embodiment of the present invention, the method comprising the following steps:

[0048] Step S001, obtaining the original animation video.

[0049] The purpose of this embodiment is to obtain a number of animation materials by segmenting the animation video. Therefore, firstly, a number of animation videos that allow secondary creation are purchased and downloaded through official channels, which are recorded as original animation videos; the original animation video includes a number of frames, and the sound frequency corresponding to each frame of the original animation video is obtained.

[0050] Step S002: Obtain the picture similarity between each frame and other frames; obtain several key frames.

[0051] It should be noted that semantic recognition of all frames of the original animation video through an artificial intelligence model can preliminarily reflect the content of each frame; the distance between the semantic feature vectors of each two frames reflects the difference between each two frames, and the similarity between each two frames is obtained by combining the differences in each channel between each two frames and the changes in pixels between each two frames; since a sudden change in the picture usually reflects the end of a segment and the beginning of the next segment, the frame with the smallest similarity to the adjacent previous frame is recorded as a key frame, and each key frame is used as a marker for preliminary segmentation. However, since the result obtained by segmentation based only on the picture is not accurate, it is necessary to further analyze several frames adjacent to the key frame.

[0052] Specifically, taking any original animation video as an example, a pre-trained artificial intelligence model for picture semantic recognition is used to perform semantic recognition on each frame of the original animation video to obtain a semantic feature vector of each frame in the original animation video; the inter-frame difference technology can obtain pixels that have undergone large changes between two frames, and a binary difference image between the two frames is obtained by the inter-frame difference technology, and the number of white area pixels in the binary difference image is recorded as the difference weight. It should be noted that the pre-trained artificial intelligence model for picture semantic recognition in this embodiment is described using the Illustration2Vec model as an example, and other embodiments may select other artificial intelligence models, which are not specifically limited in this embodiment; the use of artificial intelligence models for semantic recognition and inter-frame difference technology are well-known technologies, which are not described in detail in this embodiment;

[0053] The similarity G between the i-th frame and the j-th frame except the i-th frame i,j The calculation method is:

[0054]

[0055] Where: n i.j represents the differential weight between the i-th frame and the j-th frame, d i,j represents the Euclidean distance between the semantic feature vectors of the i-th frame and the j-th frame, g i,r represents the average value of all pixels in the i-th frame in the r-th channel, g j,r represents the average value of the values ​​of all pixels in the jth frame in the rth channel, R represents the number of channels, this embodiment takes RGB three channels as an example for description, other embodiments may select other channels, and this embodiment does not make specific limitations; | | represents an absolute value function, exp() represents an exponential function with a natural constant as the base;

[0056] It should be noted that n i.j The smaller the value of is, the fewer pixels move between the i-th frame and the j-th frame, the more similar the i-th frame and the j-th frame are, and the greater the similarity between the i-th frame and the j-th frame is; |g i,r ―g j,r | 2 It represents the difference between the values ​​of the i-th frame and the j-th frame in the r-th channel. The smaller the value, the greater the similarity between the i-th frame and the j-th frame.

[0057] A feature threshold is preset to obtain the picture similarity between each frame in the original animation video and the adjacent previous frame. In particular, the picture similarity of the first frame is recorded as 0, and all frames whose picture similarity with the adjacent previous frame is less than the feature threshold are recorded as key frames; the preset feature threshold in this embodiment is 0.5, and other embodiments can be set to other values, which are not specifically limited in this embodiment.

[0058] Step S003: Obtain the audio forward relevance and audio backward relevance of each key frame; obtain the audio weight of each key frame.

[0059] It should be noted that when segmenting is performed only by the similarity of the images, there may be a situation where the scenes are switched continuously. The situation where the scenes are switched continuously in the animation video may be the switching of the scenes in the dialogue or the transition of the scenes in the dialogue. The situation where the scenes are switched in the dialogue is usually when a speaker is speaking, the camera cuts to the speaker to show the speaker's reaction, or cuts to the scene to show the speaker's emotions.

[0060] Combine the continuity of the key frame and its adjacent frames of audio to analyze the continuous plot switching shots. When the human voice is continuous and the adjacent frames have low similarity, it is likely that there is a continuous plot switching shot. The continuous plot of switching shots in the dialogue should be divided into the same segment when segmenting. The continuous scene transition screen is usually a shot switching between multiple scenes. In animation, it usually shows scenes happening in different places at the same time, or the changes of scenes in multiple places within a period of time. It is used to transition the plot of animation or reflect the background of the story. For the segmentation of continuous scene transition screens, although there is a certain correlation between different scenes, when segmenting animation materials, scenes in different locations are usually divided into multiple segments for easy distinction;

[0061] Since the sound frequency of the human voice in the dialogue is high frequency, the more high-frequency frames there are near the key frame, the more likely the key frame is to have voice dialogue; based on the forward correlation and backward correlation of each key frame audio, analyze whether each key frame is a key frame in the continuous plot of the dialogue. When the forward correlation and backward correlation of the key frame audio are both greater, or one of the forward correlation and backward correlation is greater and the other is smaller, the frames near the key frame are more likely to be continuous dialogues, and more attention should be paid to the continuity of the audio when analyzing the segment continuity of the key frame.

[0062] Specifically, a frequency threshold and an interval upper limit are preset, and all frames whose sound frequency is greater than the frequency threshold are recorded as high-frequency frames. The frames taken forward by the interval upper limit number from each key frame are recorded as the forward interval frames of each key frame, and the frames taken backward by the interval upper limit number are recorded as the backward interval frames of each key frame. In this embodiment, the preset frequency threshold is 100 Hz, and the interval upper limit is 48 frames. Other embodiments may be set to other values, which are not specifically limited in this embodiment. It should be noted that the frame number specification of an animation video is usually 24 frames per second, and the breathing pause interval when a person speaks continuously is usually about 0.5 to 2 seconds, so the interval upper limit is set to 48 frames;

[0063] The audio forward relevance S of the t-th key frame t The calculation method is:

[0064]

[0065] Where: v t represents the number of high-frequency frames in the forward interval frames of the t-th key frame, f t represents the variance of the sound frequencies of all forward interval frames of the t-th key frame, x represents a hyperparameter used to avoid the denominator being 0, and in this embodiment, x=1;

[0066] It should be noted that Indicates the audio continuity of all frames before the t-th key frame. The larger the value, the more continuous the human voice is before the t-th key frame, and the more likely it is that there is a continuous dialogue before the t-th key frame. The smaller the value, the more likely there is no human voice before the t-th key frame, and the less likely it is that there is a continuous dialogue at the t-th key frame.

[0067] The audio backward relevance of the t-th key frame is obtained. The calculation method of the audio backward relevance of the t-th key frame is the same as the calculation method of the audio forward relevance of the t-th key frame. Similarly, the audio forward relevance and audio backward relevance of each key frame are obtained.

[0068] It should be further explained that, since high-frequency frames are more likely to be frames with human voices, key frames with more high-frequency frames are more likely to be continuous plots of switching shots in the dialogue. Therefore, when analyzing segment continuity for key frames with more high-frequency frames, more emphasis should be placed on audio relevance than on picture similarity. The greater the forward audio relevance and backward audio relevance of a key frame, the more likely the key frame is to be in the dialogue, and the key frame is likely to have strong continuity with the previous and next adjacent frames. The greater the absolute value of the difference between the forward audio relevance and backward audio relevance of a key frame, the more likely the key frame is to be at the beginning or end of the dialogue, and the key frame is likely to have strong continuity with the previous adjacent frame or the next adjacent frame.

[0069] Specifically, the audio weight L of the tth key frame t The calculation method is:

[0070] L t =Sigmoid(S t ×S′ t ×|S t ―S′ t |)

[0071] Where: S t represents the audio forward relevance of the t-th keyframe, S′ t represents the backward correlation of the audio of the t-th key frame; | | represents the absolute value function, and Sigmoid() represents the Sigmoid function;

[0072] It should be noted that St ×S′ t It represents the product of the audio forward relevance and audio backward relevance of the t-th key frame. The larger the value, the greater the audio forward relevance and audio backward relevance of the t-th key frame. The more likely the t-th key frame is a continuous plot of switching shots in the dialogue, the greater the audio weight of the t-th key frame. t ―S′ t | represents the absolute value of the difference between the audio forward relevance and the audio backward relevance of the t-th key frame. The larger the value, the more likely the t-th key frame is to be the end of a sentence or the beginning of a sentence. The more likely the frames near the t-th key frame are to be continuous dialogues, the greater the audio weight of the t-th key frame.

[0073] Similarly, the audio weight of each key frame is obtained.

[0074] Step S004, obtaining the picture similarity cardinality between each key frame and each other frame; obtaining the segment continuity between each key frame and each other frame; obtaining a number of animation material segments.

[0075] It should be noted that the fragment continuity of each frame near each key frame is calculated by combining the picture similarity of several adjacent frames before and after each key frame and the audio weight of each key frame; for a key frame with a larger audio weight, the vicinity of the key frame is more likely to be a continuous plot of switching shots in the dialogue, and when analyzing the fragment continuity of the key frame, more attention should be paid to the continuity of the audio; for a key frame with a smaller audio weight, when analyzing the fragment continuity of the key frame, more attention should be paid to the similarity of the picture; since the key frame is a mark for the preliminary segmentation of the animation video, the fragment continuity of each frame near the key frame is extended to several adjacent frames before and after each key frame to obtain the animation material fragment.

[0076] Specifically, the similarity cardinality Z of the image from the tth key frame to the mth frame is t,m The calculation method is:

[0077]

[0078] Where: G t,k represents the image similarity between the t-th key frame and the k-th frame before the t-th key frame, m represents the m-th frame before the t-th key frame, k represents the frame with a frame interval of k before the t-th key frame, and Sigmoid[] represents the Sigmoid function;

[0079] It should be noted that It represents the weighted accumulation of the similarity of all frames from the t-th key frame to the m-th frame to the t-th key frame. The larger the value, the greater the similarity base of the m-th frame from the t-th key frame to the m-th frame. Usually, the farther the distance between the frames, the greater the change. Therefore, As a weight, the farther away from the t-th key frame, the smaller the weight of the impact on the picture similarity cardinality;

[0080] The fragment continuity F from the tth key frame to the mth frame t,m The calculation method is:

[0081]

[0082] Where: L t represents the audio weight of the tth key frame, S t represents the audio forward relevance of the t-th keyframe, f t,m Indicates the sound frequency of the mth frame forward from the tth key frame, It represents the mean value of the sound frequency of all frames from the mth frame to the tth key frame, Z t,m represents the similarity cardinality of the images from the t-th key frame to the m-th frame, | | represents the absolute value function, and Sigmoid[] represents the Sigmoid function;

[0083] It should be noted that the larger the audio weight of the tth key frame, Partial for fragment continuity F t,m The greater the impact of the calculation result, the greater the value, the more continuous the audio of the segment from the t-th key frame to the m-th frame, and the greater the continuity of the segment from the t-th key frame to the m-th frame; the smaller the audio weight of the t-th key frame, (1-L t )×Z t,m Partial for fragment continuity F t,m The greater the influence of the calculation result, the greater the value, the more continuous the images of the segment from the t-th key frame forward are, and the greater the continuity of the segment from the t-th key frame to the m-th frame;

[0084] The segment continuity from the t-th key frame to the m-th frame is obtained. The calculation method of the segment continuity from the t-th key frame to the m-th frame is the same as the calculation method of the segment continuity from the t-th key frame to the m-th frame. Similarly, the segment continuity of each frame from all key frames to the front is obtained.

[0085] Further, a continuity threshold is preset, and each key frame is taken as each initial segment. When the segment continuity of the first frame from the t-th key frame is greater than or equal to the continuity threshold, the first frame from the t-th key frame is added to the initial segment where the t-th key frame is located. When the segment continuity of the second frame from the t-th key frame is greater than or equal to the continuity threshold, the second frame from the t-th key frame is added to the initial segment where the t-th key frame is located. When the segment continuity of the third frame from the t-th key frame is greater than or equal to the continuity threshold, the third frame from the t-th key frame is added to the initial segment where the t-th key frame is located. And so on, until a constant c appears, such that the segment continuity of the c-th frame from the t-th key frame is less than the continuity threshold or the c-th frame from the t-th key frame is a video boundary, the forward extension of the t-th key frame is stopped, and all frames from the c-th frame from the t-th key frame to the first frame from the t-th key frame are recorded as the forward segmentation result of the t-th key frame.

[0086] According to the method for obtaining the forward segmentation result of the t-th key frame, the backward segmentation result of the t-th key frame is obtained, and the forward segmentation result and the backward segmentation result of the t-th key frame and the video clip composed of the t-th frame are recorded as the segmentation result of the t-th key frame; the segmentation results of all key frames are obtained, and when the segmentation results of several key frames have an intersection, the several segmentation results with an intersection are merged to obtain all animation material clips.

[0087] Furthermore, semantic recognition is performed on the semantic feature vector of each frame of each animation material clip to obtain the recognition result text, and the recognition result text that appears most frequently in the recognition result texts of all frames in each animation material clip is used as the label of the animation material clip. All animation material clips are classified by several labels to facilitate the retrieval of the animation material clips when they are used, build an animation material library, and realize animation material data collection based on artificial intelligence.

[0088] It should be noted that the exp(-x) model used in this embodiment only indicates that the result of the negative correlation and constraint model output is in the interval (0,1], where x is the input of this model. It can be replaced by other models with the same purpose during specific implementation. This embodiment only takes the exp(-x) model as an example for description without specific limitation.

[0089] In a second aspect, an embodiment of the present invention provides an artificial intelligence-based animation material data collection method and system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above steps S001 to S004 when executing the computer program.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for collecting animation material data based on artificial intelligence, characterized in that: The method comprises the following steps: Obtain an original animation video, the original animation video includes a plurality of frames, and obtain a sound frequency corresponding to each frame of the original animation video; According to the changes in each color channel and the difference in picture information between each frame and other frames in the original animation video, the picture similarity between each frame and other frames is obtained; according to the picture similarity between each frame and the previous adjacent frame, several key frames are obtained; According to the sound frequency of each frame and the change of the sound frequencies of several frames near each key frame, the audio forward relevance and the audio backward relevance of each key frame are obtained; according to the audio forward relevance and the audio backward relevance of each key frame, the audio weight of each key frame is obtained; According to the picture similarity between each key frame and every other frame, the picture similarity cardinality between each key frame and every other frame is obtained; according to the audio weight of each key frame and the picture similarity cardinality of several nearby frames, as well as the degree of continuity of the audio near each key frame, the fragment continuity between each key frame and every other frame is obtained; according to the fragment continuity between each key frame and every other frame, cutting is performed to obtain several animation material fragments, thereby realizing animation material data collection based on artificial intelligence.

2. According to claim 1, a method for collecting animation material data based on artificial intelligence is characterized in that: The obtaining of the picture similarity between each frame and other frames includes: The similarity G between the i-th frame and the j-th frame except the i-th frame i,j The calculation method is: Where: n i.j represents the differential weight between the i-th frame and the j-th frame, d i,j represents the Euclidean distance between the semantic feature vectors of the i-th frame and the j-th frame, g i,r represents the average value of all pixels in the i-th frame in the r-th channel, g j,r represents the average value of the values ​​of all pixels in the jth frame in the rth channel, R represents the number of channels; || represents the absolute value function, and exp() represents the exponential function with a natural constant as the base.

3. According to claim 2, a method for collecting animation material data based on artificial intelligence is characterized in that: The specific method for obtaining the differential weight and the semantic feature vector is: A pre-trained artificial intelligence model for picture semantic recognition is used to perform semantic recognition on each frame of the original animation video to obtain the semantic feature vector of each frame in the original animation video. The binary difference image between the two frames is obtained through inter-frame difference technology, and the number of white area pixels in the binary difference image is recorded as the difference weight.

4. The method for collecting animation material data based on artificial intelligence according to claim 1, characterized in that: The obtaining of several key frames includes: A feature threshold is preset, and all frames whose picture similarity with the previous adjacent frame is less than the feature threshold are recorded as key frames.

5. The method for collecting animation material data based on artificial intelligence according to claim 1, characterized in that: The obtaining of the audio forward relevance and the audio backward relevance of each key frame includes: A frequency threshold and an interval upper limit are preset, and all frames whose sound frequency is greater than the frequency threshold are recorded as high-frequency frames. The frames with the interval upper limit number forward from each key frame are recorded as the forward interval frames of each key frame; The audio forward relevance S of the t-th keyframe t The calculation method is: Where: v t represents the number of high-frequency frames in the forward interval frames of the t-th key frame, f t Represents the variance of the sound frequency of all forward interval frames of the t-th key frame; The audio backward relevance of the t-th key frame is obtained, and the calculation method of the audio backward relevance of the t-th key frame is the same as the calculation method of the audio forward relevance of the t-th key frame.

6. The method for collecting animation material data based on artificial intelligence according to claim 1, characterized in that: The step of obtaining the audio weight of each key frame includes: The audio weight L of the tth keyframe t The calculation method is: L t =Sigmoid(S t ×S′ t ×|S t ―S′ t |) Where: S t represents the audio forward relevance of the t-th keyframe, S′ t represents the backward correlation of the audio of the t-th key frame; || represents the absolute value function, and Sigmoid() represents the Sigmoid function.

7. The method for collecting animation material data based on artificial intelligence according to claim 1, characterized in that: The obtaining of the picture similarity cardinality between each key frame and each other frame includes: The similarity cardinality Z of the image from the tth key frame to the mth frame t,m The calculation method is: Where: G t,k represents the image similarity between the t-th key frame and the k-th frame before the t-th key frame, m represents the m-th frame before the t-th key frame, k represents the frame with an interval of k before the t-th key frame, and Sigmoid[ ] represents the Sigmoid function.

8. The method for collecting animation material data based on artificial intelligence according to claim 1, characterized in that: The obtaining of the fragment continuity between each key frame and each other frame includes: The segment continuity F from the tth key frame to the mth frame t,m The calculation method is: Where: L t represents the audio weight of the tth key frame, S t represents the audio forward relevance of the t-th keyframe, f t,m Indicates the sound frequency of the mth frame forward from the tth key frame, It represents the mean value of the sound frequency of all frames from the mth frame to the tth key frame, Z t,m represents the similarity cardinality of the image from the t-th key frame to the m-th frame forward, || represents the absolute value function, and Sigmoid[ ] represents the Sigmoid function.

9. The method for collecting animation material data based on artificial intelligence according to claim 1, characterized in that: The obtaining of a plurality of animation material clips includes: A continuity threshold is preset, and each key frame is taken as an initial segment. When the segment continuity of the first frame from the t-th key frame is greater than or equal to the continuity threshold, the first frame from the t-th key frame is added to the initial segment where the t-th key frame is located. When the segment continuity of the second frame from the t-th key frame is greater than or equal to the continuity threshold, the second frame from the t-th key frame is added to the initial segment where the t-th key frame is located. When the segment continuity of the third frame from the t-th key frame is greater than or equal to the continuity threshold, the third frame from the t-th key frame is added to the initial segment where the t-th key frame is located. This is deduced by analogy until a constant c appears, such that the segment continuity of the c-th frame from the t-th key frame is less than the continuity threshold or the c-th frame from the t-th key frame is a video boundary, and the forward extension of the t-th key frame is stopped, and all frames from the c-th frame from the t-th key frame to the first frame from the t-th key frame are recorded as the forward segmentation result of the t-th key frame. According to the method for obtaining the forward segmentation result of the t-th key frame, the backward segmentation result of the t-th key frame is obtained, and the forward segmentation result and the backward segmentation result of the t-th key frame and the video clip composed of the t-th frame are recorded as the segmentation result of the t-th key frame; the segmentation results of all key frames are obtained, and when the segmentation results of several key frames have an intersection, the several segmentation results with an intersection are merged to obtain all animation material clips.

10. An animation material data acquisition system based on artificial intelligence, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images