An improved method and system for scene content detection

Abstract

The present invention provides an improved method and system for scene content detection. By extracting multiple features from multimedia data, the multiple features are input into a neural network model according to different dimensions to obtain a high-dimensional vector matrix after machine learning. Then, through a state chain model, the dominant feature distribution area is obtained, and the required semantic feature set is determined. Finally, the dividing line for different scenes is determined, thereby achieving accurate segmented content detection and reducing the amount of calculation.

Description

Technical Field

[0001] The present application relates to the field of network multimedia, and in particular to an improved method and system for scene content detection. Background Art

[0002] Existing networks contain a vast amount of scene information and rich video data. A single video often contains multiple, completely different scenes. Calling different content detection algorithms for different scenes significantly increases the computational load. Even if precise scene boundaries can be delineated, the computational load for content detection is still considerable, placing a significant burden on the system.

[0003] Therefore, there is an urgent need for a targeted and improved method and system for scene content detection. Given an accurate scene boundary, the content detection algorithm can be further improved. Summary of the Invention

[0004] The purpose of the present invention is to provide an improved method and system for scene content detection, which extracts multiple features from multimedia data, inputs the multiple features into a neural network model according to different dimensions, obtains a high-dimensional vector matrix after machine learning, and then obtains the explicit feature distribution area through a state chain model, thereby determining the required semantic feature set of the implicit feature distribution area, and finally determines the dividing line for different scenes to achieve accurate segmented content detection.

[0005] In a first aspect, the present application provides an improved method for scene content detection, the method comprising:

[0006] receiving multimedia data sent by an acquisition terminal, extracting visual features, sound features, and text features therefrom, listing the visual features, sound features, and text features into three dimensions of a first vector matrix, and sending the first vector matrix to an input end of a neural network model;

[0007] The neural network model has three input terminals, which are respectively connected to the corresponding intermediate layers. After calculation processing, they are connected to three buffers. One of the intermediate layers includes a full-state connection unit. The full-state connection unit will feed back the results of the intermediate layer to the remaining two buffers. The remaining two buffers correct the data based on the feedback. The outputs of the three buffers are weighted and summed to obtain an output terminal.

[0008] The neural network model operates on the first vector matrix, removes eigenvalues ​​whose single dimension deviation is greater than a preset threshold, and obtains a second high-dimensional vector matrix after machine learning;

[0009] Inputting the second high-dimensional vector matrix into a state chain model, determining the dominant feature distribution area corresponding to the multimedia data based on a preset probability density function, deriving possible recessive feature distribution areas, extracting multiple third vector matrices from the possible recessive feature distribution areas, and decomposing the third vector matrices to obtain recessive features;

[0010] Semantically analyzing the implicit features to obtain a plurality of undetermined semantic features, calculating correlations between the plurality of undetermined semantic features, removing undetermined semantic features whose correlations are lower than a threshold, and determining a semantic feature set corresponding to the multimedia data;

[0011] Inputting the second high-dimensional vector matrix and the semantic feature set into a calculation function, while introducing the probability density parameter of the state chain model, obtaining a conditional probability formula from the third vector matrix to the second high-dimensional vector matrix, calculating the conditional probability formula using a neural network model, and calculating the optimal third vector matrix;

[0012] Determining a dividing line for different scene segments based on the distribution of the optimal third vector matrices, dividing the multimedia data into different scene segments based on the dividing line, and performing semantic analysis in sequence to obtain semantic labels corresponding to the different scene segments;

[0013] According to the semantic tag, different content detection algorithms are called to perform content detection on the scene segment corresponding to the semantic tag.

[0014] In combination with the first aspect, in a first possible implementation of the first aspect, the semantic analysis further includes a clustering operation to centrally analyze scene segments of the same category.

[0015] In combination with the first aspect, in a second possible implementation manner of the first aspect, the receiving multimedia data stream sent by the acquisition terminal includes encoding and decoding of the multimedia data stream.

[0016] In combination with the first aspect, in a third possible implementation of the first aspect, the semantic analysis adopts a neural network model.

[0017] In a second aspect, the present application provides an improved system for scene content detection, the system comprising a processor and a memory:

[0018] The memory is used to store program code and transmit the program code to the processor;

[0019] The processor is configured to execute any one of the four possible methods of the first aspect according to instructions in the program code.

[0020] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute any one of the four possible methods in the first aspect.

[0021] Beneficial effects

[0022] The present invention provides an improved method and system for scene content detection. By inputting multiple extracted features into a neural network model according to different dimensions, a high-dimensional vector matrix after machine learning is obtained. Then, through a state chain model, the required semantic feature set is determined, and finally the dividing line for different scenes is determined, thereby achieving accurate segmented content detection, improving detection accuracy, and further reducing the amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0025] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0026] Figure 1 A flowchart of an improved scene content detection method provided in this application includes:

[0027] receiving multimedia data sent by an acquisition terminal, extracting visual features, sound features, and text features therefrom, listing the visual features, sound features, and text features into three dimensions of a first vector matrix, and sending the first vector matrix to an input end of a neural network model;

[0028] The neural network model has three input terminals, which are respectively connected to the corresponding intermediate layers. After calculation processing, they are connected to three buffers. One of the intermediate layers includes a full-state connection unit. The full-state connection unit will feed back the results of the intermediate layer to the remaining two buffers. The remaining two buffers correct the data based on the feedback. The outputs of the three buffers are weighted and summed to obtain an output terminal.

[0029] The neural network model operates on the first vector matrix, removes eigenvalues ​​whose single dimension deviation is greater than a preset threshold, and obtains a second high-dimensional vector matrix after machine learning;

[0030] Inputting the second high-dimensional vector matrix into a state chain model, determining the dominant feature distribution area corresponding to the multimedia data based on a preset probability density function, deriving possible recessive feature distribution areas, extracting multiple third vector matrices from the possible recessive feature distribution areas, and decomposing the third vector matrices to obtain recessive features;

[0031] Semantically analyzing the implicit features to obtain a plurality of undetermined semantic features, calculating correlations between the plurality of undetermined semantic features, removing undetermined semantic features whose correlations are lower than a threshold, and determining a semantic feature set corresponding to the multimedia data;

[0032] Inputting the second high-dimensional vector matrix and the semantic feature set into a calculation function, while introducing the probability density parameter of the state chain model, obtaining a conditional probability formula from the third vector matrix to the second high-dimensional vector matrix, calculating the conditional probability formula using a neural network model, and calculating the optimal third vector matrix;

[0033] Determining a dividing line for different scene segments based on the distribution of the optimal third vector matrices, dividing the multimedia data into different scene segments based on the dividing line, and performing semantic analysis in sequence to obtain semantic labels corresponding to the different scene segments;

[0034] According to the semantic tag, different content detection algorithms are called to perform content detection on the scene segment corresponding to the semantic tag.

[0035] In some preferred embodiments, the semantic analysis further includes a clustering operation, which focuses on analyzing scene segments of the same category.

[0036] In some preferred embodiments, the receiving of the multimedia data stream sent by the acquisition terminal includes encoding and decoding of the multimedia data stream.

[0037] In some preferred embodiments, the semantic analysis adopts a neural network model.

[0038] The present application provides an improved system for scene content detection, the system comprising: the system comprising a processor and a memory:

[0039] The memory is used to store program code and transmit the program code to the processor;

[0040] The processor is configured to execute the method described in any one of all embodiments of the first aspect according to instructions in the program code.

[0041] The present application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the method described in any one of the embodiments of the first aspect.

[0042] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program that, when executed, may include some or all of the steps of various embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0043] Those skilled in the art will clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain portions of the embodiments.

[0044] In particular, for the embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0045] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. An improved method for scene content detection, characterized in that: The method comprises: receiving multimedia data sent by an acquisition terminal, extracting visual features, sound features, and text features therefrom, listing the visual features, sound features, and text features into three dimensions of a first vector matrix, and sending the first vector matrix to an input end of a neural network model; The neural network model has three input terminals, which are respectively connected to the corresponding intermediate layers. After calculation processing, they are connected to three buffers. One of the intermediate layers includes a full-state connection unit. The full-state connection unit will feed back the results of the intermediate layer to the remaining two buffers. The remaining two buffers correct the data based on the feedback. The outputs of the three buffers are weighted and summed to obtain an output terminal. The neural network model operates on the first vector matrix, removes eigenvalues ​​whose single dimension deviation is greater than a preset threshold, and obtains a second high-dimensional vector matrix after machine learning; Inputting the second high-dimensional vector matrix into a state chain model, determining the dominant feature distribution area corresponding to the multimedia data based on a preset probability density function, deriving possible recessive feature distribution areas, extracting multiple third vector matrices from the possible recessive feature distribution areas, and decomposing the third vector matrices to obtain recessive features; Semantically analyzing the implicit features to obtain a plurality of undetermined semantic features, calculating correlations between the plurality of undetermined semantic features, removing undetermined semantic features whose correlations are lower than a threshold, and determining a semantic feature set corresponding to the multimedia data; Inputting the second high-dimensional vector matrix and the semantic feature set into a calculation function, while introducing the probability density parameter of the state chain model, obtaining a conditional probability formula from the third vector matrix to the second high-dimensional vector matrix, calculating the conditional probability formula using a neural network model, and calculating the optimal third vector matrix; Determining a dividing line for different scene segments based on the distribution of the optimal third vector matrices, dividing the multimedia data into different scene segments based on the dividing line, and performing semantic analysis in sequence to obtain semantic labels corresponding to the different scene segments; According to the semantic tag, different content detection algorithms are called to perform content detection on the scene segment corresponding to the semantic tag.

2. The method according to claim 1, wherein: The semantic analysis also includes a clustering operation, which focuses on analyzing scene segments of the same category.

3. The method according to claim 2, wherein: The receiving and collecting multimedia data stream sent by the terminal includes encoding and decoding of the multimedia data stream.

4. The method according to claim 3, wherein: The semantic analysis adopts a neural network model.

5. An improved scene content detection system, characterized in that: The system includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to implement the method according to any one of claims 1 to 4 according to the instructions in the program code.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the method according to any one of claims 1 to 4.

Images