Semantic analysis and recognition method and system based on weakly supervised network

Abstract

The present invention provides a semantic analysis and recognition method and system based on a weakly supervised network. By calculating the eigenvalue of the histogram of each frame in the gradient direction, the jumping point is found to start the weakly supervised network model. The weakly supervised network architecture is used to quickly process the data stream, effectively reducing the error value of data processing. The semantic analysis model and the graphic analysis model are input in parallel to determine whether the multimedia data stream involves sensitive data, thereby restricting the use of sensitive data.

Description

Technical Field

[0001] The present application relates to the field of network multimedia, and in particular to a semantic analysis and recognition method and system based on a weakly supervised network. Background Art

[0002] Existing text recognition and analysis technologies have poor real-time performance, low analysis accuracy and efficiency, and are unable to process large amounts of data in a timely manner. Applying existing text recognition and analysis methods to protecting sensitive network data will encounter insurmountable difficulties.

[0003] Therefore, there is an urgent need for a targeted semantic analysis and recognition method and system based on weakly supervised networks. Summary of the Invention

[0004] The purpose of the present invention is to provide a semantic analysis and recognition method and system based on a weakly supervised network. By calculating the eigenvalues ​​of the histogram of each frame in the gradient direction, the jumping point is found to start the weakly supervised network model, and the weakly supervised network architecture is used to quickly process the data stream, effectively reducing the error value of data processing, and inputting the semantic analysis model and the graphic analysis model in parallel to determine whether the multimedia data stream involves sensitive data, thereby restricting the use of sensitive data.

[0005] In a first aspect, the present application provides a semantic analysis and recognition method based on a weakly supervised network, the method comprising:

[0006] Acquire a multimedia data stream, calculate the eigenvalue of the histogram of each frame in the gradient direction, and when the difference between the eigenvalues ​​between the frames is greater than a preset threshold, start a weakly supervised network model, wherein the weakly supervised network model includes a plurality of input layers, hidden layers, and output layers;

[0007] Inputting the multimedia data stream into the input layer of the weakly supervised network model, assigning values ​​to the hidden layer at the current moment, outputting a first result from the output layer, summing the first result with the hidden layer at the next moment, and inputting the result into the next input layer, repeating this process until all layers have been used, and obtaining an output result of the weakly supervised network model, which is the second result;

[0008] Inputting the second result into a semantic analysis model, analyzing the text information contained therein frame by frame, obtaining field values ​​of text characters, calling a word segmenter to decompose the field values ​​into words and word attributes, matching the words and word attributes with a vocabulary, determining whether the field value is an address, name, or company name, and obtaining a first judgment result;

[0009] Inputting the second results into the graphic analysis model in parallel, identifying the object information contained in each frame, obtaining key object features, determining whether the frame contains sensitive graphic content, and whether the object information matches the first determination result, thereby obtaining a second determination result;

[0010] According to the first judgment result and the second judgment result, it is determined whether the frame of the multimedia data stream involves sensitivity. If so, the current multimedia data stream is encrypted and stored separately; otherwise, the multimedia data stream is allowed to be used arbitrarily.

[0011] In combination with the first aspect, in a first possible implementation manner of the first aspect, the characteristic value of the histogram in the gradient direction includes detecting a change in a grayscale centroid position of the image.

[0012] In combination with the first aspect, in a second possible implementation of the first aspect, if the multimedia data stream is determined to be sensitive data, its scope of use will be restricted.

[0013] In combination with the first aspect, in a third possible implementation of the first aspect, the kernels of the semantic analysis model and the graphic analysis model both use a neural network model.

[0014] In a second aspect, the present application provides a semantic analysis and recognition system based on a weakly supervised network, the system comprising a processor and a memory:

[0015] The memory is used to store program code and transmit the program code to the processor;

[0016] The processor is configured to execute any one of the four possible methods of the first aspect according to instructions in the program code.

[0017] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute any one of the four possible methods in the first aspect.

[0018] The present invention provides a method and system for semantic analysis and recognition based on a weakly supervised network. By calculating the eigenvalue of the histogram of each frame in the gradient direction, the jumping point is found to start the weakly supervised network model. The weakly supervised network architecture is used to quickly process data streams, effectively reducing the error value of data processing. The semantic analysis model and the graphic analysis model are input in parallel to determine whether the multimedia data stream involves sensitive data, thereby restricting the use of sensitive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0022] Figure 1 The flowchart of the semantic analysis and recognition method based on weakly supervised network provided in this application includes:

[0023] Acquire a multimedia data stream, calculate the eigenvalue of the histogram of each frame in the gradient direction, and when the difference between the eigenvalues ​​between the frames is greater than a preset threshold, start a weakly supervised network model, wherein the weakly supervised network model includes a plurality of input layers, hidden layers, and output layers;

[0024] Inputting the multimedia data stream into the input layer of the weakly supervised network model, assigning values ​​to the hidden layer at the current moment, outputting a first result from the output layer, summing the first result with the hidden layer at the next moment, and inputting the result into the next input layer, repeating this process until all layers have been used, and obtaining an output result of the weakly supervised network model, which is the second result;

[0025] Inputting the second result into a semantic analysis model, analyzing the text information contained therein frame by frame, obtaining field values ​​of text characters, calling a word segmenter to decompose the field values ​​into words and word attributes, matching the words and word attributes with a vocabulary, determining whether the field value is an address, name, or company name, and obtaining a first judgment result;

[0026] Inputting the second results into the graphic analysis model in parallel, identifying the object information contained in each frame, obtaining key object features, determining whether the frame contains sensitive graphic content, and whether the object information matches the first determination result, thereby obtaining a second determination result;

[0027] According to the first judgment result and the second judgment result, it is determined whether the frame of the multimedia data stream involves sensitivity. If so, the current multimedia data stream is encrypted and stored separately; otherwise, the multimedia data stream is allowed to be used arbitrarily.

[0028] In some preferred embodiments, the characteristic value of the histogram in the gradient direction includes detecting a change in the position of the grayscale centroid of the image.

[0029] In some preferred embodiments, if the multimedia data stream is determined to be sensitive data, its scope of use will be restricted.

[0030] In some preferred embodiments, the kernels of the semantic analysis model and the graphic analysis model both use neural network models.

[0031] The present application provides a semantic analysis and recognition system based on a weakly supervised network, the system comprising: the system comprising a processor and a memory:

[0032] The memory is used to store program code and transmit the program code to the processor;

[0033] The processor is configured to execute the method described in any one of all embodiments of the first aspect according to instructions in the program code.

[0034] The present application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the method described in any one of the embodiments of the first aspect.

[0035] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program that, when executed, may include some or all of the steps of various embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0036] Those skilled in the art will clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain portions of the embodiments.

[0037] In particular, for the embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0038] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A semantic analysis and recognition method based on a weakly supervised network, characterized in that: The method comprises: Acquire a multimedia data stream, calculate the characteristic value of the histogram of each frame in the gradient direction, and when the difference between the characteristic values ​​between the frames is greater than a preset threshold, start a weakly supervised network model, the weakly supervised network model including a plurality of input layers, hidden layers, and an output layer; the characteristic value of the histogram in the gradient direction includes a change in the grayscale centroid position of the detected image; Inputting the multimedia data stream into the input layer of the weakly supervised network model, assigning values ​​to the hidden layer at the current moment, outputting a first result from the output layer, summing the first result with the hidden layer at the next moment, and inputting the result into the next input layer, repeating this process until all layers have been used, and obtaining an output result of the weakly supervised network model, which is the second result; Inputting the second result into a semantic analysis model, analyzing the text information contained therein frame by frame, obtaining field values ​​of text characters, calling a word segmenter to decompose the field values ​​into words and word attributes, matching the words and word attributes with a vocabulary, determining whether the field value is an address, name, or company name, and obtaining a first judgment result; Inputting the second results into the graphic analysis model in parallel, identifying the object information contained in each frame, obtaining key object features, determining whether the frame contains sensitive graphic content, and whether the object information matches the first determination result, thereby obtaining a second determination result; According to the first judgment result and the second judgment result, it is determined whether the frame of the multimedia data stream involves sensitivity. If so, the current multimedia data stream is encrypted and stored separately; otherwise, the multimedia data stream is allowed to be used arbitrarily.

2. The method according to claim 1, wherein: If the multimedia data stream is determined to be sensitive data, its use will be restricted.

3. The method according to claim 1, wherein: The kernels of the semantic analysis model and the graphic analysis model both use neural network models.

4. A semantic analysis and recognition system based on a weakly supervised network, characterized in that: The system includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to implement the method according to any one of claims 1 to 3 according to the instructions in the program code.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the method according to any one of claims 1 to 3.

Images