Image analysis method and system based on sample selection and semantic mining

Abstract

The present invention provides a method and system for image analysis based on sample selection and semantic mining. By extracting image features and video attributes from network data streams, image features with feature values ​​greater than a threshold are sent to a sample set for clustering, thereby improving the efficiency of image and video review. Elements of different video attributes are clustered separately, and sentiment classification is performed in turn, thereby being able to quickly and accurately determine whether images and videos are compliant.

Description

Technical Field

[0001] The present application relates to the field of network multimedia, and in particular to a method and system for image analysis based on sample selection and semantic mining. Background Art

[0002] With the rapid development of online multimedia, many new demands have emerged. Platform companies need to review massive amounts of images and videos to determine whether they are compliant. Existing technologies mostly rely on manual judgment, which is not only slow but also inefficient. There is an urgent need for machines to perform automated review.

[0003] Therefore, there is an urgent need for a targeted image analysis method and system based on sample selection and semantic mining. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for image analysis based on sample selection and semantic mining. By extracting image features and video attributes from network data streams, image features with eigenvalues ​​greater than a threshold are sent to a sample set for clustering, thereby improving the efficiency of image and video review. Elements of different video attributes are clustered separately, and sentiment classification is performed in turn, so that it is possible to quickly and accurately determine whether images and videos are compliant.

[0005] In a first aspect, the present application provides an image analysis method based on sample selection and semantic mining, the method comprising:

[0006] Obtain network data stream, extract image features and video attributes therein, calculate the characteristic value of the histogram of each frame of image in the gradient direction, compare the characteristic value with a preset threshold, and if it is greater than the threshold, send the image feature corresponding to the characteristic value to the sample set; the video attribute includes three elements: video tag, user identifier and evaluation value;

[0007] Clustering the image features in the sample set according to the three elements of the video attributes to obtain three different cluster samples, and inputting the three cluster samples into the sentiment classification model in sequence;

[0008] The sentiment classification model analyzes the cluster samples item by item according to semantics, determines whether they include specified keywords, sentence meanings, and extracts context features, and determines the sentiment type of the image features according to the context features and the sentence meanings; the sentence meanings refer to the meanings referred to by the sentences contained in the image features, and the context features refer to the scenes corresponding to the image features, which are simulated and inferred based on semantic analysis;

[0009] According to the emotion type determined by the emotion classification model, it is determined whether the emotion type matches the video attribute, and whether the specified keywords carried by the network data stream conform to the reasonable range defined by the emotion type. If so, the network data stream is deemed compliant; otherwise, the network data stream is deemed non-compliant. The non-compliant result is bound to the corresponding user identifier and added to the blacklist for isolation.

[0010] In combination with the first aspect, in a first possible implementation manner of the first aspect, the characteristic value of the histogram in the gradient direction includes detecting a change in a grayscale centroid position of the image.

[0011] In combination with the first aspect, in a second possible implementation manner of the first aspect, a classifier is called to classify the sample set, including an inner product operation between an input vector and the sample set.

[0012] In combination with the first aspect, in a third possible implementation of the first aspect, the kernels of the semantic analysis model and the graphic analysis model both use a neural network model.

[0013] In a second aspect, the present application provides a system for image analysis based on sample selection and semantic mining, the system comprising a processor and a memory:

[0014] The memory is used to store program code and transmit the program code to the processor;

[0015] The processor is used to execute any one of the four possible methods of the first aspect according to the instructions in the program code.

[0016] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute any one of the four possible methods in the first aspect.

[0017] The present invention provides a method and system for image analysis based on sample selection and semantic mining. By extracting image features and video attributes from network data streams, image features with feature values ​​greater than a threshold are sent to a sample set for clustering, thereby improving the efficiency of image and video review. Elements of different video attributes are clustered separately, and sentiment classification is performed in turn, thereby being able to quickly and accurately determine whether images and videos are compliant. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0020] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0021] Figure 1 The flowchart of the method for image analysis based on sample selection and semantic mining provided in this application includes:

[0022] Obtain network data stream, extract image features and video attributes therein, calculate the characteristic value of the histogram of each frame of image in the gradient direction, compare the characteristic value with a preset threshold, and if it is greater than the threshold, send the image feature corresponding to the characteristic value to the sample set; the video attribute includes three elements: video tag, user identifier and evaluation value;

[0023] Clustering the image features in the sample set according to the three elements of the video attributes to obtain three different cluster samples, and inputting the three cluster samples into the sentiment classification model in sequence;

[0024] The sentiment classification model analyzes the cluster samples item by item according to semantics, determines whether they include specified keywords, sentence meanings, and extracts context features, and determines the sentiment type of the image features according to the context features and the sentence meanings; the sentence meanings refer to the meanings referred to by the sentences contained in the image features, and the context features refer to the scenes corresponding to the image features, which are simulated and inferred based on semantic analysis;

[0025] According to the emotion type determined by the emotion classification model, it is determined whether the emotion type matches the video attribute, and whether the specified keywords carried by the network data stream conform to the reasonable range defined by the emotion type. If so, the network data stream is deemed compliant; otherwise, the network data stream is deemed non-compliant. The non-compliant result is bound to the corresponding user identifier and added to the blacklist for isolation.

[0026] In some preferred embodiments, the characteristic value of the histogram in the gradient direction includes detecting changes in the grayscale centroid position of the image.

[0027] In some preferred embodiments, calling a classifier to classify the sample set includes an inner product operation between an input vector and the sample set.

[0028] In some preferred embodiments, the kernels of the semantic analysis model and the graphic analysis model both use neural network models.

[0029] The present application provides a system for image analysis based on sample selection and semantic mining, the system comprising: the system comprising a processor and a memory:

[0030] The memory is used to store program code and transmit the program code to the processor;

[0031] The processor is used to execute the method described in any one of all embodiments of the first aspect according to the instructions in the program code.

[0032] The present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in any one of the embodiments of the first aspect.

[0033] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the present invention. The storage medium may be a disk, an optical disk, a read-only storage memory (abbreviated as: ROM) or a random access memory (abbreviated as: RAM), etc.

[0034] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0035] The same and similar parts between the various embodiments of this specification can be referred to each other. In particular, for the embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0036] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. An image analysis method based on sample selection and semantic mining, characterized in that: The method comprises: Obtain network data stream, extract image features and video attributes therein, calculate the characteristic value of the histogram of each frame of image in the gradient direction, compare the characteristic value with a preset threshold, and if it is greater than the threshold, send the image feature corresponding to the characteristic value to the sample set; the video attribute includes three elements: video tag, user identifier and evaluation value; Clustering the image features in the sample set according to the three elements of the video attributes to obtain three different cluster samples, and inputting the three cluster samples into the sentiment classification model in sequence; The sentiment classification model analyzes the cluster samples item by item according to semantics, determines whether they include specified keywords, sentence meanings, and extracts context features, and determines the sentiment type of the image features according to the context features and the sentence meanings; the sentence meanings refer to the meanings referred to by the sentences contained in the image features, and the context features refer to the scenes corresponding to the image features, which are simulated and inferred based on semantic analysis; According to the emotion type determined by the emotion classification model, it is determined whether the emotion type matches the video attribute, and whether the specified keywords carried by the network data stream conform to the reasonable range defined by the emotion type. If so, the network data stream is deemed compliant; otherwise, the network data stream is deemed non-compliant. The non-compliant result is bound to the corresponding user identifier and added to the blacklist for isolation.

2. The method according to claim 1, characterized in that: The characteristic value of the histogram in the gradient direction includes the change of the grayscale centroid position of the detected image.

3. The method according to any one of claims 1 to 2, characterized in that: A classifier is called to classify the sample set, including an inner product operation between an input vector and the sample set.

4. The method according to any one of claims 1 to 3, characterized in that: The kernels of the semantic analysis model and the graphic analysis model both use neural network models.

5. An image analysis system based on sample selection and semantic mining, characterized in that: The system comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the method according to any one of claims 1 to 4 according to the instructions in the program code.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the method according to any one of claims 1 to 4.

Images