Image recognition method, computer device, and storage medium

By extracting feature data and differential data of multiple information elements, determining the feature type of content information, the problem of low accuracy of malicious image recognition in the prior art is solved, and the accuracy of recognition and the dimension of feature analysis are improved.

CN112580674BActive Publication Date: 2024-11-26ZHEJIANG TMALL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910926845.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-09-27
Publication Date
2024-11-26
Estimated Expiration
2039-09-27

AI Technical Summary

Technical Problem

It is difficult for the prior art to effectively identify hidden advertising information in malicious pictures, especially because the advertising information adopts blurred and hidden methods, resulting in a low recognition accuracy based on a single picture.

Method used

By obtaining multiple information elements in the content information, extracting their characteristic data, and determining the difference between the image information elements and other information elements based on the feature data and the difference data, then determining the feature type of the content information, introducing the difference analysis between pictures , avoid the problem of low accuracy when relying on a single image analysis.

Benefits of technology

The accuracy of determining the feature type of content information is improved, the problem of insufficient information elements is overcome, and the dimension of feature analysis is enhanced by increasing the number of information elements and supplementing related information is improved, and the accuracy of malicious image recognition is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112580674B_ABST
    Figure CN112580674B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a method for image recognition. The method comprises: obtaining content information, the content information comprises a plurality of information elements, the plurality of information elements comprises at least one image information element, extracting feature data of the information element in the content information, determining difference data between the image information element and other information elements according to the feature data, determining the feature type of the content information according to the feature data and the difference data, introducing the analysis of the dimension of difference between images, being able to analyze the feature type from the overall perspective of the content information, avoiding the problem of low accuracy when relying on a single image to analyze features, and improving the accuracy of determining the feature type of the content information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image recognition method, a computer device, and a computer-readable storage medium. Background Art

[0002] With the popularization of mobile Internet, the volume of online transactions is increasing, and the supervision and governance of false reviews and the creation of a good online shopping environment are becoming increasingly urgent. However, professional good and bad reviews and "collecting reviews" have formed a professional gray industry chain.

[0003] In e-commerce transactions, it is normal to evaluate the quality and user experience of the traded goods. However, if the evaluation content does not objectively describe the target goods, but other information is added for the purpose of guiding users to buy other goods, including but not limited to communication accounts, discount information, web links, etc., such advertising evaluation is abnormal and is considered risky evaluation content.

[0004] The applicant has found through research that malicious comments change very quickly, and it is more difficult to identify malicious images than malicious text. For example, images contain various hidden advertising information. Since the advertising information is blurred, hidden, and mutated in the image, the method of identifying advertisements based on a single image such as OCR (Optical Character Recognition) has a low accuracy rate. Summary of the invention

[0005] In view of the above problems, the present application is proposed to provide a picture recognition method, a computer device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] According to one aspect of the present application, a method for image recognition is provided, comprising:

[0007] Acquire content information, where the content information includes multiple information elements, and the multiple information elements include at least one picture information element;

[0008] Extracting characteristic data of information elements in the content information;

[0009] Determine difference data between the picture information element and other information elements according to the feature data;

[0010] The characteristic type of the content information is determined according to the characteristic data and the difference data.

[0011] Optionally, determining the feature type of the content information according to the feature data and the difference data includes:

[0012] Taking the characteristic data of the information element as input, determining the characteristic value of the information element according to the characteristic value recognition model;

[0013] The feature type of the content information is determined according to the feature value and the difference data.

[0014] Optionally, before taking the characteristic data of the information element as input and determining the characteristic value of the information element according to a characteristic value recognition model, the method further comprises:

[0015] The feature value recognition model is trained using information element samples and corresponding marked feature types.

[0016] Optionally, the content information includes a text information element, the feature data includes description information, and determining the difference data between the picture information element and other information elements according to the feature data includes:

[0017] The difference data between the picture information element and the text information element is determined according to the description information of the picture information element and the description information of the text information element.

[0018] Optionally, determining the difference data between the picture information element and other information elements according to the feature data includes:

[0019] The characteristic data between the image information elements are compared to obtain the difference data between the image information elements.

[0020] Optionally, determining the difference data between the picture information element and other information elements according to the feature data includes:

[0021] Clustering multiple image information elements according to the feature data;

[0022] The difference data between the picture clusters is calculated as the difference data between the picture information elements.

[0023] Optionally, before extracting the feature data of the information element in the content information, the method further includes:

[0024] Searching for associated information of the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element;

[0025] The association information is added to the content information.

[0026] Optionally, the associated information includes a picture information element, and before searching for the associated information of the content information, the method further includes:

[0027] It is determined that the number of picture information elements in the content information does not meet a preset requirement.

[0028] Optionally, the content information includes at least one of comment content information and video content information.

[0029] Optionally, when the content information is video content information, before extracting feature data of information elements in the content information, the method further includes:

[0030] A plurality of video frames in the video content information are extracted as the picture information elements.

[0031] According to another aspect of the present application, there is provided an image recognition device, comprising:

[0032] An information acquisition module, used to acquire content information, wherein the content information includes a plurality of information elements, and the plurality of information elements includes at least one picture information element;

[0033] A data extraction module, used to extract characteristic data of information elements in the content information;

[0034] A difference determination module, used to determine difference data between the picture information element and other information elements according to the feature data;

[0035] The type determination module is used to determine the characteristic type of the content information according to the characteristic data and the difference data.

[0036] Optionally, the type determination module includes:

[0037] A feature value determination submodule, used to determine the feature value of the information element according to a feature value recognition model by taking the feature data of the information element as input;

[0038] The type determination submodule is used to determine the characteristic type of the content information according to the characteristic value and the difference data.

[0039] Optionally, the device further comprises:

[0040] The training module is used to train the feature value recognition model using information element samples and corresponding marked feature types before determining the feature value of the information element according to the feature value recognition model using the feature data of the information element as input.

[0041] Optionally, the content information includes a text information element, the feature data includes description information, and the difference determination module includes:

[0042] The difference determination submodule is used to determine the difference data between the picture information element and the text information element according to the description information of the picture information element and the description information of the text information element.

[0043] Optionally, the difference determination module includes:

[0044] The comparison submodule is used to compare the feature data between the image information elements to obtain the difference data between the image information elements.

[0045] Optionally, the difference determination module includes:

[0046] A clustering submodule, used for clustering a plurality of image information elements according to the feature data;

[0047] The calculation submodule is used to calculate the difference data between the picture clusters as the difference data between the picture information elements.

[0048] Optionally, the device further comprises:

[0049] A search module, used for searching for associated information of the content information before extracting the characteristic data of the information element in the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element;

[0050] The adding module is used to add the associated information to the content information.

[0051] Optionally, the association information includes a picture information element, and the device further includes:

[0052] The determination module is used to determine whether the number of picture information elements in the content information meets a preset requirement before searching for the associated information of the content information.

[0053] Optionally, the content information includes at least one of comment content information and video content information.

[0054] Optionally, when the content information is video content information, the device further includes:

[0055] The video frame extraction module is used to extract multiple video frames from the video content information as the picture information elements before extracting the feature data of the information elements from the content information.

[0056] According to another aspect of the present application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements one or more of the above methods when executing the computer program.

[0057] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, one or more of the above methods are implemented.

[0058] According to an embodiment of the present application, content information is obtained, wherein the content information includes multiple information elements, wherein the multiple information elements include at least one picture information element, and feature data of the information elements in the content information are extracted. Based on the feature data, difference data between the picture information element and other information elements are determined. Based on the feature data and the difference data, a feature type of the content information is determined. This introduces an analysis of the dimension of differences between pictures, and is more capable of analyzing feature types from the perspective of the content information as a whole. This avoids the problem of low accuracy when relying on a single picture to analyze features, and improves the accuracy of determining the feature type of the content information.

[0059] Furthermore, by searching for associated information of the content information and adding the associated information to the content information, the acquired content information is supplemented, thereby solving the problem of insufficient information elements in the content information and overcoming the problem that the method cannot be implemented due to insufficient information elements. Alternatively, by increasing the number of information elements, the dimension of feature analysis of the content information can be increased, further improving the accuracy of feature analysis.

[0060] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0062] Figure 1 A schematic diagram showing the image recognition process;

[0063] Figure 2 A flowchart of an embodiment of a method for image recognition according to the first embodiment of the present application is shown;

[0064] Figure 3 A schematic diagram showing the risk identification process of product reviews;

[0065] Figure 4 A flowchart of an embodiment of a method for image recognition according to the second embodiment of the present application is shown;

[0066] Figure 5 A structural block diagram of an image recognition device according to the third embodiment of the present application is shown;

[0067] Figure 6 An exemplary system is shown that can be used to implement the various embodiments described in this disclosure. DETAILED DESCRIPTION

[0068] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0069] In order to enable those skilled in the art to better understand the present application, the concepts involved in the present application are explained below:

[0070] Content information includes information in the form of pictures, text, video, audio, etc. In this application, content information is composed of multiple information elements and the multiple information elements include at least one picture information element. Information elements may include pictures, text, video, audio, etc., or other information elements in any other applicable form, which is not limited in this embodiment of the present application. Content information may have a single form of information element or multiple forms of information elements. There may be one or more of various information elements.

[0071] For example, in e-commerce transactions, product reviews submitted for traded products are a type of content information, which may include image information elements, text information elements, etc., or the content of the question and answer area on the product details page is also a type of content information. Or in video network platforms, videos such as movies and TV series are a type of content information. Videos are composed of video frame sequences. Video frames can be used as image information elements, and the audio of the video can be used as audio information elements.

[0072] Feature data includes various data used to characterize information elements. For example, for image information elements, a space vector model is used to represent each image information element, and each image information element is vectorized, or other any applicable feature data, which is not limited in the embodiments of the present application. The description information of the information element can also be regarded as a kind of feature data. For example, the subject word of the text information element is a kind of description information. The TextCNN (text convolutional neural network) model can be used to obtain the subject of the text. The subject word can describe a certain characteristic of the text, so the subject word can be regarded as a kind of feature data of the text information element. Specifically, any applicable description information can be included, which is not limited in the embodiments of the present application.

[0073] The difference data between information elements is used to characterize the size of the difference between information elements. Specifically, the difference data can be obtained by comparing the characteristic data of the information elements, or by any other applicable method, which is not limited in the embodiments of the present application. The difference data can be the difference data between two information elements, or the difference data between multiple information elements, wherein the difference data between multiple information elements can include the sum or mean of the difference data between two information elements, which is not limited in the embodiments of the present application.

[0074] In the present application, in order to overcome the problem that it is difficult to identify malicious images by relying on a single image information element, the difference data that needs to be determined is between the image information element and other image information elements or information elements in non-image form. The greater the difference between the image information element and other information elements, the greater the possibility that the image information element is a malicious image, and then the greater the possibility that the content information is malicious information. Conversely, the smaller the difference between the image information element and other information elements, the smaller the possibility that the image information element is a malicious image, and then the smaller the possibility that the content information is malicious information.

[0075] For example, for the difference data between image information elements, the image information elements can be represented by vectors. One method is to use cosine similarity calculation to obtain the difference data. By calculating the cosine value of the vectors of two image information elements, the similarity between the two can be evaluated, that is, the difference data between the two is obtained; another method is to use the Pearson correlation coefficient to calculate the difference data. By calculating the quotient of the covariance and standard deviation between the vectors of two image information elements, the correlation between the two can be evaluated, that is, the difference data between the two is obtained. Specifically, any applicable method can be used to obtain the difference data, and the embodiments of the present application are not limited to this.

[0076] Content information can be classified into a certain feature type based on the feature data and difference data of the information elements. The feature type is used to characterize the characteristics of a certain dimension of the content information. For example, for product reviews, the feature types can be divided into advertising reviews and normal reviews, or can be divided into malicious reviews and non-malicious reviews, or can be divided into serious violations, minor violations, suspected violations, no violations, etc.; for videos, the feature types can be divided into replaced ads and non-replaced ads, or can be divided into face replacement or non-face replacement, or can be divided into image replacement or non-image replacement, or can be divided into video replacement or non-video replacement, etc., and can specifically include any applicable feature types, and the embodiments of the present application are not limited to this.

[0077] In an optional embodiment of the present application, in order to determine the characteristic type of content information, the degree of the information element in the content information on the characteristic dimension can be first determined, which is recorded as the characteristic value. The characteristic value of the information element is used to characterize the degree value of the information element on the corresponding characteristic. Taking the characteristic as the degree of malice as an example, in the numerical range of 0-1, the larger the value, the more serious the malice. The malice degree of a certain information element is 0.6, which indicates relatively serious malice. It is understandable that words or symbols can also be used to represent the characteristic value. For example, the malice degree of an information element is ten levels (for example, level 1-level 10 corresponds to the previous numerical range of 0-1).

[0078] To this end, a feature value recognition model can be used to determine, and the feature data of the information element is input into the feature value recognition model, and the model can output the feature value of the information element after calculation. The reason why the feature value of the information element can be output is that the feature value recognition model adopts a supervised learning method, and the model is trained based on a large number of information element samples with marked feature types, so that the feature value recognition model can evaluate the feature value of the information element.

[0079] For example, a large number of image information elements of advertising evaluations and normal evaluations are collected, the image information elements are marked with two feature types of advertising evaluations and normal evaluations, and a binary classification model is trained using a supervised learning method. The binary classification model can classify the image information elements into two feature types of advertising evaluations or normal evaluations. However, the present application does not require the image information elements output by the binary classification model to be classified into a certain feature type, but rather outputs a parameter in the binary classification model that can characterize the degree of the image information element on the feature type as a feature value of the image information element.

[0080] In an optional embodiment of the present application, when the content information includes a text information element, in order to determine the difference data between the image information element and the text information element, it is necessary to first obtain the description information of the information element, and the description information is also a kind of feature data.

[0081] For example, the description information of a text information element can be obtained by using a TextCNN model or a TF-IDF (termfrequency–inverse document frequency) algorithm to obtain the subject words in the text information element as the description information. The description information of an image information element can be obtained by using a Mask R-CNN (Mask Region-based Convolutional Neural Network) model or an image scene recognition model to obtain the main target or scene included in the image information element as the description information. Specifically, any applicable method can be used to obtain the description information of the information element, and the embodiments of the present application are not limited to this.

[0082] In an optional embodiment of the present application, the content information has information associated with it in an application system or a data system, which is recorded as associated information. The associated information includes at least one of the following: a picture information element, a text information element, a video information element, or any other applicable form, which is not limited in the present embodiment of the application.

[0083] For example, for content information such as product reviews, the product targeted by the product reviews will have detailed information about the product on the e-commerce platform, and this detailed information is a kind of associated information of the product reviews. Among them, there are text describing the product, display pictures of the product, and even demonstration videos of the use of the product. When storing product reviews, the product ID (identification) will be stored accordingly. By querying based on the product ID, the detailed information of the product can be found, and the required information elements can be extracted from it. Any applicable search method can be used, and the embodiments of the present application do not limit this.

[0084] In an optional embodiment of the present application, when the content information is video content information, that is, the acquired content information is data in a video format, the video is composed of a sequence of video frames, and the video frame can be regarded as a picture information element.

[0085] For example, the present application scheme can be used in the application scenario of tampering detection. In a film or TV drama, an advertiser inserts an advertised product into a certain scene in the video, but the infringer processes the video and tampers with the advertised product into other products in part of the video. Or it can be used in the application scenario of video advertising detection, that is, to detect whether there are video frames with embedded advertisements in the video. Or it can be used in the application scenario of personal privacy detection, that is, to detect whether there are video frames with content involving personal privacy in the video.

[0086] In one implementation method for the above application scenario: video frames at different times in the scene in the video are obtained, the obtained video frames are used as picture information elements, feature data of the picture information elements are extracted, difference data between the picture information elements are determined, and then based on the feature data and the difference data, it is determined whether the video is a tampered video or a video with embedded advertisements or a video involving personal privacy.

[0087] In another implementation method for the above application scenario: the content information includes the video to be detected and the sample video frames (such as tampered video frames, or video frames with embedded advertisements, or video frames involving personal privacy, etc.), the video frame sequence to be detected is input, tampering or advertisement detection is performed, the feature data of the video frames to be detected and the sample video frames are extracted, the difference data between the video frames to be detected and the sample video frames are determined, and then based on the feature data and the difference data, it is determined whether the video is a tampered video or a video with embedded advertisements or a video involving personal privacy. Among them, the sample video frames can have positive samples and negative samples. For example, the positive sample is a video frame that has not been tampered with, and the negative sample is a tampered video frame; the positive sample is a video frame with embedded advertisements, and the negative sample is a video frame without embedded advertisements.

[0088] According to an embodiment of the present application, when the evaluation content includes malicious images, such as malicious images containing various hidden advertising information, the method of identifying maliciousness based on a single image has a low accuracy rate. Figure 1 The schematic diagram of the picture recognition process shown in the figure, the present application provides a picture recognition mechanism, by obtaining content information, the content information includes multiple information elements, the multiple information elements include at least one picture information element, extracting feature data of the information element in the content information, determining the difference data between the picture information element and other information elements based on the feature data, determining the feature type of the content information based on the feature data and the difference data, introducing the analysis of the dimension of difference between pictures, and being able to analyze the feature type from the overall content information, avoiding the problem of low accuracy when relying on a single picture analysis feature, and improving the accuracy of determining the feature type of content information. The present application is applicable to but not limited to the above application scenarios.

[0089] Reference Figure 2 , shows a flow chart of an embodiment of a picture recognition method according to the first embodiment of the present application, the method may specifically include the following steps:

[0090] Step 101: Acquire content information, where the content information includes multiple information elements, and the multiple information elements include at least one picture information element.

[0091] In an embodiment of the present application, in order to solve the problem of difficulty in identifying malicious images, content information of a feature type to be determined is first obtained, where the content information is composed of multiple information elements and includes at least one image information element.

[0092] For example, in e-commerce transactions, when a user submits a product review for a transaction, before the product review is displayed on the review page, the product review is first obtained from the e-commerce data system for analysis to prevent problematic product reviews from being posted on the review page. Or on a video network platform, after a user uploads a self-made video, before providing the video to other users for viewing, the video is first obtained from the video network platform's data system for analysis. Of course, for video content information, video frames need to be extracted from the video as image information elements.

[0093] Step 102: extracting characteristic data of information elements in the content information.

[0094] In the embodiments of the present application, there are multiple ways to extract the feature data of information elements in the content information. For picture information elements, the picture information elements can be vectorized, and the obtained vectors can be used as the feature data of the picture information elements. If it is necessary to determine the difference data between the picture information elements and the text information elements later, the picture information elements can be subjected to target recognition or scene recognition to obtain descriptive information such as the main target or the scene in the picture as feature data. For text information elements, descriptive information such as the subject words of the text can be identified as feature data. Any applicable extraction method can be used, and the embodiments of the present application do not limit this.

[0095] For example, Figure 3 The schematic diagram of the risk identification process of a product review shown in FIG. 1 includes 1 text information element and 5 image information elements. Each image information element is vectorized to obtain a vector corresponding to each image information element as feature data of the image information element. The subject words of the text information element are identified as feature data of the text information element.

[0096] Step 103: Determine difference data between the picture information element and other information elements according to the feature data.

[0097] In an embodiment of the present application, the difference data between the picture information element and the other information elements can be determined based on the characteristic data of the information element. When the content information includes multiple picture information elements, the picture information element and the other information elements include the picture information element and other picture information elements, and may also include the picture information element and other information elements in non-picture form. When the content information includes one picture information element, the picture information element and the other information elements include the picture information element and other information elements in non-picture form.

[0098] In an embodiment of the present application, there may be multiple ways to implement determining the difference data between the picture information element and other information elements, for example, comparing feature data between the picture information elements to obtain the difference data between the picture information elements; or clustering multiple picture information elements based on the feature data, calculating the difference data between the picture clusters as the difference data between the picture information elements; or determining the difference data between the picture information element and the text information element based on the description information of the picture information element and the description information of the text information element, or any other applicable implementation method, which is not limited by the embodiment of the present application.

[0099] For example, Figure 3 As shown, the five image information elements are compared with each other, and the difference degree values ​​between the five image information elements, that is, the difference data, are obtained by comparing the feature data corresponding to the image information elements. The five difference data obtained can be used directly, or one or more difference data representing the largest difference among the five difference data can be selected for use, or the average of the five difference data can be calculated to obtain a total difference data for use, or any other applicable method can be used, and the embodiments of the present application are not limited to this.

[0100] Step 104: Determine the feature type of the content information according to the feature data and the difference data.

[0101] In the embodiment of the present application, the content information can be classified into a certain feature type according to the feature data of the information element and the difference data obtained in the previous step. Specifically, it can be determined according to the feature data of all the information elements in the content information and the difference data obtained in the previous step, or it can be determined according to the feature data of some of the information elements in the content information and the difference data obtained in the previous step, and the embodiment of the present application does not limit this.

[0102] In an embodiment of the present application, there may be multiple ways to implement determining the feature type of the content information, for example, taking the feature data of the information element as input, determining the feature value of the information element according to a feature value recognition model, determining the feature type of the content information according to the feature value and difference data, or any other applicable implementation method, which is not limited in the embodiment of the present application.

[0103] For example, Figure 3 As shown, the characteristic data of a text information element in the product review, namely the keyword, is input into the risk value recognition model for the text to obtain the risk value of the text information element, and then a weighted average calculation is performed based on the risk value and the difference degree value obtained in the previous step to obtain a comprehensive risk value. If the comprehensive risk value exceeds a predetermined threshold, it is determined that the product review belongs to a risky feature type.

[0104] According to an embodiment of the present application, content information is obtained, wherein the content information includes multiple information elements, wherein the multiple information elements include at least one picture information element, and feature data of the information elements in the content information are extracted. Based on the feature data, difference data between the picture information element and other information elements are determined. Based on the feature data and the difference data, a feature type of the content information is determined. This introduces an analysis of the dimension of differences between pictures, and is more capable of analyzing feature types from the perspective of the content information as a whole. This avoids the problem of low accuracy when relying on a single picture to analyze features, and improves the accuracy of determining the feature type of the content information.

[0105] Reference Figure 4 , shows a flow chart of an embodiment of a method for image recognition according to the second embodiment of the present application, the method may specifically include the following steps:

[0106] Step 201: Acquire content information, where the content information includes multiple information elements, and the multiple information elements include at least one picture information element.

[0107] In the embodiment of the present application, the specific implementation method of this step can be found in the description in the aforementioned embodiment and will not be further described here.

[0108] In the embodiment of the present application, optionally, the content information includes at least one of comment content information and video content information. For example, in an e-commerce transaction, a product review submitted by a user for a product belongs to a type of comment content information. In a video network platform, a homemade video uploaded by a user belongs to a type of video content information. The specific comment content information or video content information can be arbitrarily applicable, and the embodiment of the present application does not limit this.

[0109] Step 202: searching for associated information of the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element.

[0110] In the embodiment of the present application, before extracting the characteristic data of the information element in the content information, the associated information of the content information can also be searched. For example, for a product review, the detailed information of the product targeted by the product review is searched, and the pictures in the detailed information are determined as the associated information. The specific search can be performed according to actual needs, and the embodiment of the present application does not limit this.

[0111] In an embodiment of the present application, optionally, in some implementation scenarios, difference data between picture information elements is required, and the associated information to be searched includes picture information elements. Before searching for the associated information of the content information, it may also include: determining that the number of picture information elements in the content information does not meet the preset requirement. The preset required number may be set according to actual needs, and the embodiment of the present application does not limit this.

[0112] For example, a product review includes only one image information element, while the preset requirement is 2. When it is determined that the number of image information elements in the product review does not meet the preset requirement, it indicates that the content information contains insufficient image information elements. Images in the product details are searched for as related information based on the product review and added to the content information.

[0113] Step 203: Add the associated information to the content information.

[0114] In an embodiment of the present application, the found related information is added to the content information to supplement the acquired content information, thereby solving the problem of insufficient information elements in the content information and overcoming the problem that the method cannot be implemented due to insufficient information elements. Alternatively, by increasing the number of information elements, the dimension of feature analysis of the content information can be increased, further improving the accuracy of feature analysis.

[0115] Step 204: extracting characteristic data of information elements in the content information.

[0116] In the embodiment of the present application, the specific implementation method of this step can be found in the description in the aforementioned embodiment and will not be further described here.

[0117] Step 205: clustering multiple image information elements according to the feature data.

[0118] In the embodiment of the present application, when calculating the difference data between multiple picture information elements, the multiple picture information elements can be clustered. The process of dividing the multiple picture information elements into multiple classes composed of similar picture information elements is called clustering.

[0119] In the embodiment of the present application, clustering the feature data of multiple image information elements means clustering multiple image information elements. The cluster generated by clustering is a set of image information elements, which are similar to image information elements in the same cluster and different from image information elements in other clusters. The image information elements of a cluster are recorded as an image cluster. For example, clustering is performed on 5 image information elements in a product review, of which 3 image information elements belong to one image cluster and the other 2 image information elements belong to another image cluster.

[0120] Step 206: Calculate the difference data between the picture clusters as the difference data between the picture information elements.

[0121] In an embodiment of the present application, after clustering, the difference data between the image clusters can be calculated as the difference data between the image information elements. There are many ways to calculate the difference data between the image clusters. For example, the difference data between each image information element in one image cluster and each image information element in another image cluster is calculated to obtain a plurality of difference data, and then the average value of the plurality of difference data is calculated to obtain the difference data between the two image clusters. Specifically, any applicable calculation method may be included, and the embodiment of the present application does not limit this.

[0122] In an embodiment of the present application, optionally, the content information includes a text information element, the feature data includes description information, and an implementation method for determining the difference data between the image information element and other information elements based on the feature data may include: determining the difference data between the image information element and the text information element based on the description information of the image information element and the description information of the text information element.

[0123] By comparing the description information of the image information element with the description information of the text information element, the difference data between the image information element and the text information element can be obtained. The difference between the description information can represent the difference between the image information element and the text information element. For example, by performing text vectorization on two description information and calculating the distance between the vectors corresponding to the two description information, the difference data can be obtained. If the description information of the image information element is "socks", the difference data obtained when the description information of the text information element is "cotton" will be smaller than the difference data obtained when the description information of the text information element is "plastic".

[0124] Step 207: Taking the characteristic data of the information element as input, determining the characteristic value of the information element according to the characteristic value recognition model.

[0125] In the embodiment of the present application, when the characteristic value recognition model is used to determine the characteristic value of the information element, the input data is the characteristic data of the information element, and the characteristic value recognition model can output the characteristic value of the information element after calculation.

[0126] In the embodiment of the present application, optionally, before determining the characteristic value of the information element according to the characteristic value recognition model using the characteristic data of the information element as input, it may also include: using information element samples and corresponding marked characteristic types to train the characteristic value recognition model.

[0127] The feature value recognition model needs to be trained to accurately identify the feature values ​​of information elements. Collect a large number of information element samples, mark the information element samples, mark the feature type corresponding to each information element, input the information element samples into the feature value recognition model, and the parameters in the feature value recognition model are continuously learned and updated until the required performance is achieved.

[0128] Step 208: Determine the feature type of the content information according to the feature value and the difference data.

[0129] In the embodiment of the present application, after the feature value is obtained by using the feature value recognition model, the feature value is integrated with the difference data to determine the feature type of the content information. For example, after the feature value and the difference data are weighted averaged to obtain a comprehensive value, if the comprehensive value exceeds a preset threshold, the content information is determined to be a feature type, and if the comprehensive value does not exceed the preset threshold, the content information is determined to be another feature type. Any applicable implementation method can be used, and the embodiment of the present application does not limit this.

[0130] According to an embodiment of the present application, content information is obtained, the content information includes multiple information elements, the multiple information elements include at least one picture information element, feature data of the information elements in the content information are extracted, multiple picture information elements are clustered according to the feature data, difference data between picture clusters are calculated as the difference data between the picture information elements, the feature data of the information elements are used as input, feature values ​​of the information elements are determined according to a feature value recognition model, and feature types of the content information are determined according to the feature values ​​and the difference data, thereby introducing analysis of the dimension of differences between pictures, and being able to better analyze feature types from the overall perspective of the content information, thereby avoiding the problem of low accuracy when relying on a single picture to analyze features, and improving the accuracy of determining the feature types of the content information.

[0131] Furthermore, by searching for associated information of the content information and adding the associated information to the content information, the acquired content information is supplemented, thereby solving the problem of insufficient information elements in the content information and overcoming the problem that the method cannot be implemented due to insufficient information elements. Alternatively, by increasing the number of information elements, the dimension of feature analysis of the content information can be increased, further improving the accuracy of feature analysis.

[0132] Reference Figure 5 , shows a structural block diagram of an embodiment of a picture recognition device according to the third embodiment of the present application, which may specifically include:

[0133] An information acquisition module 301 is used to acquire content information, where the content information includes multiple information elements, and the multiple information elements include at least one picture information element;

[0134] A data extraction module 302 is used to extract characteristic data of information elements in the content information;

[0135] A difference determination module 303, used to determine difference data between the picture information element and other information elements according to the feature data;

[0136] The type determination module 304 is configured to determine the characteristic type of the content information according to the characteristic data and the difference data.

[0137] In the embodiment of the present application, optionally, the type determination module includes:

[0138] A feature value determination submodule, used to determine the feature value of the information element according to a feature value recognition model by taking the feature data of the information element as input;

[0139] The type determination submodule is used to determine the characteristic type of the content information according to the characteristic value and the difference data.

[0140] In the embodiment of the present application, optionally, the device further includes:

[0141] The training module is used to train the feature value recognition model using information element samples and corresponding marked feature types before determining the feature value of the information element according to the feature value recognition model using the feature data of the information element as input.

[0142] In the embodiment of the present application, optionally, the content information includes a text information element, the feature data includes description information, and the difference determination module includes:

[0143] The difference determination submodule is used to determine the difference data between the picture information element and the text information element according to the description information of the picture information element and the description information of the text information element.

[0144] In the embodiment of the present application, optionally, the difference determination module includes:

[0145] The comparison submodule is used to compare the feature data between the image information elements to obtain the difference data between the image information elements.

[0146] In the embodiment of the present application, optionally, the difference determination module includes:

[0147] A clustering submodule, used for clustering a plurality of image information elements according to the feature data;

[0148] The calculation submodule is used to calculate the difference data between the picture clusters as the difference data between the picture information elements.

[0149] In the embodiment of the present application, optionally, the device further includes:

[0150] A search module, used for searching for associated information of the content information before extracting the characteristic data of the information element in the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element;

[0151] The adding module is used to add the associated information to the content information.

[0152] In the embodiment of the present application, optionally, the associated information includes a picture information element, and the device further includes:

[0153] The determination module is used to determine whether the number of picture information elements in the content information meets a preset requirement before searching for the associated information of the content information.

[0154] In the embodiment of the present application, optionally, the content information includes at least one of comment content information and video content information.

[0155] In the embodiment of the present application, optionally, when the content information is video content information, the device further includes:

[0156] The video frame extraction module is used to extract multiple video frames from the video content information as the picture information elements before extracting the feature data of the information elements from the content information.

[0157] According to an embodiment of the present application, content information is obtained, wherein the content information includes multiple information elements, wherein the multiple information elements include at least one picture information element, and feature data of the information elements in the content information are extracted. Based on the feature data, difference data between the picture information element and other information elements are determined. Based on the feature data and the difference data, a feature type of the content information is determined. This introduces an analysis of the dimension of differences between pictures, and is more capable of analyzing feature types from the perspective of the content information as a whole. This avoids the problem of low accuracy when relying on a single picture to analyze features, and improves the accuracy of determining the feature type of the content information.

[0158] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] Embodiments of the present disclosure may be implemented as a system using any suitable hardware, firmware, software, or any combination thereof to perform the desired configuration. Figure 6 An exemplary system (or device) 700 that can be used to implement various embodiments described in the present disclosure is schematically illustrated.

[0160] For one embodiment, Figure 6 An exemplary system 700 is shown having one or more processors 702, a system control module (chip set) 704 coupled to at least one of the processor(s) 702, a system memory 706 coupled to the system control module 704, a non-volatile memory (NVM) / storage device 708 coupled to the system control module 704, one or more input / output devices 710 coupled to the system control module 704, and a network interface 712 coupled to the system control module 706.

[0161] Processor 702 may include one or more single-core or multi-core processors, and processor 702 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.) In some embodiments, system 700 can be used as a browser as described in the embodiments of the present application.

[0162] In some embodiments, the system 700 may include one or more computer-readable media (e.g., system memory 706 or NVM / storage device 708) having instructions and one or more processors 702 that are combined with the one or more computer-readable media and configured to execute the instructions to implement the modules to perform the actions described in the present disclosure.

[0163] For one embodiment, the system control module 704 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 702 and / or any suitable device or component in communication with the system control module 704 .

[0164] The system control module 704 may include a memory controller module to provide an interface to the system memory 706. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0165] The system memory 706 may be used, for example, to load and store data and / or instructions for the system 700. For one embodiment, the system memory 706 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the system memory 706 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0166] For one embodiment, the system control module 704 may include one or more input / output controllers to provide interfaces to the NVM / storage device 708 and the input / output device(s) 710 .

[0167] For example, NVM / storage device 708 may be used to store data and / or instructions. NVM / storage device 708 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0168] NVM / storage device 708 may include storage resources that are physically part of the device on which system 700 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 708 may be accessed via input / output device(s) 710 over a network.

[0169] The input / output device(s) 710 may provide an interface for the system 700 to communicate with any other appropriate device, and the input / output device 710 may include a communication component, an audio component, a sensor component, etc. The network interface 712 may provide an interface for the system 700 to communicate through one or more networks, and the system 700 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof for wireless communication.

[0170] For one embodiment, at least one of the processor(s) 702 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the system control module 704. For one embodiment, at least one of the processor(s) 702 may be packaged together with the logic of one or more controllers of the system control module 704 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 702 may be integrated on the same die with the logic of one or more controllers of the system control module 704. For one embodiment, at least one of the processor(s) 702 may be integrated on the same die with the logic of one or more controllers of the system control module 704 to form a system on chip (SoC).

[0171] In various embodiments, the system 700 may be, but is not limited to: a browser, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the system 700 may have more or fewer components and / or a different architecture. For example, in some embodiments, the system 700 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application specific integrated circuit (ASIC), and a speaker.

[0172] Wherein, if the display includes a touch panel, the display screen can be implemented as a touch screen display to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0173] An embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a terminal device, the terminal device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0174] In one example, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method as in an embodiment of the present application when executing the computer program.

[0175] In one example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, one or more methods of the embodiments of the present application are implemented.

[0176] The embodiments of the present application disclose a method and device for image recognition. Example 1 includes an image recognition method, including:

[0177] Acquire content information, where the content information includes multiple information elements, and the multiple information elements include at least one picture information element;

[0178] Extracting characteristic data of information elements in the content information;

[0179] Determine difference data between the picture information element and other information elements according to the feature data;

[0180] The characteristic type of the content information is determined according to the characteristic data and the difference data.

[0181] Example 2 may include the method of Example 1, wherein determining the feature type of the content information according to the feature data and the difference data includes:

[0182] Taking the characteristic data of the information element as input, determining the characteristic value of the information element according to the characteristic value recognition model;

[0183] The feature type of the content information is determined according to the feature value and the difference data.

[0184] Example 3 may include the method described in Example 1 and / or Example 2, wherein, before determining the feature value of the information element according to the feature value recognition model using the feature data of the information element as input, the method further includes:

[0185] The feature value recognition model is trained using information element samples and corresponding marked feature types.

[0186] Example 4 may include the method described in one or more of Examples 1 to 3, wherein the content information includes a text information element, the feature data includes description information, and determining the difference data between the picture information element and the other information elements based on the feature data includes:

[0187] The difference data between the picture information element and the text information element is determined according to the description information of the picture information element and the description information of the text information element.

[0188] Example 5 may include the method described in one or more of Examples 1 to 4, wherein determining the difference data between the picture information element and other information elements according to the feature data includes:

[0189] The characteristic data between the image information elements are compared to obtain the difference data between the image information elements.

[0190] Example 6 may include the method described in one or more of Examples 1 to 5, wherein determining the difference data between the picture information element and other information elements according to the feature data comprises:

[0191] Clustering multiple image information elements according to the feature data;

[0192] The difference data between the picture clusters is calculated as the difference data between the picture information elements.

[0193] Example 7 may include the method of one or more of Examples 1 to 6, wherein, before extracting the feature data of the information element in the content information, the method further includes:

[0194] Searching for associated information of the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element;

[0195] The association information is added to the content information.

[0196] Example 8 may include the method described in one or more of Examples 1 to 7, wherein the associated information includes a picture information element, and before searching for the associated information of the content information, the method further includes:

[0197] It is determined that the number of picture information elements in the content information does not meet a preset requirement.

[0198] Example 9 may include the methods described in one or more of Examples 1-8, wherein the content information includes at least one of comment content information and video content information.

[0199] Example 10 may include the method described in one or more of Examples 1 to 6, wherein when the content information is video content information, before extracting feature data of information elements in the content information, the method further includes:

[0200] A plurality of video frames in the video content information are extracted as the picture information elements.

[0201] Example 11 includes an image recognition device, comprising:

[0202] An information acquisition module, used to acquire content information, wherein the content information includes a plurality of information elements, and the plurality of information elements includes at least one picture information element;

[0203] A data extraction module, used to extract characteristic data of information elements in the content information;

[0204] A difference determination module, used to determine difference data between the picture information element and other information elements according to the feature data;

[0205] The type determination module is used to determine the characteristic type of the content information according to the characteristic data and the difference data.

[0206] Example 12 may include the apparatus of Example 11, wherein the type determination module includes:

[0207] A feature value determination submodule, used to determine the feature value of the information element according to a feature value recognition model by taking the feature data of the information element as input;

[0208] The type determination submodule is used to determine the characteristic type of the content information according to the characteristic value and the difference data.

[0209] Example 13 may include the apparatus of Example 11 and / or Example 12, wherein the apparatus further comprises:

[0210] The training module is used to train the feature value recognition model using information element samples and corresponding marked feature types before determining the feature value of the information element according to the feature value recognition model using the feature data of the information element as input.

[0211] Example 14 may include the apparatus of one or more of Examples 11 to 13, wherein the content information includes a text information element, the feature data includes description information, and the difference determination module includes:

[0212] The difference determination submodule is used to determine the difference data between the picture information element and the text information element according to the description information of the picture information element and the description information of the text information element.

[0213] Example 15 may include the apparatus of one or more of Examples 11-14, wherein the difference determination module includes:

[0214] The comparison submodule is used to compare the feature data between the image information elements to obtain the difference data between the image information elements.

[0215] Example 16 may include the apparatus of one or more of Examples 11-15, wherein the difference determination module includes:

[0216] A clustering submodule, used for clustering a plurality of image information elements according to the feature data;

[0217] The calculation submodule is used to calculate the difference data between the picture clusters as the difference data between the picture information elements.

[0218] Example 17 may include the apparatus of one or more of Examples 11-16, wherein the apparatus further comprises:

[0219] A search module, used for searching for associated information of the content information before extracting the characteristic data of the information element in the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element;

[0220] The adding module is used to add the associated information to the content information.

[0221] Example 18 may include the apparatus of one or more of Examples 11 to 17, wherein the associated information includes a picture information element, and the apparatus further includes:

[0222] The determination module is used to determine whether the number of picture information elements in the content information meets a preset requirement before searching for the associated information of the content information.

[0223] Example 19 may include the device described in one or more of Examples 11 to 18, wherein the content information includes at least one of comment content information and video content information.

[0224] Example 20 may include the apparatus described in one or more of Examples 11 to 19, wherein when the content information is video content information, the apparatus further comprises:

[0225] The video frame extraction module is used to extract multiple video frames from the video content information as the picture information elements before extracting the feature data of the information elements from the content information.

[0226] Example 21 includes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, one or more methods of Examples 1-10 are implemented.

[0227] Example 22 includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements one or more methods of Examples 1-10.

[0228] Although certain embodiments are for the purpose of illustration and description, various alternatives, and / or, equivalent implementations, or calculations to achieve the same purpose of the embodiment shown and described implementations do not depart from the scope of the present application. This application is intended to cover any modifications or changes of the embodiments discussed herein. Therefore, it is apparent that the embodiments described herein are limited only by the claims and their equivalents.

Claims

1. A method for image recognition, characterized in that: include: Acquire content information, where the content information includes multiple information elements, and the multiple information elements include at least one picture information element; Extracting characteristic data of information elements in the content information; Determine, based on the feature data, difference data between the picture information element and other information elements in the content information; wherein the content information includes at least one of comment content information and video content information; The content information includes a text information element, the feature data includes description information, and determining the difference data between the picture information element and other information elements based on the feature data includes: Determine difference data between the picture information element and the text information element according to the description information of the picture information element and the description information of the text information element; Wherein, determining the difference data between the picture information element and other information elements according to the feature data includes: Comparing the feature data between the image information elements to obtain difference data between the image information elements; The characteristic type of the content information is determined according to the characteristic data and the difference data.

2. The method according to claim 1, characterized in that: Determining the feature type of the content information according to the feature data and the difference data includes: Taking the characteristic data of the information element as input, determining the characteristic value of the information element according to the characteristic value recognition model; The feature type of the content information is determined according to the feature value and the difference data.

3. The method according to claim 2, characterized in that Before determining the characteristic value of the information element according to the characteristic value recognition model using the characteristic data of the information element as input, the method further includes: The feature value recognition model is trained using information element samples and corresponding marked feature types.

4. The method according to claim 1, characterized in that: The determining, based on the feature data, difference data between the picture information element and other information elements further comprises: Clustering multiple image information elements according to the feature data; The difference data between the picture clusters is calculated as the difference data between the picture information elements.

5. The method according to claim 1, characterized in that Before extracting the characteristic data of the information element in the content information, the method further includes: Searching for associated information of the content information, wherein the associated information includes at least one of the following: a picture information element, a text information element, and a video information element; The association information is added to the content information.

6. The method according to claim 5, characterized in that The associated information includes a picture information element. Before searching for the associated information of the content information, the method further includes: It is determined that the number of picture information elements in the content information does not meet a preset requirement.

7. The method according to claim 1, characterized in that When the content information is video content information, before extracting feature data of information elements in the content information, the method further includes: A plurality of video frames in the video content information are extracted as the picture information elements.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.