An opinion image information recognition method based on deep learning

Abstract

The present application relates to a kind of public opinion image information recognition method based on deep learning, comprising the following steps: collecting public opinion image and video and obtaining preprocessed image after the preprocessing of public opinion image and video;Sub-image is obtained after cutting preprocessed image, and the category label of each sub-image is set, then the geometric element contained in each sub-image is extracted, each class of geometric element is named, then the corresponding sentence information is formed based on the naming of all geometric elements in each sub-image, and finally the sentence information and the category label of corresponding sub-image are constructed as training set;Public opinion image information recognition model is constructed, and the training set is used to train the model to obtain the training completed public opinion image information recognition model;Public opinion image information is identified by the trained public opinion image information recognition model.

Description

TECHNICAL FIELD

[0001] The application relates to a public opinion image information recognition method based on deep learning and belongs to the technical field of image recognition. BACKGROUND

[0002] Image recognition has been widely applied in many fields, including autonomous driving, medical image analysis, security monitoring, object detection and recognition, facial recognition, etc. These applications have brought many conveniences to society and accelerated the digital transformation and intelligent development of many industries. However, despite the significant progress made, AI image recognition still faces some challenges. One of them is the problem of insufficient data, deep learning models usually need a large amount of labeled data for training, but in some fields and tasks, it may be very expensive and time-consuming to obtain high-quality labeled data. In addition, overfitting is also an important challenge, that is, the model performs well on the training data, but has poor generalization ability on new unseen data;

[0003] In addition, for some complex scenes and ambiguous images, the existing image recognition technology may still have the problem of low recognition accuracy. At the same time, security and privacy issues also need to be concerned, especially in the field of facial recognition, etc.

[0004] At the same time, the short video industry is in the stage of wild growth, and its information publishing threshold is low, so it leads to the explosive growth of massive content on the platform, therefore, to monitor the negative public opinion information needed from the vast amount of information sea, it is necessary to adopt modern technology means, that is, a professional public opinion monitoring system. According to the video, how to quickly and accurately identify positive and negative information also requires the system platform to have strong computing power.

[0005] The prior art, such as the invention patent with the patent number “CN110837581A”, provides a video public opinion analysis method, device and storage medium, the specific steps are as follows: acquiring a to-be-detected video; extracting key frames, video speech and first text information used to explain the to-be-detected video in the to-be-detected video; performing public opinion analysis on the second text information and determining the first public opinion analysis result of the to-be-detected video, wherein the second text information includes the first text information and the text information extracted from the key frames and the video speech; performing public opinion analysis on the images in the key and determining the second public opinion analysis result of the to-be-detected video; and determining the third public opinion analysis result of the to-be-detected video according to the first public opinion analysis result and the second public opinion analysis result.

[0006] Although the above-mentioned patent provides a video public opinion analysis method, the specific machine learning model used in the scheme is not described, and the recognition efficiency and recognition accuracy of the patent cannot be judged. SUMMARY

[0007] In order to solve the problems existing in the prior art, the application provides a public opinion image information recognition method based on deep learning.

[0008] The technical scheme of the application is as follows:

[0009] In one aspect, the application provides a public opinion image information recognition method based on deep learning, comprising the following steps:

[0010] Collecting public opinion images and videos and pre-processing the public opinion images and videos to obtain pre-processed images;

[0011] Cutting the pre-processed images to obtain sub-images, setting the class labels of each sub-image, extracting the geometric elements contained in each sub-image, naming each type of geometric element, constructing corresponding sentence information based on the names of all geometric elements in each sub-image, and finally constructing the sentence information and the class labels of the corresponding sub-images into a training set;

[0012] Constructing a public opinion image information recognition model, training the model through the training set to obtain a trained public opinion image information recognition model;

[0013] Recognizing public opinion image information through the trained public opinion image information recognition model.

[0014] As a preferred embodiment of the application, each frame image of the public opinion video is obtained by frame acquisition of the public opinion video, and each frame image is extracted as a public opinion image.

[0015] As a preferred embodiment of the application, the pre-processing step of the public opinion image is as follows:

[0016] A filter template of a preset size is set based on a convolutional neural network, for each pixel point of the public opinion image, the filter template and the pixel point and its neighborhood pixel points are convolved to generate a new pixel value, all pixel points of the public opinion image are convolved through the filter, and then the public opinion image is subjected to blurring, sharpening and edge detection operations to obtain a clear public opinion image.

[0017] As a preferred embodiment of the application, the public opinion image information recognition model is constructed based on a long short-term memory network;

[0018] Let the input feature information of the long short-term memory network at time t be x t , and the output result be m t , then the input gate, the forget gate, the output gate and the cell state of the long short-term memory network at time t are as follows:

[0019] i t =σ(W i x t +Wi m t-1 +W i c t-1 +b i )

[0020] f t =σ(W f x t +W f m t-1 +W f c t-1 +b f )

[0021] c t =f t ×c t-1 +i t ×g(W c x t +W f m t-1 +b c )

[0022] o t =σ(W o x t +W o m t-1 +W o c t-1 +b o )

[0023] m t =o t ×h(c t )

[0024] Where: i t W represents the input gate at time t; i b represents the weight matrix of the input gate; i Indicates the input gate bias; f t W represents the forget gate at time t; f The weight matrix representing the forget gate; b f Indicates the bias of the forget gate; c t W represents the cell state at time t; c The weight matrix represents the cell state; g(·) represents the activation function; o t Indicates the output gate; W o b represents the weight matrix of the output gate; o The output gate bias is represented by σ; the sigmoid function is represented by m. t-1 This represents the output result at time t-1.

[0025] In another aspect, the present application also provides a public opinion image information recognition system based on deep learning, comprising a data acquisition module, a training set construction module, a model training module and a public opinion image information recognition module.

[0026] The data acquisition module is configured to acquire public opinion images and videos, and obtain preprocessed images after preprocessing the public opinion images and videos.

[0027] The training set construction module is configured to obtain sub-images after cropping the preprocessed images, set a class label for each sub-image, extract geometric elements contained in each sub-image, name each class of geometric elements, form corresponding sentence information based on the names of all geometric elements in each sub-image, and finally construct the sentence information and the class label of the corresponding sub-image as a training set.

[0028] The model training module is configured to construct a public opinion image information recognition model, and train the model through the training set to obtain a trained public opinion image information recognition model.

[0029] The public opinion image information recognition module is configured to recognize public opinion image information through the trained public opinion image information recognition model.

[0030] As a preferred embodiment of the present application, each frame image of a public opinion video is obtained by frame acquisition of the public opinion video, and each frame image is extracted as a public opinion image.

[0031] As a preferred embodiment of the present application, the preprocessing step of the public opinion image comprises:

[0032] A filter template of a preset size is set based on a convolutional neural network, for each pixel point of the public opinion image, the filter template is convoluted with the pixel point and its neighborhood pixel points to generate a new pixel value, all pixel points of the public opinion image are convoluted through the filter, and then the public opinion image is subjected to blurring, sharpening and edge detection operations to obtain a clear public opinion image.

[0033] As a preferred embodiment of the present application, the public opinion image information recognition model is constructed based on a long short-term memory network.

[0034] Let the input feature information of the long short-term memory network at time t be x t , and the output result be m t , then the input gate, the forget gate, the output gate and the cell state of the long short-term memory network at time t are as follows:

[0035] i t =σ(W i x t +W i m t-1 +Wi c t-1 +b i )

[0036] f t =σ(W f x t +W f m t-1 +W f c t-1 +b f )

[0037] c t =f t ×c t-1 +i t ×g(W c x t +W f m t-1 +b c )

[0038] o t =σ(W o x t +W o m t-1 +W o c t-1 +b o )

[0039] m t =o t ×h(c t )

[0040] Where: i t W represents the input gate at time t; i b represents the weight matrix of the input gate; i Indicates the input gate bias; f t W represents the forget gate at time t; f The weight matrix of the forget gate; b f Indicates the bias of the forget gate; c t W represents the cell state at time t; c The weight matrix represents the cell state; g(·) represents the activation function; o t Indicates the output gate; W o b represents the weight matrix of the output gate; o The output gate bias is represented by σ; the sigmoid function is represented by m. t-1 This represents the output result at time t-1.

[0041] In still another aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable in the processor, wherein the processor implements the method according to any one of the embodiments of the present application when executing the program.

[0042] In still another aspect, the present application provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the method according to any one of the embodiments of the present application.

[0043] The present application has the following beneficial effects:

[0044] 1、The present application combines convolutional neural network and long short-term memory network to identify public opinion image information, first removes picture defects through the convolutional neural network, and then inputs the data after extracting features into the long short-term memory network for training, thereby improving the accuracy and recognition efficiency of image output. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The present application has the following beneficial effects: DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0047] It should be understood that the step numbers used herein are only for the convenience of description, and are not limited to the execution sequence of the steps.

[0048] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0049] The terms "comprise" and "include" indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0050] The term "and / or" means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0051] Embodiment one:

[0052] Referring to Figure 1 A public opinion image information recognition method based on deep learning, comprising the following steps:

[0053] Collecting public opinion images and videos and pre-processing the public opinion images and videos to obtain pre-processed images;

[0054] According to the relationship between the recognition target and the overall scene size of the image, the pre-processed images are cut to obtain M*N size sub-images, and the class labels of each sub-image are set, then the geometric elements contained in each sub-image are extracted, each type of geometric element is named, and finally the sentence information is constructed based on the naming of all geometric elements in each sub-image, and the sentence information and the class labels of the corresponding sub-images are constructed as a training set;

[0055] Constructing a public opinion image information recognition model, training the model through the training set to obtain a trained public opinion image information recognition model;

[0056] Recognizing public opinion image information through the trained public opinion image information recognition model.

[0057] As a preferred embodiment of the present embodiment, the public opinion video is frame-captured to obtain each frame image of the public opinion video, and each frame image is extracted as a public opinion image.

[0058] As a preferred embodiment of the present embodiment, the pre-processing step of the public opinion image is:

[0059] A filter template of a preset size is set based on a convolutional neural network, for each pixel point of the public opinion image, the filter template and the pixel point and its neighborhood pixel points are convolved to generate a new pixel value, all pixel points of the public opinion image are convolved through the filter, and then the public opinion image is subjected to blurring, sharpening and edge detection operations to obtain a clear public opinion image, the convolutional network is used to remove the defect information of the picture, and then the picture is converted back to the original space and differenced with the original picture, thereby obtaining an enhanced image.

[0060] As a preferred embodiment of the present embodiment, the public opinion image information recognition model is constructed based on a long short-term memory network;

[0061] Let the input feature information of the long short-term memory network at time t be x t , and the output result be m t , then the input gate, the forget gate, the output gate and the cell state of the long short-term memory network at time t are as follows:

[0062] i t =σ(W i x t +W i m t-1+W i c t-1 +b i )

[0063] f t =σ(W f x t +W f m t-1 +W f c t-1 +b f )

[0064] c t =f t ×c t-1 +i t ×g(W c x t +W f m t-1 +b c )

[0065] o t =σ(W o x t +W o m t-1 +W o c t-1 +b o )

[0066] m t =o t ×h(c t )

[0067] Where: i t W represents the input gate at time t; i b represents the weight matrix of the input gate; i Indicates the input gate bias; f t W represents the forget gate at time t; f The weight matrix representing the forget gate; b f Indicates the bias of the forget gate; c t W represents the cell state at time t; c The weight matrix represents the cell state; g(·) represents the activation function; o t Indicates the output gate; W o b represents the weight matrix of the output gate; o The output gate bias is represented by σ; the sigmoid function is represented by m. t-1 This represents the output result at time t-1.

[0068] Simulation results show that as interference with the acquired images is reduced, the Long Short-Term Memory (LSTM) network has the best recognition accuracy, which is a high level for a deep neural network.

[0069] Embodiment Two

[0070] A deep learning-based public opinion image information recognition system comprises a data acquisition module, a training set construction module, a model training module, and a public opinion image information recognition module.

[0071] The data acquisition module is configured to acquire public opinion images and videos, and obtain preprocessed images by preprocessing the public opinion images and videos.

[0072] The training set construction module is configured to obtain sub-images by cropping the preprocessed images, set a class label for each sub-image, extract geometric elements contained in each sub-image, name each type of geometric element, form corresponding sentence information based on the names of all geometric elements in each sub-image, and finally construct a training set from the sentence information and the class label of the corresponding sub-image.

[0073] The model training module is configured to construct a public opinion image information recognition model, and train the model using the training set to obtain a trained public opinion image information recognition model.

[0074] The public opinion image information recognition module is configured to recognize public opinion image information using the trained public opinion image information recognition model.

[0075] As a preferred embodiment of the present embodiment, each frame of image of the public opinion video is obtained by frame-by-frame acquisition of the public opinion video, and each frame of image is extracted as a public opinion image.

[0076] As a preferred embodiment of the present embodiment, the preprocessing step of the public opinion image comprises:

[0077] A filter template of a preset size is set based on a convolutional neural network, for each pixel point of the public opinion image, the filter template is convoluted with the pixel point and its neighborhood pixel points to generate a new pixel value, all pixel points of the public opinion image are convoluted by the filter, and the public opinion image is subjected to blurring, sharpening, and edge detection operations to obtain a clear public opinion image.

[0078] As a preferred embodiment of the present embodiment, the public opinion image information recognition model is constructed based on a long short-term memory network.

[0079] Let the input feature information of the long short-term memory network at time t be x t , and the output result be m t , then the input gate, the forget gate, the output gate, and the cell state of the long short-term memory network at time t are as follows:

[0080] i t = σ(W i xt +W i m t-1 +W i c t-1 +b i )

[0081] f t =σ(W f x t +W f m t-1 +W f c t-1 +b f )

[0082] c t =f t ×c t-1 +i t ×g(W c x t +W f m t-1 +b c )

[0083] o t =σ(W o x t +W o m t-1 +W o c t-1 +b o )

[0084] m t =o t ×h(c t )

[0085] Where: i t W represents the input gate at time t; i b represents the weight matrix of the input gate; i Indicates the input gate bias; f t W represents the forget gate at time t; f The weight matrix of the forget gate; b f Indicates the bias of the forget gate; c t W represents the cell state at time t; c The weight matrix represents the cell state; g(·) represents the activation function; o t Indicates the output gate; W o b represents the weight matrix of the output gate; o The output gate bias is represented by σ; the sigmoid function is represented by m. t-1 This represents the output result at time t-1.

[0086] Example 3:

[0087] The embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any one of the embodiments of the present application when executing the program.

[0088] Embodiment four:

[0089] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executable on a processor to implement the method according to any one of the embodiments of the present application.

[0090] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the cases of A alone, A and B together, and B alone. Wherein A and B can be singular or plural. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. "At least one of the following" and the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, wherein a, b, and c can be single or multiple.

[0091] Those skilled in the art can realize that the units and algorithm steps described in the embodiments disclosed in the present application can be realized by electronic hardware, computer software and combination of electronic hardware and computer software. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0093] In several embodiments provided in the present application, any function, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0094] The above description is only some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A deep learning-based public opinion image information recognition method, characterized by, The method comprises the following steps: Collecting public opinion images and videos and pre-processing the public opinion images and videos to obtain pre-processed images; The pre-processing step of the public opinion images is: A filter template of a preset size is set based on a convolutional neural network, for each pixel point of the public opinion image, the filter template is convoluted with the pixel point and its neighborhood pixel points to generate a new pixel value, all pixel points of the public opinion image are convoluted by the filter, and then the public opinion image is subjected to blurring, sharpening and edge detection to obtain a clear public opinion image; The pre-processed images are cut to obtain sub-images, the class labels of each sub-image are set, the geometric elements contained in each sub-image are extracted, each type of geometric element is named, the corresponding sentence information is formed based on the naming of all geometric elements in each sub-image, and finally the sentence information and the class labels of the corresponding sub-images are constructed into a training set; A public opinion image information recognition model is constructed, and the model is trained by the training set to obtain a trained public opinion image information recognition model; The public opinion image information recognition model is constructed based on a long short-term memory network; Let the input feature information of the long short-term memory network at time t be x t , and the output result be m t , then the input gate, the forget gate, the output gate and the cell state of the long short-term memory network at time t are as follows: i t = σ(W i x t + W i m t-1 + W i c t-1 + b i ) f t = σ(W f x t + W f m t-1 + W f c t-1 + b f ) c t = f t × c t-1 + i t × g(W c x t + W f m t-1 + b c ) o t = σ(W o x t + W o m t-1 + W o c t-1 + b o ) m t = o t x h(c t ) wherein: i t denotes the input gate at time t; W i denotes the weight matrix of the input gate; b i denotes the bias of the input gate; f t denotes the forget gate at time t; W f denotes the weight matrix of the forget gate; b f denotes the bias of the forget gate; c t denotes the cell state at time t; W c denotes the weight matrix of the cell state; g(·) denotes an activation function; o t denotes the output gate; W o denotes the weight matrix of the output gate; b o denotes the bias of the output gate; σ denotes a Sigmoid function; m t-1 denotes the output result at time t-1; The trained public opinion image information recognition model is used to recognize public opinion image information. 2.The deep learning-based public opinion image information recognition method according to claim 1, characterized in that, Each frame of image of the public opinion video is obtained by frame-by-frame acquisition of the public opinion video, and each frame of image is extracted as a public opinion image.

3. A public opinion image information recognition system based on deep learning, characterized in that, The system comprises a data acquisition module, a training set construction module, a model training module and a public opinion image information recognition module; The data acquisition module is used to collect public opinion images and videos and pre-process the public opinion images and videos to obtain pre-processed images; The pre-processing step of the public opinion images is: A filter template of a preset size is set based on a convolutional neural network, for each pixel point of the public opinion image, the filter template is convoluted with the pixel point and its neighborhood pixel points to generate a new pixel value, all pixel points of the public opinion image are convoluted by the filter, and then the public opinion image is subjected to blurring, sharpening and edge detection to obtain a clear public opinion image; The training set construction module is used to cut the pre-processed images to obtain sub-images, set the class labels of each sub-image, extract the geometric elements contained in each sub-image, name each type of geometric element, form the corresponding sentence information based on the naming of all geometric elements in each sub-image, and finally construct the sentence information and the class labels of the corresponding sub-images into a training set; The model training module is used to construct a public opinion image information recognition model, and the model is trained by the training set to obtain a trained public opinion image information recognition model; the public opinion image information recognition model is constructed based on a long short-term memory network; Let the input feature information of the long short-term memory network at time t be x t , and the output result be m t , then the input gate, the forget gate, the output gate and the cell state of the long short-term memory network at time t are as follows: i t = σ(W i x t + W i m t-1 + W i c t-1 + b i ) f t = σ(W f x t + W f m t-1 + W f c t-1 + b f ) c t = f t × c t-1 + i t × g(W c x t + W f m t-1 + b c ) o t = σ(W o x t + W o m t-1 + W o c t-1 + b o ) m t = o t x h(c t ) wherein: i t denotes the input gate at time t; W i denotes the weight matrix of the input gate; b i denotes the bias of the input gate; f t denotes the forget gate at time t; W f denotes the weight matrix of the forget gate; b f denotes the bias of the forget gate; c t denotes the cell state at time t; W c denotes the weight matrix of the cell state; g(·) denotes an activation function; o t denotes the output gate; W o denotes the weight matrix of the output gate; b o denotes the bias of the output gate; σ denotes a Sigmoid function; m t-1 denotes the output result at time t-1; The public opinion image information recognition module is used to recognize public opinion image information by the trained public opinion image information recognition model.

4. The public opinion image information recognition system based on deep learning according to claim 3, characterized in that, Each frame of image of the public opinion video is obtained by frame-by-frame acquisition of the public opinion video, and each frame of image is extracted as a public opinion image.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any one of claims 1 to 2.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1 to 2.

Images