Public opinion event entity relationship identification method and device, equipment and medium

Abstract

The invention provides a public opinion event entity relationship identification method and device, equipment and a medium. The method comprises the steps of obtaining first public opinion data to be processed, wherein the first public opinion data comprises matched first image data and first text data; determining an entity set related to a public opinion event from the first text data according to the first image feature of the first image data and the first text feature of the first text data; determining entity features of a plurality of entities according to the entity set and the position association relationship of the plurality of entities in the entity set in the first text data, and determining a causal relationship among the entities according to the entity features of the plurality of entities, so that the multi-modal information of the image and the text is fused for entity recognition, and the recognition efficiency is improved. And in combination with the position association relationship of the entity, the feature representation of the entity is enhanced, the accuracy of identifying the causal relationship of the entity is improved, and the efficiency of analyzing public opinion data is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of public opinion analysis, and in particular to a method, apparatus, device, and medium for identifying entity relationships of public opinion events. Background Art

[0002] Agricultural product safety has become a hot topic of widespread social concern. Agricultural product public opinion, as the public's response and discussion to agricultural product safety incidents, will affect consumer trust and corporate reputation. However, agricultural product public opinion analysis usually involves complex and diverse data, and the evolution and impact of public opinion events are also dynamic. How to effectively conduct public opinion analysis and identify causal relationships is very important and challenging. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, device and medium for identifying entity relationships of public opinion events.

[0004] In a first aspect, the present disclosure provides a method for identifying entity relationships of public opinion events, the method comprising:

[0005] Acquire first public opinion data to be processed, where the first public opinion data includes matched first image data and first text data;

[0006] Determining an entity set related to the public opinion event from the first text data based on a first image feature of the first image data and a first text feature of the first text data;

[0007] According to the entity set and the position association relationship between multiple entities in the entity set in the first text data, entity features of multiple entities are determined, and according to the entity features of multiple entities, causal relationships between entities are determined.

[0008] In a second aspect, the present disclosure provides a device for identifying entity relationships of public opinion events, the device comprising:

[0009] A preprocessing module, configured to obtain first public opinion data to be processed, wherein the first public opinion data includes matched first image data and first text data;

[0010] A first determining module, configured to determine an entity set related to a public opinion event from the first text data based on a first image feature of the first image data and a first text feature of the first text data;

[0011] The second determination module is used to determine the entity features of multiple entities based on the entity set and the position association relationship between multiple entities in the entity set in the first text data, and determine the causal relationship between the entities based on the entity features of the multiple entities.

[0012] In a third aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned entity relationship identification method for public opinion events.

[0013] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned entity relationship identification method for public opinion events when executed by a processor.

[0014] The entity relationship identification method for public opinion events provided by the embodiment of the present disclosure obtains first public opinion data including matched first image data and first text data, and can fuse the first image features of the first image data and the first text features of the first text data to determine the entity set, thereby improving the information richness and the accuracy of entity identification, and based on the positional association relationship of the entities in the entity set in the first text data, determines the entity features and enhances the entity feature representation, thereby performing causal reasoning based on the enhanced entity features, determining the causal relationship between entities, and improving the efficiency and accuracy of entity causal relationship identification, thereby providing a better basis for monitoring public opinion events based on the causal relationship between entities.

[0015] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0017] Figure 1 An application scenario diagram provided for an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of a method for identifying entity relationships of public opinion events provided by an embodiment of the present disclosure;

[0019] Figure 3 This is a principle block diagram of the text generated by the diagram in the embodiment of the present disclosure;

[0020] Figure 4A block diagram of the principle of determining an entity set in an embodiment of the present disclosure;

[0021] Figure 5 A block diagram of the principle of determining entity causal relationships in an embodiment of the present disclosure;

[0022] Figure 6 A block diagram of an entity relationship identification device for public opinion events provided by an embodiment of the present disclosure;

[0023] Figure 7 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] To enable those skilled in the art to better understand the technical solutions of the present disclosure, exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0026] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0027] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0028] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0029] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution complies with relevant national laws and regulations (for example, the "Information Security Technology Personal Information Security Specification", etc.). For example: corresponding prescribed measures are taken to control access to personal information; the display of personal information is subject to prescribed restrictions; the purpose of using personal information does not exceed the scope of direct or reasonable connection; when using personal information, clear identity reference is eliminated to avoid precise positioning of specific individuals.

[0030] In public opinion analysis scenarios, such as agricultural product public opinion and food safety public opinion, the data involved is usually complex, diverse and dynamically changing. How to effectively conduct public opinion analysis and identify causal relationships is very important and challenging.

[0031] In an embodiment of the present disclosure, a method for identifying entity relationships of public opinion events is provided, which obtains first public opinion data, the first public opinion data including matched first image data and first text data, and combines the first image data and the first text data to determine an entity set related to the public opinion event from the first text data. In this way, the multimodal information of the image and text is integrated, which improves the information richness and the accuracy of entity recognition. Then, entity features can be determined based on the entity set and the positional association relationship of the entities in the entity set in the first text data. Based on the entity features, the causal relationship between the entities is determined. In this way, feature fusion is performed through the positional association relationship, and the entity feature representation is enhanced. Therefore, entity causal reasoning is performed based on the enhanced entity feature representation, which not only improves the efficiency of entity causal relationship recognition, but also improves the accuracy of entity causal relationship recognition in public opinion data, thereby providing a basis for monitoring public opinion events.

[0032] Of course, in the embodiments of the present disclosure, there is no limitation on the application fields of public opinion analysis, and all fields are applicable.

[0033] Figure 1 The application scenario diagram of the entity relationship identification method and device for public opinion events provided by the embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, an application scenario of an embodiment of the present disclosure may include a terminal device 101, a network 103, and a server 102. The network 103 is used as a medium for providing a communication link between the terminal device 101 and the server 102. The network 103 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0035] The user can use the terminal device 101 to interact with the server 102 via the network 103 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0036] The terminal device 101 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0037] The server 102 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the terminal device 101. The background management server may analyze and process received user requests and other data, and feed back the processing results (e.g., web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0038] It should be noted that the entity relationship identification method and apparatus for public opinion events provided by the embodiments of the present disclosure can be executed by the server 102. Accordingly, the entity relationship identification method and apparatus for public opinion events provided by the embodiments of the present disclosure can be set in the server 102. The entity relationship identification method and apparatus for public opinion events provided by the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102. Accordingly, the entity relationship identification method and apparatus for public opinion events provided by the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0040] Figure 2 This is a flow chart of a method for identifying entity relationships of public opinion events provided by an embodiment of the present disclosure. Figure 2 , the method comprising:

[0041] S210: Acquire first public opinion data to be processed, where the first public opinion data includes matched first image data and first text data.

[0042] In the disclosed embodiment, when conducting public opinion analysis, relevant public opinion data can be collected first. In order to ensure the comprehensiveness and diversity of the data, web crawler technology can be used to crawl public opinion data from multiple network platforms by setting keywords. Usually, the presentation of network information includes images and text forms, and text data and its corresponding image data can be obtained.

[0043] S220: Determine an entity set related to the public opinion event from the first text data based on the first image feature of the first image data and the first text feature of the first text data.

[0044] In the embodiments of the present disclosure, text data usually contains a lot of descriptions about public opinion events, such as the background, participants, time, place, cause, result and other information of the public opinion events. This information can be structured to help understand the development of the public opinion events. Among them, entities can represent these structured information or keywords of the text data. For example, the first text data is "The sales of a certain brand of agricultural products dropped sharply after being complained about." The entities in the first text data can be: "a certain brand", "agricultural products", "complaints", "sales", "sudden drop", etc.

[0045] S230: Determine entity features of the plurality of entities based on the entity set and position association relationships of the plurality of entities in the entity set in the first text data, and determine causal relationships between the entities based on the entity features of the plurality of entities.

[0046] In the disclosed embodiments, entity characteristics are determined and causal reasoning is performed through the location association relationship of entities. For example, through analysis, it is determined that there is a causal relationship between the entities "complaints" and "sales", which can be used for subsequent public opinion event analysis and causal judgment. For example, if there is an event in which sales of a certain product decreases, by determining the causal relationship between "complaints" and "sales", it can be analyzed that the sales decrease may be due to complaints, and then the public opinion event can be quickly responded to and handled, thereby improving the speed of handling public opinion events.

[0047] In an embodiment of the present disclosure, first public opinion data including matched first image data and first text data is obtained, and a set of entities related to the public opinion event is determined from the first text data based on the first image features of the first image data and the first text features of the first text data. The first image data and the first text data can be combined to identify entities. The first image data can provide richer context and visual information for the analysis of public opinion events, which can improve accuracy. Therefore, entity features can be determined based on the positional association relationship of the entities in the first text data, and the causal relationship between entities can be determined based on the entity features. Through reasoning based on the positional association relationship, the potential causal relationship between entities can be better understood and captured, and the representation of entity features can be enhanced, thereby improving the accuracy of identifying causal relationships between entities.

[0048] The following is a detailed description of the entity relationship identification method for public opinion events in an embodiment of the present disclosure.

[0049] As described above, with respect to the above step S210, the present disclosure provides a possible implementation method for obtaining the first public opinion data to be processed, including:

[0050] S211. Acquire second public opinion data, where the second public opinion data includes second image data and second text data.

[0051] In the embodiment of the present disclosure, during the process of collecting public opinion data, there may be some information information in which the pictures and texts are not relevant. In order to ensure the relevance of the pictures and texts, it is also necessary to perform preliminary screening on the collected second public opinion data to obtain the first public opinion data containing matching picture and text data.

[0052] S212: Determine a second text feature of the second text data.

[0053] For example, based on the encoder representation of the bidirectional encoder transformer (BERT), feature extraction can be performed on the second text data, and the second text data can be converted into a word vector representation, such as the second text feature representation obtained as X = {X1, X2, ..., X n}, where X i is the feature vector of each word in the second text data.

[0054] S213: Determine the first text description information corresponding to the second image data, and generate a third text feature of the first text description information.

[0055] Regarding determining the first text description information corresponding to the second image data in this step, a possible embodiment specifically includes:

[0056] 1) Performing entity region feature extraction and global feature extraction on the second image data to obtain entity region features and global features corresponding to the second image data.

[0057] See Figure 3 As shown in FIG, it is a principle block diagram of the embodiment of the present disclosure. Figure 3 As shown, entity region feature extraction and global feature extraction are performed on the second image data respectively. For example, entity region features are extracted by a region detection method based on convolutional neural network features (Regions with Convolutional Neural Network features, R-CNN) to obtain entity region features. The entity region features can represent the local features of the entity contained in the second image data, and fuse the position features of the entity region. The position features include the center position coordinates, height and width of the bounding box of the entity region. For example, the global features can be extracted by the YOLO (You Only Look Once) method. The YOLO method can divide the second image data into multiple grids and perform feature extraction on each grid. When extracting the features of each grid, sine and cosine embedding can be used to jointly obtain the position encoding of the grid.

[0058] 2) Performing entity recognition on the second image data to obtain entity text recognized from the second image data, and encoding the entity text to obtain entity text features.

[0059] Among them, the entity in the image refers to the object that can be individually identified and classified in the image. Entity recognition can mean identifying and classifying different objects in the image, such as people, animals, vehicles, buildings, etc., without limitation. For example, Figure 3 As shown in the figure, image entity recognition can be performed using the YOLOv8 method, and the recognized entity text can be input into a text encoder (such as a BERT model) for encoding to obtain entity text features.

[0060] 3) Obtaining a first fusion feature based on the entity region feature, the global feature, and the entity text feature, and obtaining first text description information corresponding to the second image data based on the first fusion feature.

[0061] For example, Figure 3 As shown in Figure 1, the entity region features and global features can be input into the collaborative encoder to interact with the entity region features and global features, eliminate semantic noise, and obtain the entity region features and global features after encoding transformation. The collaborative encoder includes a self-attention module, a feedforward neural network (FNN) module, and a local constraint cross attention module. The calculation formula of the attention mechanism in the collaborative encoder can be expressed as:

[0062] CA(Q,K,V)=Concat(head1,···,head n )W O

[0063]

[0064] Among them, the attention mechanism CA(Q,K,V) of the collaborative encoder is merged by multiple heads, W O is the weight after merging multiple heads, represents the weight of the query of the i-th head, represents the weight of the key of the i-th head, Indicates the weight of the value of the i-th head, pos q The absolute position code of the Query value, pos k is the absolute position encoding of the Key value, Ω represents the spatial relationship of the bounding box or grid of the entity area, and d k The size of the dimension representing the key is finally normalized using the softmax function. In this way, through transformation operations such as the attention mechanism in the collaborative encoder, the dependency between entity region features and global features can be further explored to improve the accuracy of feature representation.

[0065] The entity region features and global features, as well as the entity text features, transformed by the collaborative encoder are input into the dynamic gate module for feature fusion, and the final first text description information is generated by the decoder, wherein the dynamic gate module is used to perform weighted fusion of the entity region features, global features and entity text features according to corresponding weight values, and the decoder is used to perform a decoding operation on the first fused features to generate the first text description information. The decoder can use a Transformer decoder, for example, and there is no restriction on this.

[0066] In this way, in the embodiment of the present disclosure, features are extracted from different levels of the second image data, and features are fused to obtain the first text description information of the second image data, which can cover more semantic information and improve the accuracy of the first text description information, thereby improving the accuracy of subsequent screening and matching image and text pairs.

[0067] S4. Determine a matching degree between the second image data and the second text data according to the second text feature and the third text feature.

[0068] For example, the first text description information is encoded, and the third text feature obtained is Y={Y1, Y2, ..., Y n}, the second text feature is X={X1,X2,...,X n}, the matching degree can be calculated by the weighted comprehensive similarity method of cosine similarity and Jaccard index, where cosine similarity is expressed as WCS cosine (X,Y), Jaccard index expressed as WCS Jaccard (X, Y), the matching degree calculated by weighted summation of the two can be expressed as WCS(X, Y):

[0069] WCS(X,Y)=α·WCS cosine (X,Y)+(1-α)·WCS Jaccard (X,Y)

[0070]

[0071] Among them, α is a weight coefficient, which is between 0 and 1 and is used to balance the contribution of cosine similarity and Jaccard index. The cross-validation method can be used to obtain the value of α by conducting experiments on the validation set in advance; w i is the weight of each element in the second text feature and the third text feature, which is usually a non-negative value and can be used to reflect the importance or contribution of each element in calculating the similarity. It can be set in advance and there is no restriction on this.

[0072] Of course, only cosine similarity or Jsccard index may be used to determine the matching degree, or other similarity calculation methods may be used to determine the matching degree based on the second text feature and the third text feature, which is not limited in the embodiments of the present disclosure.

[0073] S5. Filter out the first image data and the first text data whose matching degree meets the matching condition from the second public opinion data.

[0074] For example, if the matching condition is that the matching degree is greater than or equal to a preset threshold, then the first image data and first text data whose matching degree is greater than or equal to the preset threshold are screened out through the matching degree between the second image data and the second text data, so as to obtain the first public opinion data. Furthermore, for the image-text pairs in the second public opinion data that do not meet the matching conditions, they can be deleted, or they can be manually reviewed again to improve the data quality and data quantity.

[0075] In this way, in the embodiment of the present disclosure, the initially acquired second public opinion data is preprocessed, the second image data is converted into first text description information, and the semantic similarity between the first text description information and the second text data is calculated to screen out the first public opinion data including the matching first image data and the first text data, thereby improving the data quality of the first public opinion data, and thus performing subsequent entity causal relationship analysis based on the matching first image data and the first text data, which can also improve the accuracy of the entity causal relationship analysis.

[0076] In a possible embodiment, with respect to the above step S220, determining an entity set related to the public opinion event from the first text data based on the first image feature of the first image data and the first text feature of the first text data includes:

[0077] S221: Determine second text description information corresponding to the first image data, and determine third image data corresponding to the first text data.

[0078] See Figure 4 As shown in FIG, it is a principle block diagram of determining an entity set in an embodiment of the present disclosure, as shown in FIG. Figure 4 As shown, the first image data is processed into text to generate second text description information of the first image data. The specific implementation method is similar to that shown in FIG. Figure 3 The illustrated implementation method of generating the first text description information will not be described in detail here.

[0079] The first text data is processed into a text-to-image format to generate third image data of the first text data. Specifically, in one possible embodiment, 1) the first text data can be segmented to obtain a segmentation set corresponding to the first text data, and feature encoding is performed on each segmentation set to obtain a text embedding representation of the first text data. For example, the WordPiece segmentation algorithm can be used. The WordPiece segmentation algorithm can split words into common roots, prefixes, or suffixes according to frequency and context, thereby achieving effective processing of unknown words, and mapping the words processed by WordPiece to their corresponding identification codes (Token ID), and input it into the embedding layer to obtain word embedding representation, and encode it through the Transformer encoder to obtain text embedding representation; 2) According to the word segmentation set, the entity relationship extraction model is adopted to obtain entity embedding representation and relationship embedding representation; 3) According to the sequential splicing of text embedding representation, entity embedding representation and relationship embedding representation, conditional embedding is obtained, and the conditional embedding is used as the conditional input for subsequent image generation; 4) With the conditional embedding as the input condition, the third image data of the first text data is generated. For example, the U-Net (U-Net is an image segmentation model based on convolutional neural network) downsampling module can be used to extract the input features at each level, and at the same time, conditional embedding is introduced to combine text, entity, relationship, and so on through the cross attention layer. The information of the system is used to make feature extraction take into account multimodal information. When merging features in the downsampling process, the resolution is gradually halved, and the feature dimension is expanded to achieve more efficient multi-scale feature learning. The resolution is restored by gradually upsampling through a symmetrical decoder, and the multi-scale features in the encoder are introduced into the decoder and fused with the upsampled features to help restore more detailed information. The decoder uses a feature amplification module to gradually increase the image resolution and reduce the feature dimension to eventually generate an image of the target resolution. It can generate higher quality and higher resolution images based on the joint use of multiple U-Net modules. For example, based on two U-Net modules, it can gradually generate images with a resolution of 256*256 and images with a resolution of 1024*1024.

[0080] The text-to-image conversion method is not limited in the embodiments of the present disclosure. For example, a diffusion model, a generative adversarial network (GANs), etc. may also be used to implement the conversion.

[0081] S222: Obtain a comprehensive text feature based on the fourth text feature of the second text description information and the first text feature of the first text data, and obtain a comprehensive image feature based on the second image feature of the third image data and the first image feature of the first image data.

[0082] like Figure 4 As shown, based on the second text description information and the first text data, feature extraction and fusion can obtain comprehensive text features, for example: 1) the first text data X={X1, X2, ..., X n} and the second text description information Y={Y1,Y2,...,Y n}, combine according to the preset format to obtain the input text input_text, such as input_text = [CLS] + X i +[SEP]+Y i +[SEP], CLS tags the beginning and SEP tags the end; 2) Segment the combined input text input_text based on the BERT tokenizer. The tokenizer can convert the input text into a token sequence and convert each token into its unique token ID in the preset vocabulary; 3) Generate an attention mask (Attention Mask) to indicate which tokens are valid (that is, the parts related to the actual tokens in the input text), where the Attention Mask is a list of 0s and 1s, 1 represents a valid token and 0 represents a filler token; 4) Input the obtained token ID and Attention Mask together into the encoding model for forward propagation to obtain comprehensive text features.

[0083] Based on the third image data and the first image data, feature extraction and fusion can be performed to obtain comprehensive image features. For example, global feature extraction can be performed on the first image data based on the YOLO method, and entity region feature extraction can be performed on the third image data based on the fast region-based convolutional neural network (Faster Region-based Convolutional Neural Networks, Faster R-CNN) method, and then comprehensive image features can be obtained based on the second image features corresponding to the third image data and the first image features corresponding to the first image data.

[0084] S223: Fusing the comprehensive text feature and the comprehensive image feature to obtain a second fused feature.

[0085] For example, Figure 4As shown, the comprehensive text features and the comprehensive image features are aligned and input into the dynamic gate module. The dynamic gate module is used to, for example, use a sigmoid activation function to determine the weight values corresponding to the comprehensive text features and the comprehensive image features. The weight values represent the importance in the final fusion feature, and then based on the corresponding weight values, the comprehensive text features and the comprehensive image features are fused to obtain a second fusion feature. In this way, the multimodal information of the image and text can be fused to obtain the fused second fusion feature.

[0086] Among them, the weight values corresponding to the comprehensive text features and the comprehensive image features can be pre-trained and dynamically adjusted, which can enhance the performance of feature fusion. For example, when the image information is rich but the text information is less, the weight value of the comprehensive image feature may be higher to improve the accuracy of the fusion feature.

[0087] S224. Based on a preset prompt sentence template and a second fusion feature, semantic analysis is performed on the words in the first text data, and an entity set related to the public opinion event is determined from the first text data. The prompt sentence template includes at least one preset entity filling position.

[0088] In the embodiment of the present disclosure, in order to enhance the association relationship between entities and text data in an image, entity screening is performed based on a preset prompt sentence template. The prompt sentence template is mainly used as external semantic information to guide the model to focus on specific entities or targets.

[0089] Among them, the prompt statement template can be set in advance, and its number can be multiple. For example, based on the public opinion data samples in the target field, and based on the image data samples and text data samples in the public opinion data samples, feature extraction and fusion are performed to obtain a set of entity samples identified from the text data samples, and then based on the category of each entity sample in the entity sample set, a prompt statement template is set. The prompt statement template includes an entity filling position, and the entity filling position can be set according to the category of the identified entity sample.

[0090] In one possible embodiment, since the entity categories that different public opinion event categories focus on may be different, for example, if the public opinion event category is unqualified product quality, more attention may be paid to the product name, time, and events that occurred. Therefore, in the embodiment of the present disclosure, corresponding prompt statement templates can be set for different public opinion event categories. For example, when analyzing public opinion data in a target field, the categories of public opinion events in the target field are pre-set, so that corresponding prompt statement templates are set for different categories of public opinion events. In this way, corresponding prompt statement templates can be set based on the categories of different public opinion events, and semantic analysis can be performed on the words in the first text data to identify the entity set from the first text data.

[0091] For this step S224, in a possible embodiment, specifically: based on a preset prompt sentence template, the words in the first text data are filled into the corresponding entity filling positions in the prompt sentence template to obtain a prompt sentence, wherein the prompt sentence template includes at least one preset entity filling position; according to the prompt sentence and the second fusion feature, the prompt sentence is semantically analyzed to determine the entity set related to the public opinion event from the first text data.

[0092] For example, the prompt statement template is: at _(time filling position), _(product name filling position) was detected as _(reason filling position) and caused it to be unqualified. The words are searched from the first text data, and the words in the first text data are filled into the corresponding entity filling position to obtain a complete prompt statement, so that the prompt statement can be semantically analyzed. For example, whether the prompt statement is complete and correct, its coherence and logic can be analyzed, etc., so as to obtain the semantic score of the prompt statement, so that the words corresponding to the prompt statement with a semantic score greater than the set threshold can be determined as entities related to the public opinion event, and the index position of the entity in the first text data can be determined for use in subsequent determination of entity features.

[0093] like Figure 4 As shown in the figure, after the prompt sentence template passes through the encoder, the entity filling position in the prompt sentence template is filled and then input into the decoder for decoding. Then, the second fusion feature and the decoded prompt sentence are extracted through the Transformer to extract semantic features, and then input into the selector for entity recognition. The principle formula of the selector can be expressed as: represents the span in the first text data, l and m represent the index of any token in the first text data, and represents the distribution of the start and end positions of the context of the entity filling position k for each entity, and L represents the context length.

[0094] In this way, in the embodiment of the present disclosure, the features of the image and text are fused to improve the information richness of the obtained second fused feature, and based on the preset prompt sentence template and the second fused feature, the entity set related to the public opinion event is determined from the first text data. The prompt sentence template can be used to guide attention and understanding of specific entities in a certain field, so that the identified entity set is more in line with the needs of the field, and the accuracy of identifying the causal relationship of entities related to public opinion events in the field is also improved.

[0095] In a possible embodiment, determining entity features of multiple entities in step S230 based on the entity set and the positional association relationship between multiple entities in the entity set in the first text data includes:

[0096] 1) Split the first text data into multiple sentences.

[0097] 2) Constructing an interaction graph based on the entity set and the position association relationship of multiple entities in the entity set in the first text data.

[0098] Among them, the interaction graph includes nodes and connecting edges, the nodes include entity nodes, sentence nodes and text nodes corresponding to the first text data, and the connecting edges include connecting edges between sentence nodes and text nodes, connecting edges between sentence nodes and entity nodes belonging to entities in the sentence, connecting edges between all entity nodes in the same sentence, connecting edges between nodes belonging to the same entity, and connecting edges between sentence nodes.

[0099] See Figure 5 As shown, this is a principle block diagram for determining entity causal relationships in an embodiment of the present disclosure. An interaction graph can be constructed based on a graph convolutional network. The length of the first text data may be long. For ease of analysis, it can be split into multiple sentences, and the sentences and entities can be used as sentence nodes and entity nodes of the interaction graph, and the first text data can be used as a text node, so that multiple entity nodes, multiple sentence nodes and one text node can be obtained, and different types of connection edges can be constructed through position association relationships, thereby obtaining an interaction graph.

[0100] 3) According to the neighbor nodes corresponding to each entity node in the interaction graph, determine the entity features corresponding to each entity node.

[0101] In the disclosed embodiment, feature encoding can be used to obtain the initial entity features of each entity node, the initial sentence features of the sentence node, and the initial text features of the text node. To facilitate subsequent calculations, the feature vector dimensions of the initial entity features, initial sentence features, and initial text features are the same. Then, based on the connection relationship represented by the connecting edges, the final entity features corresponding to each entity node can be generated by aggregating the feature representations of the neighboring nodes corresponding to the entity nodes.

[0102] For example, the calculation principle formula of entity features can be expressed as:

[0103]

[0104] in, represents the entity feature representation of entity e in the l+1th layer, R is the index set of the neighbor structure of entity e, N r (e) represents the set of neighbor nodes of entity e, c e,r represents the normalized coefficient of the neighborhood structure associated with entity e, is the weight matrix corresponding to the neighborhood structure r, It represents the entity feature representation of the i-th entity after l layers of graph convolution. Represents the bias term of the rth neighbor structure, and Relu is a nonlinear activation function.

[0105] In this way, by constructing an interaction graph and performing feature fusion based on the neighbor nodes of the entity node, the entity features of the entity corresponding to the entity node can be obtained, which can enhance the entity feature representation and improve the accuracy of the entity feature.

[0106] In a possible embodiment, determining the causal relationship between entities based on entity features of multiple entities in step S230 includes:

[0107] 1) Based on the entity features of multiple entities and the connection relationships represented by the connecting edges in the interaction graph, candidate entity pairs with causal relationships are determined.

[0108] For example, Figure 5 As shown in the figure, based on the generated entity features and connection relationships, causal reasoning is performed through a progressive strategy to construct a progressive causal reasoning graph. In the progressive causal reasoning graph, attention is first paid to the entity causal relationship within the sentence, and then to the entity causal relationship across sentences. The causal chain can be gradually inferred for complex text data, and the implicit entity causal relationship can be determined. Specifically, this can be achieved based on the attention mechanism. The attention mechanism can not only dynamically pay attention to the causal relationship of common entities contained in different sentences, but also introduce position encoding to enhance the relationship between entity nodes. The calculation formula of the attention score of the attention mechanism can be expressed as:

[0109]

[0110] Among them, W q Convert node feature representation to Q vector, W k Convert node feature representation into K vector, It represents the connection between the feature representations of node i and node k and the feature representations of node j and node k, which is used to reflect the connection between node i and node k. d represents the feature dimension, and PE j It represents the position encoding of node j, which is used to provide information about the relative position of the node in the interaction graph, and helps to understand the context of the node in the first text data or the interaction graph.

[0111] Therefore, in the embodiment of the present disclosure, the entities corresponding to the entity nodes whose attention scores are greater than the preset value can be determined as candidate entity pairs with causal relationships by calculating the attention scores. In this way, entities with connecting edge relationships can be determined as candidate entity pairs, and candidate entity pairs that do not have direct connecting edges but may have implicit causal relationships can also be inferred. For example, the candidate entity pairs (e i ,e0) and (e0,e j ), so we can get a causal chain ei -e0-e j , we can infer the candidate entity pairs that may have causal relationships (e i , e j ).

[0112] 2) Encode the semantic features of the candidate entity pairs to generate the third fusion features of the candidate entity pairs.

[0113] For example, Figure 5 As shown, for each candidate entity pair, a dynamic gate module is used to perform a weighted fusion operation on the entity features of each candidate entity according to the weight value corresponding to each candidate entity in the candidate entity pair to generate a third fusion feature of the candidate entity pair.

[0114] For example, the dynamic gate module can use 12 dynamic gates, each corresponding to an encoder, to perform layered and refined processing on text data, so that each dynamic gate can focus on different semantic dimensions. The principle formula can be expressed as:

[0115] Among them, V i is the fusion feature generated by dynamic gate i, Indicates that the intention feature and slot feature are connected and combined into a new feature vector, w i Represents the weight vector associated with the i-th dynamic gate, which is used to weight the fused features according to their importance. b is the bias term, which is used to fine-tune the fused features.

[0116] Then we can use V i The encoding and decoding operations are then performed through the encoder and decoder to further enhance the fusion feature representation. For example, in the encoding and decoding process, nonlinear information can be introduced through the activation function, more complex pattern information can be captured, and the fusion features can be converted through the attention mechanism. There is no restriction on the specific implementation method adopted by the encoder and decoder.

[0117] 3) Based on the third fusion feature, a preset classifier is used to determine whether the causal relationship between the candidate entity pairs is a true causal relationship or a false causal relationship. The classifier is used to identify whether the category has a causal relationship.

[0118] For example, Figure 5 As shown, the third fusion feature is input into the classifier, and the classifier performs semantic analysis based on the third fusion feature, classifies and judges whether there is a causal relationship between the entities in the candidate entity pair, and obtains the final causal relationship judgment result.

[0119] Among them, the classifier can be implemented using a fully connected classifier, for example. The classifier can be trained in advance. When training the classifier, it can be trained based on a training sample set and using a focal loss function. The training sample set includes multiple entity sample pairs, and a causal relationship label or a non-causal relationship label for each entity sample pair. Then, during the training process, the value of the focal loss function is calculated based on the classifier's predicted causal relationship results for the entity sample pairs and their corresponding labels. The model parameters of the classifier are updated through back propagation until the focal loss function converges or reaches a preset number of iterative training times, that is, the trained classifier is obtained. The focal loss function can reduce the weight of samples that are easy to classify and increase the weight of samples that are difficult to classify, so that the classifier model can focus on samples that are more difficult to classify, thereby improving the classification effect and performance of the overall classifier model.

[0120] In the embodiment of the present disclosure, an interaction graph is constructed based on the positional association relationship between entity sets and entities in the first text data, and the entity features of each entity node are determined by aggregating the features of neighboring nodes in the interaction graph, thereby enhancing the entity feature representation and improving accuracy. Furthermore, based on the entity features, progressive causal reasoning is performed, considering the causal chain formed between multiple entities, to determine candidate entity pairs that may have a causal relationship. Semantic feature encoding and feature fusion are then performed based on the candidate entity pairs, and the causal relationship of the candidate entity pairs is further judged by a classifier to determine whether it is a true causal relationship, thereby improving the accuracy of the entity causal relationship judgment. In this way, entity pairs with a true causal relationship can be obtained, and then public opinion events can be subsequently analyzed based on the entity pairs with a true causal relationship, facilitating the processing and control of public opinion. For example, if a public opinion event occurs in which a company is punished, and the exceeding of standards has a causal relationship with the punishment, it may be preliminarily judged that the cause may be the product exceeding the standards, thereby enabling timely analysis of whether the company's related products exceed the standards, thereby improving the speed of public opinion handling and facilitating management and control.

[0121] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0122] In addition, the present disclosure also provides an entity relationship identification device, an electronic device, a computer-readable storage medium and a computer program product for public opinion events. The above can all be used to implement any entity relationship identification method for public opinion events provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0123] Figure 6A block diagram of an entity relationship identification device for public opinion events provided in an embodiment of the present disclosure.

[0124] Reference Figure 6 The present disclosure provides an entity relationship identification device for public opinion events, and the entity relationship identification device for public opinion events includes:

[0125] A pre-processing module 61 is configured to obtain first public opinion data to be processed, wherein the first public opinion data includes matched first image data and first text data;

[0126] A first determining module 62 is configured to determine an entity set related to a public opinion event from the first text data based on a first image feature of the first image data and a first text feature of the first text data;

[0127] The second determination module 63 is used to determine the entity features of multiple entities based on the entity set and the position association relationship between multiple entities in the entity set in the first text data, and determine the causal relationship between the entities based on the entity features of the multiple entities.

[0128] In a possible embodiment, when obtaining the first public opinion data to be processed, the preprocessing module 61 is used to:

[0129] Acquire second public opinion data, where the second public opinion data includes second image data and second text data;

[0130] determining a second text feature of the second text data;

[0131] determining first text description information corresponding to the second image data, and generating a third text feature of the first text description information;

[0132] determining a degree of matching between the second image data and the second text data based on the second text feature and the third text feature;

[0133] The first image data and the first text data whose matching degree meets the matching condition are screened out from the second public opinion data.

[0134] In a possible embodiment, when determining the first text description information corresponding to the second image data, the preprocessing module 61 is configured to:

[0135] Performing entity region feature extraction and global feature extraction on the second image data to obtain entity region features and global features corresponding to the second image data;

[0136] Performing entity recognition on the second image data to obtain entity text recognized from the second image data, and encoding the entity text to obtain entity text features;

[0137] A first fusion feature is obtained according to the entity region feature, the global feature and the entity text feature, and first text description information corresponding to the second image data is obtained according to the first fusion feature.

[0138] In a possible embodiment, when determining the entity set related to the public opinion event from the first text data based on the first image feature of the first image data and the first text feature of the first text data, the first determination module 62 is configured to:

[0139] Determining second text description information corresponding to the first image data, and determining third image data corresponding to the first text data;

[0140] Obtaining a comprehensive text feature based on the fourth text feature of the second text description information and the first text feature of the first text data, and obtaining a comprehensive image feature based on the second image feature of the third image data and the first image feature of the first image data;

[0141] Fusing the comprehensive text feature and the comprehensive image feature to obtain a second fused feature;

[0142] Based on a preset prompt sentence template and the second fusion feature, semantic analysis is performed on the words in the first text data, and an entity set related to the public opinion event is determined from the first text data. The prompt sentence template includes at least one preset entity filling position.

[0143] In a possible embodiment, when performing semantic analysis on words in the first text data based on a preset prompt statement template and the second fusion feature and determining an entity set related to a public opinion event from the first text data, the first determination module 62 is configured to:

[0144] Based on a preset prompt sentence template, words in the first text data are filled into corresponding entity filling positions in the prompt sentence template to obtain a prompt sentence, wherein the prompt sentence template includes at least one preset entity filling position;

[0145] According to the prompt sentence and the second fusion feature, a semantic analysis is performed on the prompt sentence, and an entity set related to the public opinion event is determined from the first text data.

[0146] In a possible embodiment, when determining entity features of a plurality of entities based on the entity set and the positional association relationship between the plurality of entities in the entity set in the first text data, the second determining module 63 is configured to:

[0147] Splitting the first text data into a plurality of sentences;

[0148] An interaction graph is constructed based on the entity set and the positional association relationship between multiple entities in the entity set in the first text data, wherein the interaction graph includes nodes and connecting edges, the nodes include entity nodes, sentence nodes, and text nodes corresponding to the first text data, and the connecting edges include connecting edges between the sentence nodes and the text nodes, connecting edges between the sentence nodes and entity nodes belonging to entities in the sentence, connecting edges between all entity nodes belonging to the same sentence, connecting edges between nodes belonging to the same entity, and connecting edges between sentence nodes;

[0149] According to the neighbor nodes corresponding to each entity node in the interaction graph, the entity feature corresponding to each entity node is determined.

[0150] In a possible embodiment, when determining the causal relationship between entities based on the entity features of the plurality of entities, the second determining module 63 is configured to:

[0151] Determining candidate entity pairs having a causal relationship based on entity features of the plurality of entities and connection relationships represented by connection edges in the interaction graph;

[0152] Performing semantic feature encoding on the candidate entity pair to generate a third fusion feature of the candidate entity pair;

[0153] According to the third fusion feature, a preset classifier is used to determine whether the causal relationship between the candidate entity pairs is a true causal relationship or a false causal relationship. The classifier is used to identify whether a category has a causal relationship.

[0154] Each module in the aforementioned apparatus for identifying entity relationships in public opinion events can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device's memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0155] Figure 7 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0156] Reference Figure 7An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702, and one or more I / O interfaces 703, connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 so that the at least one processor 701 can execute the above-mentioned entity relationship identification method of public opinion events.

[0157] Each module in the above-mentioned electronic device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0158] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for identifying entity relationships in public opinion events. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0159] An embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned entity relationship identification method for public opinion events.

[0160] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium).

[0161] As is well known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those skilled in the art, communication media typically contains computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0162] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0163] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0164] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0165] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0166] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0167] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0168] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0169] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A method for identifying entity relationships of public opinion events, characterized in that: include: Acquire first public opinion data to be processed, where the first public opinion data includes matched first image data and first text data; Determining an entity set related to the public opinion event from the first text data based on a first image feature of the first image data and a first text feature of the first text data; According to the entity set and the position association relationship between multiple entities in the entity set in the first text data, entity features of multiple entities are determined, and according to the entity features of multiple entities, causal relationships between entities are determined.

2. The method according to claim 1, characterized in that The obtaining of the first public opinion data to be processed includes: Acquire second public opinion data, where the second public opinion data includes second image data and second text data; determining a second text feature of the second text data; determining first text description information corresponding to the second image data, and generating a third text feature of the first text description information; determining a degree of matching between the second image data and the second text data based on the second text feature and the third text feature; The first image data and the first text data whose matching degree meets the matching condition are screened out from the second public opinion data.

3. The method according to claim 2, characterized in that The determining the first text description information corresponding to the second image data includes: Performing entity region feature extraction and global feature extraction on the second image data to obtain entity region features and global features corresponding to the second image data; Performing entity recognition on the second image data to obtain entity text recognized from the second image data, and encoding the entity text to obtain entity text features; A first fusion feature is obtained according to the entity region feature, the global feature and the entity text feature, and first text description information corresponding to the second image data is obtained according to the first fusion feature.

4. The method according to claim 1, wherein The determining, from the first text data, an entity set related to the public opinion event based on the first image feature of the first image data and the first text feature of the first text data includes: Determining second text description information corresponding to the first image data, and determining third image data corresponding to the first text data; Obtaining a comprehensive text feature based on the fourth text feature of the second text description information and the first text feature of the first text data, and obtaining a comprehensive image feature based on the second image feature of the third image data and the first image feature of the first image data; Fusing the comprehensive text feature and the comprehensive image feature to obtain a second fused feature; Based on a preset prompt sentence template and the second fusion feature, semantic analysis is performed on the words in the first text data, and an entity set related to the public opinion event is determined from the first text data. The prompt sentence template includes at least one preset entity filling position.

5. The method according to claim 4, characterized in that The method of performing semantic analysis on words in the first text data based on a preset prompt statement template and the second fusion feature, and determining an entity set related to the public opinion event from the first text data, includes: Based on a preset prompt sentence template, words in the first text data are filled into corresponding entity filling positions in the prompt sentence template to obtain a prompt sentence, wherein the prompt sentence template includes at least one preset entity filling position; According to the prompt sentence and the second fusion feature, a semantic analysis is performed on the prompt sentence, and an entity set related to the public opinion event is determined from the first text data.

6. The method according to any one of claims 1 to 5, characterized in that Determining entity features of a plurality of entities based on the entity set and the positional association relationship between the plurality of entities in the entity set in the first text data includes: Splitting the first text data into a plurality of sentences; An interaction graph is constructed based on the entity set and the positional association relationship between multiple entities in the entity set in the first text data, wherein the interaction graph includes nodes and connecting edges, the nodes include entity nodes, sentence nodes, and text nodes corresponding to the first text data, and the connecting edges include connecting edges between the sentence nodes and the text nodes, connecting edges between the sentence nodes and entity nodes belonging to entities in the sentence, connecting edges between all entity nodes belonging to the same sentence, connecting edges between nodes belonging to the same entity, and connecting edges between sentence nodes; According to the neighbor nodes corresponding to each entity node in the interaction graph, the entity feature corresponding to each entity node is determined.

7. The method according to claim 6, characterized in that The determining of the causal relationship between entities based on the entity characteristics of the plurality of entities includes: Determining candidate entity pairs having a causal relationship based on entity features of the plurality of entities and connection relationships represented by connection edges in the interaction graph; Performing semantic feature encoding on the candidate entity pair to generate a third fusion feature of the candidate entity pair; According to the third fusion feature, a preset classifier is used to determine whether the causal relationship between the candidate entity pairs is a true causal relationship or a false causal relationship. The classifier is used to identify whether a category has a causal relationship.

8. A device for identifying entity relationships of public opinion events, characterized in that: include: A preprocessing module, configured to obtain first public opinion data to be processed, wherein the first public opinion data includes matched first image data and first text data; A first determining module, configured to determine an entity set related to a public opinion event from the first text data based on a first image feature of the first image data and a first text feature of the first text data; The second determination module is used to determine the entity features of multiple entities based on the entity set and the position association relationship between multiple entities in the entity set in the first text data, and determine the causal relationship between the entities based on the entity features of the multiple entities.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor so that the at least one processor can execute the entity relationship identification method for public opinion events as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the entity relationship identification method of public opinion events as described in any one of claims 1 to 7.

Images