Event analysis method and device, terminal equipment and computer program product

Abstract

The invention is suitable for the technical field of computers, and provides an event analysis method and device, terminal equipment and a computer program product, and the method comprises the steps: obtaining a first pre-training feature of a target event; the target event comprises different types of cross-media event data; obtaining a first mapping feature of the target event in the same high-dimensional space according to the first pre-training feature; and based on the first mapping feature, obtaining a tag category of the target event and outputting the tag category. According to the embodiment of the invention, the target function representing the clustering characteristics and the local neighborhood relationship is constructed, the target function is solved, and when different types of target events are analyzed by using the solved function, all tag categories to which the target events belong can be obtained, so that the real situation of the target events can be reflected more accurately, and the accuracy of the target events is improved. And the accuracy of the target event analysis result is effectively improved.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to an event analysis method, apparatus, terminal device and computer program product. Background Art

[0002] The development of the Internet and social media has led to an explosive growth in various types of event data. At present, when classifying and analyzing event data, traditional classification methods are mostly based on a single label category for classification and analysis, resulting in the final analysis results mostly containing only one label category, with low accuracy. For example, an event may involve multiple label categories such as environmental protection and public safety. When using traditional classification methods for classification and analysis, most of the time, the event can only be classified into one of the environmental protection category or public safety category, resulting in low accuracy of the analysis results. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an event analysis method, apparatus, terminal device and computer program product to solve the problem in the prior art that when classifying and analyzing event data, the accuracy of the analysis results is low.

[0004] A first aspect of an embodiment of the present application provides an event analysis method, including: Acquire a first pre-trained feature of a target event; the target event includes different types of cross-media event data; Acquire, according to the first pre-trained feature, a first mapping feature of the target event in the same high-dimensional space; Based on the first mapping feature, a label category of the target event is obtained and output.

[0005] In one embodiment, before obtaining the first pre-trained feature of the target event, the method includes: Acquire a second pre-trained feature of a sample event; the sample event includes different types of cross-media event data; According to the second pre-trained feature, obtaining a second mapping feature of the sample event in the same high-dimensional space; According to the second mapping feature, construct an objective function that characterizes clustering characteristics and local neighborhood relationships; The objective function is iteratively solved to obtain the corresponding projection matrix, fuzzy membership matrix and cluster center.

[0006] In one embodiment, constructing an objective function characterizing clustering characteristics and local neighborhood relationships according to the second mapping feature includes: Based on the second mapping feature, construct a first function characterizing clustering characteristics and a second function characterizing local neighborhood relations; The objective function is constructed according to the first function and the second function.

[0007] In one embodiment, constructing a first function characterizing clustering characteristics and a second function characterizing local neighborhood relationships based on the second mapping feature includes: Based on the second mapping feature, a first function characterizing the clustering characteristic is constructed, where the first function is: , ; in, represents the first function; They represent projection matrix variable, fuzzy membership matrix variable, and cluster center variable respectively; Indicates The samples belong to The probability of a class is greater than or equal to 0; Indicates The sum of the probabilities that a sample belongs to all classes is 1; represents the fuzziness parameter, and ; P represents the low-dimensional projection matrix, , ; Representation sample projection; Represents the first Cluster centers, express The projection of express The projection of express projection; Indicates that the projection matrix is ​​a column-orthogonal matrix, Represents the projection matrix The number of non-zero rows is ; represents the first regularization parameter, and ; Based on the second mapping feature, a second function characterizing the local neighborhood relationship is constructed, where the second function is: , ; in, represents the second function; They represent projection matrix variables, fuzzy membership matrix variables, and similarity matrix variables respectively; P represents the low-dimensional projection matrix, , ; Representation sample The projection of Representation sample projection; is the similarity matrix The elements in represent samples and samples The similarity between , ; Indicates The probability that a sample belongs to each class; Indicates The probability that a sample belongs to each class; represents the second regularization parameter, and .

[0008] In one embodiment, constructing the objective function according to the first function and the second function includes: The first function and the second function are integrated to obtain an objective function, which is: ; in, represents the objective function, represents the first function, represents the second function, represents the third regularization parameter.

[0009] In one embodiment, the iterative solution of the objective function to obtain the corresponding projection matrix, fuzzy membership matrix and cluster center includes: Get the initial projection matrix, the initial fuzzy membership matrix, the initial cluster center and the initial similarity matrix; The objective function is optimized by an alternating iteration method, and when the objective function converges, the corresponding projection matrix, fuzzy membership matrix and cluster center are obtained.

[0010] In one embodiment, obtaining and outputting the label category of the target event based on the first mapping feature includes: Projecting the first mapping feature to the same low-dimensional space through the projection matrix to obtain a third mapping feature; Obtaining a target fuzzy membership matrix according to the third mapping feature, the fuzzy membership matrix and the cluster center; Based on whether the element value in the target fuzzy membership matrix is ​​greater than a preset membership threshold, the label category of the target event is determined and output.

[0011] A second aspect of an embodiment of the present application provides an event analysis device, including: A first acquisition module is used to acquire a first pre-trained feature of a target event; the target event includes different types of cross-media event data; A second acquisition module, used for acquiring a first mapping feature of the target event in the same high-dimensional space according to the first pre-trained feature; A category acquisition module is used to obtain and output the label category of the target event based on the first mapping feature.

[0012] A third aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the event analysis method described in the first aspect of the embodiment of the present application are implemented.

[0013] A fourth aspect of the embodiments of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the event analysis method described in the first aspect of the embodiments of the present application are implemented.

[0014] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the event analysis method as described in the first aspect of the embodiment of the present application are implemented.

[0015] The event analysis method provided by the first aspect of the embodiment of the present application is by obtaining a first pre-trained feature of the target event; the target event includes different types of cross-media event data; according to the first pre-trained feature, a first mapping feature of the target event in the same high-dimensional space is obtained; based on the first mapping feature, the label category of the target event is obtained and output. The embodiment of the present application constructs an objective function that characterizes clustering characteristics and local neighborhood relationships, and solves the objective function. When the solved function is used to analyze different types of target events, all label categories to which the target event belongs can be obtained, thereby more accurately reflecting the actual situation of the target event and effectively improving the accuracy of the target event analysis results.

[0016] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a first flow chart of the event analysis method provided in the embodiment of the present application; Figure 2 This is a second flow chart of the event analysis method provided in the embodiment of the present application; Figure 3 This is a third flow chart of the event analysis method provided in the embodiment of the present application; Figure 4 This is a schematic diagram of the first structure of the event analysis device provided in the embodiment of the present application; Figure 5 This is a second structural diagram of the event analysis device provided in an embodiment of the present application; Figure 6 It is a structural diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0020] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0021] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. "Multiple" means "two" or "more than two".

[0023] The development of the Internet and social media has led to an explosive growth in various types of event data. At present, when analyzing event data, due to the complexity and diversity of event data, the analysis efficiency and the accuracy of the obtained analysis results are low. For example, a piece of information may involve multiple aspects such as environmental protection and public safety at the same time, but the existing analysis methods can usually only identify a single category label (for example, identifying that the information belongs to the environmental protection category), so it is impossible to accurately describe the true situation of the information, resulting in limited accuracy and practicality of the analysis results.

[0024] Therefore, an embodiment of the present application provides an event analysis method, by obtaining a first pre-trained feature of a target event; the target event includes different types of cross-media event data; according to the first pre-trained feature, a first mapping feature of the target event in the same high-dimensional space is obtained; based on the first mapping feature, a label category of the target event is obtained and output. The embodiment of the present application constructs an objective function that characterizes clustering characteristics and local neighborhood relationships, and solves the objective function. When the solved function is used to analyze different types of target events, all label categories to which the target event belongs can be obtained, thereby more accurately reflecting the actual situation of the target event and effectively improving the accuracy of the target event analysis results.

[0025] Embodiment 1 like Figure 1 Shown or Figure 2 As shown, the event analysis method provided in the embodiment of the present application includes the following steps S1 to S3, and before step S1 includes the following steps S4 to S7: Step S4, obtain the second pre-trained feature of the sample event, and proceed to step S5.

[0026] In the application, sample events can refer to any events occurring in a social environment that may have an impact on social order, public safety, social stability, etc., including but not limited to public safety events (for example, food safety events, etc.), natural disaster events (for example, earthquakes, floods, typhoons, droughts, etc.) and other social events.

[0027] In the application, the sources of sample events include but are not limited to traditional media platforms (for example, various news reporting platforms, various broadcasting platforms, newspapers, etc.), new media platforms (for example, online news platforms, various websites and forums, and social media platforms including Weibo, WeChat, Douyin, Kuaishou, etc.), etc.

[0028] In the application, the sample events include different types of cross-media event data, specifically, the sample events include but are not limited to part or all of text media data, audio media data, and video media data. It should be noted that the number of sample events can be set according to actual needs and is not limited here.

[0029] In the application, obtaining the second pre-trained features of the sample event includes but is not limited to obtaining part or all of the second pre-trained features of the text media data in the sample event, the second pre-trained features of the audio media data, and the second pre-trained features of the video media data.

[0030] In the application, a pre-trained model based on Bert (Bidirectional Encoder Representations from Transformers) can be used to obtain the second pre-trained features of text media data, or some other pre-trained models, such as CNN (Convolutional Neural Network) model, RNN (Recurrent Neural Network) model, CRNN (Convolutional Recurrent Neural Network) model, CTC (Connectionist Temporal Classification) model, Attention Mechanism model, Transformer model, Seq2Seq (Sequence-to-Sequence) model, etc. can be used to obtain the second pre-trained features of text media data. This is just an example and does not constitute a limitation.

[0031] In the application, the second pre-trained features of audio media data can be obtained by using a Conformer-based pre-trained model, or by using some other pre-trained models, such as MFCC (Mel Frequency Cepstral Coefficients) model, Spectral Features model, Filter Banks model, Delta Features model, Log-Mel Spectrogram model, CNN (Convolutional Neural Network) model, RNN (Recurrent Neural Network) model, etc. The second pre-trained features of audio media data can be obtained by using the following examples only and are not intended to be limiting.

[0032] In the application, a pre-trained model based on ResNet-101 can be used to obtain the second pre-trained features of video media data, or some other pre-trained models, such as Scene Parsing Models, DeepLab, FCN (Fully Convolutional Network) model, EfficientNet model, MobileNet model, DenseNet model, AlexNet model, VGGNet model, etc. can be used to obtain the second pre-trained features of video media data. This is just an example and does not constitute a limitation.

[0033] Step S5: According to the second pre-training feature, obtain the second mapping feature of the sample event in the same high-dimensional space, and proceed to step S6.

[0034] In the application, after obtaining the second pre-trained features of the sample event, they can be mapped to the same high-dimensional space so that feature data of different modalities have a unified feature representation to facilitate subsequent data processing and application.

[0035] Specifically, after obtaining the second pre-trained features of text media data, audio media data, and video media data, the pre-trained features corresponding to each media data can be mapped to a feature space of the same dimension through a mapping function to obtain the corresponding second mapping features. It should be noted that the mapping function can be any form of function that has the function of mapping the pre-trained features corresponding to each media data to a feature space of the same dimension, which is not limited here. In practical applications, a fully connected layer can also be used as a specific implementation of a mapping function, which is just an example and not limited here.

[0036] In the application, after obtaining the second mapping features of various media data in the same space, a feature data matrix It indicates that, Indicates The second mapping feature of the sample event, Represents the characteristic dimension of sample events, Indicates the number of sample events.

[0037] Step S6: construct an objective function that characterizes clustering characteristics and local neighborhood relationships based on the second mapping features, and proceed to step S7.

[0038] In applications, by constructing an objective function that characterizes clustering characteristics and local neighborhood relationships, we can improve the compactness of clustering (ensuring that the data points within each cluster have similar characteristics and improving the internal consistency of the cluster), separability (ensuring that there is sufficient distinction between different clusters and avoiding overlap between clusters), local structure preservation (ensuring that the local similarity of data points in high-dimensional space can still be preserved in low-dimensional feature space), and data manifold preservation (i.e., ensuring that data points still maintain their inherent geometric structure in low-dimensional feature space), thereby improving the quality and robustness of clustering. Specifically, the detailed steps for constructing the objective function based on the second mapping feature are shown below.

[0039] In one embodiment, step S6 includes the following steps S61 to S62: Step S61: Based on the second mapping feature, construct a first function that characterizes clustering characteristics and a second function that characterizes local neighborhood relations, and proceed to step S62.

[0040] In the application, the first function is the maximum center fuzzy clustering function (Fuzzy C-Means, FCM) that characterizes the clustering characteristics. It aims to find the optimal cluster center and membership matrix by minimizing the weighted distance from the data point to the cluster center and maximizing the distance between the cluster centers. The first function can not only improve the compactness and separation of clustering, but also enhance the robustness and quality of clustering; the second function is a function that characterizes the local neighborhood preservation relationship. In the low-dimensional projection space, the local manifold structure and similarity relationship of the sample event and its membership can be maintained. That is, if the sample event and There is a great similarity between them, and their fuzzy membership and There are also similarities.

[0041] In one embodiment, step S61 specifically includes: Based on the second mapping feature, a first function characterizing the clustering characteristic is constructed, where the first function is: , ; in, represents the first function; They represent projection matrix variable, fuzzy membership matrix variable, and cluster center variable respectively; Indicates The samples belong to The probability of a class is greater than or equal to 0; Indicates The sum of the probabilities that a sample belongs to all classes is 1; represents the fuzziness parameter, and ; P represents the low-dimensional projection matrix, , ; Representation sample projection; Represents the first Cluster centers, express The projection of express The projection of express projection; Indicates that the projection matrix is ​​a column-orthogonal matrix, Represents the projection matrix The number of non-zero rows is ; represents the first regularization parameter, and ; Based on the second mapping feature, a second function characterizing the local neighborhood relationship is constructed, where the second function is: , ; in, represents the second function; They represent projection matrix variables, fuzzy membership matrix variables, and similarity matrix variables respectively; P represents the low-dimensional projection matrix, , ; Representation sample The projection of Representation sample projection; is the similarity matrix The elements in represent samples and samples The similarity between , ; Indicates The probability that a sample belongs to each class; Indicates The probability that a sample belongs to each class; represents the second regularization parameter, and .

[0042] Step S62: construct the objective function according to the first function and the second function.

[0043] In applications, existing fuzzy clustering methods often ignore the distance interval between cluster centers and fail to fully consider the distribution characteristics of events on the spatial manifold. The embodiment of the present application proposes a fuzzy clustering method that maximizes the projection distance of cluster centers. This method can not only maximize the distance between cluster centers in the low-dimensional projection space, but also consider the local neighborhood relationship between sample events and fuzzy membership, thereby enhancing the clustering effect. In addition, by introducing the sparsity and orthogonality constraints of the projection matrix, the algorithm's discrimination ability is effectively improved, and the demand for storage space is reduced. Only non-zero elements and their position information need to be stored.

[0044] In one embodiment, step S62 specifically includes: The first function and the second function are integrated to obtain an objective function, which is: ; in, represents the objective function, represents the first function, represents the second function, represents the third regularization parameter, and .

[0045] In the application, the objective function obtained by integrating the first function and the second function is:

[0046] ; in, The term is used to minimize the sample points In low-dimensional projection space to the center of the cluster to which it belongs The weighted distance is used to ensure that each sample point is as close as possible to the cluster center to which it belongs, thus improving the compactness of the cluster; The term is used to maximize the distance between different cluster centers in the low-dimensional projection space, thereby ensuring a higher degree of distinction between different clusters and improving the separation of clusters. is the first regularization parameter, which is used to balance the minimization of the distance from the sample point to the cluster center and the maximization of the distance between the cluster centers; The term is used to maintain the local domain relationship and membership similarity relationship between sample points; The term is used to ensure that the relative position relationship of the sample points in the low-dimensional projection space is as consistent as possible with their relative position relationship in the high-dimensional space; The term is used to ensure that the local neighborhood relationship of the membership matrix in the low-dimensional space is as consistent as possible with its local neighborhood relationship in the high-dimensional space; is the second regularization parameter, which is used to balance the local neighborhood relationship and the membership relationship; is the third regularization parameter, used to balance Item and item.

[0047] Step S7: iteratively solve the objective function to obtain the corresponding projection matrix, fuzzy membership matrix and cluster center.

[0048] In the application, in the objective function obtained by step S6, there are four variables to be solved, namely , which are projection matrix variables, fuzzy membership matrix variables, cluster center variables, and similarity matrix variables, respectively. They can be solved by alternating iteration method, that is, three of the variables are fixed each time and the fourth variable is solved until the objective function converges.

[0049] In one embodiment, step S7 specifically includes: Get the initial projection matrix, the initial fuzzy membership matrix, the initial cluster center and the initial similarity matrix; The objective function is optimized by an alternating iteration method, and when the objective function converges, the corresponding projection matrix, fuzzy membership matrix and cluster center are obtained.

[0050] In the application, before optimizing the objective function by the alternating iteration method, the variables in the objective function need to be initialized, including: initializing the projection matrix P as The unit matrix of ;make , solve , to initialize the similarity matrix Initialize the cluster centers using clustering methods (including but not limited to any of the clustering methods such as k-means, K-medoids, Agglomerative Clustering, Divisive Clustering, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), OPTICS (Ordering Points To Identify the Clustering Structure), STING (Statistical Information Grid), and DPC (Density Peaks Clustering)) ; , Set up according to your needs; Then the objective function is optimized using the alternating iterative method. , iteratively find the unknown variables , specifically including: In the variable When the variable U is fixed, the variable The function is: , ; In the variable Under the condition of fixed, find the variable C, the function of variable C is: ; In the variable When the variable P is fixed, the variable The function is: ; ; In the variable When the variable W is fixed, the variable The function is: , ; Repeat the above four variables until the objective function converges (that is, the change of the objective function is less than a certain threshold range), and finally get a specific .

[0051] It should be noted that the meanings of the parameters in the function expressions in this section refer to the relevant description in step S6 and will not be repeated here.

[0052] In one embodiment, the first function characterizing the clustering characteristics can also be expressed as: , ; Among them, a, b, r, , are parameters, and , , The term is used to minimize the typicality or possibility that the sample point pair does not belong to the cluster to which it belongs. Indicates The samples relative to the typicality or likelihood of a category; Representation sample projection; Represents the first Cluster centers, express The projection of express The projection of express projection; Indicates that the projection matrix is ​​a column-orthogonal matrix; Represents the projection matrix The number of non-zero rows is .

[0053] The second function that characterizes the local neighborhood relationship can also be expressed as: , ; in, Representation sample For samples The reconstruction contribution of The larger the sample For samples The greater the reconstruction contribution of and samples The greater the similarity between , ; Indicates The probability that a sample belongs to each class; Indicates The probability that a sample belongs to each class.

[0054] Correspondingly, the constructed objective function can also be expressed as:

[0055] ; in, , , denotes the regularization parameter, and , , The meanings of other parameters in the objective function expression refer to the above-mentioned related content and will not be repeated here.

[0056] The corresponding process of solving the objective function is also similar to the aforementioned step S7, that is, first obtain the initialization projection matrix, the initialization fuzzy membership matrix, the initialization cluster center and the initialization similarity matrix; then optimize the objective function through the alternating iteration method, and when the objective function converges, obtain the corresponding projection matrix, fuzzy membership matrix and cluster center, which will not be repeated here.

[0057] In one embodiment, the first function characterizing the clustering characteristics can also be expressed as:

[0058] ; in, represents the regularization parameter, and , log is the logarithmic function, Indicates The samples belong to The probability of a class is greater than or equal to 0; Indicates The sum of the probabilities that a sample belongs to all classes is 1; express The projection of express The projection of express projection; Indicates that the projection matrix is ​​a column-orthogonal matrix, Represents the projection matrix The number of non-zero rows in is .

[0059] It should be noted that any of the first functions or any of the second functions The term represents the square of the Euclidean norm (i.e. The superscript 2 indicates square, and the subscript 2 indicates 2-norm); The subscript 2 represents the 2-norm, and the subscript 0 represents the 0-norm, which is used to calculate The number of non-zero rows in .

[0060] It should be noted that the objective function may be a combination of any one of the first functions and any one of the second functions, which is not limited here.

[0061] Step S1, obtain the first pre-trained feature of the target event, and proceed to step S2.

[0062] In the application, the target event can be any event to be analyzed. The sources of the target event include but are not limited to traditional media platforms (for example, various news reporting platforms, various broadcasting platforms, newspapers, etc.), new media platforms (for example, online news platforms, various websites and forums, and social media platforms including Weibo, WeChat, Douyin, Kuaishou, etc.), etc.

[0063] In the application, the target event includes different types of cross-media event data. Specifically, the target event includes but is not limited to part or all of text media data, audio media data, and video media data.

[0064] In the application, obtaining the first pre-trained features of the target event includes but is not limited to obtaining part or all of the first pre-trained features of the text media data, the audio media data, and the video media data in the target event.

[0065] In the application, a pre-trained model based on Bert (Bidirectional Encoder Representations from Transformers) can be used to obtain the first pre-trained features of text media data, or some other pre-trained models, such as CNN (Convolutional Neural Network) model, RNN (Recurrent Neural Network) model, CRNN (Convolutional Recurrent Neural Network) model, CTC (Connectionist Temporal Classification) model, Attention Mechanism model, Transformer model, Seq2Seq (Sequence-to-Sequence) model, etc. can be used to obtain the first pre-trained features of text media data. This is just an example and does not constitute a limitation.

[0066] In the application, a Conformer-based pre-trained model can be used to obtain the first pre-trained features of audio media data, or some other pre-trained models, such as MFCC (Mel Frequency Cepstral Coefficients) model, Spectral Features model, Filter Banks model, Delta Features model, Log-Mel Spectrogram model, CNN (Convolutional Neural Network) model, RNN (Recurrent Neural Network) model, etc. can be used to obtain the first pre-trained features of audio media data. This is just an example and does not constitute a limitation.

[0067] In the application, a pre-trained model based on ResNet-101 can be used to obtain the first pre-trained features of video media data, or some other pre-trained models, such as Scene Parsing Models, DeepLab, FCN (Fully Convolutional Network) model, EfficientNet model, MobileNet model, DenseNet model, AlexNet model, VGGNet model, etc. can be used to obtain the first pre-trained features of video media data. This is just an example and does not constitute a limitation.

[0068] Step S2: According to the first pre-trained feature, obtain the first mapping feature of the target event in the same high-dimensional space, and proceed to step S3.

[0069] In the application, after obtaining the first pre-trained features of the target event, it can be mapped to the same high-dimensional space so that the feature data of different modalities have a unified feature representation, which is convenient for subsequent data processing and application. Specifically, after obtaining the first pre-trained features of text media data, audio media data, and video media data, the pre-trained features corresponding to each media data can be mapped to the feature space of the same dimension through a mapping function to obtain the corresponding first mapping features. It should be noted that the mapping function can be any form of function that has the function of mapping the pre-trained features corresponding to each media data to the feature space of the same dimension. This is not limited here. In actual applications, a fully connected layer can also be used as a specific implementation of a mapping function. This is just an example and is not limited.

[0070] In the application, after obtaining the first mapping features of various types of media data in the same space, a feature data matrix It indicates that, represents the mapping function, The first pre-trained feature representing the target event, Represents the feature dimension of the target event.

[0071] It should be noted that the same high-dimensional space in step S5 and step S2 refers to the same feature space.

[0072] Step S3: Based on the first mapping feature, obtain and output the label category of the target event.

[0073] In the application, through the aforementioned step S7, the projection matrix has been obtained , cluster center and the fuzzy membership matrix , we can now directly use the cluster centers Cluster the target events according to the fuzzy membership matrix The target event is labeled and classified based on the value of the column, and the classification result is output to the user terminal.

[0074] In applications, user terminals can be mobile phones, tablet computers, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and other devices.

[0075] In one embodiment, Figure 3 As shown, step S3 specifically includes the following steps S31 to S33: Step S31: projecting the first mapping feature to the same low-dimensional space through the projection matrix to obtain a third mapping feature; Step S32, obtaining a target fuzzy membership matrix according to the third mapping feature, the fuzzy membership matrix and the cluster center; Step S33: Based on whether the element value in the target fuzzy membership matrix is ​​greater than a preset membership threshold, determine the label category of the target event and output it.

[0076] In the application, the preset membership threshold can be set according to actual needs and is not limited here.

[0077] In the application, after obtaining the first mapping feature of the target event After that, the projection matrix P can be used to transform Projecting to low-dimensional space, we get The third mapping feature representation y in the low-dimensional space is ; Then, the expression of the fuzzy membership matrix is: ,(or ; >0) ; or, through the expression of the fuzzy membership matrix , obtain the target fuzzy membership matrix of the target event (the target fuzzy membership matrix is ​​a value matrix calculated according to the expression u of the fuzzy membership matrix); Finally, the target event is labeled and classified according to the value in the target fuzzy membership matrix. If the kth element in the target fuzzy membership matrix is ​​greater than the preset membership threshold, , it indicates that the target event belongs to the kth label category; if there are multiple element values ​​greater than the preset membership threshold, it indicates that the target event belongs to multiple label categories at the same time, and the label category to which the target event belongs can be output.

[0078] The event analysis method provided in the embodiment of the present application adopts a method of unified representation of cross-media data. By mapping different types of media data to the same modal space for analysis, it achieves a joint unified representation of different types of media data, breaking through the limitations of traditional event analysis methods that rely on a single data source or multimodal data fusion.

[0079] In addition, the event analysis method provided in the embodiment of the present application is different from the existing method of classifying events into a single label type based only on clustering results. The present application can classify events into multiple types simultaneously according to their actual attributes through clustering centers and membership functions, which can better meet the analysis requirements of highly complex events and has a wider range of applications.

[0080] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0081] Embodiment 2 The present application also provides an event analysis device for executing the method steps in the above event analysis method embodiment. The device may be a virtual appliance in a terminal device, which is run by a processor of the terminal device, or the terminal device itself.

[0082] like Figure 4 As shown, the event analysis device 100 provided in the embodiment of the present application includes a first acquisition module 101 , a second acquisition module 102 , and a category acquisition module 103 .

[0083] A first acquisition module 101 is used to acquire a first pre-trained feature of a target event; the target event includes different types of cross-media event data; A second acquisition module 102, configured to acquire a first mapping feature of the target event in the same high-dimensional space according to the first pre-trained feature; The category acquisition module 103 is used to obtain and output the label category of the target event based on the first mapping feature.

[0084] In one embodiment, Figure 5 As shown, the event analysis device 100 provided in the embodiment of the present application further includes a third acquisition module 104 , a fourth acquisition module 105 , a function construction module 106 and a solution module 107 .

[0085] The third acquisition module 104 is used to acquire a second pre-trained feature of a sample event; the sample event includes different types of cross-media event data; A fourth acquisition module 105, configured to acquire a second mapping feature of the sample event in the same high-dimensional space according to the second pre-trained feature; A function construction module 106, configured to construct an objective function representing clustering characteristics and local neighborhood relationships according to the second mapping feature; The solving module 107 is used to iteratively solve the objective function to obtain the corresponding projection matrix, fuzzy membership matrix and cluster center.

[0086] In one embodiment, the function building module 106 is further configured to: Based on the second mapping feature, construct a first function characterizing clustering characteristics and a second function characterizing local neighborhood relations; The objective function is constructed according to the first function and the second function.

[0087] In one embodiment, the function building module 106 is further configured to: Based on the second mapping feature, a first function characterizing the clustering characteristic is constructed, where the first function is: , ; in, represents the first function; They represent projection matrix variable, fuzzy membership matrix variable, and cluster center variable respectively; Indicates The samples belong to The probability of a class is greater than or equal to 0; Indicates The sum of the probabilities that a sample belongs to all classes is 1; represents the fuzziness parameter, and ; P represents the low-dimensional projection matrix, , ; Representation sample projection; Represents the first Cluster centers, express The projection of express The projection of express projection; Indicates that the projection matrix is ​​a column-orthogonal matrix, Represents the projection matrix The number of non-zero rows is ; represents the first regularization parameter, and ; Based on the second mapping feature, a second function characterizing the local neighborhood relationship is constructed, where the second function is: , ; in, represents the second function; They represent projection matrix variables, fuzzy membership matrix variables, and similarity matrix variables respectively; P represents the low-dimensional projection matrix, , ; Representation sample The projection of Representation sample projection; is the similarity matrix The elements in represent samples and samples The similarity between , ; Indicates The probability that a sample belongs to each class; Indicates The probability that a sample belongs to each class; represents the second regularization parameter, and .

[0088] In one embodiment, the function building module 106 is further configured to: The first function and the second function are integrated to obtain an objective function, which is: ; in, represents the objective function, represents the first function, represents the second function, represents the third regularization parameter.

[0089] In one embodiment, the solution module 107 is further configured to: Get the initial projection matrix, the initial fuzzy membership matrix, the initial cluster center and the initial similarity matrix; The objective function is optimized by an alternating iteration method, and when the objective function converges, the corresponding projection matrix, fuzzy membership matrix and cluster center are obtained.

[0090] In one embodiment, the category acquisition module 103 is further configured to: Projecting the first mapping feature to the same low-dimensional space through the projection matrix to obtain a third mapping feature; Obtaining a target fuzzy membership matrix according to the third mapping feature, the fuzzy membership matrix and the cluster center; Based on whether the element value in the target fuzzy membership matrix is ​​greater than a preset membership threshold, the label category of the target event is determined and output.

[0091] In application, each module in the above device may be a software program module, or may be implemented by different logic circuits integrated in a processor or independent physical components connected to a processor, or may be implemented by multiple distributed processors.

[0092] Embodiment 3 like Figure 6 As shown, the embodiment of the present application further provides a terminal device 200, including: at least one processor 201 ( Figure 6 Only one processor is shown in the figure), memory 202, and a computer program 203 stored in the memory 202 and executable on at least one processor 201. When the processor 201 executes the computer program 203, the steps in the above-mentioned method embodiments are implemented.

[0093] In the application, the terminal device may include but is not limited to a processor, a memory, Figure 6 It is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as human-computer interaction devices, input and output devices, network access devices, etc. The network access device may include a communication module for the terminal device to communicate with the user terminal.

[0094] In applications, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. For example, the processor may be a timing controller (TCON). A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0095] In applications, the memory may be an internal storage unit of a terminal device in some embodiments, such as a hard disk or memory of the terminal device. In other embodiments, the memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device. The memory may also include both an internal storage unit and an external storage device of the terminal device. The memory is used to store operating systems, applications, boot loaders, data, and other programs, such as program codes of computer programs. The memory may also be used to temporarily store data that has been output or is to be output.

[0096] In the application, the communication module can be set as any device that can directly or indirectly communicate with the user terminal over a long distance by wire or wireless communication according to actual needs. For example, the communication module can provide communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth, Zigbee, mobile communication networks, global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to network devices. The communication module can include an antenna, which can have only one array element or an antenna array including multiple array elements. The communication module can receive electromagnetic waves through the antenna, frequency modulate and filter the electromagnetic wave signals, and send the processed signals to the processor. The communication module can also receive the signal to be sent from the processor, frequency modulate and amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0097] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0098] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The functional modules in the embodiment can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. In addition, the specific names of the functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0099] The embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0100] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned various method embodiments.

[0101] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0102] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0105] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An event analysis method, characterized in that: include: Obtaining a first pre-trained feature of a target event; The target event includes different types of cross-media event data; Acquire, according to the first pre-trained feature, a first mapping feature of the target event in the same high-dimensional space; Based on the first mapping feature, a label category of the target event is obtained and output.

2. The event analysis method according to claim 1, characterized in that: Before obtaining the first pre-trained feature of the target event, the method includes: Acquire a second pre-trained feature of a sample event; the sample event includes different types of cross-media event data; According to the second pre-trained feature, obtaining a second mapping feature of the sample event in the same high-dimensional space; According to the second mapping feature, construct an objective function that characterizes clustering characteristics and local neighborhood relationships; The objective function is iteratively solved to obtain the corresponding projection matrix, fuzzy membership matrix and cluster center.

3. The event analysis method according to claim 2, characterized in that: The step of constructing an objective function that characterizes clustering characteristics and local neighborhood relationships according to the second mapping feature includes: Based on the second mapping feature, construct a first function characterizing clustering characteristics and a second function characterizing local neighborhood relations; The objective function is constructed according to the first function and the second function.

4. The event analysis method according to claim 3, characterized in that: The step of constructing a first function characterizing clustering characteristics and a second function characterizing local neighborhood relationships based on the second mapping feature includes: Based on the second mapping feature, a first function characterizing the clustering characteristic is constructed, where the first function is: ， ； in, represents the first function; They represent projection matrix variable, fuzzy membership matrix variable, and cluster center variable respectively; Indicates The samples belong to The probability of a class is greater than or equal to 0; Indicates The sum of the probabilities that a sample belongs to all classes is 1; represents the fuzziness parameter, and ; P represents the low-dimensional projection matrix, , ; Representation sample projection; Represents the first Cluster centers, express The projection of express The projection of express projection; Indicates that the projection matrix is ​​a column-orthogonal matrix, Represents the projection matrix The number of non-zero rows is ; represents the first regularization parameter, and ; Based on the second mapping feature, a second function characterizing the local neighborhood relationship is constructed, where the second function is: ， ； in, represents the second function; They represent projection matrix variables, fuzzy membership matrix variables, and similarity matrix variables respectively; P represents the low-dimensional projection matrix, , ; Representation sample The projection of Representation sample projection; is the similarity matrix The elements in represent samples and samples The similarity between , ; Indicates The probability that a sample belongs to each class; Indicates The probability that a sample belongs to each class; represents the second regularization parameter, and .

5. The event analysis method according to claim 4, characterized in that: The constructing the objective function according to the first function and the second function comprises: The first function and the second function are integrated to obtain an objective function, which is: ； in, represents the objective function, represents the first function, represents the second function, represents the third regularization parameter.

6. The event analysis method according to any one of claims 2 to 5, characterized in that: The iterative solution of the objective function to obtain the corresponding projection matrix, fuzzy membership matrix and cluster center includes: Get the initial projection matrix, the initial fuzzy membership matrix, the initial cluster center and the initial similarity matrix; The objective function is optimized by an alternating iteration method, and when the objective function converges, the corresponding projection matrix, fuzzy membership matrix and cluster center are obtained.

7. The event analysis method according to claim 6, characterized in that: The obtaining and outputting the label category of the target event based on the first mapping feature includes: Projecting the first mapping feature to the same low-dimensional space through the projection matrix to obtain a third mapping feature; Obtaining a target fuzzy membership matrix according to the third mapping feature, the fuzzy membership matrix and the cluster center; Based on whether the element value in the target fuzzy membership matrix is ​​greater than a preset membership threshold, the label category of the target event is determined and output.

8. An event analysis device, characterized in that: include: A first acquisition module, used to acquire a first pre-trained feature of a target event; The target event includes different types of cross-media event data; A second acquisition module, used for acquiring a first mapping feature of the target event in the same high-dimensional space according to the first pre-trained feature; A category acquisition module is used to obtain and output the label category of the target event based on the first mapping feature.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the event analysis method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the event analysis method according to any one of claims 1 to 7 are implemented.

Images