Event identification method and device, readable medium and electronic equipment
The event recognition method enhances training efficiency and accuracy by using a model trained on positive and negative sample data, characterized by entity identifiers and types, to automatically classify events.
Patent Information
- Application Number
- CN202310756069.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-25
- Publication Date
- 2024-12-27
AI Technical Summary
Existing machine learning models for event recognition require manual labeling of sample data, leading to low training efficiency and accuracy.
An event recognition method that utilizes a pre-generated model trained on positive and negative sample data, characterized by entity identifiers and entity types determined from data distribution characteristics, to automatically classify events.
Improves the efficiency of model training and recognition accuracy by efficiently obtaining and utilizing positive and negative sample data based on entity identification and types.
Smart Images
Figure CN119202776A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular, to an event recognition method, device, readable medium and electronic device. Background Art
[0002] With the development of artificial intelligence technology, machine learning models for event recognition can be generated based on existing data training, and events corresponding to the data can be identified based on the input data, so that the data or events can be processed accordingly. In related technologies, in order to train the model, the sample data used for training needs to be manually labeled, which results in low training efficiency and low model recognition accuracy. Summary of the invention
[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an event recognition method, the method comprising:
[0005] Get target data;
[0006] Inputting the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model;
[0007] Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0008] According to a second aspect of an embodiment of the present disclosure, there is provided an event recognition device, the device comprising:
[0009] An acquisition module is used to acquire target data;
[0010] A recognition module, used for inputting the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model;
[0011] Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0012] According to a third aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method described in the first aspect of the present disclosure are implemented.
[0013] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:
[0014] a storage device having a computer program stored thereon;
[0015] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.
[0016] By adopting the above technical solution, the target data is obtained, and the target data is input into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model; wherein, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, and the target event recognition model can be a model generated according to positive sample data and negative sample data, and the positive sample data and negative sample data can be data obtained from multiple first historical data according to entity identification and entity type, and the entity identification can be used to characterize the entity that sends the first historical data, and the entity type can be a type determined according to the data distribution characteristics of multiple first historical data in a preset dimension, and the entity type can include a first type and a second type, and the event corresponding to the first historical data sent by the first type of entity identification is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identification is not a specified type event. Thus, the positive sample data and negative sample data can be efficiently obtained, and the target event recognition model can be generated according to the positive sample data and negative sample data, thereby improving the efficiency of model training, and then improving the recognition accuracy of the target data.
[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:
[0019] Figure 1 The figure is a flow chart of an event recognition method according to an exemplary embodiment of the present disclosure.
[0020] Figure 2 The figure is a flowchart of a method for generating a target event recognition model according to an exemplary embodiment of the present disclosure.
[0021] Figure 3 It is a schematic diagram showing a data distribution feature according to an exemplary embodiment of the present disclosure.
[0022] Figure 4 The figure is a flowchart of a method for evaluating a target event recognition model according to an exemplary embodiment of the present disclosure.
[0023] Figure 5 The flowchart of another method for evaluating a target event recognition model is shown according to an exemplary embodiment of the present disclosure.
[0024] Figure 6 It is a block diagram of an event recognition system according to an exemplary embodiment of the present disclosure.
[0025] Figure 7 It is a block diagram of an event recognition device according to an exemplary embodiment of the present disclosure.
[0026] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0029] The term "including" and its variations used in the present disclosure are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0031] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly indicated in the context, they should be understood as "one or more". In the description of the present disclosure, unless otherwise specified, "multiple" refers to two or more than two, and other quantifiers are similar; "at least one item", "one or more items" or similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one item a can represent any number of a; for another example, one or more items of a, b and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple; "and / or" is a kind of association relationship that describes the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character “ / ” indicates that the preceding and following objects are in an “or” relationship. The singular forms “a”, “an”, “an”, “said” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.
[0032] Although operations or steps are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood that it is required to perform these operations or steps in the specific order shown or in a serial order, or to perform all the operations or steps shown to obtain the desired results. In the embodiments of the present disclosure, these operations or steps can be performed in series; these operations or steps can also be performed in parallel; or some of these operations or steps can be performed.
[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0034] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0035] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0036] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0037] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0038] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0039] The present disclosure is described below in conjunction with specific embodiments.
[0040] Figure 1 is a flow chart of an event recognition method according to an exemplary embodiment. The method can be applied to electronic devices, which may include terminal devices, such as smart phones, smart wearable devices, smart speakers, smart tablets, PDAs (Personal Digital Assistants), CPEs (Customer Premise Equipment), personal computers, vehicle-mounted terminals, etc.; the electronic devices may also include servers, such as local servers or cloud servers. Figure 1 As shown, the method may include:
[0041] S101. Obtain target data.
[0042] The target data may include one or more data packets, and the data packets may be packets based on the TCP protocol, the UDP protocol, the HTTP protocol or other protocols.
[0043] In some embodiments, the target data may be data corresponding to the target event, or the target data may be data for executing the target event. The target event may be a request event, which is used to represent an event in which a user requests any one of the following: accessing a webpage or server, downloading a file, uploading a file, modifying a file or data, or deleting a file or data.
[0044] S102: Input the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model.
[0045] In some embodiments, the above-mentioned target event recognition model can be a model generated based on positive sample data and negative sample data, and the positive sample data and negative sample data can be data obtained from multiple first historical data based on entity identifiers and entity types. The entity identifier can be used to characterize the entity that sends the first historical data. The entity type can be a type determined based on data distribution characteristics of multiple first historical data in a preset dimension. The entity type can include a first type and a second type. The event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0046] The target recognition result can be used to indicate whether the event corresponding to the target data is a specified type event. The specified type event can be any type of event preset by the user.
[0047] For example, request events can be divided into normal request events and abnormal request events. The specified type of event can be an abnormal request event, which can also be called a risk request event. Through the target identification result, it can be determined whether the event corresponding to the target data is an abnormal request event.
[0048] For another example, the request event can be divided into an internal request event and an external request event. The specified type of event can be an external request event. The target identification result can be used to determine whether the event corresponding to the target data is an external request event. The organization can be an enterprise or an organization.
[0049] For another example, the specified type of event may also be used to characterize a request event from an abnormal user.
[0050] In some embodiments, corresponding processing can also be performed according to the target event identification model. For example, an abnormal request event can be intercepted and service can be refused for the abnormal request event. For another example, an alarm can be performed on the abnormal request event, and the abnormal request event can be reported to the user (such as operation and maintenance personnel) for processing by the user.
[0051] The above method is adopted to obtain target data, input the target data into a pre-generated target event recognition model, and obtain a target recognition result output by the target event recognition model; wherein the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, and the target event recognition model can be a model generated according to positive sample data and negative sample data, and the positive sample data and negative sample data can be data obtained from multiple first historical data according to entity identification and entity type, and the entity identification can be used to characterize the entity sending the first historical data, and the entity type can be a type determined according to the data distribution characteristics of multiple first historical data in a preset dimension, and the entity type can include a first type and a second type, and the event corresponding to the first historical data sent by the first type of entity identification is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identification is not a specified type event. Thus, positive sample data and negative sample data can be efficiently obtained, and a target event recognition model can be generated according to the positive sample data and negative sample data, thereby improving the efficiency of model training, and then improving the recognition accuracy of target data.
[0052] Figure 2 FIG. 1 is a flow chart showing a method for generating a target event recognition model according to an exemplary embodiment. Figure 2 As shown, the method may include:
[0053] S201: Acquire multiple first historical data and an entity identifier corresponding to each first historical data.
[0054] The first historical data may be historical data within a first historical time range, and the first historical time range may be any pre-set time range, for example, the most recent day, the most recent week, the most recent 15 days, the most recent month, or a certain absolute time range.
[0055] Likewise, the first historical data may include historical data corresponding to the target event within the first historical time range. For example, the first historical data may include historical data corresponding to the request event within the first historical time range.
[0056] In some embodiments, the entity identifier may include an IP address, a MAC address, or an electronic device identifier of the electronic device that sends the first historical data.
[0057] In other embodiments, the entity identifier may include a user identifier (eg, user name, user account, or encrypted mobile phone number, etc.) of the user who sent the first historical data.
[0058] S202: Determine an entity type of the entity identifier according to data distribution characteristics of the plurality of first historical data in a preset dimension.
[0059] In some embodiments, first designated data can be determined from a plurality of first historical data according to data distribution characteristics of a plurality of first historical data in a preset dimension, and the event corresponding to the first designated data can be a designated type event; the identification type of the entity identification corresponding to the first designated data is taken as the first type, that is, the event corresponding to the first historical data sent by the first type entity identification is a designated type event. In this way, the first type entity identification can be automatically obtained according to the data distribution characteristics.
[0060] In other embodiments, the user may determine the second designated data by judging the first historical data, and the event corresponding to the second designated data may not be a designated type event; the identification type of the entity identification corresponding to the second designated data is taken as the second type, that is, the event corresponding to the first historical data sent by the second type entity identification is not a designated type event. In this way, the second type of entity identification may be determined based on user experience (e.g., expert experience).
[0061] It should be noted that the above-mentioned preset dimensions can be fields extracted from the first historical data. For example, the first historical data is a field corresponding to a request event. Each request event may include at least one of the following fields: user ID, client IP address, region where the client IP is located, target server visited, target website visited, etc. More statistical fields can be obtained by counting the above-mentioned fields of multiple first historical data, for example: the number of client IPs with the same user ID within a specified time range, the number of user IDs with the same client IP address within a specified time range, etc.
[0062] Figure 3 FIG. 1 is a schematic diagram showing a data distribution feature according to an exemplary embodiment. Figure 3 As shown, the data distribution characteristics of the plurality of first historical data in the preset dimension may include aggregation.
[0063] For example, with the first dimension as the horizontal axis and the second dimension as the vertical axis, and each first historical data as a data node, it can be seen that the first historical data in the 301 area has obvious clustering.
[0064] For another example, with the third dimension as the horizontal coordinate and the fourth dimension as the vertical coordinate, and each first historical data as a data node, it can be seen that the first historical data in the 302 area has obvious clustering.
[0065] In some embodiments, when the first historical data is the historical data corresponding to the request event, the historical data corresponding to the abnormal request event can be called risk traffic, and the historical data corresponding to the normal request event can be called normal traffic. The data distribution characteristics of the risk traffic may be different from the data distribution characteristics of the normal process. For example, the risk traffic has obvious clustering in certain specific dimensions, and the risk traffic can be obtained by mining the traffic with obvious data distribution characteristics. For example, requests with obvious clustering can be mined as risk requests.
[0066] In some embodiments, the data distribution characteristics of the plurality of first historical data in the preset dimension may be determined by the following formula (1):
[0067]
[0068] in, represents the data distribution characteristics of the first historical data, L(X, XZ) is the loss function, which can be used to constrain the difference between X and XZ to be the smallest, λ represents the regularization term coefficient, which can be a pre-set value greater than 0, Ω(Z) is the regularization term, which can also be called a regularization function, used to sparse Z. X represents a data matrix, which can be a matrix generated according to a plurality of first historical data, and each column item x of the data matrix X i It can be a dimension of the first historical data, Z represents a coefficient matrix, and the z of each column item of the coefficient matrix Z i , which can be the column entries x of the data matrix X i By other items x j , where j is not equal to i. In some embodiments, each column item z of the coefficient matrix Z i The L1 norm of can be 1.
[0069] In some embodiments, the step S202 may include the following sub-steps:
[0070] First, comprehensive data distribution characteristics of a plurality of first historical data in a preset dimension are determined.
[0071] For example, the comprehensive data distribution characteristics of all first historical data in a preset dimension can be obtained according to the above formula (1).
[0072] Then, for each entity identifier, entity data distribution characteristics of the first historical data corresponding to the entity identifier in a preset dimension are obtained, and the entity type of the entity identifier is determined according to the entity data distribution characteristics and the comprehensive data distribution characteristics.
[0073] For example, for each entity identifier, entity data distribution characteristics of the first historical data corresponding to the entity identifier in a preset dimension may be obtained according to the above formula (1).
[0074] In one implementation, if the difference between the entity data distribution feature of a certain entity identifier and the comprehensive data distribution feature is greater than or equal to a preset difference, the entity type of the entity identifier can be determined to be the first type. Further, if the difference between the entity data distribution feature of a certain entity identifier and the comprehensive data distribution feature is less than a preset difference, the entity type of the entity identifier can also be determined to be the second type.
[0075] In another implementation, the difference between the entity data distribution feature and the comprehensive data distribution feature of each entity identifier can be obtained, the difference values are sorted from large to small, and the entity types of the first N entity identifiers with the largest difference values are set as the first type. Furthermore, the entity types of the first M entity identifiers with the smallest difference values can also be set as the second type. The sum of N and M is less than or equal to the total number of entity identifiers.
[0076] S203: Acquire positive sample data and negative sample data according to the entity identifier and entity type.
[0077] The entity type of the entity identifier corresponding to the positive sample data is the first type, and the entity type of the entity identifier corresponding to the negative sample data is the second type.
[0078] In some embodiments, positive sample data and negative sample data may be acquired from the third historical data according to the entity identifier and the entity type.
[0079] The third historical data and the first historical data may be data within the same time range, or data within different time ranges.
[0080] For example, historical data sent by entity identifiers whose entity type is the first type in the third historical data can be used as positive sample data; and historical data sent by entity identifiers whose entity type is the second type in the third historical data can be used as negative sample data.
[0081] In some embodiments, online data in a recent period of time may be associated according to entity identifiers and entity types, thereby obtaining positive sample data and negative sample data.
[0082] S204: Train a preset event recognition model according to the positive sample data and the negative sample data to obtain a target event recognition model.
[0083] The preset event recognition model can be any neural network model, such as a convolutional neural network (CNN), a recursive neural network (RNN), or other neural network models. The specific method of training the preset event recognition model can also refer to the model training method in the relevant technology, and the present disclosure will not repeat it.
[0084] By adopting the above method, positive sample data and negative sample data can be obtained, and a target event recognition model can be generated according to the positive sample data and the negative sample data, thereby improving the training efficiency.
[0085] In some embodiments of the present disclosure, after the preset event recognition model is trained according to the positive sample data and the negative sample data to obtain the target event recognition model, the target event recognition model can be deployed online at time T. From time T to time T+K, the prediction effect of the model can be evaluated according to the entity identifier and entity type. After the preset indicators (for example, recall rate, precision rate, etc.) are met, the target event recognition model can continue to be deployed.
[0086] Figure 4 FIG. 1 is a flow chart showing a method for evaluating a target event recognition model according to an exemplary embodiment. Figure 4 As shown, the method may include:
[0087] S401, obtaining prediction results of the target event recognition model for the second historical data.
[0088] Among them, the prediction result can be used to indicate whether the event corresponding to the second historical data predicted by the target event recognition model is a specified type of event.
[0089] S402: Determine a historical data label for each second historical data according to data distribution characteristics of the second historical data in a preset dimension.
[0090] The historical data tag may also be used to indicate whether the event corresponding to the second historical data is an event of a specified type.
[0091] In some embodiments, each second historical data may correspond to an entity identifier. The comprehensive data distribution characteristics of the plurality of second historical data in a preset dimension may be first determined. Then, for each entity identifier, the entity data distribution characteristics of the second historical data corresponding to the entity identifier in the preset dimension are obtained, and the entity type of the entity identifier is determined based on the entity data distribution characteristics and the comprehensive data distribution characteristics, and finally the historical data label of the second historical data is determined based on the entity type.
[0092] For example, the comprehensive data distribution characteristics of all second historical data in a preset dimension can be obtained according to the above formula (1); the entity data distribution characteristics of the second historical data corresponding to the entity identifier in a preset dimension can also be obtained according to the above formula (1).
[0093] In one implementation, if the entity type of the entity identifier is the first type, the historical data tag of the second historical data sent by the entity identifier may be a first type tag, and the first type tag may be used to characterize that the event corresponding to the second historical data is a specified type event. If the entity type of the entity identifier is the second type, the historical data tag of the second historical data sent by the entity identifier may be a second type tag, and the second type tag may be used to characterize that the event corresponding to the second historical data is not a specified type event.
[0094] S403: Determine a first accuracy of the target event recognition model in recognizing the second historical data according to the prediction result and the historical data label.
[0095] For each second historical data, if the prediction result is the same as the historical data label, it can be determined that the target event recognition model accurately recognizes the second historical data; if the prediction result is the same as the historical data label, it can be determined that the target event recognition model incorrectly recognizes the second historical data; in this way, the ratio of the number of accurately recognized second historical data to the total number of all second historical data can be used as the first accuracy.
[0096] In some embodiments, when the first accuracy is greater than or equal to a first preset accuracy threshold, the target event recognition model is deployed online; conversely, when the first accuracy is less than the first preset accuracy threshold, the target event recognition model can continue to be trained and deployed online after the accuracy is improved.
[0097] In this way, the target event recognition model can be evaluated based on the second historical data.
[0098] Figure 5 FIG. 1 is a flow chart showing another method for evaluating a target event recognition model according to an exemplary embodiment. Figure 5 As shown, the method may include:
[0099] S501, obtaining multiple target evaluation data identified by a target event recognition model within a preset time range.
[0100] Among them, the event corresponding to the target evaluation data is an event of a specified type.
[0101] The preset time range may be any preset time, for example, the most recent day, the most recent week, the most recent 15 days, the most recent month, or an absolute time range.
[0102] S502: Determine a second accuracy of the target event recognition model according to the target evaluation data.
[0103] In some embodiments, there may be multiple target event recognition models, and the target evaluation data may include first evaluation data of each target event recognition model. In this step, the first recognition vector of the target event recognition model can be obtained based on the first evaluation data of each target event recognition model, and the second accuracy of the target event recognition model can be determined based on the difference between the first recognition vector of the target event recognition model and the first recognition vectors of other target event recognition models.
[0104] For example, if there are N target event recognition models, the first evaluation data recognized by each target event recognition model within a specified time range, that is, the recalled data D1, D2, ..., D N , vectorize the data recalled by each target event recognition model to obtain the first recognition vectors M1, M2, ..., M N The first recognition vector M of the i-th target event recognition model is i , obtain the first recognition vector M of other target event recognition models j (j includes other natural numbers from 1 to N except i) and add and transform to obtain vector M t , calculate M i With M t The difference between the target event recognition models (e.g., vector distance) can be used to determine the second accuracy of the target event recognition model. For example, the inverse of the difference between the target event recognition models can be normalized to any value between 0 and 1. The greater the difference, the lower the second accuracy.
[0105] It should be noted that the first identification vector may be a two-dimensional matrix, or a vector obtained by transforming a two-dimensional matrix into a one-dimensional matrix.
[0106] In one implementation, if the difference is greater than or equal to a preset difference threshold or the second accuracy is less than or equal to a second preset accuracy threshold, it can be determined that the target event recognition model has misidentified. At this time, the target event recognition model can be trained offline and then deployed online after training.
[0107] In some other embodiments, second evaluation data among the plurality of target evaluation data may be determined, and a second accuracy of the target event recognition model may be determined according to the proportion of the second evaluation data among the plurality of target evaluation data, wherein the entity type of the entity identifier corresponding to the second evaluation data is the second type.
[0108] For example, data sent by an entity identifier whose entity type is the second type in the target evaluation data may be used as the second evaluation data.
[0109] In this way, through any of the above methods, the target event recognition model can be evaluated to obtain its second accuracy.
[0110] In some embodiments of the present disclosure, the second accuracy may also be sent or displayed to the user.
[0111] For example, the model identification information of the target event identification model can be sent to the user's mailbox via email, or the model identification information can be displayed to the user via a display device. The model identification information can include the second accuracy, and further, the model identification information can also include other auxiliary information, such as the number of target evaluation data identified within a preset time range.
[0112] In some embodiments of the present disclosure, when the second accuracy is less than or equal to the second preset accuracy threshold, the target event recognition model can be trained offline and then deployed online after the training.
[0113] In this way, the recognition accuracy of the target event recognition model can be further improved.
[0114] In some embodiments of the present disclosure, the methods in one or more of the above embodiments may be implemented by an event recognition system. Figure 6 FIG. 6 is a block diagram of an event recognition system 600 according to an exemplary embodiment. Figure 6 As shown, the event recognition system 600 may include an entity type recognition module 601, a model training module 602, an online model effect evaluation module 603, an evaluation index push module 604 and a model reasoning module 605.
[0115] The entity type identification module 601 can be used to perform Figure 2 In the steps S201 and S202, the model training module 602 can be used to perform Figure 2 In the steps S203 and S204, the online model effect evaluation module 603 can be used to perform Figure 4 and / or Figure 5 In the steps of the embodiment shown in the figure, the evaluation index push module 604 can be used to send or display the second accuracy to the user, and the model reasoning module 605 can be used to execute Figure 5 Steps of the illustrated embodiment.
[0116] Figure 7 FIG. 1 is a block diagram of an event recognition device 1100 according to an exemplary embodiment. Figure 7 As shown, the event recognition device 1100 may include:
[0117] An acquisition module 1101 is used to acquire target data;
[0118] The recognition module 1102 is used to input the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model;
[0119] Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0120] According to one or more embodiments of the present disclosure, the event recognition device 1100 may further include:
[0121] A generation module 1103 is used to obtain multiple first historical data and entity identifiers corresponding to each first historical data; determine the entity type of the entity identifier based on the data distribution characteristics of the multiple first historical data in a preset dimension; obtain positive sample data and negative sample data from the first historical data based on the entity identifier and the entity type, the entity type of the entity identifier corresponding to the positive sample data is the first type, and the entity type of the entity identifier corresponding to the negative sample data is the second type; train a preset event recognition model based on the positive sample data and the negative sample data to obtain the target event recognition model.
[0122] According to one or more embodiments of the present disclosure, the generation module 1103 is used to determine the comprehensive data distribution characteristics of the multiple first historical data in a preset dimension; for each entity identifier, obtain the entity data distribution characteristics of the first historical data corresponding to the entity identifier in the preset dimension, and determine the entity type of the entity identifier based on the entity data distribution characteristics and the comprehensive data distribution characteristics.
[0123] According to one or more embodiments of the present disclosure, the event recognition device 1100 may further include:
[0124] Evaluation module 1104 is used to obtain the prediction result of the target event recognition model for the second historical data; the prediction result is used to indicate whether the event corresponding to the second historical data is a specified type of event; determine the historical data label of each second historical data according to the data distribution characteristics of the second historical data in a preset dimension; determine the first accuracy of the target event recognition model in identifying the second historical data according to the prediction result and the historical data label.
[0125] According to one or more embodiments of the present disclosure, the evaluation module 1104 is further configured to deploy the target event recognition model online when the first accuracy is greater than or equal to a first preset accuracy threshold.
[0126] According to one or more embodiments of the present disclosure, the evaluation module 1104 is used to obtain multiple target evaluation data identified by the target event recognition model within a preset time range; wherein the event corresponding to the target evaluation data is an event of a specified type; and determine the second accuracy of the target event recognition model based on the target evaluation data.
[0127] According to one or more embodiments of the present disclosure, there are multiple target event recognition models, and the target evaluation data includes first evaluation data of each target event recognition model;
[0128] The evaluation module 1104 is further used to obtain a first recognition vector of each target event recognition model based on the first evaluation data of the target event recognition model, and determine a second accuracy of the target event recognition model based on the difference between the first recognition vector of the target event recognition model and the first recognition vectors of other target event recognition models.
[0129] According to one or more embodiments of the present disclosure, the evaluation module 1104 is used to determine second evaluation data among the multiple target evaluation data; wherein the entity type of the entity identifier corresponding to the second evaluation data is the second type; and based on the proportion of the second evaluation data in the multiple target evaluation data, determine the second accuracy of the target event recognition model.
[0130] According to one or more embodiments of the present disclosure, the evaluation module 1104 is further configured to send or display the second accuracy to a user.
[0131] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0132] Reference below Figure 8, which shows a schematic diagram of the structure of an electronic device 2000 (e.g., a terminal device or a server) suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The server in the embodiment of the present disclosure may include, but is not limited to, local servers, cloud servers, single servers, distributed servers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0133] like Figure 8 As shown, the electronic device 2000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 2001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2002 or a program loaded from a storage device 2008 into a random access memory (RAM) 2003. In the RAM 2003, various programs and data required for the operation of the electronic device 2000 are also stored. The processing device 2001, the ROM 2002, and the RAM 2003 are connected to each other via a bus 2004. An input / output (I / O) interface 2005 is also connected to the bus 2004.
[0134] Typically, the following devices may be connected to the input / output interface 2005: an input device 2006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 2007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 2008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 2009. The communication device 2009 may allow the electronic device 2000 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 2000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0135] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 2009, or installed from the storage device 2008, or installed from the ROM 2002. When the computer program is executed by the processing device 2001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0136] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0137] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0138] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0139] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains target data; inputs the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model; wherein the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on an entity identifier and an entity type, the entity identifier is used to characterize an entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0140] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0142] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, an acquisition module may also be described as a "module for acquiring target data".
[0143] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0144] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0145] According to one or more embodiments of the present disclosure, there is provided an event recognition method, the method comprising:
[0146] Get target data;
[0147] Inputting the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model;
[0148] Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0149] According to one or more embodiments of the present disclosure, the target event recognition model is a model generated based on the following method:
[0150] Acquire multiple first historical data and an entity identifier corresponding to each first historical data;
[0151] Determining the entity type of the entity identifier according to data distribution characteristics of the plurality of first historical data in a preset dimension;
[0152] According to the entity identifier and the entity type, obtaining positive sample data and negative sample data from the first historical data, the entity type of the entity identifier corresponding to the positive sample data being the first type, and the entity type of the entity identifier corresponding to the negative sample data being the second type;
[0153] The preset event recognition model is trained according to the positive sample data and the negative sample data to obtain the target event recognition model.
[0154] According to one or more embodiments of the present disclosure, determining the entity type of the entity identifier according to data distribution characteristics of the plurality of first historical data in a preset dimension includes:
[0155] Determining comprehensive data distribution characteristics of the plurality of first historical data in a preset dimension;
[0156] For each entity identifier, entity data distribution characteristics of the first historical data corresponding to the entity identifier in the preset dimension are obtained, and the entity type of the entity identifier is determined according to the entity data distribution characteristics and the comprehensive data distribution characteristics.
[0157] According to one or more embodiments of the present disclosure, the method further includes:
[0158] Obtaining a prediction result of the target event recognition model for the second historical data; the prediction result is used to indicate whether the event corresponding to the second historical data is a specified type of event;
[0159] determining a historical data label for each of the second historical data according to a data distribution feature of the second historical data in a preset dimension;
[0160] A first accuracy of the target event recognition model in recognizing the second historical data is determined according to the prediction result and the historical data label.
[0161] According to one or more embodiments of the present disclosure, the method further includes:
[0162] When the first accuracy is greater than or equal to a first preset accuracy threshold, the target event recognition model is deployed online.
[0163] According to one or more embodiments of the present disclosure, the method further includes:
[0164] Acquire multiple target evaluation data identified by the target event recognition model within a preset time range; wherein the event corresponding to the target evaluation data is a specified type of event;
[0165] A second accuracy of the target event recognition model is determined based on the target evaluation data.
[0166] According to one or more embodiments of the present disclosure, there are multiple target event recognition models, and the target evaluation data includes first evaluation data of each target event recognition model;
[0167] Determining the second accuracy of the target event recognition model according to the target evaluation data comprises:
[0168] According to the first evaluation data of each target event recognition model, a first recognition vector of the target event recognition model is obtained, and according to the difference between the first recognition vector of the target event recognition model and the first recognition vectors of other target event recognition models, a second accuracy of the target event recognition model is determined.
[0169] According to one or more embodiments of the present disclosure, determining the second accuracy of the target event recognition model according to the target evaluation data includes:
[0170] Determine second evaluation data among the plurality of target evaluation data; wherein the entity type of the entity identifier corresponding to the second evaluation data is the second type;
[0171] A second accuracy of the target event recognition model is determined according to a proportion of the second evaluation data in the multiple target evaluation data.
[0172] According to one or more embodiments of the present disclosure, the method further includes:
[0173] The second accuracy is sent or displayed to the user.
[0174] According to one or more embodiments of the present disclosure, there is provided an event recognition device, the device comprising:
[0175] An acquisition module is used to acquire target data;
[0176] A recognition module, used for inputting the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model;
[0177] Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
[0178] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0179] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0180] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Claims
1. An event recognition method, characterized in that: The method comprises: Get target data; Inputting the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model; Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
2. The method according to claim 1, characterized in that The target event recognition model is a model generated based on the following method: Acquire multiple first historical data and an entity identifier corresponding to each first historical data; Determining the entity type of the entity identifier according to data distribution characteristics of the plurality of first historical data in a preset dimension; According to the entity identifier and the entity type, obtaining positive sample data and negative sample data from the first historical data, the entity type of the entity identifier corresponding to the positive sample data being the first type, and the entity type of the entity identifier corresponding to the negative sample data being the second type; The preset event recognition model is trained according to the positive sample data and the negative sample data to obtain the target event recognition model.
3. The method according to claim 2, characterized in that The determining the entity type of the entity identifier according to the data distribution characteristics of the plurality of first historical data in a preset dimension includes: Determining comprehensive data distribution characteristics of the plurality of first historical data in a preset dimension; For each entity identifier, entity data distribution characteristics of the first historical data corresponding to the entity identifier in the preset dimension are obtained, and the entity type of the entity identifier is determined according to the entity data distribution characteristics and the comprehensive data distribution characteristics.
4. The method according to claim 2, characterized in that: The method further comprises: Obtaining a prediction result of the target event recognition model for the second historical data; the prediction result is used to indicate whether the event corresponding to the second historical data is a specified type of event; determining a historical data label for each of the second historical data according to a data distribution feature of the second historical data in a preset dimension; A first accuracy of the target event recognition model in recognizing the second historical data is determined according to the prediction result and the historical data label.
5. The method according to claim 4, characterized in that The method further comprises: When the first accuracy is greater than or equal to a first preset accuracy threshold, the target event recognition model is deployed online.
6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Acquire multiple target evaluation data identified by the target event recognition model within a preset time range; wherein the event corresponding to the target evaluation data is a specified type of event; A second accuracy of the target event recognition model is determined based on the target evaluation data.
7. The method according to claim 6, characterized in that There are multiple target event recognition models, and the target evaluation data includes first evaluation data of each target event recognition model; Determining the second accuracy of the target event recognition model according to the target evaluation data comprises: According to the first evaluation data of each target event recognition model, a first recognition vector of the target event recognition model is obtained, and according to the difference between the first recognition vector of the target event recognition model and the first recognition vectors of other target event recognition models, a second accuracy of the target event recognition model is determined.
8. The method according to claim 6, characterized in that Determining the second accuracy of the target event recognition model according to the target evaluation data comprises: Determine second evaluation data among the plurality of target evaluation data; wherein the entity type of the entity identifier corresponding to the second evaluation data is the second type; A second accuracy of the target event recognition model is determined according to a proportion of the second evaluation data in the multiple target evaluation data.
9. The method according to claim 6, characterized in that The method further comprises: The second accuracy is sent or displayed to the user.
10. An event recognition device, characterized in that: The device comprises: An acquisition module is used to acquire target data; A recognition module, used for inputting the target data into a pre-generated target event recognition model to obtain a target recognition result output by the target event recognition model; Among them, the target recognition result is used to indicate whether the event corresponding to the target data is a specified type event, the target event recognition model is a model generated based on positive sample data and negative sample data, the positive sample data and the negative sample data are data obtained from multiple first historical data based on entity identifiers and entity types, the entity identifier is used to characterize the entity that sends the first historical data, the entity type is a type determined based on data distribution characteristics of multiple first historical data in a preset dimension, the entity type includes a first type and a second type, the event corresponding to the first historical data sent by the first type of entity identifier is a specified type event, and the event corresponding to the first historical data sent by the second type of entity identifier is not a specified type event.
11. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 9 are implemented.
12. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 9.