Model self-learning and event dispatching method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the application discloses a model self-learning and event dispatching method, an electronic device and a computer storage medium. The method comprises the following steps: obtaining the event features of a new event and the event features of historical events by using a feature detection model, and determining similar events similar to the new event from the historical events according to the event features of the new event and the event features of the historical events; obtaining the recommended dispatching department of the new event according to the real dispatching department of the similar events; performing sample construction according to the new event, the event features of the new event and the recommended dispatching department, the similar events, the event features of the similar events and the real dispatching department, and obtaining sample data; and performing model self-learning of the feature detection model by using the sample data. According to the technical scheme, the sustainable self-learning of the feature detection model can be realized by using the new event in the dispatching process, so as to improve the prediction accuracy of the model features and improve the event dispatching processing efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model self-learning and event dispatching method, apparatus, electronic device and computer storage medium. Background Technology

[0002] With the development of computer technology and the widespread application of artificial intelligence, new approaches have emerged to solve urban governance challenges and enhance urban governance capabilities. For example, in event dispatch systems, events (such as river pollution, damage to urban infrastructure, and other urban governance events) can be manually dispatched to appropriate departments based on their descriptions. However, when the volume of events to be dispatched is large, this highly manual approach results in high processing costs and low efficiency. Therefore, automated intelligent dispatch methods are needed to replace traditional manual dispatch methods.

[0003] In practical applications of automatic event dispatch, the complexity of event dispatch processing is relatively high. Various event dispatch scenarios differ in preconditions and dispatch requirements. Furthermore, during the dispatch process, in addition to the event's description, other dimensions such as the event's location must be considered to more accurately determine the dispatching department. Moreover, once the event dispatch system is officially online, new types of dispatched events will continuously emerge. How to enable the event dispatch system to continuously learn from historical dispatched events to handle various new types of event dispatch and continuously improve event dispatch efficiency is also a pressing issue that needs to be addressed. Summary of the Invention

[0004] In view of this, embodiments of this application provide a model self-learning and event dispatching scheme to at least partially solve the above problems.

[0005] According to a first aspect of the embodiments of this application, a model self-learning method is provided, comprising:

[0006] Using a feature detection model, the event features of the new event and the event features of multiple historical events are obtained. Based on the event features of the new event and the event features of the multiple historical events, at least one similar event that is similar to the new event is determined among the multiple historical events.

[0007] Based on the actual dispatching departments of the similar events, the recommended dispatching department for the new events is obtained;

[0008] Based on the newly added events, the event characteristics of the newly added events and the recommended distribution departments, the similar events, the event characteristics of the similar events and the actual distribution departments, sample construction is performed to obtain sample data;

[0009] Using the sample data, perform model self-learning of the feature detection model.

[0010] According to a second aspect of the embodiments of this application, an event dispatching method is provided, including:

[0011] Using a feature detection model, feature detection is performed on a target event and multiple historical events to obtain the target event features and multiple historical event features of the multiple historical events. Then, a similarity comparison is performed based on the target event features and the multiple historical event features to determine at least one similar event that is similar to the target event among the multiple historical events.

[0012] Based on the actual dispatching departments of the similar events, a recommended dispatching department for the target event is obtained, and the dispatching process of the target event is performed according to the recommended dispatching department.

[0013] The feature detection model is trained using the model self-learning method described in the first aspect.

[0014] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first or second aspect.

[0015] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method as described in the first or second aspect.

[0016] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform an operation corresponding to the method described in the first or second aspect.

[0017] According to the model self-learning and event dispatching schemes provided in the embodiments of this application, the feature detection model is used to detect the event features of new events and historical events, thereby identifying similar events that are similar to the new events. Based on the event features and dispatching departments of the new events and similar events, sample data is automatically constructed so that the feature detection model can achieve sustainable self-learning based on the continuously expanding sample data. This not only improves the prediction performance of the feature detection model, but also improves the efficiency of event dispatching. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 A schematic diagram of an application system for the model self-learning method or event dispatching method applicable to the embodiments of this application;

[0020] Figure 2 This is a flowchart illustrating a model self-learning method as an exemplary embodiment of this application.

[0021] Figures 3A to 3B This is a schematic diagram illustrating the application of a model self-learning method as an exemplary embodiment of this application.

[0022] Figure 4 This is a schematic diagram illustrating another application of a model self-learning method, which is an exemplary embodiment of this application.

[0023] Figure 5 This is a flowchart of a model self-learning method, which is another exemplary embodiment of this application.

[0024] Figure 6 This is a flowchart of a model self-learning method, which is another exemplary embodiment of this application.

[0025] Figure 7 This is a flowchart of a model self-learning method, which is another exemplary embodiment of this application.

[0026] Figure 8 This is a flowchart of an event dispatching method as an exemplary embodiment of this application.

[0027] Figure 9 This is a schematic diagram of the structure of an electronic device that is an exemplary embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0029] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. The steps in the following method embodiments are for illustrative purposes only and are not intended to limit the invention.

[0030] As described in the background section, current automatic event dispatch systems primarily rely on classification techniques for intelligent event distribution, resulting in low dispatch accuracy. Furthermore, current online event dispatch models are static and lack self-learning capabilities. Therefore, when the distribution of events to be dispatched changes, the event dispatch model trained on historical events cannot adaptively adjust its parameters, leading to a decline in the accuracy of event dispatch results over time. In view of this, this application proposes a continuously learning event dispatch solution to improve the accuracy of event dispatch results.

[0031] To facilitate understanding, the following will be combined with Figure 1 The concept of event dispatching technology in the embodiments of this application will be explained.

[0032] Figure 1 An exemplary system for a model self-learning method and an event dispatching method applicable to various embodiments of this application is shown. For example... Figure 1 As shown, the system 100 may include a server 102, a communication network 104, a client 106, and an event handling department 108. The number of clients 106 and event handling departments 108 may be one or more. For example, clients 106 may include client A, client B, etc., and event handling departments 108 may include event handling department A, event handling department B, event handling department C, etc.

[0033] Server 102 may include a cloud server, any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, server 102 may perform any suitable function. For example, in some embodiments, server 102 may be used to obtain events to be dispatched (e.g., traffic light malfunction events) transmitted by client 106, perform prediction on the event handling department of the events to be dispatched, and dispatch the events to be dispatched to the corresponding event handling department 108 (e.g., traffic management department) based on the prediction results.

[0034] In this embodiment, the service terminal 102 can perform feature prediction on the event to be assigned transmitted by the client 106 to obtain the event features of the event to be assigned, and perform similarity comparison between the event features of the event to be assigned and the event features of each historical event in the historical event database to obtain similar events of the event to be assigned, and obtain the recommended dispatch department of the event to be assigned based on the actual dispatch department of the similar events, and transmit the event to be assigned to the corresponding event handling department 108 of the recommended dispatch department.

[0035] Communication network 104 provides a communication connection between server 102 and client 106 or incident handling department 108. In some embodiments, communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, communication network 104 can include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. Client 106 or incident handling department 108 can connect to communication network 104 via one or more communication links, and communication network 104 can be linked to server 102 via one or more communication links. The communication link can be any communication link suitable for transmitting data between the client 106, the incident handling department 108, and the server 102, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.

[0036] Client 106 may include any one or more electronic devices suitable for uploading event data (e.g., event description information, address description information) to be dispatched, and event handling unit 108 may include any one or more electronic devices suitable for receiving event data (e.g., event description information, address description information) to be dispatched. In some embodiments, client 106 and event handling unit 108 may include any suitable type of device. For example, client 106 may include mobile devices, tablet computers, laptop computers, desktop computers, wearable computers, game consoles, media players, vehicle entertainment systems, and / or any other suitable type of user equipment.

[0037] In various embodiments of the present invention, the new or historical events uploaded by the client 106 can be obtained from various public platforms, such as hotline platforms, WeChat official account platforms, or any other service platforms. In various embodiments of the present invention, the new or historical events uploaded by the client 106 may include one or more of text information, image information, voice information, and audio-visual information.

[0038] Based on the above system, this application provides a model self-learning method and an event dispatching method, which are described below through several embodiments.

[0039] Example 1

[0040] Figure 2 The processing flow of the model self-learning method according to an exemplary embodiment of this application is illustrated. As shown in the figure, this embodiment mainly includes the following steps:

[0041] Step S202: Using a feature detection model, obtain the event features of the new event and the event features of multiple historical events, and based on the event features of the new event and the event features of multiple historical events, determine at least one similar event that is similar to the new event among the multiple historical events.

[0042] Optionally, feature extraction can be performed on the event description information and address description information of the newly added event using a feature detection model to obtain the event features of the newly added event, and feature extraction can be performed on the event description information and address description information of multiple historical events to obtain the event features of multiple historical events.

[0043] In some embodiments, new events or historical events can be applied to various application scenarios, including but not limited to: urban governance scenarios, office and study scenarios, medical service scenarios, and financial service scenarios. The event description information describes the content of the new or historical event, such as a road congestion event, a community power outage event, a shop operation event, or a river pollution event; the address description information describes the location where the new or historical event occurred.

[0044] Optionally, for any current event among new events and historical events, a feature detection model can be used to extract features from the event description information and address description information of the current event, so as to obtain the text number vector corresponding to the event description information and the address number vector corresponding to the address description information.

[0045] Optionally, the length of the text numeric vector in the current event can be the same as or different from the length of the address numeric vector.

[0046] In some embodiments, the text number vector and address number vector of the current event can be concatenated to obtain the event features of the current event; or, the event features of the current event can be determined based on the text number vector and address number vector of the current event respectively.

[0047] Optionally, the feature detection model may include a text vectorization module and an address parsing and vectorization module (see reference). Figure 3A ).

[0048] The text vectorization module is used to perform feature prediction based on the event description information of the current event, so as to transform event description information of inconsistent lengths into text number vectors with fixed lengths.

[0049] In some embodiments, a deep learning model can be used to perform pre-training of the deep learning model based on sample data composed of historical events using contrastive learning techniques, so that the deep learning model can better learn the knowledge in the historical events, and operations such as "averaging" and "pooling" can be added to specific layers in the deep learning model to obtain the text vectorization module in the feature detection model.

[0050] The address resolution and vectorization module can obtain address information (i.e., the address where the event occurred) from the current event, and perform standardization and vectorization processing on the address information to obtain a fixed-length address numeric vector.

[0051] In some embodiments, the address parsing and vectorization module may be constructed with an address extraction submodule, an address standardization submodule, and an address vectorization submodule. For example, the address extraction submodule may be a language processing model incorporating multiple natural language processing techniques and may be trained based on a public address dataset to extract the event address from the current event; the address standardization submodule is used to perform standardization processing on the event address obtained by the address extraction submodule to obtain address description information with a standard format, such as "×× Province ×× City ×× District" or "×× County ×× Street ×× Number", etc.; the address vectorization submodule may be trained in the same way as the text vectorization module described above, and is used to perform conversion processing on the address description information with a standard format to obtain a fixed-length address numeric vector.

[0052] In some embodiments, a language processing model that combines multiple natural language processing techniques, such as the BERT-Bilstm-CRF model, can be used to identify entities with specific meanings in text, including but not limited to names of people, places, organizations, and proper nouns. Here, BERT stands for Bidirectional Encoder Representation from Transformers, BiLSTM for Bi-directional Long Short-Term Memory, and CRF (Conditional Random Field) for Conditional Random Field.

[0053] Optionally, a similarity calculation can be performed based on the event characteristics of the new event and the event characteristics of multiple historical events to determine at least one similar event of the new event from the multiple historical events.

[0054] In some embodiments, for any current historical event among multiple historical events, a similarity calculation can be performed based on the event characteristics of the new event and the event characteristics of the current historical event. For example, the distance between the new event and the current historical event can be calculated to obtain the similarity value between the new event and the current historical event.

[0055] Optionally, based on the similarity value between the second similarity threshold and each historical event, historical events whose similarity value with the new event is greater than the second similarity threshold can be identified as similar events to the new event.

[0056] In some embodiments, a baseline model can be trained using contrastive learning techniques to obtain a feature detection model. (Reference) Figure 3B In some embodiments, the Chinese-BERT-wwm pre-trained model (Chinese name: Chinese BERT full-character masking pre-trained model) can be used as the baseline model, and the baseline model can be trained using sample event 1, sample event 2, and sample event 3, wherein sample event 1 and sample event 2 constitute a positive sample pair, and sample event 1 and sample event 3 constitute a negative sample pair.

[0057] Specifically, a baseline model can be used to perform feature prediction on sample events 1, 2, and 3 respectively. Based on the feature prediction results of sample events 1, 2, and 3 within the sample time period, the similarity of positive sample pairs (e.g., by calculating the similarity between sample events 1 and 2) and the similarity of negative sample pairs (e.g., by calculating the similarity between sample events 1 and 3) can be calculated. Based on the similarity of positive and negative sample pairs, the model loss value of the baseline model can be obtained. In this embodiment, the infoNiseContrastiveEstimation loss function value (infoNCE loss) of the baseline model can be obtained by performing cross-entropy calculation on the similarity of positive and negative sample pairs. The model parameters of the baseline model are then iteratively updated based on the model loss value, so that the similarity between positive sample pairs continuously increases and the similarity between negative sample pairs continuously decreases until the preset model training termination condition is met, thereby obtaining the feature detection model.

[0058] Step S204: Based on the actual dispatching departments of similar events, obtain the recommended dispatching department for the new event.

[0059] Optionally, a department recommendation module can be used (see reference). Figure 3A Based on the actual dispatch address of at least one similar event to the new event, the dispatch department of the new event is recommended, and the recommended dispatch department of the new event is obtained.

[0060] In this embodiment, the actual dispatching department of a historical event can be obtained based on the event processing feedback log of the historical event.

[0061] In some embodiments, multiple candidate dispatching departments for a new event can be determined based on the actual dispatching address of at least one similar event to the new event. Then, based on given factors such as dispatching time and the link relationship between different dispatching departments (e.g., obtaining the upstream department link of the end-processing department through reverse lookup based on the end-processing department), the multiple candidate dispatching departments for the new event are subjected to screening, filtering, merging, and other processing to obtain the recommended dispatching department link for the new event (which includes multiple recommended dispatching departments with hierarchical relationships). By generating the recommended dispatching department link for the new event, the corresponding organizer and co-organizer departments can be recommended for the new event at the same time, which is beneficial to improving the processing efficiency of the new event.

[0062] Step S206: Based on the new events, the event characteristics of the new events and the recommended distribution departments, similar events, the event characteristics of similar events and the actual distribution departments, perform sample construction to obtain sample data.

[0063] Optionally, based on the event characteristics of the new event and the event characteristics of multiple similar events, the new event can be identified as either a positive feedback event or a negative feedback event, thereby constructing sample data in the positive sample data pool and sample data in the negative sample data pool.

[0064] In some embodiments, similar events can be identified as positive or negative samples of the new event based on the recommended distribution department of the new event and the actual distribution department of similar events. Based on the determination result that the new event is identified as a positive or negative feedback event, when the new event is identified as a positive feedback event, sample data in the positive sample data pool is constructed using the new event, the positive samples of the new event, and the negative samples of the new event. When the new event is identified as a negative feedback event, sample data in the negative sample data pool is constructed using the new event, the positive samples of the new event, and the negative samples of the new event.

[0065] In some embodiments, a sample building module (see reference) can be used. Figure 3A ), construct sample data.

[0066] In some embodiments, the constructed sample data may include a training sample set, a validation sample set, and an evaluation sample set (see reference). Figure 3A ).

[0067] Step S208: Using sample data, perform model self-learning of the feature detection model.

[0068] In some embodiments, training and validation sample sets from the sample data can be used to perform model self-learning of the feature detection model (see reference). Figure 3A ).

[0069] In summary, the model self-learning method provided in this embodiment queries similar events from historical events to obtain recommended distribution departments for the new event based on the actual distribution departments of the similar events. It then constructs sample data by comparing the event characteristics and distribution departments of the new event and the similar events, thereby achieving the sustainable self-learning function of the feature detection model. This embodiment enables automatic iteration of the feature detection model, gradually improving its predictive accuracy. Especially when the distribution of new events changes, the automatically iteratively updated feature detection model can adapt to these changing trends to maintain high accuracy in prediction results.

[0070] Example 2

[0071] Figure 4 This is a flowchart illustrating a model self-learning method as another exemplary embodiment of this application. This embodiment is an exemplary implementation of step S206 described above. Figure 4 As shown, this embodiment mainly includes the following steps:

[0072] Step S402: Based on the recommended dispatching department of the new event and the actual dispatching department of similar events, obtain the positive and negative samples of the new event.

[0073] Optionally, based on the recommended distribution department of the new event and the actual distribution department of similar events, similar events whose actual distribution department is the same as the recommended distribution department of the new event can be identified as positive samples of the new event, and similar events whose actual distribution department is different from the recommended distribution department of the new event can be identified as negative samples of the new event.

[0074] Step S404: Based on the event characteristics of the new event and the event characteristics of similar events, determine whether the new event is a positive feedback event or a negative feedback event.

[0075] Optionally, a similarity value between the new event and similar events can be obtained based on the event characteristics of the new event and the event characteristics of similar events. Based on the similarity value between the new event and similar events, at least one reference event can be determined among the similar events. Matching can be performed based on the recommended distribution department of the new event and the actual distribution department of the reference event to determine whether the new event is a positive feedback event or a negative feedback event.

[0076] Step S406: Construct sample data based on the positive and negative samples of the newly added event, and the determination result that the newly added event is a positive feedback event or a negative feedback event.

[0077] Optionally, when a new event is determined to be a positive feedback event, sample data in the positive sample data pool is constructed using the new event, its positive samples, and its negative samples.

[0078] In some embodiments, when a new event is determined to be a positive feedback event, the first positive sample pair in the positive sample data pool can be constructed using the new event and any positive sample of the new event, and the first negative sample pair in the positive sample data pool can be constructed using the new event and any negative sample of the new event.

[0079] Optionally, when a new event is determined to be a negative feedback event, sample data in the negative sample data pool is constructed using the new event, positive samples of the new event, and negative samples of the new event.

[0080] In some embodiments, when a new event is determined to be a negative feedback event, a second positive sample pair in the negative sample data pool can be constructed using the new event and any positive sample of the new event, and a second negative sample pair in the negative sample data pool can be constructed using the new event and any negative sample of the new event.

[0081] Optionally, sample building blocks (see reference) can be used. Figure 3ABased on the positive and negative samples of the newly added event, and the determination result that the newly added event is a positive feedback event or a negative feedback event, construct the first positive sample pair and the first negative sample pair in the positive sample data pool, and the second positive sample pair and the second negative sample pair in the negative sample data pool.

[0082] In summary, the model self-learning method provided in this embodiment constructs positive and negative sample pairs in the positive sample data pool and positive and negative sample pairs in the negative sample data pool based on the event characteristics and dispatching departments of the newly added events and similar events, which can improve the model self-learning effect of the feature detection model and enhance the model detection performance.

[0083] Figure 5 This is a flowchart of a model self-learning method as another exemplary embodiment of this application. This embodiment is an exemplary implementation of step S404 above, and the following is combined with... Figure 3A and Figure 3B The technical solution of this embodiment will be described in detail below. Figure 5 As shown, this embodiment mainly includes the following steps:

[0084] Step S502: Based on the event characteristics of the newly added event and the event characteristics of similar events, obtain the similarity value between the newly added event and the similar events.

[0085] Optionally, for any current event among the newly added events and similar events, feature extraction can be performed based on the event description information and address description information of the current event to obtain the text feature vector and address feature vector of the current event.

[0086] In some embodiments, similarity calculation can be performed based on the text feature vector of the new event and the text feature vector of similar events to obtain the text similarity value between the new event and the similar events. Similarly, similarity calculation can be performed based on the address feature vector of the new event and the address feature vector of similar events to obtain the address similarity value between the new event and the similar events. And based on the text similarity value and the address similarity value, the similarity value between the new event and the similar events can be determined.

[0087] In other embodiments, the text feature vector and address feature vector of the newly added event can be concatenated to obtain the event feature vector of the newly added event, the text feature vector and address feature vector of the similar event can be concatenated to obtain the event feature vector of the similar event, and a similarity calculation can be performed based on the event feature vector of the newly added event and the event feature vector of the similar event to obtain the similarity value between the newly added event and the similar event.

[0088] Step S504: Based on the similarity values ​​between the newly added event and the similar events, determine at least one reference event among the similar events.

[0089] In this embodiment, the similarity value between the reference event and the newly added event exceeds a first similarity threshold. For example, among N similar events, M similar events whose similarity value with the newly added event exceeds the first similarity threshold are determined as reference events.

[0090] Since the reference event is a further screening result of similar events, the first similarity threshold used to determine the reference event should be higher than the second similarity threshold used to determine the similar events (refer to the description of step S202).

[0091] In other embodiments, similar events can be arranged in descending order of their similarity values ​​to the newly added event to generate a similar event sequence. Then, based on a reference event quantity threshold, one or more similar events that meet the threshold are selected from the similar event sequence as reference events. For example, if the reference event quantity threshold is set to M, the M similar events from the 1st to the Mth position in a similar event sequence containing N similar events are selected as reference events. Figure 3B ).

[0092] Step S506: Based on the department category, classify and statistically analyze at least one real distribution department of at least one reference event to obtain at least one statistical value of at least one real distribution department.

[0093] Specifically, it can be based on department category (e.g., Figure 1 The event handling departments (such as A, B, and C) are shown. The actual dispatching departments of the reference events are classified and statistically analyzed to obtain the statistical values ​​of the actual dispatching departments. For example, in at least one actual dispatching department of at least one reference event, the statistical value of event handling department A is 10, the statistical value of event handling department B is 2, and the statistical value of event handling department C is 4.

[0094] Step S508: Based on at least one statistical value of at least one real distribution department, determine at least one reference distribution department among at least one real distribution department.

[0095] In some embodiments, the statistical value of the reference distribution department is not lower than the statistical threshold, that is, the statistical value of the reference distribution department should be higher than or equal to the statistical threshold.

[0096] In some embodiments, a statistical threshold may be determined based on a set value of a reference distribution department and at least one statistical value of at least one actual distribution department.

[0097] In one embodiment, when the setting value of the reference distribution department is 1, the statistical value with the largest value can be determined as the statistical threshold based on at least one statistical value of at least one real distribution department, thereby determining the real distribution department with the largest statistical value as the reference distribution department.

[0098] In another embodiment, when the reference distribution department is set to M, the statistical values ​​can be arranged in descending order based on at least one statistical value of at least one real distribution department, and the statistical value ranked in the Mth position can be determined as the statistical threshold, thereby determining the M real distribution departments ranked in the top M positions as reference distribution departments.

[0099] For example, if the statistical value of incident handling department A is 10, the statistical value of incident handling department B is 2, and the statistical value of incident handling department C is 4, then if the setting value of the reference dispatch department is 1, incident handling department A can be determined as the reference dispatch department; if the setting value of the reference dispatch department is 2 (M equals 2), both incident handling department A and incident handling department C can be determined as reference dispatch departments.

[0100] Step S510: Match the new event with the recommended dispatching department and the actual dispatching department of the reference event to determine whether the new event is a positive feedback event or a negative feedback event.

[0101] Optionally, if the recommended dispatching department of a new event matches at least one reference dispatching department, the new event is identified as a positive feedback event; if the recommended dispatching department of a new event does not match the reference dispatching department, the new event is identified as a negative feedback event.

[0102] In some embodiments, when the reference distribution department is set to 1 (that is, there is only 1 reference distribution department), if the recommended distribution department of the new event is the same as the reference distribution department, the new event is determined as a positive feedback event; if the recommended distribution department of the new event is different from the reference distribution department, the new event is determined as a negative feedback event.

[0103] In other embodiments, when the reference dispatch department is set to M (that is, there are M reference dispatch departments), if the recommended dispatch department of the new event is the same as any one of the M reference dispatch departments, the new event is determined as a positive feedback event; if the recommended dispatch department of the new event is different from all of the M reference dispatch departments, the new event is determined as a negative feedback event.

[0104] In summary, the model self-learning method provided in this embodiment further determines reference events among similar events based on the similarity values ​​of similar events. By comparing the actual dispatching department of the reference event with the recommended dispatching department of the newly added event, the newly added event is determined to be a positive feedback event or a negative feedback event. Sample data in the positive sample data pool is constructed based on the positive feedback event, and sample data in the negative sample data pool is constructed based on the negative feedback event. In this way, this embodiment can automatically expand the sample data in the sample database using the continuously increasing new events, so that the feature detection model can continuously perform model self-learning based on the newly added sample data, thereby ensuring the accuracy of the model prediction results.

[0105] Example 3

[0106] Figure 6 This is a flowchart of a model self-learning method as another exemplary embodiment of this application. This embodiment is an exemplary implementation of step S208 above, and the following is combined with... Figure 3B The technical solution of this embodiment is described in detail, such as... Figure 6 As shown, this embodiment mainly includes the following steps:

[0107] Step S602: Using the feature detection model, perform feature prediction on the sample data pairs in the positive sample data pool to obtain the first prediction result of the positive sample data pool, and perform feature prediction on the sample data in the negative sample data pool to obtain the second prediction result of the negative sample data pool.

[0108] Optionally, the sample data in the positive sample data pool includes a first positive sample pair and a first negative sample pair, and the sample data in the negative sample data pool includes a second positive sample pair and a second negative sample pair.

[0109] Optionally, a feature detection model can be used to perform similarity calculation on the first positive sample pair in the positive sample data pool to obtain a first similarity value of the two events in the first positive sample pair; perform similarity calculation on the first negative sample pair in the positive sample data pool to obtain a second similarity value of the two events in the first negative sample pair; perform similarity calculation on the second positive sample pair in the negative sample data pool to obtain a third similarity value of the two events in the second positive sample pair; and perform similarity calculation on the second negative sample pair in the negative sample data pool to obtain a fourth similarity value of the two events in the second negative sample pair.

[0110] In some embodiments, a feature detection model can be used to perform feature prediction on the two events in the first positive sample pair to obtain the event features of the two events, and a similarity calculation can be performed based on the event features of the two events to obtain the first similarity value of the two events in the first positive sample pair. Furthermore, the methods for obtaining the second, third, and fourth similarity values ​​are the same as those for obtaining the first similarity value, and will not be elaborated here.

[0111] Step S604: Based on the first prediction result and the second prediction result, obtain the first loss function value, and based on the first prediction result, obtain the second loss function value.

[0112] In some embodiments, cross-entropy calculation can be performed based on a first similarity value and a second similarity value to obtain a first cross-entropy, and cross-entropy calculation can be performed based on a third similarity value and a fourth similarity value to obtain a second cross-entropy.

[0113] In some embodiments, the first cross-entropy and the second cross-entropy can be recalculated to obtain the first loss function value, and the second loss function value can be determined based on the first cross-entropy.

[0114] Optionally, the first loss function value may include the info Noise Contrastive Estimation loss function value (infoNCE loss for short), and the second loss function value may include the distillation loss function value.

[0115] Step S606: Obtain the total loss function value based on the sum of the first loss function value and the second loss function value.

[0116] Optionally, the values ​​of the first loss function and the second loss function can be summed based on preset weight coefficients to obtain the total loss function of the feature detection model.

[0117] Step S608: Determine whether the total loss function value meets the preset model training termination condition. If not, proceed to step S610; if so, proceed to step S612.

[0118] In some embodiments, when the total loss function value meets a preset convergence value, a judgment result can be obtained that the total loss function value meets the preset model training termination condition.

[0119] In some embodiments, when the total loss function value (θ) is updated in the i-th iteration i ) and the total loss function value (θ) updated in the (i-1)th iteration i-1 When the difference between the two values ​​is less than the preset difference threshold, the result of the judgment that the total loss function value meets the preset model training termination condition can be obtained.

[0120] Step S610: Update the model parameters of the feature detection model according to the total loss function value, and return to execute step S602.

[0121] Specifically, when the total loss function value does not meet the preset model training termination condition, it means that the model training has not yet ended. Then, the model parameters of the feature detection model are updated based on the total loss function value, and the execution step S602 is returned based on the updated feature detection model.

[0122] Step S612: End the training of the feature detection model and obtain the model to be evaluated.

[0123] Specifically, when the total loss function value meets the preset model training termination condition, the training of the feature detection model can be terminated, and the model to be evaluated can be obtained.

[0124] In summary, the model self-learning method provided in this embodiment performs self-learning training of the model based on the first loss function value and the second loss function value. The first loss function value, determined by combining the prediction results of the feature detection model in the positive and negative sample data pools, allows the feature detection model to learn new knowledge from more new events. This enables the model to adaptively adjust its parameters according to the distribution changes of new events, thereby improving the model's prediction performance for new events. The second loss function value, determined by the prediction results of the feature detection model in the positive sample data pool, ensures that the feature detection model does not forget the historical knowledge learned from historical events when using new events for training, thus steadily improving the model's prediction performance.

[0125] Example 4

[0126] Figure 7 This is a flowchart of a model self-learning method as another exemplary embodiment of this application. This embodiment can continue execution from step 612 described above. The following will combine... Figure 3A and Figure 3B The technical solution of this embodiment is described in detail.

[0127] Step S702: Construct new validation sets for multiple new events and historical validation sets for multiple historical events.

[0128] Specifically, a historical verification set can be constructed based on each historical event, and a new verification set can be constructed based on each new event;

[0129] Step S704: Use the feature detection model to perform predictions on the new validation set and the historical validation set respectively to obtain the first model evaluation value of the feature detection model, and use the model to be evaluated to perform predictions on the new validation set and the historical validation set respectively to obtain the second model evaluation value of the model to be evaluated.

[0130] In some embodiments, a feature detection model can be used to perform predictions on a new validation set, and based on the prediction results, evaluation metrics such as accuracy, coverage, and F1 score of the feature detection model corresponding to the new validation set can be calculated to obtain a first new evaluation value of the feature detection model corresponding to the new validation set. The feature detection model can also be used to perform predictions on a historical validation set, and based on the prediction results, evaluation metrics such as accuracy, coverage, and F1 score of the feature detection model corresponding to the historical validation set can be calculated to obtain a first historical evaluation value of the feature detection model corresponding to the historical validation set. Finally, based on the first new evaluation value and the first historical evaluation value, a first model evaluation value of the feature detection model can be obtained.

[0131] Similarly, the model to be evaluated can be used to perform predictions on the new validation set, and based on the prediction results, evaluation metrics such as accuracy, coverage, and F1 score of the model to be evaluated corresponding to the new validation set can be calculated to obtain the second new evaluation value of the model to be evaluated corresponding to the new validation set. The model to be evaluated can be used to perform predictions on the historical validation set, and based on the prediction results, evaluation metrics such as accuracy, coverage, and F1 score of the model to be evaluated corresponding to the historical validation set can be calculated to obtain the second historical evaluation value of the model to be evaluated corresponding to the historical validation set. Based on the second new evaluation value and the second historical evaluation value, the second model evaluation value of the model to be evaluated can be obtained.

[0132] Step S706: Determine whether the evaluation value of the second model is greater than the evaluation value of the first model. If yes, proceed to step S708; otherwise, proceed to step S710.

[0133] In this embodiment, the second model evaluation value of the model to be evaluated and the first model evaluation value of the feature detection model can be compared. If the second model evaluation value is greater than or equal to the first model evaluation value, it means that the model performance of the model to be evaluated is better than that of the feature detection model, and step S708 is executed. If the second model evaluation value is less than the first model evaluation value, step S710 is executed.

[0134] Optionally, if the second historical evaluation value of the model to be evaluated is greater than or equal to the first historical model evaluation value of the feature detection model, and the second new evaluation value of the model to be evaluated is greater than the first new evaluation value of the feature detection model (for example, the accuracy of the model to be evaluated corresponding to the new validation set is increased by 1% compared to the accuracy of the feature detection model corresponding to the new validation set), the evaluation result that the model performance of the model to be evaluated is better than that of the feature detection model can be obtained.

[0135] Step S708: Replace the feature detection model with the model to be evaluated to obtain a new feature detection model.

[0136] In some embodiments, the current feature detection model can be replaced with a higher-performing model to be evaluated (e.g., using...). Figure 3AThe update module shown updates the model parameters of the feature detection model based on the model parameters of the model to be evaluated, thereby obtaining a new feature detection model.

[0137] Step S710: Do not replace the feature detection model with the model to be evaluated.

[0138] Specifically, if the second historical evaluation value and the second newly added evaluation value of the model to be evaluated are both lower than the first historical evaluation value and the first newly added evaluation value of the feature detection model, then the current feature detection model will be retained and will not be updated.

[0139] In summary, the model self-learning method provided in this embodiment utilizes historical validation sets constructed from historical events and newly constructed validation sets from newly added events to evaluate the performance of the model to be evaluated and the feature detection model. This can improve the objectivity and accuracy of the model performance evaluation results and ensure the model optimization effect.

[0140] Example 5

[0141] Figure 8 A flowchart illustrating an exemplary embodiment of the event dispatch method of this application is shown, which mainly includes the following steps:

[0142] Step S802: Using a feature detection model, feature prediction is performed on the target event and multiple historical events to obtain the target event features and multiple historical event features of the target event. A similarity comparison is then performed based on the target event features and multiple historical event features to identify at least one similar event that is similar to the target event among the multiple historical events.

[0143] In this embodiment, the feature detection model is trained using the model self-learning method described in any of the above embodiments.

[0144] refer to Figure 1 The feature detection model can be set up on the server 102. The server 102 can obtain the target event uploaded by the client 106 through the communication network 104, and perform feature detection on the target event by the feature detection model to obtain the event features of the target event. Then, by calculating the similarity between the target event and each historical event, at least one similar event of the target event can be determined from each historical event.

[0145] In some embodiments, the acquired target event may include at least one of text data, image data, audio data, and video / audio data.

[0146] Step S804: Based on the actual dispatching departments of similar events, obtain the recommended dispatching department for the target event, and perform the dispatching process of the target event according to the recommended dispatching department.

[0147] Specifically, the department recommendation module set up on server 102 can be used (refer to the reference). Figure 3A The system obtains similar events of the target event from the feature detection model, and obtains the recommended dispatch department of the target event based on the real dispatch department of each similar event. The system then provides service score 102 to dispatch the target event to the event handling department 108 corresponding to the recommended dispatch department via communication network 104.

[0148] Therefore, the event dispatching method provided in this embodiment utilizes the feature detection model trained by the model self-learning method described in any of the above embodiments to collaboratively perform the dispatching and processing of the target task, which can effectively improve the event dispatching and processing efficiency and the event dispatching accuracy.

[0149] In summary, the model self-learning and event dispatching schemes provided in the embodiments of this application can, after the feature detection model is officially launched, determine the actual dispatching department of each event based on the online feedback logs of each event, thereby automatically expanding the training sample data of the feature detection model to enable sustainable model self-learning. Furthermore, based on multiple dimensions (e.g., accuracy, coverage, F1 score, etc.), the model performance of the feature detection model is compared with that of the model to be evaluated, so that the model to be evaluated with better performance can replace the feature detection model. Therefore, this invention not only allows the detection effect of the feature detection model to gradually improve as the number of dispatched events increases, but also enables adaptive adjustment of model parameters according to changes in the distribution of newly added events, ensuring the accuracy of event dispatching.

[0150] Example 6

[0151] Reference Figure 9 The diagram shows a structural schematic of an electronic device according to Embodiment 5 of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0152] like Figure 9 As shown, the electronic device may include: a processor 902, a communications interface 904, a memory 906, and a communications bus 908.

[0153] in:

[0154] The processor 902, communication interface 904, and memory 906 communicate with each other via communication bus 908.

[0155] Communication interface 904 is used to communicate with other electronic devices or servers.

[0156] The processor 902 is used to execute program 910, which can specifically execute the relevant steps in the above-described verification code generation method embodiment.

[0157] Specifically, program 910 may include program code that includes computer operation instructions.

[0158] The processor 902 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0159] Memory 906 is used to store program 910. Memory 906 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0160] Program 910 may include multiple computer instructions. Specifically, program 910 can cause processor 902 to execute the operations corresponding to the model self-learning method and event dispatching method described in any of the aforementioned multiple method embodiments through multiple computer instructions.

[0161] The specific implementation of each step in program 910 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0162] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0163] This application also provides a computer program product, including computer instructions that instruct a computing device to perform operations corresponding to the model self-learning method and event dispatching method described in any of the above embodiments.

[0164] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0165] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0166] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0167] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0168] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A model self-learning method, comprising: Using a feature detection model, the event features of the new event and the event features of multiple historical events are obtained. Based on the event features of the new event and the event features of the multiple historical events, at least one similar event that is similar to the new event is determined among the multiple historical events. Based on the actual dispatching departments of the similar events, the recommended dispatching department for the new events is obtained; Based on the recommended dispatch department of the new event and the actual dispatch department of the similar event, similar events whose actual dispatch department is the same as the recommended dispatch department of the new event are determined as positive samples of the new event, and similar events whose actual dispatch department is different from the recommended dispatch department of the new event are determined as negative samples of the new event. Based on the event characteristics of the newly added event and the event characteristics of the similar events, the similarity value between the newly added event and the similar events is obtained; Based on the similarity value between the newly added event and the similar events, at least one reference event is determined among the similar events, wherein the similarity value between the at least one reference event and the newly added event exceeds a first similarity threshold; Based on the department category, at least one real dispatching department of the at least one reference event is classified and statistically analyzed to obtain at least one statistical value of the at least one real dispatching department; Based on at least one statistical value of the at least one real distribution department, at least one reference distribution department is determined among the at least one real distribution department, wherein the statistical value of the reference distribution department is not lower than a statistical threshold. If the recommended dispatching department of the newly added event matches the at least one reference dispatching department, the newly added event is determined as a positive feedback event; if the recommended dispatching department of the newly added event does not match the reference dispatching department, the newly added event is determined as a negative feedback event. When the newly added event is determined to be the positive feedback event, sample data in the positive sample data pool is constructed using the newly added event, the positive samples and the negative samples of the newly added event; when the newly added event is determined to be the negative feedback event, sample data in the negative sample data pool is constructed using the newly added event, the positive samples and the negative samples of the newly added event. Using the sample data, perform model self-learning of the feature detection model.

2. The method according to claim 1, wherein, The step of using a feature detection model to obtain event features of a new event and event features of multiple historical events, and determining at least one similar event of the new event from the multiple historical events based on the event features of the new event and the event features of the multiple historical events, includes: Using the feature detection model, feature extraction is performed on the event description information and address description information of the newly added event to obtain the event features of the newly added event; and feature extraction is performed on the event description information and address description information of the multiple historical events to obtain the event features of the multiple historical events. A similarity calculation is performed based on the event characteristics of the newly added event and the event characteristics of the plurality of historical events to determine the similar events of the newly added event from the plurality of historical events.

3. The method according to claim 1, wherein, When the newly added event is determined to be the positive feedback event, sample data in the positive sample data pool is constructed using the newly added event, its positive samples, and negative samples. When the newly added event is determined to be the negative feedback event, sample data in the negative sample data pool is constructed using the newly added event, its positive samples, and negative samples, including: When the newly added event is determined to be the positive feedback event, the first positive sample pair in the positive sample data pool is constructed using the newly added event and any positive sample of the newly added event, and the first negative sample pair in the positive sample data pool is constructed using the newly added event and any negative sample of the newly added event. When the newly added event is determined to be the negative feedback event, a second positive sample pair in the negative sample data pool is constructed using the newly added event and any positive sample of the newly added event, and a second negative sample pair in the negative sample data pool is constructed using the newly added event and any negative sample of the newly added event.

4. The method according to claim 1, wherein, The sample data includes sample data in the positive sample data pool and sample data in the negative sample data pool of the feature detection model; The step of using the sample data to perform model self-learning of the feature detection model includes: Using the feature detection model, feature prediction is performed on the sample data pairs in the positive sample data pool to obtain the first prediction result of the positive sample data pool, and feature prediction is performed on the sample data in the negative sample data pool to obtain the second prediction result of the negative sample data pool. Based on the first prediction result and the second prediction result, a first loss function value is obtained, and based on the first prediction result, a second loss function value is obtained; The total loss function value is obtained by summing the first loss function value and the second loss function value. The model parameters of the feature detection model are updated according to the total loss function value, and the following steps are performed: using the feature detection model to perform feature prediction on sample data pairs in the positive sample data pool to obtain the first prediction result of the positive sample data pool, and performing feature prediction on sample data in the negative sample data pool to obtain the second prediction result of the negative sample data pool, until the total loss function value meets the preset model training termination condition, at which point the training of the feature detection model ends and the model to be evaluated is obtained.

5. The method according to claim 4, wherein, The sample data in the positive sample data pool includes a first positive sample pair and a first negative sample pair, and the sample data in the negative sample data pool includes a second positive sample pair and a second negative sample pair. The step of using the feature detection model to perform feature prediction on sample data pairs in the positive sample data pool to obtain a first prediction result for the positive sample data pool, and performing feature prediction on sample data in the negative sample data pool to obtain a second prediction result for the negative sample data pool, includes: Using the feature detection model, similarity calculation is performed on two events in the first positive sample pair in the positive sample data pool to obtain a first similarity value of the two events in the first positive sample pair. Similarity calculation is performed on two events in the first negative sample pair in the positive sample data pool to obtain a second similarity value of the two events in the first negative sample pair. Similarity calculation is performed on two events in the second positive sample pair in the negative sample data pool to obtain a third similarity value of the two events in the second positive sample pair. Similarity calculation is performed on two events in the second negative sample pair in the negative sample data pool to obtain a fourth similarity value of the two events in the second negative sample pair. The first prediction result is obtained based on the first similarity value and the second similarity value, and the second prediction result is obtained based on the third similarity value and the fourth similarity value.

6. The method according to claim 5, wherein, The step of obtaining a first loss function value based on the first prediction result and the second prediction result, and obtaining a second loss function value based on the first prediction result, includes: Cross-entropy is calculated based on the first similarity value and the second similarity value to obtain the first cross-entropy; cross-entropy is calculated based on the third similarity value and the fourth similarity value to obtain the second cross-entropy. Cross-entropy calculation is performed based on the first cross-entropy and the second cross-entropy to obtain the first loss function value, and the second loss function value is determined based on the first cross-entropy.

7. An event dispatching method, comprising: Using a feature detection model, feature detection is performed on a target event and multiple historical events to obtain the target event features and multiple historical event features of the multiple historical events. Then, a similarity comparison is performed based on the target event features and the multiple historical event features to determine at least one similar event that is similar to the target event among the multiple historical events. Based on the actual dispatching departments of the at least one similar event, a recommended dispatching department for the target event is obtained, and the dispatching process for the target event is performed based on the recommended dispatching department. The feature detection model is trained using the model self-learning method as described in any one of claims 1 to 6.

8. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform an operation corresponding to the method as described in any one of claims 1-6, or to perform an operation corresponding to the method as described in claim 7.

9. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-6, or implements the method as described in claim 7.

10. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods of claim 1-6, or to perform an operation corresponding to the method of claim 7.

Images