Log-based information reply method and device

Abstract

The invention provides a log-based information reply method and device. The method comprises the steps that user expression information is acquired, and log coding features of log data are acquired; synthesizing the obtained log coding features and information features extracted by a first pre-training language model for the user expression information to obtain a comprehensive result; and determining reply information which is output by the first pre-training language model based on the comprehensive result and responds to the user expression information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer application technology, and in particular to a log-based information reply method and device. Background Art

[0002] Currently, language models can be used to assist human-computer interaction. For example, a large language model can be used to respond to a user's input text or a user's input voice, specifically providing response information to the user.

[0003] In specific application scenarios, language models can be used to answer users' questions in natural language. For example, when users ask questions about specific concepts, language models can answer them in combination with relevant information on the Internet and provide users with answer information in natural language. Summary of the invention

[0004] In view of this, the present disclosure provides a log-based information reply method and device, which can improve the accuracy and comprehensiveness of the language model's reply.

[0005] According to a first aspect of an embodiment of the present disclosure, a log-based information reply method is provided, the method comprising:

[0006] Obtain user expression information and obtain log encoding features of log data;

[0007] Combining the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a comprehensive result;

[0008] Determine reply information output by the first pre-trained language model based on the comprehensive result in response to the user's expressed information.

[0009] Optionally, the method for determining the log coding feature includes:

[0010] The log data is input into a pre-trained log coding model, and log coding features output by the log coding model are determined.

[0011] Optionally, the training method of the log coding model includes:

[0012] Obtain an operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text is used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; train the log encoding model based on the log data samples in the log data sample set and the corresponding operation behavior text features.

[0013] Optionally, the training of the log encoding model according to the log data samples in the log data sample set and the corresponding operation behavior text features includes any of the following:

[0014] Determine the operation behavior text feature as a log coding feature label corresponding to the log data sample; train the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature label;

[0015] The log coding model is trained according to the log data samples in the log data sample set and the corresponding operation behavior text features to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features; the log current features include: the corresponding log data samples are input into the current log coding model, and the log coding features output by the current log coding model.

[0016] Optionally, the operation behavior text feature has the same size as the log current feature;

[0017] Improving the similarity between the operation behavior text feature corresponding to the same log data sample and the corresponding log current feature includes:

[0018] Improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reduce the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0019] Optionally, the operation behavior text features include: text features extracted from the operation behavior text using a second pre-trained language model.

[0020] Optionally, inputting the log data into a pre-trained log encoding model comprises:

[0021] Preprocess log data;

[0022] The preprocessed log data is input into the pre-trained log encoding model.

[0023] Optionally, the preprocessing of the log data includes at least one of the following:

[0024] Perform word segmentation on log data;

[0025] Set the beginning delimiter before the first character of the log data;

[0026] Set the end separator after the end character of the log data;

[0027] When it is determined that the log data includes different sentences, a separator is set between the different sentences;

[0028] In the case where it is determined that the log data includes time information, setting a time separator for the time information;

[0029] In a case where it is determined that the log data includes application information, setting an application separator for the application information;

[0030] In the case where the behavior keywords are extracted based on the log data, the extracted behavior keywords are added to the log data; the behavior keywords are used to describe the operation behavior information represented by the log data.

[0031] Optionally, the log coding feature of the acquired log data includes at least one of the following:

[0032] Obtain log encoding characteristics of pre-stored log data;

[0033] Obtaining a pre-stored log coding feature; the obtained log coding feature is determined based on the log data;

[0034] Determine the target log data required by the user according to the user expression information; and obtain the log encoding features of the target log data.

[0035] Optionally, the log encoding features obtained by synthesis and the information features extracted by the first pre-trained language model for the user expression information include any of the following:

[0036] Concatenating the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information;

[0037] Based on the cross attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

[0038] Optionally, the method further comprises:

[0039] The acquired log encoding features are input into a pre-trained log decoding model to determine the operation behavior text prediction result output by the log decoding model.

[0040] Optionally, the training method of the log decoding model includes:

[0041] Obtaining the log encoding features of any log data sample in the decoding sample set, and the operation behavior text corresponding to any log data sample;

[0042] The log decoding model is trained according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set.

[0043] Optionally, the training of the log decoding model according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set includes any of the following:

[0044] Determine the operation behavior text corresponding to any log data sample as the sample label corresponding to the log encoding feature of the same log data sample; train the log decoding model according to the log encoding feature and the corresponding sample label;

[0045] The log decoding model is trained according to the corresponding operation behavior text and log coding features of the same log data sample in the decoding sample set to improve the similarity between the current predicted text corresponding to the same log data sample and the corresponding operation behavior text; the current predicted text includes: the text prediction result output by the current log decoding model for the log coding features of the corresponding log data sample.

[0046] According to a second aspect of an embodiment of the present disclosure, a log-based information reply device is provided, the device comprising:

[0047] An acquisition unit, used to acquire user expression information and acquire log coding features of log data;

[0048] A synthesis unit, used to synthesize the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a synthesis result;

[0049] A determination unit is used to determine the reply information output by the first pre-trained language model based on the comprehensive result in response to the user's expression information.

[0050] Optionally, the method for determining the log coding feature includes:

[0051] The log data is input into a pre-trained log coding model, and log coding features output by the log coding model are determined.

[0052] Optionally, the training method of the log coding model includes:

[0053] Obtain an operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text is used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; train the log encoding model based on the log data samples in the log data sample set and the corresponding operation behavior text features.

[0054] Optionally, the training of the log encoding model according to the log data samples in the log data sample set and the corresponding operation behavior text features includes any of the following:

[0055] Determine the operation behavior text feature as a log coding feature label corresponding to the log data sample; train the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature label;

[0056] The log coding model is trained according to the log data samples in the log data sample set and the corresponding operation behavior text features to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features; the log current features include: the corresponding log data samples are input into the current log coding model, and the log coding features output by the current log coding model.

[0057] Optionally, the operation behavior text feature has the same size as the log current feature;

[0058] Improving the similarity between the operation behavior text feature corresponding to the same log data sample and the corresponding log current feature includes:

[0059] Improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reduce the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0060] Optionally, the operation behavior text features include: text features extracted from the operation behavior text using a second pre-trained language model.

[0061] Optionally, inputting the log data into a pre-trained log encoding model comprises:

[0062] Preprocess log data;

[0063] The preprocessed log data is input into the pre-trained log encoding model.

[0064] Optionally, the preprocessing of the log data includes at least one of the following:

[0065] Perform word segmentation on log data;

[0066] Set the beginning delimiter before the first character of the log data;

[0067] Set the end separator after the end character of the log data;

[0068] When it is determined that the log data includes different sentences, a separator is set between the different sentences;

[0069] In the case where it is determined that the log data includes time information, setting a time separator for the time information;

[0070] In a case where it is determined that the log data includes application information, setting an application separator for the application information;

[0071] In the case where the behavior keywords are extracted based on the log data, the extracted behavior keywords are added to the log data; the behavior keywords are used to describe the operation behavior information represented by the log data.

[0072] Optionally, the acquisition unit is configured to perform at least one of the following:

[0073] Obtain log encoding characteristics of pre-stored log data;

[0074] Obtaining a pre-stored log coding feature; the obtained log coding feature is determined based on the log data;

[0075] Determine the target log data required by the user according to the user expression information; and obtain the log encoding features of the target log data.

[0076] Optionally, the integration unit is configured to perform any of the following:

[0077] Concatenating the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information;

[0078] Based on the cross attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

[0079] Optionally, the device further comprises:

[0080] The decoding unit is used to input the acquired log encoding features into a pre-trained log decoding model to determine the operation behavior text prediction result output by the log decoding model.

[0081] Optionally, the training method of the log decoding model includes:

[0082] Obtaining the log encoding features of any log data sample in the decoding sample set, and the operation behavior text corresponding to any log data sample;

[0083] The log decoding model is trained according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set.

[0084] Optionally, the training of the log decoding model according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set includes any of the following:

[0085] Determine the operation behavior text corresponding to any log data sample as the sample label corresponding to the log encoding feature of the same log data sample; train the log decoding model according to the log encoding feature and the corresponding sample label;

[0086] The log decoding model is trained according to the corresponding operation behavior text and log coding features of the same log data sample in the decoding sample set to improve the similarity between the current predicted text corresponding to the same log data sample and the corresponding operation behavior text; the current predicted text includes: the text prediction result output by the current log decoding model for the log coding features of the corresponding log data sample.

[0087] According to a third aspect of an embodiment of the present disclosure, there is provided a non-temporary computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0088] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0089] processor;

[0090] a memory for storing processor-executable instructions;

[0091] Wherein, the processor is configured to implement the steps of any method described in the first aspect.

[0092] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by integrating user expression information and log encoding features of log data to predict reply information, the pre-trained language model can be combined with information in the log data to assist in replying, thereby improving the accuracy and comprehensiveness of the reply.

[0093] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0095] Figure 1 This is a schematic diagram of an interface for user interaction based on logs, showing an exemplary embodiment of the present disclosure;

[0096] Figure 2 A flowchart of a log-based information reply method is shown as an exemplary embodiment of the present disclosure;

[0097] Figure 3 A flow chart of a log encoding method is shown as an exemplary embodiment of the present disclosure;

[0098] Figure 4 A schematic diagram of a feature similarity matrix shown in an exemplary embodiment of the present disclosure;

[0099] Figure 5 A flow chart of a log decoding method shown as an exemplary embodiment of the present disclosure;

[0100] Figure 6 This is a schematic diagram of an interface of a log operation text prediction method according to an exemplary embodiment of the present disclosure;

[0101] Figure 7 A flowchart of another log-based information reply method according to an exemplary embodiment of the present disclosure;

[0102] Figure 8 A flowchart of another log-based information reply method according to an exemplary embodiment of the present disclosure;

[0103] Fig. 9 This is a schematic diagram of another interface for user interaction based on logs according to an exemplary embodiment of the present disclosure;

[0104] Fig.10 A block diagram of a log-based information reply device according to an exemplary embodiment of the present disclosure;

[0105] Fig.11 A block diagram of a log encoding device according to an exemplary embodiment of the present disclosure;

[0106] Fig.12 A block diagram of a log decoding device according to an exemplary embodiment of the present disclosure;

[0107] Fig.13 A block diagram of another log-based information reply device according to an exemplary embodiment of the present disclosure;

[0108] Fig.14 A block diagram of another log-based information reply device according to an exemplary embodiment of the present disclosure;

[0109] Fig.15 is a structural diagram of a log-based information reply device 1500 according to an exemplary embodiment;

[0110] Fig.16It is a schematic structural diagram of another log-based information reply device 1600 according to an exemplary embodiment. DETAILED DESCRIPTION

[0111] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0112] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0113] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0115] Currently, language models can be used to assist human-computer interaction. For example, a large language model can be used to respond to a user's input text or a user's input voice, specifically providing response information to the user.

[0116] In specific application scenarios, language models can be used to answer users' questions in natural language. For example, when users ask questions about specific concepts, language models can answer them in combination with relevant information on the Internet and provide users with answer information in natural language.

[0117] In the present disclosure, it is found that the current language model has less information that can be used to help answer questions, and the accuracy and comprehensiveness of the answers are low.

[0118] To this end, the present disclosure proposes a log-based information reply method.

[0119] In this method, it is found that language models can often only be learned and trained based on information in natural language form, so that the information used to help answer also needs to be in natural language form. For information in non-natural language form, for example, information in machine language form such as code and pseudocode, the language model often has difficulty understanding.

[0120] Furthermore, the method finds that log data in non-natural language form can be incorporated into a language model as information to assist in replying, which can improve the accuracy and comprehensiveness of the reply.

[0121] Among them, the log data can contain information about operational behaviors, thereby facilitating the language model to combine the information in the log data to improve the accuracy and comprehensiveness of the response.

[0122] Log data is important data for analyzing operational behaviors. In many applications, log data can be queried and filtered to facilitate users to view and analyze historical operations.

[0123] For example, search for logs about screen-off operations in the log data to analyze the duration of the screen-off operation; search for logs about application usage operations in the log data to analyze the usage duration of the application, etc.

[0124] Combined with log data, the language model can help respond based on information about operational behavior and provide corresponding responses to user needs.

[0125] For example, in response to users' needs to analyze historical operation behaviors, the language model can combine the information about historical operation behaviors in the log data to provide responses, thereby improving the accuracy and comprehensiveness of the responses.

[0126] For example, in response to users' needs to analyze their interests and hobbies, the language model can also combine information about the frequency of operational behaviors in the log data to provide responses, thereby improving the accuracy and comprehensiveness of the responses.

[0127] In specific applications, log data can be encoded.

[0128] In this method, the log encoding features obtained by encoding the log data can be used to participate in the prediction process of the language model, so that the language model can combine the information in the log data to help reply, thereby improving the accuracy and comprehensiveness of the reply.

[0129] The present disclosure provides multiple embodiments, and the various embodiments in the present disclosure can be combined with each other.

[0130] Each embodiment is explained in detail below.

[0131] For ease of understanding, the present disclosure provides a specific application example.

[0132] like Figure 1 As shown, Figure 1 The figure is a schematic diagram of an interface for user interaction based on logs, showing an exemplary embodiment of the present disclosure.

[0133] The form of a conversation interface may be adopted, and the conversation interface may include the conversation content between the user and the device, and may specifically include the user conversation content on the right and the device reply content on the left.

[0134] This interface can be an interface displayed on a mobile phone or on a computer, and the specific application device is not limited.

[0135] Users can enter "Please provide the number of times you have opened the food delivery app in the past 3 days."

[0136] The device can reply “The number of times the food delivery app has been opened in the past 3 days is 10 times”.

[0137] Among them, the device background can respond based on a log-based information reply method, combining user input text and log coding features.

[0138] Since the user's demand is about the number of operations, a log-based information reply method can be used to enable the language model to combine the operation behavior information in the log data to determine that the number of times the food delivery application is opened is 10 times, and then provide reply information to the user.

[0139] Users can also enter "Please recommend apps that you are interested in."

[0140] The device may reply, "Based on the analysis of your historical operating behavior in the logs, the top three application types you frequently use are games, shopping, and communications. Therefore, we recommend the xxx application to you."

[0141] Among them, the device background can respond based on a log-based information reply method, combining user input text and log coding features.

[0142] Since the user's demand is to recommend applications of interest, it is necessary to first analyze the user's interests. Through the log-based information reply method, the language model can be combined with the operational behavior information in the log data to determine the frequency of users opening various types of applications, and then provide reply information to the user.

[0143] The log-based information reply method can be applied to a device or an operating system or a program, so as to realize the corresponding function. In other words, the reply function of the conversation interface can be a function of a device or an operating system or a program.

[0144] The log-based information reply method can enable the language model to understand the information in the log data, so that the information in the log data can be used to help reply, thereby improving the accuracy and comprehensiveness of the reply.

[0145] like Figure 2 As shown, Figure 2 The present invention is a flowchart of a log-based information reply method according to an exemplary embodiment of the present invention.

[0146] The embodiments of the present disclosure do not limit the execution subject of the method. Optionally, the method can be applied to any computing device, specifically a mobile terminal device, a server device, a client device, etc. For example, a mobile phone, a tablet computer, a smart bracelet, VR glasses, etc.

[0147] The embodiments of the present disclosure do not limit the actual business meaning and source of the log data. Optionally, the log data may be a device log, an application log, a user operation log, a network log, a mobile phone log, a smart bracelet log, etc. Optionally, the log data may be obtained from the device, or the network log may be obtained from the server, etc.

[0148] The method may include at least the following steps:

[0149] S101: Obtain user expression information and obtain log encoding features of log data.

[0150] S102: Comprehensively obtain the acquired log encoding features and the information features extracted by the first pre-trained language model from the user expression information to obtain a comprehensive result.

[0151] S103: Determine reply information output by the first pre-trained language model based on the comprehensive result in response to the user's expressed information.

[0152] This method process can be used to predict reply information by integrating user expression information and log encoding features of log data, so that the pre-trained language model can be combined with information in the log data to help reply, thereby improving the accuracy and comprehensiveness of the reply.

[0153] The method process can use the log coding features obtained by encoding the log data to participate in the prediction process of the language model, so that the language model can combine the information in the log data to help reply, thereby improving the accuracy and comprehensiveness of the reply.

[0154] Each aspect is explained below.

[0155] 1. User expression information

[0156] This method does not limit the form and content of the information expressed by the user.

[0157] Optionally, the user expression information may be text or voice, so as to facilitate analysis and reply according to a multimodal language model. Specifically, the user expression information may be user input text, specifically text in natural language form.

[0158] Optionally, the user expression information may represent the user's needs or the user's questions.

[0159] For ease of understanding, an example of user expression information is given below.

[0160] For example, the user expression information may be a text input by the user, “Which application is used most frequently recently?”; the user expression information may also be a recorded user voice, in which the user expresses “How often do you open the shopping application?”.

[0161] 2. Log encoding characteristics.

[0162] This method does not limit the specific way of encoding log data, nor does it limit the specific way of determining log encoding features.

[0163] Optionally, a machine learning approach may be used to extract log encoding features from log data using a trained model.

[0164] Optionally, the log data may be encoded using a preset encoding rule to obtain a log encoding feature.

[0165] Optionally, the method for determining the log coding feature may specifically be: inputting the log data into a pre-trained log coding model, and determining the log coding feature output by the log coding model.

[0166] This embodiment can improve the representation effect of log coding features on log data through a log coding model.

[0167] This method does not limit the specific training method of the log coding model.

[0168] Optionally, the log data samples and the operation behavior information therein may be used to train the log encoding model.

[0169] Optionally, the log encoding model may be trained using log data samples and text describing the operation behavior in natural language.

[0170] Optionally, the training method of the log coding model may specifically be: obtaining the operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text may be used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; and train the log coding model based on the log data samples in the log data sample set and the corresponding operation behavior text features.

[0171] This embodiment can achieve encoding of log data by utilizing a log encoding model that learns the association between log data and operation behavior text, thereby improving the representation effect of log encoding features on log data.

[0172] Specifically, the information of operational behaviors can be introduced into model training so that the log coding model can learn the relationship between log data and operational behaviors, which can improve the log coding features extracted by the log coding model and the representation effect of log data.

[0173] In an optional embodiment, the operation behavior text may correspond to the log data sample, and the operation behavior text may be used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample.

[0174] For ease of understanding, several specific examples of operation behavior texts are given below.

[0175] "At 10:12, the screen automatically turns off based on the set screen-on time with no operation"; "At 10:15, a food delivery app was opened by clicking the app icon"; "At 10:19, an item was purchased on a shopping app", etc.

[0176] This embodiment does not limit the natural language form of the operation behavior text. Optionally, it can be described in natural language forms such as Chinese, English, and French.

[0177] This embodiment does not limit the content format of the operation behavior text. Optionally, the operation behavior text may include at least one of the following: time information, application information, triggering method, and actual operation information.

[0178] The process of this method does not limit the method of extracting the text features of the operation behavior.

[0179] Optionally, since the operation behavior text is required to participate in the training process of the log coding model, but because the log coding model needs to encode the log data into the form of a feature vector, the operation behavior text can also be converted into the form of a feature vector, so as to facilitate the training of the log coding model as a guide.

[0180] Therefore, optionally, the operation behavior text features can be directly extracted from the operation behavior text.

[0181] Optionally, in order to better characterize the operation behavior text, a model may be used to extract text features.

[0182] Optionally, a language model or a text model may be used to extract the operation behavior text features from the operation behavior text. The text model may specifically be a text feature model, such as a text convolution model. The language model may be, for example, a BERT model, a large language model, and the like.

[0183] Optionally, the second pre-trained language model may be used to extract operation behavior text features from the operation behavior text. The operation behavior text features may specifically be: text features extracted from the operation behavior text using the second pre-trained language model.

[0184] This embodiment can use the pre-trained language model to improve the representation ability of the operation behavior text features, which is convenient for further improving the training effect and representation ability of the log coding model in the future.

[0185] This embodiment does not limit the specific method of using the model to extract text features.

[0186] Optionally, the hidden layer output in the model can be used as the extracted text features.

[0187] Optionally, the operation behavior text may be input into a language model or a text model, and the output of any hidden layer in the input model may be determined as the extracted operation behavior text feature.

[0188] In a specific example, a second pre-trained language model can be used to extract operation behavior text features for the operation behavior text. The second pre-trained language model can include several Transformer layers, extract features based on the attention mechanism, and can also include a feature layer that further characterizes the output features of several Transformer layers, which can specifically be a fully connected layer or a linear layer. Therefore, optionally, the operation behavior text can be input into the second pre-trained language model, and the output of the fully connected layer or the linear layer in the input model is determined as the extracted operation behavior text features.

[0189] This method process does not limit the specific way of training the log encoding model based on the operation behavior text.

[0190] Optionally, a sample label may be determined based on the operation behavior text, and a model loss may also be determined based on the operation behavior text for training.

[0191] Optionally, the log encoding model is trained according to the log data samples in the log data sample set and the corresponding operation behavior text features, including any of the following:

[0192] 1) Determine the operation behavior text features as log coding feature labels corresponding to the log data samples; and train the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature labels.

[0193] 2) Based on the log data samples in the log data sample set and the corresponding operation behavior text features, the log encoding model is trained to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features.

[0194] Optionally, the current log feature may specifically be: a corresponding log data sample is input into a current log coding model, and the current log coding model outputs a log coding feature.

[0195] Optionally, specifically, for each log data sample in the log data sample set, the similarity between the operation behavior text feature corresponding to the same log data sample and the corresponding log current feature is improved.

[0196] This embodiment can combine the operation behavior text to train the log coding model to improve the representation ability of the log coding model.

[0197] Specifically, the initial log coding model may be trained according to the log data samples in the log data sample set and the corresponding operation behavior text features, and the trained initial log coding model is determined as the log coding model.

[0198] The current log feature may include: inputting the corresponding log data sample into the current log coding model, and the log coding feature output by the current log coding model.

[0199] The current log coding model may specifically be a current log coding model updated during the training update process.

[0200] Optionally, the operation behavior text feature and the log encoding feature may have the same size; or, the operation behavior text feature and the log encoding feature may have the same vector dimension.

[0201] Optionally, the operation behavior text feature and the log current feature may have the same size; or, the operation behavior text feature and the log current feature may have the same vector dimension.

[0202] This embodiment does not limit the calculation method of feature similarity. Optionally, the feature similarity can be calculated by using cosine similarity, Euclidean distance, etc.

[0203] In addition, the method flow does not limit the specific training method. Optionally, the training method of contrast learning can be adopted, and other training methods can also be adopted.

[0204] Optionally, positive and negative samples may be collected for training the log coding model. Specifically, the log data samples in the log data sample set and the corresponding log coding feature labels may be determined as positive samples; the log data samples in the log data sample set and the log coding feature labels corresponding to other log data samples may be combined as negative samples; and the log coding model may be trained based on the positive and negative samples.

[0205] Optionally, the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features is improved, specifically, it can be: improving the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reducing the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0206] This embodiment can train the log coding model by combining contrast learning, increase the number of samples, and improve the training effect and representation ability of the log coding model.

[0207] This method does not limit the input form of the log coding model.

[0208] Optionally, sample features in the log data sample, that is, the log data, may be preprocessed, and then the preprocessed log data may be input into the log encoding model.

[0209] Accordingly, the log data can be preprocessed and then input into the log coding model to extract the log coding features.

[0210] Optionally, the input form of the log encoding model may be adjusted according to the form of the preprocessed log data.

[0211] The method flow does not limit the specific pre-processing method.

[0212] In an optional embodiment, the serialized log data may be input into a log encoding model to facilitate subsequent processing.

[0213] For example, for log data, you can perform word segmentation and set separators to obtain serialized log data, which is convenient for subsequent transformer layer processing.

[0214] Optionally, preprocessing the log data may include at least one of the following:

[0215] 1) Perform word segmentation on the log data; 2) Set a beginning delimiter before the first character of the log data; 3) Set a ending delimiter after the last character of the log data; 4) When it is determined that the log data includes different sentences, set a spacing delimiter between different sentences; 5) When it is determined that the log data includes time information, set a time delimiter for the time information; 6) When it is determined that the log data includes application information, set an application delimiter for the application information; 7) When behavioral keywords are extracted based on the log data, add the extracted behavioral keywords to the log data; behavioral keywords are used to describe the operational behavior information represented by the log data.

[0216] 1) This embodiment does not limit the specific word segmentation method. Optionally, word segmentation can be performed based on spaces or preset rules.

[0217] 2) This embodiment does not limit the format of the beginning delimiter, and it may be cls.

[0218] 3) This embodiment does not limit the format of the end delimiter, and it may be end.

[0219] 4) This embodiment does not limit the format of the separator, and it may be sep.

[0220] 5) This embodiment does not limit the format of the time separator, and can be time. This embodiment does not limit the way to set the time separator for the time information. Optionally, the time separator can be set before or after the time information.

[0221] In this embodiment, by setting a time separator, the log coding model can be facilitated to locate the time information in the log data, thereby improving the representation capability of the log coding model.

[0222] 6) This embodiment does not limit the format of the application separator, and optionally, it can be app. This embodiment does not limit the way to set the application separator for application information. Optionally, the application separator can be set before or after the application information.

[0223] In this embodiment, the application delimiter can be set to facilitate the log coding model to locate the application information in the log data, thereby improving the representation capability of the log coding model.

[0224] 7) Optionally, behavior keywords can be used to describe the operation behavior information represented by the log data.

[0225] In an optional embodiment, the behavior keywords can be used to roughly describe the operation behavior represented by the log data, so as to facilitate the distinction between log data representing different operation behaviors, improve the degree of distinction between log data, and improve the prediction accuracy.

[0226] Optionally, the behavior keyword may describe information about the operation behavior, and the operation behavior information may specifically include at least one of the following: operation behavior type, operation behavior classification, operation behavior initiator, operation behavior triggering method, and the like.

[0227] The method flow is not limited to the method of extracting behavior keywords based on log data.

[0228] In an optional embodiment, the log data may contain keywords representing the operation behavior information, so that the keywords in the log data can be conveniently and directly extracted as behavior keywords.

[0229] For example, the log data may record the type of operation behavior, specifically whether the screen is turned off or on, so that the behavior keywords can be directly extracted.

[0230] Optionally, the log data may contain operation behavior information, and keywords in the log data need to be extracted through analysis as behavior keywords.

[0231] For example, the execution function name of the operation behavior may be recorded in the log data, so that the corresponding operation behavior type may be determined based on the execution function name, whether the screen is turned off or on.

[0232] Therefore, optionally, behavior keywords may be directly extracted from the log data, or operation behavior information in the log data may be analyzed to determine behavior keywords based on the analyzed operation behavior information.

[0233] It should be noted that for the relevant explanation of the process of this method, including the relevant explanation of the log coding features, please refer to the explanation in the subsequent method embodiments.

[0234] Different embodiments in the present disclosure may be explained with reference to each other and combined with each other.

[0235] 3. S101: Obtain user expression information and obtain log encoding features of log data.

[0236] This method does not limit the specific method of obtaining user expression information.

[0237] Optionally, the user expression information input by the user may be obtained, the collected user expression information may be obtained, or the user expression information pre-stored by the user may be obtained.

[0238] This method does not limit the way of obtaining the log coding features.

[0239] Optionally, a pre-stored log coding feature may be obtained, or a log coding feature determined in real time may be obtained.

[0240] This method does not limit the specific range of the log encoding features obtained.

[0241] Optionally, the log coding features of the log data within a preset range may be obtained, or the log coding features extracted from the log data meeting preset conditions may be obtained.

[0242] This embodiment does not limit the specific content and form of the preset range or preset conditions.

[0243] Optionally, the pre-stored log data may be directly obtained and the log coding features may be extracted in real time; pre-stored log coding features may be obtained; or log data or log coding features may be obtained according to specific requirements.

[0244] Optionally, obtaining the log coding feature of the log data may include at least one of the following:

[0245] 1) Obtain the log encoding characteristics of the pre-stored log data.

[0246] 2) Obtaining pre-stored log coding features; the obtained log coding features are determined based on the log data.

[0247] 3) Determine the target log data required by the user based on the information expressed by the user; and obtain the log encoding features of the target log data.

[0248] The method flow does not limit the specific way of determining the target log data according to the user expression information. Optionally, the target log data can be determined from the user expression information by preset rules, or the target log data required in the user expression information can be analyzed based on other pre-trained language models.

[0249] Optionally, when the target log data cannot be determined based on the user expression information, the target log data may be randomly selected, or log data within a fixed range may be determined as the target log data, for example, log data within the past 7 days may be determined as the target log data.

[0250] For example, the user expression information may specifically be "Please provide the number of screen-off times". If the target log data cannot be determined, the log data of the last 7 items may be selected for analysis.

[0251] 4. S102: Comprehensively obtain the log encoding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a comprehensive result.

[0252] The process of this method does not limit the specific synthesis method. Optionally, the synthesis can be performed by feature concatenation, channel stacking, addition, multiplication, cross attention mechanism, etc.

[0253] Optionally, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information may include any of the following:

[0254] 1) Concatenate the acquired log encoding features and the information features extracted by the first pre-trained language model based on the user expression information.

[0255] 2) Based on the cross-attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

[0256] This embodiment can integrate the log encoding features of the target log data with the user expression information to facilitate the prediction and analysis of the language model and improve the prediction accuracy.

[0257] It should be noted that for relevant explanations about the process of this method, including relevant explanations about the first pre-trained language model, please refer to the explanations in the subsequent method embodiments.

[0258] Different embodiments in the present disclosure may be explained with reference to each other and combined with each other.

[0259] 5. Log decoding.

[0260] In an optional embodiment, in order to facilitate users to view log data, the log data may be converted into a natural language form.

[0261] Based on the log encoding features in the above method flow, log decoding can be further introduced to obtain the operation behavior text corresponding to the log data, that is, the specific operation behavior represented by the corresponding log data described in natural language.

[0262] This method does not limit the specific log decoding method.

[0263] Optionally, a machine learning model may be used to decode the logs, or the logs may be decoded in combination with preset decoding rules.

[0264] Optionally, the above method flow may further include the following steps: inputting the acquired log encoding features into a pre-trained log decoding model, and determining an operation behavior text prediction result output by the log decoding model.

[0265] This embodiment does not limit the specific training method of the log decoding model.

[0266] Optionally, the training may be performed in combination with the operation behavior text, or in combination with the information in the log data.

[0267] Optionally, the training method of the log decoding model may include the following steps: obtaining the log coding features of any log data sample in the decoding sample set, and the operation behavior text corresponding to the same log data sample; training the log decoding model according to the corresponding operation behavior text and log coding features of the same log data sample in the decoding sample set.

[0268] Among them, optionally, in order to facilitate adaptation of the above method flow, the log coding feature can be determined based on the log coding model trained in any of the above method embodiments.

[0269] This embodiment does not limit the specific method of using the operation behavior text to train the log decoding model.

[0270] Optionally, training a log decoding model according to corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set may include any of the following:

[0271] 1) The operation behavior text corresponding to any log data sample is determined as the sample label corresponding to the log encoding feature of the same log data sample; and the log decoding model is trained according to the log encoding feature and the corresponding sample label.

[0272] 2) According to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set, the log decoding model is trained to improve the similarity between the current prediction text corresponding to the same log data sample and the corresponding operation behavior text.

[0273] Optionally, the current predicted text may be: a text prediction result output by a current log decoding model for a log encoding feature of a corresponding log data sample.

[0274] This embodiment can combine the operation behavior text to train the log decoding model to improve the decoding ability and training effect of the log decoding model.

[0275] This embodiment does not limit the calculation method of text similarity. Optionally, Jaccard similarity can be used, or the proportion of the same character string between two texts can be directly calculated.

[0276] It should be noted that for relevant explanations about the process of this method, please refer to the explanations in the subsequent method embodiments.

[0277] Different embodiments in the present disclosure may be explained with reference to each other and combined with each other.

[0278] In addition, the present disclosure proposes a log encoding method, which can be combined with a machine learning method and a model training method to train a log encoding model, and then the trained log encoding model can be used to encode log data.

[0279] When training the model, the operation behavior corresponding to the log data sample can be combined. Specifically, the operation behavior can be described in text using natural language. By extracting text features as a guide, it can be used to train the log encoding model.

[0280] This method can introduce the information of operation behavior to participate in model training, so that the log coding model can learn the relationship between log data and operation behavior, and can improve the log coding features extracted by the log coding model and the representation effect of log data.

[0281] like Figure 3 As shown, Figure 3 The figure is a flowchart of a log encoding method according to an exemplary embodiment of the present disclosure.

[0282] The embodiments of the present disclosure do not limit the execution subject of the method. Optionally, the method can be applied to any computing device, specifically a mobile terminal device, a server device, a client device, etc. For example, a mobile phone, a tablet computer, a smart bracelet, VR glasses, etc.

[0283] The embodiments of the present disclosure do not limit the actual business meaning and source of the log data. Optionally, the log data may be a device log, an application log, a user operation log, a network log, a mobile phone log, a smart bracelet log, etc. Optionally, the log data may be obtained from the device, or the network log may be obtained from the server, etc.

[0284] The method may include at least the following steps:

[0285] S201: inputting log data into a pre-trained log coding model, and determining log coding features output by the log coding model.

[0286] Among them, the training method of the log coding model can be: obtaining the operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text can be used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; train the log coding model according to the log data samples in the log data sample set and the corresponding operation behavior text features.

[0287] The method process can achieve the encoding of log data by utilizing the log encoding model that learns the association between log data and operation behavior text, thereby improving the representation effect of log encoding features on log data.

[0288] Specifically, the information of operational behaviors can be introduced into model training so that the log coding model can learn the relationship between log data and operational behaviors, which can improve the log coding features extracted by the log coding model and the representation effect of log data.

[0289] The above method flow is explained in detail below.

[0290] 1. About the log coding model.

[0291] 1. Training dataset.

[0292] This method does not limit the format and source of the training data set used to train the log encoding model.

[0293] Optionally, the training data set may be a log data sample set, including one or more log data samples.

[0294] Optionally, the sample features in the log data sample may include log data, which may be original log data or cleaned log data.

[0295] This embodiment does not limit the specific cleaning process. Optionally, the original log data may contain redundant information, such as automatically assigned process numbers, meaningless values ​​added to fill in the content, user-defined names or user inputs, etc. These redundant information are irrelevant or less related to the operation behavior represented by the log data, so the redundant information can be cleaned and processed to reduce the amount of data and improve the model training effect.

[0296] Optionally, redundant information in the original log data may be cleaned. Optionally, the redundant information may be information in the original log data that has no actual meaning. Optionally, the redundant information may specifically be information in the original log data that has a correlation with the operation behavior represented by the log data that is less than a preset correlation threshold.

[0297] This embodiment does not limit the cleaning process, and optionally, the redundant information may be directly deleted or fuzzy processed. This embodiment does not limit the fuzzy process, and specifically, the redundant information may be replaced with a fixed value or a null value.

[0298] Optionally, the log data samples can be labeled samples or unlabeled samples.

[0299] This embodiment does not limit the form and content of the label of the log data sample. Optionally, the label can be the operation behavior text corresponding to the log data sample.

[0300] Optionally, a pre-stored log data sample set may be directly acquired, or the log data sample set may be generated in real time by acquiring log data.

[0301] For ease of understanding, in a specific example, raw log data can be collected for a mobile phone. For example, for a screen-off operation, the raw log data can be collected as follows:

[0302] 05-29 09:58:47.600 1000 3346 3346 I ViewIcon: turnOffScreen;

[0303] 05-29 09:58:48.572 1000 3346 3346 I ViewIcon: setGxzwIconOpaque;

[0304] 05-29 09:58:55.209 1000 3346 3346 I ViewIcon: turnOnScreen mDozing=false; state=2.

[0305] After that, the original log data can be cleaned and analyzed. Specifically, the prefixes and suffixes of each original log data contain redundant information.

[0306] For example, each piece of raw log data in the ViewIcon service has numbers such as 1000, 3346, etc. These are process numbers automatically assigned by the current service and have no practical meaning. Therefore, you can perform cleaning processing on the automatically assigned process numbers, such as deleting them.

[0307] For another example, for the light perception data in the log, only the first one is the light perception intensity value, and the rest have no practical meaning, so the second light perception data and the subsequent light perception data can be cleaned, for example, deleted.

[0308] 2. Operation behavior text and operation behavior text features.

[0309] In an optional embodiment, the operation behavior text may correspond to the log data sample, and the operation behavior text may be used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample.

[0310] For ease of understanding, several specific examples of operation behavior texts are given below.

[0311] "At 10:12, the screen automatically turns off based on the set screen-on time with no operation"; "At 10:15, a food delivery app was opened by clicking the app icon"; "At 10:19, an item was purchased on a shopping app", etc.

[0312] This embodiment does not limit the natural language form of the operation behavior text. Optionally, it can be described in natural language forms such as Chinese, English, and French.

[0313] This embodiment does not limit the content format of the operation behavior text. Optionally, the operation behavior text may include at least one of the following: time information, application information, triggering method, and actual operation information.

[0314] The operation behavior text may correspond to the log data sample, and the present embodiment does not limit the form of this correspondence. Optionally, the log data sample and the operation behavior text may be in the form of a data pair.

[0315] The method flow does not limit the method for determining the operation behavior text. Optionally, the corresponding operation behavior text can be determined for the log data sample by manual summary. Optionally, the original log data can be analyzed by preset rules to determine the corresponding operation behavior text.

[0316] This method does not limit the method for obtaining the operation behavior text.

[0317] Optionally, a pre-determined and stored operation behavior text may be directly acquired, or the operation behavior text corresponding to the log data sample may be determined in real time.

[0318] Optionally, obtaining the operation behavior text corresponding to any log data sample in the log data sample set can specifically be directly obtaining the operation behavior text corresponding to any log data sample in the log data sample set from the pre-stored operation behavior text. The method for determining the pre-stored operation behavior text is not limited. Optionally, it can be determined manually or automatically by a machine.

[0319] Optionally, obtaining an operation behavior text corresponding to any log data sample in the log data sample set may specifically be determining the corresponding operation behavior text according to any log data sample in the log data sample set based on a preset text determination method.

[0320] This embodiment does not limit the specific implementation of the preset text determination method. Optionally, the log data sample can be analyzed based on preset rules to determine the corresponding operation behavior text.

[0321] Optionally, the corresponding operation behavior text may be determined by analyzing the keywords in the log data sample, for example, analyzing the time data and application data in the log data sample to determine that an operation involving a certain application is performed at a certain time.

[0322] The process of this method does not limit the method of extracting the text features of the operation behavior.

[0323] Optionally, since the operation behavior text is required to participate in the training process of the log coding model, but because the log coding model needs to encode the log data into the form of a feature vector, the operation behavior text can also be converted into the form of a feature vector, so as to facilitate the training of the log coding model as a guide.

[0324] Therefore, optionally, the operation behavior text features can be directly extracted from the operation behavior text.

[0325] Optionally, in order to better characterize the operation behavior text, a model may be used to extract text features.

[0326] Optionally, a language model or a text model may be used to extract the operation behavior text features from the operation behavior text. The text model may specifically be a text feature model, such as a text convolution model. The language model may be, for example, a BERT model, a large language model, and the like.

[0327] Optionally, a second pre-trained language model may be used to extract operation behavior text features for the operation behavior text. The operation behavior text features may specifically be: text features extracted from the operation behavior text using the second pre-trained language model. This embodiment may improve the characterization capability of the operation behavior text features with the help of a pre-trained language model, thereby facilitating further improvement of the training effect and characterization capability of the log coding model in the future.

[0328] This embodiment does not limit the specific method of using the model to extract text features.

[0329] Optionally, the hidden layer output in the model can be used as the extracted text features.

[0330] Optionally, the operation behavior text may be input into a language model or a text model, and the output of any hidden layer in the input model may be determined as the extracted operation behavior text feature.

[0331] In a specific example, a second pre-trained language model can be used to extract operation behavior text features for the operation behavior text. The second pre-trained language model can include several Transformer layers, extract features based on the attention mechanism, and can also include a feature layer that further characterizes the output features of several Transformer layers, which can specifically be a fully connected layer or a linear layer. Therefore, optionally, the operation behavior text can be input into the second pre-trained language model, and the output of the fully connected layer or the linear layer in the input model is determined as the extracted operation behavior text features.

[0332] Among them, optionally, in order to facilitate the control of the size of the operation behavior text feature, the output feature size of the hidden layer that outputs the operation behavior text feature can be controlled, specifically, the output feature size of the fully connected layer or the linear layer in the second pre-trained language model can be controlled.

[0333] Optionally, in order to facilitate the control of the vector dimension of the operation behavior text feature, the output feature vector dimension of the hidden layer that outputs the operation behavior text feature can be controlled, specifically, the output feature vector dimension of the fully connected layer or linear layer in the second pre-trained language model can be controlled.

[0334] 3. The training process of the log encoding model.

[0335] This method does not limit the specific training process of the log coding model.

[0336] In an optional embodiment, the log encoding model may be trained using text features of the operation behavior as guidance.

[0337] Optionally, the operation behavior text features corresponding to the log data sample may be determined as the log coding feature labels corresponding to the log data sample, and then the log coding model may be trained according to the log data sample and the corresponding log coding feature labels.

[0338] Optionally, the training direction of the model may be determined to improve the similarity between the operation behavior text features corresponding to the log data samples and the log encoding features.

[0339] Optionally, the log encoding model is trained according to the log data samples in the log data sample set and the corresponding operation behavior text features, including any of the following:

[0340] 1) Determine the operation behavior text features as log coding feature labels corresponding to the log data samples; and train the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature labels.

[0341] 2) Based on the log data samples in the log data sample set and the corresponding operation behavior text features, the log encoding model is trained to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features.

[0342] Optionally, specifically, for each log data sample in the log data sample set, the similarity between the operation behavior text feature corresponding to the same log data sample and the corresponding log current feature is improved.

[0343] This embodiment can combine the operation behavior text to train the log coding model to improve the representation ability of the log coding model.

[0344] Specifically, the initial log coding model may be trained according to the log data samples in the log data sample set and the corresponding operation behavior text features, and the trained initial log coding model is determined as the log coding model.

[0345] The current log feature may include: inputting the corresponding log data sample into the current log coding model, and the log coding feature output by the current log coding model.

[0346] The current log coding model may specifically be a current log coding model updated during the training update process.

[0347] Optionally, the operation behavior text feature and the log encoding feature may have the same size; or, the operation behavior text feature and the log encoding feature may have the same vector dimension.

[0348] Optionally, the operation behavior text feature and the log current feature may have the same size; or, the operation behavior text feature and the log current feature may have the same vector dimension.

[0349] This embodiment does not limit the calculation method of feature similarity. Optionally, the feature similarity can be calculated by using cosine similarity, Euclidean distance, etc.

[0350] In addition, the method flow does not limit the specific training method. Optionally, the training method of contrast learning can be adopted, and other training methods can also be adopted.

[0351] Optionally, positive and negative samples may be collected for training the log coding model. Specifically, the log data samples in the log data sample set and the corresponding log coding feature labels may be determined as positive samples; the log data samples in the log data sample set and the log coding feature labels corresponding to other log data samples may be combined as negative samples; and the log coding model may be trained based on the positive and negative samples.

[0352] Optionally, the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features is improved, specifically, it can be: improving the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reducing the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0353] This embodiment can train the log coding model by combining contrast learning, increase the number of samples, and improve the training effect and representation ability of the log coding model.

[0354] 4. The structure of the log coding model.

[0355] The process of this method does not limit the specific structure of the log coding model. The log coding model can encode the log data to obtain the log coding features.

[0356] Optionally, the log encoding model may adopt the structure of a language model or the structure of a neural network model.

[0357] In an optional embodiment, since the operation behavior text features are required to participate in the model training when training the log coding model, the output of the log coding model can be set according to the operation behavior text features.

[0358] Optionally, during the training process of the above-mentioned log coding model, the operation behavior text features can be determined as the log coding feature labels corresponding to the log data samples for model training; or the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features can be improved for model training, so that the output of the log coding model can be calculated accordingly with the operation behavior text features, that is, they can have the same size or vector dimension.

[0359] Optionally, the size of the operation behavior text feature may be set to the feature size output by the log coding model; or the vector dimension of the operation behavior text feature may be set to the feature vector dimension output by the log coding model.

[0360] Based on the model output settings, the operation behavior text features can be adapted to the log encoding model.

[0361] For ease of understanding, this method also provides an example of a log coding model structure.

[0362] Optionally, the log coding model may include a serial word embedding layer, a transformer layer, and a linear feature layer (fully connected layer). By using the fully connected layer as the output layer, the log coding model can output log coding features.

[0363] Specifically, when constructing the log encoding model, the size or vector dimension of the output feature of the output layer can be determined.

[0364] This method does not limit the source of the log coding model.

[0365] Optionally, a log encoding model may be directly constructed, or a structure adjustment may be performed on a pre-trained language model to obtain a log encoding model.

[0366] Optionally, a pre-trained language model can be obtained, which includes a series of word embedding layers, transformer layers, linear feature layers, and classification decoding layers. Then, the language model can be structurally adjusted, retaining the series of word embedding layers, transformer layers, and linear feature layers, and adjusting the output of the linear feature layer according to the text features of the operation behavior to obtain the log encoding model to be trained. For details, please refer to the above explanation.

[0367] Then, through further training, a trained log encoding model can be obtained. The specific training process can be found above.

[0368] 5. About the input of the log coding model.

[0369] This method does not limit the input form of the log coding model.

[0370] Optionally, sample features in the log data sample, that is, log data (the log data here may be original log data or original log data after cleaning), may be preprocessed, and then the preprocessed log data may be input into the log encoding model.

[0371] Optionally, the input form of the log encoding model may be adjusted according to the form of the preprocessed log data.

[0372] The method flow does not limit the specific pre-processing method.

[0373] In an optional embodiment, the serialized log data may be input into a log encoding model to facilitate subsequent processing.

[0374] For example, for log data, you can perform word segmentation and set separators to obtain serialized log data, which is convenient for subsequent transformer layer processing.

[0375] For ease of understanding, an example of a specific preprocessing is given below.

[0376] For the log data: "05-29 09:58:42.932 1000 3346 3346 I GxzwViewIcon: turnOnScreen mDozing = true; state = 1; BackTapTouchHelper: touch pointer listener has unregistered!", after cleaning and preprocessing, the following serialized log data can be obtained:

[0377] [cls]turnOnScreen[time]05-29 09:58:42.932[app]GxzwViewIcon[sep]turnOnScreen mDozing=true; state=1,[sep]BackTapTouchHelper: touch pointerlistener has unregistered! [sep].

[0378] Optionally, the preprocessing method may include word segmentation and adding separators.

[0379] Optionally, preprocessing the log data sample may include at least one of the following:

[0380] 1) Perform word segmentation on the log data sample. Optionally, the word segmentation may be performed on the sample features in the log data sample, that is, the log data.

[0381] This embodiment does not limit the specific word segmentation method. Optionally, word segmentation can be performed based on spaces or preset rules.

[0382] 2) Set the start delimiter before the first character of the log data sample.

[0383] Optionally, specifically, for the sample feature in the log data sample, that is, the log data, a start separator is set before the first character.

[0384] This embodiment does not limit the format of the beginning delimiter, and may be cls.

[0385] 3) Set the end delimiter after the last character of the log data sample.

[0386] Optionally, specifically, for the sample feature in the log data sample, that is, the log data, a terminal separator is set after the terminal character.

[0387] This embodiment does not limit the format of the end delimiter, and optionally, it may be end.

[0388] 4) When it is determined that the log data sample includes different sentences, a spacing separator is set between the different sentences.

[0389] Optionally, specifically, for the sample features in the log data sample, that is, the log data, when it is determined that different sentences are included therein, a spacing separator is set between different sentences.

[0390] This embodiment does not limit the format of the interval separator, and optionally, it may be sep.

[0391] 5) When it is determined that the log data sample includes time information, a time separator is set for the time information.

[0392] Optionally, specifically, for a sample feature in a log data sample, that is, the log data, when it is determined that the log data includes time information, a time separator is set for the time information.

[0393] This embodiment does not limit the format of the time separator, and can be time. This embodiment does not limit the way of setting the time separator for the time information. Optionally, the time separator can be set before or after the time information.

[0394] In this embodiment, by setting a time separator, the log coding model can be facilitated to learn and locate the time information in the log data, thereby improving the representation ability of the log coding model.

[0395] 6) When it is determined that the log data sample includes application information, an application delimiter is set for the application information.

[0396] Optionally, specifically, for sample features in the log data sample, that is, the log data, when it is determined that the log data includes application information, an application separator is set for the application information.

[0397] This embodiment does not limit the format of the application separator, and optionally, it can be app. This embodiment does not limit the way to set the application separator for application information. Optionally, the application separator can be set before or after the application information.

[0398] In this embodiment, the application delimiter can be set to facilitate the log coding model to learn and locate application information in the log data, thereby improving the representation ability of the log coding model.

[0399] 7) When the behavior keywords are extracted based on the log data sample, the extracted behavior keywords are added to the log data sample.

[0400] Optionally, specifically, for sample features in the log data sample, that is, the log data, when behavior keywords are extracted based on the log data, the extracted behavior keywords are added to the log data.

[0401] Optionally, the behavior keyword may be used to describe the operation behavior information represented by the log data sample.

[0402] In an optional embodiment, behavioral keywords can be used to make a rough description of the operational behaviors represented by the log data samples, so as to facilitate the distinction between log data samples representing different operational behaviors, improve the degree of sample distinction, facilitate subsequent model training, improve the model training effect, and also improve the prediction accuracy by adding behavioral keywords during the model prediction process.

[0403] Optionally, the behavior keyword may describe information about the operation behavior, and the operation behavior information may specifically include at least one of the following: operation behavior type, operation behavior classification, operation behavior initiator, operation behavior triggering method, and the like.

[0404] The method flow does not limit the way of extracting behavior keywords based on log data samples.

[0405] In an optional embodiment, the log data may contain keywords representing the operation behavior information, so that the keywords in the log data can be conveniently and directly extracted as behavior keywords.

[0406] For example, the log data may record the type of operation behavior, specifically whether the screen is turned off or on, so that the behavior keywords can be directly extracted.

[0407] Optionally, the log data may contain operation behavior information, and keywords in the log data need to be extracted through analysis as behavior keywords.

[0408] For example, the execution function name of the operation behavior may be recorded in the log data, so that the corresponding operation behavior type may be determined based on the execution function name, whether the screen is turned off or on.

[0409] Therefore, optionally, behavior keywords may be directly extracted from the log data sample, or operation behavior information in the log data sample may be analyzed to determine behavior keywords based on the analyzed operation behavior information.

[0410] This embodiment does not limit the specific direct extraction method and analysis method.

[0411] Optionally, the behavior keyword can be extracted from a fixed position of the log data sample, or the operation behavior information can be analyzed based on the information in the log data sample, and the behavior keyword corresponding to the analyzed operation behavior information can be further determined based on a preset correspondence between the operation behavior information and the behavior keyword.

[0412] This embodiment can add behavioral keywords to facilitate the distinction of log data samples representing different operating behaviors, improve the degree of sample distinction, facilitate subsequent model training, and improve the model training effect. It can also improve the prediction accuracy by adding behavioral keywords during the model prediction process.

[0413] 6. About losses.

[0414] This method does not limit the loss calculation method of the log encoding model during the training process.

[0415] Alternatively, cross entropy loss, similarity loss, Euclidean distance loss, etc. may be used.

[0416] In an optional embodiment, the operation behavior text feature may be determined as a log coding feature label corresponding to the log data sample; and a log coding model may be trained based on the log data samples in the log data sample set and the corresponding log coding feature labels.

[0417] This embodiment does not limit the specific loss calculation method. Optionally, the loss between the log coding feature prediction result and the log coding feature label can be directly calculated, which can be a similarity loss or a distance loss.

[0418] In another optional embodiment, a log encoding model can be trained based on log data samples in a log data sample set and corresponding operation behavior text features to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features.

[0419] This embodiment does not limit the specific loss calculation method. Optionally, the loss can be set to be negatively correlated with the above feature similarity. The above feature similarity can be the similarity between the operation behavior text feature corresponding to the same log data sample and the corresponding log current feature.

[0420] Optionally, the loss may specifically be the reciprocal of the above feature similarity, or its opposite.

[0421] Optionally, the feature similarities corresponding to each log data sample in the log data sample set may be summed up, and the reciprocal or opposite of the sum may be determined as the loss. The feature similarity corresponding to a single log data sample may specifically be the similarity between the operation behavior text feature corresponding to the single log data sample and the log current feature corresponding to the same log data sample.

[0422] This embodiment does not limit the calculation method of feature similarity. Optionally, the feature similarity can be calculated by using cosine similarity, Euclidean distance, etc.

[0423] Optionally, the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features is improved, specifically, it can be: improving the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reducing the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0424] Optionally, the loss may be calculated in combination with the similarity of negative samples, where the similarity of negative samples is specifically the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0425] For ease of understanding, this embodiment also provides an example of a method for calculating loss.

[0426] Optionally, for N log data samples, corresponding log current features I and corresponding operation behavior text features T may be obtained respectively.

[0427] Specifically, for the first log data sample, the corresponding log current feature I1 and the corresponding operation behavior text feature T1 can be obtained respectively.

[0428] For the second log data sample, the corresponding log current feature I2 and the corresponding operation behavior text feature T2 can be obtained respectively.

[0429] For the third log data sample, the corresponding log current feature I3 and the corresponding operation behavior text feature T3 can be obtained respectively.

[0430] By analogy, N operation behavior text features T1-TN and N log current features I1-IN can be obtained.

[0431] Then, T1 and I1 can be used as positive samples, and so on, Ti and Ii can be used as positive samples to train the log coding model and improve the similarity between Ti and Ii, which can be cosine similarity.

[0432] T1 and I2 can also be used as negative samples, and so on, Ti and Ij can be used as negative samples, i is not equal to j, and the log coding model can be trained to reduce the similarity between Ti and Ij. Specifically, the cosine similarity can be used.

[0433] Optionally, N operation behavior text features T1-TN and N log current features I1-IN may be combined to obtain N*N similarities for determining the loss and training the model.

[0434] like Figure 4 As shown, Figure 4 The figure is a schematic diagram of a feature similarity matrix according to an exemplary embodiment of the present disclosure.

[0435] Among them, for each text feature in the N operation behavior text features T1-TN, they are combined with the N log current features I1-IN to obtain N*N similarities, which are filled into the N*N matrix respectively. The cosine similarity is calculated specifically. For the convenience of description, the cosine similarity is described as (T; I).

[0436] The label corresponding to (Ti; Ii) can be 1, the label corresponding to (Ti; Ij) can be 0, and i is not equal to j.

[0437] Therefore, the matrix label of the overall feature similarity matrix can be that the elements on the diagonal from the upper left to the lower right are 1, and the remaining elements are 0. For details, see Figure 4 .

[0438] 2. Regarding S201: inputting log data into a pre-trained log coding model to determine log coding features output by the log coding model.

[0439] 1. About input.

[0440] The method flow does not limit the specific input method.

[0441] Optionally, the log data may be directly input into the log coding model, or the log data may be adjusted according to the input format of the log coding model.

[0442] In an optional embodiment, in the process of training the log encoding model, the log data samples may be preprocessed, and correspondingly, the log data may also be preprocessed accordingly.

[0443] Optionally, after preprocessing the log data samples, when the preprocessed log data samples are used for log coding model training, the same preprocessing can be performed on the log data, and the preprocessed log data can be input into the pre-trained log coding model to determine the log coding features output by the log coding model.

[0444] Optionally, the log data is input into a pre-trained log encoding model, which may specifically include: pre-processing the log data; and inputting the pre-processed log data into the pre-trained log encoding model.

[0445] This embodiment does not limit the preprocessing method. Optionally, the preprocessing of log data may include at least one of the following:

[0446] 1) Perform word segmentation on the log data; 2) Set a beginning delimiter before the first character of the log data; 3) Set a ending delimiter after the last character of the log data; 4) When it is determined that the log data includes different sentences, set a spacing delimiter between different sentences; 5) When it is determined that the log data includes time information, set a time delimiter for the time information; 6) When it is determined that the log data includes application information, set an application delimiter for the application information; 7) When behavioral keywords are extracted based on the log data, add the extracted behavioral keywords to the log data; behavioral keywords are used to describe the operational behavior information represented by the log data.

[0447] For detailed explanation, please refer to the above embodiments.

[0448] 2. About the output log encoding characteristics.

[0449] In an optional embodiment, since the model training is performed in combination with the operation behavior text, and the operation behavior text features can be used as a guide for model training, the log coding features encoded by the log coding model for the input log data can characterize the operation behavior text corresponding to the log data to a certain extent, and include relevant information of the operation behavior text.

[0450] Therefore, the output log encoding feature can be applied to a variety of scenarios. This method process is not limited to the application of the log encoding feature.

[0451] Optionally, the data volume of the output log coding feature may be smaller than the input log data, so that the log data can be stored conveniently in the form of coding features, thereby improving storage efficiency.

[0452] Optionally, since the log coding feature may include information of the operation behavior text, the operation behavior text corresponding to the log data may be determined by decoding, thereby achieving natural language interpretation of the log data.

[0453] Optionally, since the log encoding features can contain information about the operation behavior text, it is possible to combine with the language model to determine the user's needs based on the text input by the user, analyze the log data, and reply the analysis results to the user in the form of natural language to achieve interactive dialogue with the user.

[0454] 3. Log decoding.

[0455] The following further applications can be performed for the log encoding features. Specifically, the logs can be further decoded according to the log encoding features to obtain the corresponding operation behavior text.

[0456] like Figure 5 As shown, Figure 5 The figure is a flow chart of a log decoding method according to an exemplary embodiment of the present disclosure.

[0457] The embodiments of the present disclosure do not limit the execution subject of the method. Optionally, the method can be applied to any computing device, specifically a mobile terminal device, a server device, a client device, etc. For example, a mobile phone, a tablet computer, a smart bracelet, VR glasses, etc.

[0458] The embodiments of the present disclosure do not limit the actual business meaning and source of the log data. Optionally, the log data may be a device log, an application log, a user operation log, a network log, a mobile phone log, a smart bracelet log, etc. Optionally, the log data may be obtained from the device, or the network log may be obtained from the server, etc.

[0459] The method may include at least the following steps:

[0460] S301: Obtain log coding features corresponding to log data.

[0461] Optionally, the log encoding feature may be determined based on any of the above log encoding method embodiments. Optionally, the log encoding feature may be determined based on a pre-trained log encoding model.

[0462] S302: Input the acquired log encoding features into a pre-trained log decoding model to determine the operation behavior text prediction result output by the log decoding model.

[0463] Optionally, the operation behavior text prediction result can be used to: use natural language to describe the specific operation behavior represented by the corresponding log data.

[0464] The method process can predict the operation behavior text corresponding to the log data based on the log decoding model. The operation behavior text is in the form of natural language, which can facilitate users to understand and analyze the log data.

[0465] This method does not limit the way of obtaining the log coding features.

[0466] Optionally, a predetermined log coding feature may be directly acquired, or the log coding feature may be determined in real time based on any of the above method embodiments according to log data.

[0467] Optionally, the log encoding feature may be determined based on a pre-trained log encoding model.

[0468] This method does not limit the training method of the log decoding model.

[0469] Optionally, the log encoding model can be combined to train the log decoding model using the log encoding features output by the log encoding model and the corresponding operation behavior text labels.

[0470] Optionally, the training method of the log decoding model can specifically be: obtaining the log coding features of any log data sample in the decoding sample set, and the operation behavior text corresponding to the same log data sample; training the log decoding model according to the corresponding operation behavior text and log coding features of the same log data sample in the decoding sample set.

[0471] Among them, optionally, in order to facilitate adaptation to any of the above method embodiments, the sample coding features can be determined based on the log coding model trained in any of the above method embodiments.

[0472] This embodiment does not limit the decoding sample set. Optionally, the decoding sample set may include one or more log data samples, and specifically may also include the log data sample set in the above method embodiment.

[0473] This embodiment does not limit the method for obtaining the sample coding feature. Optionally, a predetermined sample coding feature may be obtained, or the sample coding feature may be determined and obtained in real time for the log data sample in the decoding sample set.

[0474] This embodiment does not limit the method for obtaining the operation behavior text. Optionally, a predetermined operation behavior text may be obtained, or the operation behavior text may be obtained in real time based on a preset method and a log data sample.

[0475] In an optional embodiment, training a log decoding model according to corresponding operation behavior text and log encoding features of the same log data sample in a decoding sample set may include any of the following:

[0476] 1) The operation behavior text corresponding to any log data sample is determined as the sample label corresponding to the log encoding feature of the same log data sample; and the log decoding model is trained according to the log encoding feature and the corresponding sample label.

[0477] 2) According to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set, the log decoding model is trained to improve the similarity between the current prediction text corresponding to the same log data sample and the corresponding operation behavior text.

[0478] The current predicted text may include: the text prediction result output by the current log decoding model for the log encoding feature of the corresponding log data sample.

[0479] This embodiment can combine the operation behavior text to train the log decoding model to improve the decoding ability and training effect of the log decoding model.

[0480] This embodiment does not limit the calculation method of text similarity. Optionally, Jaccard similarity can be used, or the proportion of the same character string between two texts can be directly calculated.

[0481] This embodiment does not limit the calculation method of the loss during the training of the log decoding model. For specific explanations, please refer to the above embodiment.

[0482] The method flow does not limit the structure of the log decoding model. Optionally, the log decoding model can adopt the structure of a language model or the structure of a neural network.

[0483] In a specific example, the log decoding model may include at least one transformer layer and an output layer, which may output predicted text.

[0484] In another specific example, the log decoding model can adopt the output layer in the pre-trained language model to output predicted text to facilitate subsequent training.

[0485] In another specific example, the operation behavior text feature may be output by any hidden layer in the model, specifically any hidden layer in the second pre-trained language model, and optionally, the log decoding model may include a model structure after the hidden layer for outputting the operation behavior text feature. Optionally, the log decoding model may also add a feature layer, such as a transformer layer and a linear feature layer, to the model structure after the hidden layer for outputting the operation behavior text feature.

[0486] In addition, the embodiment of the present disclosure also provides a method embodiment.

[0487] In an optional embodiment, the log encoding model and the log decoding model are combined to obtain a model for predicting the corresponding operation behavior text for the log data.

[0488] For the convenience of description, this model can be called a log operation text prediction model.

[0489] Optionally, the log operation text prediction model can be used to predict the corresponding operation behavior text for the input log data. The predicted operation behavior text can be used to: use natural language to describe the specific operation behavior represented by the corresponding log data.

[0490] This embodiment does not limit the content and form of the training data set.

[0491] Optionally, for the log data sample, the corresponding operation behavior text may be determined as a label for training the log operation text prediction model.

[0492] This embodiment does not limit the structure of the log operation text prediction model.

[0493] Optionally, the log operation text prediction model may include a log encoding model and a log decoding model, and the specific explanation may refer to the above embodiment.

[0494] In an optional embodiment, a log operation text prediction method may be provided, and the method flow may include the following steps: inputting log data into a pre-trained log operation text prediction model, and determining an operation behavior text prediction result output by the log operation text prediction model.

[0495] Optionally, the training method of the log operation text prediction model can be: obtaining the operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text can be used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; based on the log data samples in the log data sample set and the corresponding operation behavior text, train the log operation text prediction model.

[0496] For detailed explanation, please refer to the above embodiments.

[0497] The above-mentioned log decoding embodiment can further decode the log data or log coding features into operation behavior text, thereby facilitating user viewing and analysis, and improving the viewing efficiency and analysis efficiency of the log.

[0498] Optionally, target log data determined by the user may be obtained, the target log data may be input into a pre-trained log operation text prediction model, the operation behavior text prediction result output by the log operation text prediction model may be determined, and the determined operation behavior text prediction result may be provided to the user.

[0499] Specifically, the operation behavior text prediction result may be displayed to the user.

[0500] This embodiment does not limit the method by which the user determines the target log data. Optionally, the target log data may be log data specified by the user or may be log data input by the user.

[0501] This embodiment can improve the efficiency of users viewing log data through interaction with users.

[0502] For easier understanding, a specific application example is provided below.

[0503] When log analysis is required, the log data to be analyzed is passed through the log encoding model to obtain log encoding features, and the log encoding features are further passed through the log decoding model to obtain the predicted operation behavior text as auxiliary information for log analysis.

[0504] For example, when a user needs to view his or her historical operations, directly displaying the original log data is cumbersome and difficult for the user to understand. However, by converting the log data into corresponding operation behavior text and describing the log operations in natural language, it is convenient for users to view historical operations.

[0505] For example, when developers need to view log analysis problems, they can convert log data into corresponding operation behavior texts and describe the log operations in natural language as auxiliary information, which can facilitate developers to view and analyze. In addition, since the operation behavior text in natural language form has a smaller data volume than log data, it can improve storage efficiency and facilitate developers to view more operation behavior texts.

[0506] Specifically, the log data that meets the specified conditions selected by the developer can be converted into corresponding operation behavior text, for example, log data within a certain time period, log data of a certain application, log data of a certain user, etc.

[0507] Optionally, in order to facilitate user interaction, a correspondence can be established between the log data and the corresponding operation behavior text prediction results, so that the operation behavior text prediction results can be located to the corresponding log data, thereby facilitating viewing the original log data for subsequent detailed analysis.

[0508] This embodiment does not limit the specific corresponding relationship establishment method. Optionally, the storage address of the corresponding log data can be set for the operation behavior text prediction result. Optionally, the storage location information of the corresponding log data can be set for the operation behavior text prediction result. Optionally, the query link of the corresponding log data can be set for the operation behavior text prediction result.

[0509] For ease of understanding, Figure 6 As shown, Figure 6The figure is a schematic diagram of an interface of a log operation text prediction method shown as an exemplary embodiment of the present disclosure.

[0510] Among them, a query interface may be used, which may include a condition query panel and a result display panel. The log condition may be set in the condition query panel, and the operation behavior text corresponding to the queried log data may be displayed in the result display panel.

[0511] Specific log conditions may include: time period, application, and user.

[0512] After setting the log condition and clicking query, the operation behavior text corresponding to the queried log data can be displayed in the result display panel, which can be the operation behavior text predicted by the background for the queried log data based on the log encoding model and the log decoding model. The text prediction can be performed in the manner of the above method embodiment.

[0513] Specifically, the operation behavior texts such as "22:01, the device automatically turns off the screen" and "22:04, click the screen to turn on the screen" can be displayed to facilitate user viewing and analysis.

[0514] For the operation behavior text that needs to be viewed, the corresponding original log data can be displayed by clicking the operation behavior text.

[0515] This interface may be an interface displayed on a mobile phone or an interface displayed on a computer.

[0516] 4. Responses based on language models.

[0517] The current language model can be used to respond to user input text, specifically to answer questions or meet needs, etc.

[0518] Based on the log encoding features, since it contains information about the operation behavior text, it can be combined with the language model to respond to and analyze user needs regarding logs.

[0519] For example, in response to the user input text "Please provide the number of times the screen has been on in the past three days", the language model can combine the log encoding features to count the screen-on operation logs in the past three days and get a corresponding reply.

[0520] like Figure 7 As shown, Figure 7 The present invention is a flowchart of another log-based information reply method according to an exemplary embodiment of the present invention.

[0521] The embodiments of the present disclosure do not limit the execution subject of the method. Optionally, the method can be applied to any computing device, specifically a mobile terminal device, a server device, a client device, etc. For example, a mobile phone, a tablet computer, a smart bracelet, VR glasses, etc.

[0522] The embodiments of the present disclosure do not limit the actual business meaning and source of the log data. Optionally, the log data may be a device log, an application log, a user operation log, a network log, a mobile phone log, a smart bracelet log, etc. Optionally, the log data may be obtained from the device, or the network log may be obtained from the server, etc.

[0523] The method may include at least the following steps:

[0524] S401: Determine target log data required by the user based on the information expressed by the user.

[0525] S402: For the determined target log data, obtain corresponding log coding features.

[0526] Optionally, the log encoding feature may be determined based on any of the above log encoding method embodiments. Optionally, the log encoding feature may be determined based on a pre-trained log encoding model.

[0527] S403: Comprehensively obtain the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a comprehensive result.

[0528] S404: Determine a reply output by the first pre-trained language model based on the comprehensive result in response to the user's expressed information.

[0529] This method process can be based on a pre-trained language model and combined with log encoding features to facilitate analysis and response based on log data.

[0530] 1. About S401.

[0531] This method does not limit the form and content of the information expressed by the user.

[0532] Optionally, the user expression information may be text or voice, so as to facilitate analysis and reply according to a multimodal language model. Specifically, the user expression information may be user input text, specifically text in natural language form.

[0533] Optionally, the user expression information may include user requirements, thereby facilitating determination of target log data that meets the user requirements.

[0534] The method flow does not limit the specific way of determining the target log data according to the user expression information. Optionally, the target log data can be determined from the user expression information by preset rules, or the target log data required in the user expression information can be analyzed based on other pre-trained language models.

[0535] Optionally, when the target log data cannot be determined based on the user expression information, the target log data may be randomly selected, or log data within a fixed range may be determined as the target log data, for example, log data within the past 7 days may be determined as the target log data.

[0536] For example, the user expression information may specifically be "Please provide the number of screen-off times". If the target log data cannot be determined, the log data of the last 7 items may be selected for analysis.

[0537] 2. About S402.

[0538] This method does not limit the way of obtaining the log coding features.

[0539] Optionally, a predetermined log coding feature may be directly acquired, or the log coding feature may be determined in real time based on any of the above log coding method embodiments according to target log data.

[0540] 3. About S403.

[0541] The process of this method does not limit the specific synthesis method. Optionally, the synthesis can be performed by feature concatenation, channel stacking, addition, multiplication, cross attention mechanism, etc.

[0542] Optionally, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information may include any of the following:

[0543] 1) Concatenate the acquired log encoding features and the information features extracted by the first pre-trained language model based on the user expression information.

[0544] 2) Based on the cross-attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

[0545] This embodiment can integrate the log encoding features of the target log data with the user expression information to facilitate the prediction and analysis of the language model and improve the prediction accuracy.

[0546] 4. About S404.

[0547] The method flow does not limit the prediction process of the first pre-trained language model.

[0548] Optionally, the first pre-trained language model may first extract information features from the input user expression information, further integrate the information features with the input log encoding features, and then perform subsequent predictions based on the integrated results.

[0549] Optionally, the first pre-trained language model may also directly integrate the input user expression information and log encoding features to perform subsequent predictions.

[0550] Optionally, the first pre-trained language model may output a reply in response to the user's expressed information based on the comprehensive result through subsequent prediction. The reply may include the situation of the target log data.

[0551] 5. About the structure and training process of the first pre-trained language model.

[0552] The method flow does not limit the structure of the first pre-trained language model.

[0553] Optionally, the first pre-trained language model may include a word embedding layer, a transformer layer, a linear feature layer, an output layer, and the like. Among them, for the log coding features that need to be input, an additional input of the log coding features may be set for the first pre-trained language model. And, for the comprehensive results, due to the expansion of the feature size, the input of the corresponding layer may be further adjusted, and a pooling layer may be added to adjust the feature size.

[0554] In a specific example, since the cost of training a language model, such as a large language model, is relatively high, optionally, a language model capable of implementing information reply may be first obtained, and the structure may be further adjusted.

[0555] Specifically, for a certain hidden layer, a feature input can be set to input the log coding feature, and a model layer of comprehensive information features and log coding features can be further added. The input size of the next layer of the added model layer can also be adjusted.

[0556] In addition, for the added model layer, a feature layer and / or pooling layer may be further added later, and the feature layer may specifically be a transformer layer or a linear feature layer, etc. The pooling layer may be used to unify the feature size to the original size, so as to facilitate adaptation to the feature size of the original language model.

[0557] Optionally, the first pre-trained language model may include a word embedding layer, a transformer layer, and subsequent model results in series. A comprehensive layer, a feature layer, and a pooling layer may be added in series between the word embedding layer and the transformer layer. The comprehensive layer may be used to integrate the information features and log coding features of the user's expression information. The feature layer may be used for further characterization, specifically a new transformer layer or a linear layer. The pooling layer may be used to adjust the feature size to the original feature size, specifically the size of the information feature. The features output by the pooling layer may then be directly connected in series to the original transformer layer.

[0558] The method flow does not limit the training process of the first pre-trained language model.

[0559] Optionally, the first pre-trained language model may be trained and updated as a whole, or a newly added model layer in the first pre-trained model may be trained and its parameters updated.

[0560] In addition, the method does not limit the form of the reply, which can be in the form of text, voice or chart.

[0561] like Figure 8 As shown, Figure 8 The present invention is a flowchart of another log-based information reply method according to an exemplary embodiment of the present invention.

[0562] In the process of this method, the pre-stored log coding features can be directly analyzed.

[0563] The embodiments of the present disclosure do not limit the execution subject of the method. Optionally, the method can be applied to any computing device, specifically a mobile terminal device, a server device, a client device, etc. For example, a mobile phone, a tablet computer, a smart bracelet, VR glasses, etc.

[0564] The embodiments of the present disclosure do not limit the actual business meaning and source of the log data. Optionally, the log data may be a device log, an application log, a user operation log, a network log, a mobile phone log, a smart bracelet log, etc. Optionally, the log data may be obtained from the device, or the network log may be obtained from the server, etc.

[0565] The method may include at least the following steps.

[0566] S501: Obtain user expression information and pre-stored log encoding features.

[0567] Optionally, the log encoding feature may be determined based on any of the above log encoding method embodiments. Optionally, the log encoding feature may be determined based on a pre-trained log encoding model.

[0568] S502: Comprehensively obtain a comprehensive result by integrating the pre-stored log encoding features and the information features of the first pre-trained language model for the user's expression information.

[0569] S503: Determine a reply output by the first pre-trained language model based on the comprehensive result in response to the user's expressed information.

[0570] This method process can be based on a pre-trained language model and combined with log encoding features to facilitate analysis and response based on log data.

[0571] This method does not limit the way of obtaining the log coding features.

[0572] Optionally, a predetermined log coding feature may be directly acquired, or the log coding feature may be determined in real time based on any of the above log coding method embodiments according to target log data.

[0573] The method flow does not limit the specific content of the pre-stored log coding features.

[0574] Optionally, the log coding features corresponding to the log data within a specified time period may be stored; or the log coding features corresponding to the log data that meets a preset condition may be stored.

[0575] This embodiment can combine pre-stored log coding features to perform analysis and reply based on user expression information.

[0576] The explanation of the comprehensive result can be found in the above embodiment, and the explanation of the first pre-trained language model can be found in the above embodiment.

[0577] For other explanations of the process of this method, please refer to the above method embodiment.

[0578] In addition, it can be understood that, for the log coding features in the above method flow S101-S103, optionally, the log coding features can be determined based on any of the above log coding method embodiments. Optionally, the log coding features can be determined based on a pre-trained log coding model.

[0579] For ease of understanding, Fig. 9 As shown, Fig. 9 The present invention is a schematic diagram of another interface for user interaction based on logs, showing an exemplary embodiment of the present invention.

[0580] The form of a conversation interface may be adopted, and the conversation interface may include the conversation content between the user and the device, and may specifically include the user conversation content on the right and the device reply content on the left.

[0581] Users can enter "Please provide the number of times you have opened the food delivery app in the past 3 days."

[0582] The device may reply "The number of times the food delivery app has been opened in the past three days is 10 times". The device backend may reply based on the above method embodiment, combining the user input text and the log coding features.

[0583] Users can also enter "What is the average time your phone's screen is on every day?"

[0584] The device may reply, "Because no time period is specified, the average daily screen-on time of the user's mobile phone in the past 7 days is currently 5 hours." The device backend may reply based on the above method embodiment, combining the user input text and the log coding features.

[0585] This interface may be an interface displayed on a mobile phone or an interface displayed on a computer.

[0586] 5. Application examples

[0587] For ease of understanding, the present disclosure also provides an application embodiment for learning and applying logs on a mobile phone.

[0588] There are many kinds of historical behaviors of users on their mobile phones, but it is difficult to distinguish them.

[0589] This embodiment can develop a function, when the user wants to retrieve a certain behavior in the log, such as a certain picture taken, a certain file stored, this function can be used to find what the user wants to find by communicating with the user.

[0590] Example 1:

[0591] 1. User: Find the time spent on TikTok in the past three days.

[0592] 2. Function: Report the length of time you have spent using TikTok in the past three days, and report it in the form of text or table.

[0593] The specific steps of this embodiment are explained below.

[0594] 1. Encode the possible behaviors in the mobile phone. This part of the code does not exist in the ordinary general large model because the data set is different from natural language.

[0595] A) There are different apps at different layers of the mobile phone architecture. Each action of the mobile phone has an app (the app here can be understood as a mobile phone service) corresponding to it, and there are different logs. The log data is read out as the original data set. The feature of this original data is the log data, and the label is the corresponding operation behavior.

[0596] i. For example, a string of logs will appear once the screen is turned off:

[0597] a.GxzwViewIcon: turnOffScreen

[0598] b.GxzwViewIcon:setGxzwIconOpaque

[0599] c.GxzwViewIcon: turnOnScreen mDozing=false; state=2

[0600] ii. Then we need to regard this string of log data as a raw data feature, which corresponds to only one operation behavior label. Here, the operation behavior label is screen off, so that we can get a data pair.

[0601] B) Data needs to be cleaned because log data may contain unnecessary information such as prefixes and suffixes.

[0602] i. For example, each log in the GxzwViewIcon service has a number such as 1000, 3346, etc. This is the process number automatically assigned by the current service and has no practical meaning.

[0603] ii. For example, only the first light sensitivity data is the light intensity value, and the rest have no practical meaning.

[0604] C) After getting a clean dataset, start designing encoding methods and models.

[0605] 2. Log data needs to correspond to real natural language so that users can read it clearly.

[0606] First, there is a large model for processing natural language. Assuming that the large language model is used as an example from here on, the text features can be extracted for the operation behavior label (in natural language form), which is T, and the feature vector dimension is k.

[0607] Then there is a current log encoding model. Assume that the architecture here adopts the architecture of Llama2, but modify its word embedding layer and output layer, and retain the calculation method of transformer block.

[0608] The word embedding layer needs to conform to the way the log data is input, which can be done in a serialized manner.

[0609] Word embedding layer: As shown in the figure, [cls] is the predicted result y, and [sep] is the separator between sentences. For the log encoding model, you can use [cls] and the identifiers [time][app][sep].

[0610] The original linear feature layer is changed to an output layer so that the feature vector can be output.

[0611] The dimension of the feature vector output by the output layer is changed to k, which is equal to the dimension of the text feature vector extracted by the above large language model and recorded as I.

[0612] For N samples, two methods can be used to calculate and maximize the feature similarity of TN and IN. 1) Directly calculate Loss = MSE (T, I), and then minimize loss by back propagation. 2) Through comparative learning method: a. Assume that the first pair of data pairs Log1 "turnOffScreen..." and Text1 "turnOffScreen..." are calculated through the model to obtain I_1 and T_1, and I_1 and T1 are both features of k vector dimensions. b. Then, for N pairs of log data and operation text, N I_i and N T_i can be obtained. Each I_i and T_i should be the most similar, but not similar to all other text features or log encoding features. c. The label of similar feature pairs is 1, and the label of other feature pairs is 0. For details, please refer to Figure 4 .

[0613] Now a set of log data can be input into the log encoding model to obtain information such as time and behavior, but it is stored in an encoded form. This method simplifies the original log data with a large number of characters into a very small number of codes.

[0614] 3. Application mode 1: The above process can obtain the encoding characteristics of log data, which meets the storage requirements, but it still needs to be parsed into natural language so that users can understand it.

[0615] A) The model architecture after the feature layer is to input the encoded features into the output head of the large language model through several transformer blocks.

[0616] B) The output head of the large language model remains fixed and only the transformer block is trained and updated.

[0617] C) The model output is natural language.

[0618] D) At this point, this model can convert log data into natural language and can be applied in the following scenarios:

[0619] i. More log information can be stored in the same storage space, so that engineers can debug more information when problems occur.

[0620] ii. Log information can be directly read through natural language, allowing novice engineers to debug easily.

[0621] iii. Storing information in this way makes it possible to record long-term user behavior, allowing various applications to obtain a large amount of user historical behavior in the application, thereby optimizing their behavior patterns.

[0622] 4. Application mode 2: Add the ability to read logs to the existing big model, allowing users / engineers to communicate with the big language model through natural language.

[0623] A) This requires modifying the large language model so that it can understand the log data and communicate. Here we mainly modify the position of the Embedding layer and the subsequent connection Embedding layer. Correspondingly, the input needs to be processed into a way to splice the input text features and the log encoding features.

[0624] B) The modification method is to use the log coding features output by the log coding model as input and concatenate them with the Embedding calculated by the large language model. Since the log coding features are already the mapping of the log data in the semantic space, they can be concatenated.

[0625] C) At this time, the Embedding will not change except the dimension expansion. After the Embedding with expanded dimension passes through several transformer blocks, it is reduced to the original dimension by pooling and then connected to the original transformer block.

[0626] D) So the overall approach is to expand the Embedding layer, and then add several new transformer blocks and a pooling layer for dimensionality reduction between the Embedding and transformer blocks. After that, you can train, update, adjust, or add new model layers.

[0627] E) At this point, the trained large language model can understand the log data information. Users can communicate with the log data, and the log data can represent the operation behavior of the mobile phone in the past period of time.

[0628] The above embodiments can achieve at least the following effects: 1) log data can be naturally converted into natural language; 2) storage efficiency is improved through log encoding features; 3) users / engineers can communicate with mobile phone historical data.

[0629] Specifically, mobile phone logs can be processed through natural language, allowing users to easily understand obscure log data information; the memory usage of a large amount of historical records can be reduced, so a large amount of historical information can be saved for query use; the semantic understanding of log data can be added to the large language model, making it easier for the large language model to communicate and interact with users based on logs.

[0630] For the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously.

[0631] Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0632] Corresponding to the aforementioned method embodiments, the present disclosure also provides device embodiments.

[0633] like Fig.10 As shown, Fig.10 The present invention is a block diagram of a log-based information reply device according to an exemplary embodiment of the present invention.

[0634] The apparatus may include the following units.

[0635] The acquisition unit 601 is used to acquire user expression information and acquire log coding features of log data;

[0636] A synthesis unit 602 is used to synthesize the acquired log coding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a synthesis result;

[0637] The determination unit 603 is used to determine the reply information output by the first pre-trained language model based on the comprehensive result in response to the user's expression information.

[0638] Optionally, the method for determining the log coding feature includes:

[0639] The log data is input into a pre-trained log encoding model, and the log encoding features output by the log encoding model are determined.

[0640] Optionally, the training method of the log encoding model includes:

[0641] Obtain the operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text is used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; train the log encoding model according to the log data samples in the log data sample set and the corresponding operation behavior text features.

[0642] Optionally, the log encoding model is trained according to the log data samples in the log data sample set and the corresponding operation behavior text features, including any of the following:

[0643] Determine the operation behavior text feature as the log coding feature label corresponding to the log data sample; train the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature label;

[0644] A log coding model is trained based on log data samples in a log data sample set and corresponding operation behavior text features to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features; the log current features include: the corresponding log data samples are input into the current log coding model, and the log coding features output by the current log coding model.

[0645] Optionally, the operation behavior text feature has the same size as the log current feature;

[0646] Improve the similarity between the text features of the operation behavior corresponding to the same log data sample and the corresponding current features of the log, including:

[0647] Improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reduce the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0648] Optionally, the operation behavior text features include: text features extracted from the operation behavior text using a second pre-trained language model.

[0649] Optionally, feed the log data into a pre-trained log encoding model, including:

[0650] Preprocess log data;

[0651] The preprocessed log data is input into the pre-trained log encoding model.

[0652] Optionally, the log data is preprocessed, including at least one of the following:

[0653] Perform word segmentation on log data;

[0654] Set the beginning delimiter before the first character of the log data;

[0655] Set the end separator after the end character of the log data;

[0656] When it is determined that the log data includes different sentences, a separator is set between the different sentences;

[0657] When it is determined that the log data includes time information, setting a time separator for the time information;

[0658] When it is determined that the log data includes application information, setting an application separator for the application information;

[0659] When the behavior keywords are extracted based on the log data, the extracted behavior keywords are added to the log data; the behavior keywords are used to describe the operation behavior information represented by the log data.

[0660] Optionally, the acquiring unit 601 is configured to perform at least one of the following:

[0661] Obtain log encoding characteristics of pre-stored log data;

[0662] Obtaining a pre-stored log coding feature; the obtained log coding feature is determined based on the log data;

[0663] According to the information expressed by the user, the target log data required by the user is determined; and the log encoding characteristics of the target log data are obtained.

[0664] Optionally, the integration unit 602 is configured to perform any of the following:

[0665] Concatenate the obtained log encoding features and the information features extracted by the first pre-trained language model based on the user expression information;

[0666] Based on the cross-attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

[0667] Optionally, the device further includes: a decoding unit 604, which is used to input the acquired log encoding features into a pre-trained log decoding model to determine the operation behavior text prediction result output by the log decoding model.

[0668] Optionally, the training method of the log decoding model includes:

[0669] Obtain the log encoding features of any log data sample in the decoding sample set, as well as the operation behavior text corresponding to any log data sample;

[0670] The log decoding model is trained based on the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set.

[0671] Optionally, the log decoding model is trained according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set, including any of the following:

[0672] Determine the operation behavior text corresponding to any log data sample as the sample label corresponding to the log encoding feature of the same log data sample; train the log decoding model according to the log encoding feature and the corresponding sample label;

[0673] According to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set, the log decoding model is trained to improve the similarity between the current predicted text corresponding to the same log data sample and the corresponding operation behavior text; the current predicted text includes: the text prediction result output by the current log decoding model for the log encoding features of the corresponding log data sample.

[0674] The explanation of the embodiment of the present device can refer to the above method embodiment.

[0675] like Fig.11 As shown, Fig.11 The present invention is a block diagram of a log encoding device according to an exemplary embodiment of the present invention.

[0676] The apparatus may include the following units.

[0677] The encoding unit 701 is used to input the log data into a pre-trained log encoding model and determine the log encoding features output by the log encoding model.

[0678] The training method of the log coding model may include: obtaining the operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text is used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; and train the log coding model based on the log data samples in the log data sample set and the corresponding operation behavior text features.

[0679] Optionally, the log encoding model is trained according to the log data samples in the log data sample set and the corresponding operation behavior text features, including any of the following:

[0680] Determine the operation behavior text feature as the log coding feature label corresponding to the log data sample; train the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature label;

[0681] A log coding model is trained based on log data samples in a log data sample set and corresponding operation behavior text features to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features; the log current features include: the corresponding log data samples are input into the current log coding model, and the log coding features output by the current log coding model.

[0682] Optionally, the operation behavior text feature has the same size as the log current feature;

[0683] Improve the similarity between the text features of the operation behavior corresponding to the same log data sample and the corresponding current features of the log, including:

[0684] Improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reduce the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

[0685] Optionally, the operation behavior text features include: text features extracted from the operation behavior text using a second pre-trained language model.

[0686] Optionally, the encoding unit 701 is used for:

[0687] Preprocess log data;

[0688] The preprocessed log data is input into the pre-trained log encoding model.

[0689] Optionally, the encoding unit 701 is configured to perform at least one of the following:

[0690] Perform word segmentation on log data;

[0691] Set the beginning delimiter before the first character of the log data;

[0692] Set the end separator after the end character of the log data;

[0693] When it is determined that the log data includes different sentences, a separator is set between the different sentences;

[0694] When it is determined that the log data includes time information, setting a time separator for the time information;

[0695] When it is determined that the log data includes application information, setting an application separator for the application information;

[0696] When the behavior keywords are extracted based on the log data, the extracted behavior keywords are added to the log data; the behavior keywords are used to describe the operation behavior information represented by the log data.

[0697] The explanation of the embodiment of the present device can refer to the above method embodiment.

[0698] like Fig.12 As shown, Fig.12 The present invention is a block diagram of a log decoding device according to an exemplary embodiment of the present invention.

[0699] The apparatus may include the following units.

[0700] The feature acquisition unit 801 is used to acquire the log coding feature corresponding to the log data; the log coding feature is determined based on any of the above-mentioned log coding device embodiments;

[0701] The feature decoding unit 802 is used to input the acquired log encoding features into a pre-trained log decoding model to determine the operation behavior text prediction result output by the log decoding model.

[0702] Optionally, the training method of the log decoding model includes:

[0703] Obtain sample coding features and corresponding operation behavior text corresponding to the same log data sample in the decoding sample set; the sample coding features are determined based on a pre-trained log coding model; train the log decoding model according to the sample coding features and corresponding operation behavior text corresponding to the same log data sample in the decoding sample set.

[0704] Optionally, the log decoding model is trained according to the sample encoding features corresponding to the same log data sample in the decoding sample set and the corresponding operation behavior text, including any of the following:

[0705] Determine the operation behavior text corresponding to any log data sample as the sample label of the sample encoding feature corresponding to the same log data sample; train the log decoding model according to the sample encoding feature and the corresponding sample label;

[0706] The log decoding model is trained according to the sample encoding features corresponding to the same log data sample in the decoding sample set, and the corresponding operation behavior text, so as to improve the similarity between the operation behavior text corresponding to the same log data sample and the corresponding current prediction text; the current prediction text includes: the sample encoding features corresponding to the corresponding log data sample are input into the current log decoding model, and the text prediction result output by the current log decoding model.

[0707] The explanation of the embodiment of the present device can refer to the above method embodiment.

[0708] like Fig.13 As shown, Fig.13 The present invention is a block diagram of another log-based information reply device according to an exemplary embodiment of the present invention.

[0709] The apparatus may include the following units.

[0710] The first determining unit 901 is used to determine the target log data required by the user according to the user expression information;

[0711] The first feature unit 902 is used to obtain corresponding log coding features for the determined target log data.

[0712] Optionally, the log encoding feature may be determined based on any of the above-mentioned log encoding device embodiments; or may be determined based on a pre-trained log encoding model.

[0713] A first synthesis unit 903 is used to synthesize the acquired log coding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a synthesis result;

[0714] The first response unit 904 is used to determine a reply output by the first pre-trained language model based on the comprehensive result in response to the user's expression information.

[0715] Optionally, the first integration unit 903 is configured to perform any of the following:

[0716] Concatenate the obtained log encoding features and the information features extracted by the first pre-trained language model based on the user expression information;

[0717] Based on the cross-attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

[0718] The explanation of the embodiment of the present device can refer to the above method embodiment.

[0719] like Fig.14 As shown, Fig.14 The present invention is a block diagram of another log-based information reply device according to an exemplary embodiment of the present invention.

[0720] The apparatus may include the following units.

[0721] The second feature unit 1001 is used to obtain user expression information and pre-stored log coding features; the log coding features are determined based on any of the above-mentioned log coding device embodiments;

[0722] The second synthesis unit 1002 is used to synthesize the pre-stored log coding features and the information features of the first pre-trained language model for the user's expression information to obtain a synthesis result;

[0723] The second response unit 1003 is used to determine a reply output by the first pre-trained language model based on the comprehensive result in response to the user's expressed information.

[0724] The explanation of the embodiment of the present device can refer to the above method embodiment.

[0725] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art may understand and implement it without creative work.

[0726] Accordingly, on the one hand, an embodiment of the present disclosure provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: obtain user expression information, and obtain log coding features of log data; integrate the obtained log coding features and information features extracted by the first pre-trained language model for the user expression information to obtain a comprehensive result; determine the reply information output by the first pre-trained language model based on the comprehensive result in response to the user expression information.

[0727] On the one hand, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute any of the above method embodiments.

[0728] Fig.15 1 is a schematic diagram of a log-based information reply device 1500 according to an exemplary embodiment. For example, the device 1500 may be a user device, which may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, a wearable device such as a smart watch, a smart glasses, a smart bracelet, a smart running shoe, etc.

[0729] Reference Fig.15 , device 1500 may include one or more of the following components: a processing component 1502 , a memory 1504 , a power component 1506 , a multimedia component 1508 , an audio component 1510 , an input / output (I / O) interface 1512 , a sensor component 1514 , and a communication component 1516 .

[0730] The processing component 1502 generally controls the overall operation of the device 1500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1502 may include one or more processors 1520 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1502 may include one or more modules to facilitate the interaction between the processing component 1502 and other components. For example, the processing component 1502 may include a multimedia module to facilitate the interaction between the multimedia component 1508 and the processing component 1502.

[0731] The memory 1504 is configured to store various types of data to support operations on the device 1500. Examples of such data include instructions for any application or method operating on the device 1500, contact data, phone book data, messages, pictures, videos, etc. The memory 1504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0732] The power supply component 1506 provides power to the various components of the device 1500. The power supply component 1506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1500.

[0733] The multimedia component 1508 includes a screen that provides an output interface between the above-mentioned device 1500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The above-mentioned touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the above-mentioned touch or slide operation. In some embodiments, the multimedia component 1508 includes a front camera and / or a rear camera. When the device 1500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0734] The audio component 1510 is configured to output and / or input audio signals. For example, the audio component 1510 includes a microphone (MIC), and when the device 1500 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1504 or sent via the communication component 1516. In some embodiments, the audio component 1510 also includes a speaker for outputting audio signals.

[0735] I / O interface 1512 provides an interface between processing component 1502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0736] The sensor assembly 1514 includes one or more sensors for providing various aspects of the status assessment of the device 1500. For example, the sensor assembly 1514 can detect the open / closed state of the device 1500, the relative positioning of components, such as the display and keypad of the device 1500, the sensor assembly 1514 can also detect the position change of the device 1500 or a component of the device 1500, the presence or absence of user contact with the device 1500, the orientation or acceleration / deceleration of the device 1500, and the temperature change of the device 1500. The sensor assembly 1514 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1514 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1514 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0737] The communication component 1516 is configured to facilitate wired or wireless communication between the device 1500 and other devices. The device 1500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR or a combination thereof. In an exemplary embodiment, the communication component 1516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0738] In an exemplary embodiment, the apparatus 1500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform any of the above method embodiments.

[0739] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 1504 including instructions, which, when executed by a processor 1520 of the device 1500, enables the device 1500 to execute any of the above method embodiments.

[0740] The non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0741] like Fig.16 As shown, Fig.16 FIG. 1 is a schematic diagram of another log-based information reply device 1600 according to an exemplary embodiment. For example, the device 1600 may be provided as an application server. Fig.16 , the apparatus 1600 includes a processing component 1622, which further includes one or more processors, and a memory resource represented by a memory 1616 for storing instructions, such as an application, that can be executed by the processing component 1622. The application stored in the memory 1616 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1622 is configured to execute instructions to perform any of the above-mentioned method embodiments.

[0742] The device 1600 may also include a power supply component 1626 configured to perform power management of the device 1600, a wired or wireless network interface 1650 configured to connect the device 1600 to a network, and an input / output (I / O) interface 1658. The device 1600 may operate based on an operating system stored in the memory 1616, such as Android, iOS, Windows Server, etc. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0743] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1616 including instructions, which can be executed by a processing component 1622 of the apparatus 1600 to perform the above method embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0744] When the instructions in the memory 1616 are executed by the processing component 1622, the device 1600 is enabled to execute any of the above method embodiments.

[0745] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0746] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A log-based information reply method, characterized in that: include: Obtain user expression information and obtain log encoding features of log data; Combining the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a comprehensive result; Determine reply information output by the first pre-trained language model based on the comprehensive result in response to the user's expressed information.

2. The method according to claim 1, characterized in that The method for determining the log coding feature includes: The log data is input into a pre-trained log coding model, and the log coding features output by the log coding model are determined.

3. The method according to claim 2, characterized in that The training method of the log coding model includes: Obtain an operation behavior text corresponding to any log data sample in the log data sample set; the operation behavior text is used to: use natural language to describe the specific operation behavior represented by the corresponding log data sample; train the log encoding model based on the log data samples in the log data sample set and the corresponding operation behavior text features.

4. The method according to claim 3, characterized in that The training of the log encoding model according to the log data samples in the log data sample set and the corresponding operation behavior text features includes any of the following: Determine the operation behavior text feature as the log coding feature label corresponding to the log data sample; Training the log coding model according to the log data samples in the log data sample set and the corresponding log coding feature labels; According to the log data samples in the log data sample set and the corresponding operation behavior text features, the log encoding model is trained to improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features; The current log feature includes: inputting the corresponding log data sample into the current log coding model, and the log coding feature output by the current log coding model.

5. The method according to claim 4, characterized in that The action text feature has the same size as the log current feature; Improving the similarity between the operation behavior text feature corresponding to the same log data sample and the corresponding log current feature includes: Improve the similarity between the operation behavior text features corresponding to the same log data sample and the corresponding log current features, and reduce the similarity between the operation behavior text features corresponding to any log data sample and the log current features corresponding to different log data samples.

6. The method according to claim 3, characterized in that The operation behavior text features include: text features extracted from the operation behavior text using a second pre-trained language model.

7. The method according to claim 2, characterized in that The step of inputting the log data into a pre-trained log encoding model comprises: Preprocess log data; The preprocessed log data is input into the pre-trained log encoding model.

8. The method according to claim 7, characterized in that The preprocessing of the log data includes at least one of the following: Perform word segmentation on log data; Set the beginning delimiter before the first character of the log data; Set the end delimiter after the end character of the log data; When it is determined that the log data includes different sentences, a separator is set between the different sentences; In the case of determining that the log data includes time information, setting a time separator for the time information; In a case where it is determined that the log data includes application information, setting an application separator for the application information; In the case where the behavior keywords are extracted based on the log data, the extracted behavior keywords are added to the log data; The behavior keywords are used to describe the operation behavior information represented by the log data.

9. The method according to claim 1, characterized in that: The log encoding feature of the obtained log data includes at least one of the following: Obtain log encoding characteristics of pre-stored log data; Obtaining a pre-stored log coding feature; the obtained log coding feature is determined based on the log data; Determine the target log data required by the user according to the user expression information; and obtain the log encoding features of the target log data.

10. The method according to claim 1, characterized in that The log encoding features obtained by the synthesis and the information features extracted by the first pre-trained language model for the user expression information, Include any of the following: Concatenating the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information; Based on the cross attention mechanism, the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information are integrated.

11. The method according to claim 1, characterized in that: Also includes: The acquired log encoding features are input into a pre-trained log decoding model to determine the operation behavior text prediction result output by the log decoding model.

12. The method according to claim 11, characterized in that The training method of the log decoding model includes: Obtaining the log encoding features of any log data sample in the decoding sample set, and the operation behavior text corresponding to any log data sample; The log decoding model is trained according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set.

13. The method according to claim 12, characterized in that The step of training the log decoding model according to the corresponding operation behavior text and log encoding features of the same log data sample in the decoding sample set includes any one of the following: Determine the operation behavior text corresponding to any log data sample as the sample label corresponding to the log encoding feature of the same log data sample; train the log decoding model according to the log encoding feature and the corresponding sample label; The log decoding model is trained according to the corresponding operation behavior text and log coding features of the same log data sample in the decoding sample set to improve the similarity between the current predicted text corresponding to the same log data sample and the corresponding operation behavior text; the current predicted text includes: the text prediction result output by the current log decoding model for the log coding features of the corresponding log data sample.

14. A log-based information reply device, characterized in that: include: An acquisition unit, used to acquire user expression information and acquire log coding features of log data; A synthesis unit, used to synthesize the acquired log encoding features and the information features extracted by the first pre-trained language model for the user expression information to obtain a synthesis result; A determination unit is used to determine the reply information output by the first pre-trained language model based on the comprehensive result in response to the user's expression information.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 13.

Images