Log analysis method, device and equipment, readable storage medium and program product

Abstract

The invention discloses a log analysis method and device, equipment, a readable storage medium and a program product, and the method comprises the steps: determining instruction type requests and non-instruction type requests in a plurality of target requests for the plurality of target requests with unknown intentions in a to-be-analyzed log; clustering the instruction type requests into a plurality of categories by adopting a first clustering model, and performing intention recognition on the instruction type requests under each category by adopting a large language model to obtain an intention corresponding to each category; clustering the non-instruction type requests into a plurality of categories by adopting a second clustering model, and determining whether the non-instruction type requests in each category contain entities or not by adopting a large language model; extracting keywords of non-instruction type requests in corresponding categories under the condition that the categories cannot determine whether the entities are contained or not; and taking the keyword as the entity contained in the non-instruction type request under the corresponding category under the condition that the corresponding category is determined to contain the entity based on the keyword. By adopting the method, the log analysis efficiency can be improved.

Description

Technical Field

[0001] This application relates to the technical field of display devices, and in particular, to a log analysis method, apparatus, device, readable storage medium, and program product. Background Art

[0002] With the rapid development of the functions of display devices, the functions that display devices can provide to users are becoming increasingly rich. Currently, display devices include smart TVs, smart set-top boxes, smart boxes, and products with smart display screens, etc. Taking the voice assistant function supported by smart TVs as an example, the smart TV is interacted with through the input voice commands. During the operation of the smart TV, logs are often generated, and by analyzing the logs, the processing process of the voice commands by the smart TV can be located.

[0003] In the traditional method of log analysis using deep learning technology, in the case where the log contains a large number of requests, the resources required for log analysis are relatively large, resulting in a long time-consuming for log analysis. In this scenario, methods such as random sampling or pre-duplication can be used to reduce the amount of logs. However, in the random sampling method, only part of the logs are analyzed, which may lead to the loss of important logs; in the pre-duplication method, pre-duplication of logs is performed through algorithms such as edit distance, and it is often difficult to determine the distance threshold. If the distance threshold is too small, it may not be possible to effectively reduce the logs, and if the distance threshold is too large, logs that are semantically unrelated may be removed. Therefore, the traditional log analysis method has the problem of low log analysis efficiency. Summary of the Invention

[0004] This application provides a log analysis method, apparatus, computer device, readable storage medium, and program product to solve the problem of low log analysis efficiency of the traditional log analysis method.

[0005] In a first aspect, some embodiments provide a log analysis method, and the method includes:

[0006] For multiple target requests with unknown intents in the log to be analyzed, determine the instruction type requests and non-instruction type requests among the multiple target requests;

[0007] Cluster the instruction type requests into multiple categories using a first clustering model, and use a large language model to identify the intents of the instruction type requests under each category to obtain the intent corresponding to each category;

[0008] Use a second clustering model to cluster non-instruction type requests into multiple categories, and use a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where it is impossible to determine whether a category contains entities, extract the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined based on the keywords that the corresponding category contains entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding category.

[0009] In a second aspect, some embodiments also provide a log analysis device, including:

[0010] A determination module, configured to determine instruction type requests and non-instruction type requests among a plurality of target requests with unknown intentions in the log to be analyzed;

[0011] A first recognition module, configured to use a first clustering model to cluster instruction type requests into multiple categories, and use a large language model to perform intention recognition on the instruction type requests under each category to obtain the intention corresponding to each category;

[0012] A second recognition module, configured to use a second clustering model to cluster non-instruction type requests into multiple categories, and use a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where it is impossible to determine whether a category contains entities, extract the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined based on the keywords that the corresponding category contains entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding category.

[0013] In a third aspect, some embodiments also provide a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0014] For a plurality of target requests with unknown intentions in the log to be analyzed, determine instruction type requests and non-instruction type requests among the plurality of target requests;

[0015] Use a first clustering model to cluster instruction type requests into multiple categories, and use a large language model to perform intention recognition on the instruction type requests under each category to obtain the intention corresponding to each category;

[0016] Use a second clustering model to cluster non-instruction type requests into multiple categories, and use a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where it is impossible to determine whether a category contains entities, extract the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined based on the keywords that the corresponding category contains entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding category.

[0017] Fourthly, some embodiments also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0018] For multiple target requests with unknown intentions in the log to be analyzed, determine the instruction type requests and non-instruction type requests among the multiple target requests;

[0019] Use a first clustering model to cluster the instruction type requests into multiple categories, and use a large language model to identify the intentions of the instruction type requests under each category to obtain the corresponding intention for each category;

[0020] Use a second clustering model to cluster the non-instruction type requests into multiple categories, and use a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where there are categories for which it is impossible to determine whether they contain entities, extract the keywords in the non-instruction type requests under the corresponding categories; in the case where it is determined based on the keywords that the corresponding categories contain entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding categories.

[0021] Fifthly, some embodiments also provide a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0022] For multiple target requests with unknown intentions in the log to be analyzed, determine the instruction type requests and non-instruction type requests among the multiple target requests;

[0023] Use a first clustering model to cluster the instruction type requests into multiple categories, and use a large language model to identify the intentions of the instruction type requests under each category to obtain the corresponding intention for each category;

[0024] Use a second clustering model to cluster the non-instruction type requests into multiple categories, and use a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where there are categories for which it is impossible to determine whether they contain entities, extract the keywords in the non-instruction type requests under the corresponding categories; in the case where it is determined based on the keywords that the corresponding categories contain entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding categories.

[0025] Technical effects: Some embodiments provide a log analysis method, apparatus, computer device, computer-readable storage medium, and computer program product. By identifying multiple target requests with unknown intentions in the logs to be analyzed, since requests with known intentions can be quickly identified and responded to, further analyzing the target requests with unknown intentions is beneficial to improving the log analysis efficiency; Instruction type requests are requests with clear intentions among the target requests, and non-instruction type requests are requests with unclear intentions among the target requests, that is, the semantic focuses of different types of target requests are different. By identifying the instruction type requests and non-instruction type requests among the multiple target requests, it is beneficial to guiding the clustering model to cluster different types of target requests according to different semantic focuses; Using the first clustering model to cluster the instruction type requests, each category has similar intentions, and using a large language model to identify the intentions corresponding to each category, using the second clustering model to cluster the non-instruction type requests, and identifying the entities corresponding to each category based on the large language model and the keywords in the non-instruction type requests. This method of classifying and clustering the target requests in the logs to be analyzed and identifying the intentions and entities according to the categories, compared with using a large language model to analyze a single target request, greatly reduces the number of calls to the large language model and the time-consuming of log analysis, which is beneficial to improving the log analysis efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0027] Figure 1 It is a schematic diagram of the operation scenario between a display device and a control device provided by some embodiments of the present application;

[0028] Figure 2 It is a schematic diagram of the hardware configuration of a display device provided by some embodiments of the present application;

[0029] Figure 3 It is a schematic diagram of the hardware configuration of a control device provided by some embodiments of the present application;

[0030] Figure 4 It is a schematic diagram of the software configuration of a display device provided by some embodiments of the present application;

[0031] Figure 5 It is a schematic flowchart of the log analysis method provided by some embodiments of the present application;

[0032] Figure 6Schematic diagram for entity recognition of non-instruction type requests provided by some embodiments of the present application;

[0033] Figure 7 Schematic diagram of the structure of a pre-trained classification model provided by some embodiments of the present application;

[0034] Figure 8 Schematic diagram of the structure of a first clustering model provided by some embodiments of the present application;

[0035] Figure 9 Schematic diagram for identifying the intent of instruction type requests by category provided by some embodiments of the present application;

[0036] Figure 10 Overall process schematic diagram of a log analysis method provided by some embodiments of the present application;

[0037] Figure 11 Structural block diagram of a log analysis device provided by some embodiments of the present application;

[0038] Figure 12 Internal structure diagram of a computer device provided by some embodiments of the present application. Detailed implementation manners

[0039] The embodiments will be described in detail below, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all implementation manners consistent with the present application. They are merely examples of systems and methods consistent with some aspects of the present application detailed in the claims.

[0040] It should be noted that the brief description of the terms in the present application is only for facilitating the understanding of the following described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0041] The terms "first", "second", "third", etc. in the specification, claims and above-mentioned drawings of the present application are used to distinguish similar or same-kind objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such used terms can be interchanged under appropriate circumstances.

[0042] The terms "including" and "having" and any variations thereof are intended to cover but not exclusively include. For example, a product or device including a series of components does not necessarily have to be limited to all the clearly listed components, but may include other components not clearly listed or inherent to these products or devices.

[0043] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware and / or software code that can perform functions related to that element.

[0044] In the embodiments of the present application, the log analysis method can be applied to computer devices, including display devices, servers, etc. Taking the log analysis method applied to the display device 200 as an example, the display device 200 generally refers to a device with the ability to display pictures and process data. For example, the display device 200 includes but is not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0045] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided for some embodiments of the present application. As Figure 1 shown, the user can operate the display device 200 through touch operations, the mobile terminal 300, and the control device 100. For example, the control device 100 can be a remote control, a stylus, a handle, etc.

[0046] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 to perform data interaction. In some embodiments, the mobile terminal 300 and the display device 200 can install software applications and achieve connection communication through network communication protocols to achieve the purpose of one-to-one control operations and data communication. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve the synchronous display function.

[0047] As Figure 1 also shown, the display device 200 also performs data communication with the server 400 through various communication methods. The display device 200 is allowed to establish a communication connection through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0048] The display device 200 can provide a broadcast receiving TV function, and can also additionally provide a smart network TV function with computer support functions, including but not limited to, network TV, smart TV, Internet Protocol TV (IPTV), etc.

[0049] Figure 2 Provided for some embodiments of the present application Figure 1 is the hardware configuration block diagram of the display device 200 in

[0050] In some embodiments, the display device 200 may include at least one of a tuner-demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0051] In some embodiments, the detector 230 is used to collect signals of the external environment or for external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the ambient light intensity; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0052] In some embodiments, the display 260 includes a display function component for presenting a picture and a driving component for driving image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, components of a menu control interface, and a user control UI interface, etc.

[0053] In some embodiments, the communication device 220 is a component for communicating with external devices or a server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0054] The communication device 220 can enable the display device 200 to communicate with external devices or the server 400 through wireless or wired connection. Among them, the wired connection can connect the display device 200 with an external device through components such as a data cable and an interface. The wireless connection can connect the display device 200 with an external device through a wireless signal or a wireless network. The display device 200 can directly establish a connection relationship with an external device, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0055] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0056] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0057] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and then the user input interface receives the user input command through the graphical user interface (GUI).

[0058] In some embodiments, the audio output device 270 may be the built-in speaker of the display device 200, or may be an external audio output device connected to the display device 200. Among them, for the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0059] In some embodiments, the user input interface 280 can be used to receive instructions input by the user.

[0060] Figure 3 The hardware configuration block diagram of the control device provided in some embodiments of this application. As Figure 1 shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply. Figure 3

[0061] The control device 100 is configured to control the display device 200, and can receive input operation instructions from the user, and convert the operation instructions into instructions recognizable and responsive by the display device 200, playing an intermediary role in the interaction between the user and the display device 200.

[0062] In some embodiments, the control device 100 may be an intelligent device. For example: the control device 100 can install various applications for controlling the display device 200 according to user needs.

[0063] Figure 1 In some embodiments, as shown, after the mobile terminal 300 or other intelligent electronic devices install the application for controlling the display device 200, they can play a similar function to the control device 100.

[0064] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and the data processing functions inside and outside.

[0065] Under the control of the controller 110, the communication interface 130 enables the communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as a WiFi chip 131, a Bluetooth module 132, an NFC module 133, etc.

[0066] The user input / output interface 140, where the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a button 144, and other input interfaces.

[0067] In some embodiments, the control device 100 includes at least one of the communication interface 130 and the input / output interface 140. The communication interface 130 configured in the control device 100, such as modules like WiFi, Bluetooth, NFC, etc., can encode user input instructions through the WiFi protocol, or the Bluetooth protocol, or the NFC protocol and send them to the display device 200.

[0068] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0069] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0070] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling the hardware resources and software resources in the display device 200. The operating system can (control the display device) provide a user interface, allowing the user to interact with the display device 200 and support the running of various application programs.

[0071] It should be noted that the operating system can be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0072] The operating system can be divided into different modules or layers according to the functions implemented. For example, as Figure 4 shown, Figure 4 For some embodiments provided in this application Figure 1 is a schematic diagram of the software configuration in the display device. In some embodiments, the system of the display device 200 can be divided into three layers, from top to bottom, namely the application layer, the middleware layer, and the hardware layer.

[0073] The application layer mainly includes common applications on the TV and the Application Framework. Among them, the common applications are mainly applications developed based on the Browser, such as HTML5 APPs, and native applications (Native APPs).

[0074] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the usage interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0075] Native applications (Native APPs) can support online or offline, message push or local resource access.

[0076] The middleware layer includes various middleware such as TV protocols, multimedia protocols, and system components. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system on the network or different applications, and can achieve the purpose of resource sharing and function sharing.

[0077] The hardware layer mainly includes the Hardware Abstraction Layer (HAL) interface, hardware, and drivers. Among them, the HAL interface is the unified interface for all TV chips to dock, and the specific logic is implemented by each chip. The drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power supply drivers, etc.

[0078] It should be noted that the above examples are only simple divisions of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. According to factors such as the functions of the display device and the type of the operating system, the number of levels and the specific level types included in the operating system can be in other forms.

[0079] With the rapid development of the functions of display devices, the functions that display devices can provide to users are becoming more and more abundant. Currently, display devices include smart TVs, smart set-top boxes, smart boxes, and products with smart display screens, etc. Taking the voice assistant function supported by smart TVs as an example, it interacts with the smart TV through the input voice commands. The running process of a smart TV often generates logs, and by analyzing the logs, the processing process of the smart TV for voice commands can be located.

[0080] Log analysis mainly conducts intent analysis and entity analysis on voice commands in the log. For example, in the voice assistant scenario, intent analysis mainly focuses on analyzing intents not covered by the voice assistant to help the voice assistant expand new vertical domain services in the future; entity analysis mainly aims to find entity names that the voice assistant cannot recognize, such as movie names, actor names, etc., thereby enhancing the knowledge scope of the voice assistant.

[0081] Traditional methods for log analysis using deep learning techniques, such as using large language models (LLMs) for log analysis, have a long inference time for the large language model itself. Once the online request volume is large, the log analysis time is even longer than the time consumed by the service itself. In this scenario, using an LLM for log analysis requires too many resources, resulting in a long log analysis time. In this situation, methods such as random sampling or pre-de-duplication can be used to reduce the log volume. However, in the random sampling method, only part of the logs are analyzed, which may lead to the loss of important logs; in the pre-de-duplication method, pre-de-duplication of logs is performed through algorithms such as edit distance, and it is often difficult to determine the distance threshold. If the distance threshold is too small, it may not effectively reduce the logs, and if the distance threshold is too large, it may remove semantically unrelated logs. Therefore, traditional log analysis methods have the problem of low log analysis efficiency.

[0082] Based on this, the embodiments of the present application propose a log analysis method, device, computer device, readable storage medium, and program product. By identifying multiple target requests with unknown intents in the log to be analyzed, since requests with known intents can quickly identify the intent and obtain a response, further analyzing the target requests with unknown intents is beneficial to improving the log analysis efficiency; instruction type requests are requests with clear intents among the target requests, and non-instruction type requests are requests with unclear intents among the target requests, that is, the semantic focuses of different types of target requests are different. By identifying instruction type requests and non-instruction type requests among multiple target requests, it is beneficial to guide the clustering model to cluster different types of target requests according to different semantic focuses; the first clustering model is used to cluster instruction type requests, and each category has similar intents, and a large language model is used to identify the intent corresponding to each category. The second clustering model is used to cluster non-instruction type requests, and entities corresponding to each category are identified based on the large language model and keywords in the non-instruction type requests. This method of classifying and clustering target requests in the log to be analyzed and identifying intents and entities according to categories greatly reduces the number of calls to the large language model and the log analysis time compared to analyzing a single target request using a large language model, which is beneficial to improving the log analysis efficiency.

[0083] In an exemplary embodiment, as Figure 5As shown, a log analysis method is provided. Taking the application of this method to a computer device as an example, it includes the following steps 202 to 206. Among them:

[0084] Step 202, for multiple target requests with unknown intentions in the log to be analyzed, determine the instruction type requests and non-instruction type requests among the multiple target requests.

[0085] Among them, the log to be analyzed is the object that needs to be analyzed for logs. It can be the logs generated during the operation of the computer device, including the logs generated when the computer device responds to the input requests. The input requests are used to instruct the computer device to perform corresponding tasks or operations. According to different application scenarios, the content indicated by the input requests is different. Exemplarily, in the voice assistant scenario, the input requests are used to perform operations on the computer device that match the requests, such as turning on the device, querying the weather, etc.

[0086] The input requests can be in the form of natural language, such as voice or text form, etc. The log to be analyzed can be the offline log obtained every preset time interval. The log to be analyzed contains the text corresponding to the input requests. In some embodiments, when the computer device generates the log corresponding to each input request, it saves the text corresponding to the input request into the log and adds a preset identifier to the text corresponding to the input request. After the computer device obtains the log to be analyzed, it extracts the text corresponding to each of the multiple input requests according to the preset identifier.

[0087] Intention refers to the task or operation that the input request instructs the computer device to perform. For example, in the voice assistant scenario, the intention refers to the operation that the user instructs the computer device to perform by inputting a voice request. The computer device can perform a preliminary intention classification on the text corresponding to each of the multiple input requests in the log to be analyzed, identify the requests with known intentions and the target requests with unknown intentions among the multiple input requests, and obtain the intentions of the requests with known intentions. For example, a pre-trained BERT model can be used for preliminary intention classification.

[0088] The target requests with unknown intentions are the requests that the current computer device cannot cover. It is necessary to further analyze the intentions of the target requests to expand the knowledge scope of the computer device and help the computer device to expand new vertical domain services in the future.

[0089] An instruction type request refers to a request with a clear intention in the target request. Instruction type requests generally have a relatively fixed syntax pattern, such as "Open xxx", "Set xxx as xxx", "Search xxx". An entity refers to the specific information contained in the target request, which can be a noun, a noun phrase, or other words with specific meanings. The entity provides the details required to complete the intention. For example, for the request "Turn on the lights in the living room", the intention is "Control the device", and the entities are "living room" and "lights". For instruction type requests, a natural language understanding (NLU) model is generally used. The entity category can be inferred through the sentence pattern and slot position, and the requirement for entity knowledge is not high. Moreover, the intention of instruction type requests is more explicit and suitable for intention mining.

[0090] A non-instruction type request refers to a request with an unclear intention in the target request. It is often difficult to distinguish the intention or even prone to misjudgment through the sentence pattern for non-instruction type requests, and certain entity knowledge is required. For example, a non-instruction type request is "The Wandering Earth". For such requests, the computer device needs to have the prior knowledge that "The Wandering Earth" is the name of a movie to accurately infer the intention. Therefore, for non-instruction type requests, it is more suitable for entity mining.

[0091] In the solution of the embodiments of the present application, for a target request with an unknown intention, the computer device may not be able to extract the correct parameters, resulting in the failure of the target request response. By classifying the target request into instruction type and non-instruction type, and then performing log analysis separately, it is beneficial to improve the coverage of intention and entity, and improve the recognition ability of the computer device for the input request.

[0092] Step 204: Use the first clustering model to cluster the instruction type requests into multiple categories, and use a large language model to identify the intention of the instruction type requests under each category to obtain the intention corresponding to each category.

[0093] Among them, the first clustering model refers to a machine learning model that groups instruction type requests according to semantic similarity. Since there are intention-related words in the instruction type requests, when using the first clustering model to cluster the instruction type requests, the instruction type requests under each obtained category have the same or similar intention.

[0094] A large language model (LLM) refers to a natural language processing model trained through deep learning technology and with a huge number of parameters. By training on a large amount of text data, the large language model can learn the complex structure and pattern of natural language, and thus has the ability to generate high-quality text and understand natural language.

[0095] Intent recognition refers to inputting a first prompt word into a large language model, and the large language model outputs the intent corresponding to each category. The first prompt word is used to instruct the large language model to output the intent corresponding to the instruction type request under each category. For example, using a first clustering model to cluster into M categories, and each category includes at least one instruction type request. The large language model summarizes the intent of the instruction type requests in category i and concludes that the intent under category i is "turn on the device", where M and i are positive integers.

[0096] In some embodiments, the large language model can summarize the intents corresponding to multiple categories respectively to obtain the expansion direction of the new business field.

[0097] Step 206: Use a second clustering model to cluster non-instruction type requests into multiple categories, and use the large language model to determine whether the non-instruction type requests under each category contain entities; in the case where there is a category for which it is impossible to determine whether it contains entities, extract the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined that the corresponding category contains entities based on the keywords, use the keywords as the entities contained in the non-instruction type requests under the corresponding category.

[0098] Among them, the second clustering model refers to a machine learning model that groups non-instruction type requests according to semantic similarity. Since there are no words related to intent in non-instruction type requests, but there may be words related to entities, therefore, clustering non-instruction type requests using the second clustering model, the non-instruction type requests under each obtained category may have the same or similar entities, or may not have entities.

[0099] The computer device can input a second prompt word into the large language model, and the large language model determines whether the non-instruction type requests under each category contain entities. The second prompt word is used to instruct the large language model to output the determination result of whether the non-instruction type requests under each category contain entities.

[0100] In some embodiments, the computer device determines the number of non-instruction types included in multiple categories. For categories with a number greater than a preset number threshold, use the large language model to determine whether the non-instruction type requests under the corresponding category contain entities; for categories with a number not greater than the preset number threshold, in the next round of log analysis, integrate the non-instruction type requests included in the corresponding category with the newly determined non-instruction type requests, so as to use the second clustering model to cluster the integrated non-instruction type requests.

[0101] As Figure 6 shown is a schematic diagram of entity recognition for non-instruction type requests provided by some embodiments of the present application. Refer to Figure 6, when the definite result output by the large language model indicates that there is a category that definitely contains entities, obtain the entities included in the non-instruction type requests under this category output by the large language model, and save the non-instruction type requests and the included entities under this category to a preset database.

[0102] When the definite result output by the large language model indicates that there is a category that definitely does not contain entities, add a label of invalid data to the non-instruction type requests under this category, and save them to the preset database.

[0103] When the definite result output by the large language model indicates that there is a category where it is impossible to determine whether it contains entities, entity recognition needs to be further performed on this category. The specific method is to extract the keywords in the non-instruction type requests under this category. For example, a large language model or a keyword extraction algorithm can be used for extraction, and based on the keywords, it is determined whether the corresponding category contains entities. Exemplarily, the computer device searches for knowledge content matching the keywords in the preset knowledge base, and the preset knowledge base can be an online knowledge base. Input the third prompt word into the large language model, and the large language model determines whether the corresponding category contains entities based on this knowledge content. Since the knowledge content contains rich information related to the keywords and can represent the specific meaning of the keywords, therefore, based on the method of the large language model and keywords, it can accurately determine whether the corresponding category contains entities and reduce the possibility of misjudgment. Among them, the preset knowledge base can have an online update function to ensure that the knowledge content in the preset knowledge base can be updated and expanded in real time. In some embodiments, the computer device can also search for knowledge content matching the keywords through online networking to ensure the timeliness of the knowledge content.

[0104] When the large language model determines based on the knowledge content that the corresponding category does not contain entities, add a label of invalid data to the non-instruction type requests under this category, and save them to the preset database.

[0105] When the large language model determines based on the knowledge content that the corresponding category contains entities, obtain the entities included in the non-instruction type requests under this category output by the large language model, and save the non-instruction type requests and the included entities under this category to the preset database. In some embodiments, the keyword of the non-instruction type request is the included entity.

[0106] In some embodiments, the preset database can be used for speech assistant model training or as an external knowledge base, thereby improving the semantic understanding ability of the speech assistant.

[0107] Alternatively, the invalid data in the preset database can be merged into the next non-instruction type request to be clustered. When the new non-instruction type request belongs to the same category as the non-instruction type request marked as invalid data, it can be directly considered as invalid data without being submitted to the large language model for judgment. The advantage of doing this is that it can reduce the time consumption, and the disadvantage is that a small amount of data will be misjudged as invalid data.

[0108] In the above log analysis method, by identifying multiple target requests with unknown intentions in the log to be analyzed, since requests with known intentions can quickly identify the intentions and obtain responses, therefore, further analyzing the target requests with unknown intentions is beneficial to improving the log analysis efficiency; instruction type requests are requests with clear intentions among the target requests, and non-instruction type requests are requests with unclear intentions among the target requests, that is, the semantic focuses of different types of target requests are different. By identifying the instruction type requests and non-instruction type requests among the multiple target requests, it is beneficial to guide the clustering model to cluster different types of target requests according to different semantic focuses; using the first clustering model to cluster the instruction type requests, each category has similar intentions, and using the large language model to identify the intentions corresponding to each category, using the second clustering model to cluster the non-instruction type requests, and identifying the entities corresponding to each category based on the large language model and the keywords in the non-instruction type requests. This method of classifying and clustering the target requests in the log to be analyzed and identifying the intentions and entities according to the categories greatly reduces the number of calls to the large language model and the log analysis time consumption compared with using the large language model to analyze a single target request, which is beneficial to improving the log analysis efficiency.

[0109] In an exemplary embodiment, the instruction type requests and non-instruction type requests among the multiple target requests are determined by a pre-trained classification model. The training steps of the pre-trained classification model include: obtaining multiple request samples and the type labels corresponding to the multiple request samples respectively; selecting target request samples from the multiple request samples, classifying the target request samples using the initial model, and obtaining the confidence that the target request samples belong to the instruction type; calculating the first model loss based on the confidence that the target request samples belong to the instruction type and the type label, and adjusting the parameters of the classification module in the initial model based on the first model loss until the first stop condition is met and then stopping to obtain the initially trained initial model; for the remaining request samples other than the target request samples among the multiple request samples, classifying the remaining request samples using the initially trained initial model, and obtaining the confidence that the remaining request samples belong to the instruction type; calculating the second model loss based on the confidence that the remaining request samples belong to the instruction type and the type label, and adjusting the parameters of the vector conversion module and the classification module in the initially trained initial model based on the second model loss until the second stop condition is met and then stopping to obtain the pre-trained classification model.

[0110] Among them, the instruction type requests and non-instruction type requests in multiple target requests are determined by a pre-trained classification model. For example, a computer device inputs multiple target requests into the pre-trained classification model respectively to obtain the confidence levels of each of the multiple target requests belonging to the instruction type requests; when the confidence level of any target request belonging to the instruction type request exceeds a preset confidence level (for example, the preset confidence level is 0.5), the corresponding target request is regarded as an instruction type request, so that the instruction type requests and non-instruction type requests in the multiple target requests can be determined.

[0111] In some embodiments, multiple target requests can be processed simultaneously, and this batch processing method is beneficial to improving the classification efficiency.

[0112] The training steps of the pre-trained classification model are obtained by training based on an initial model. The pre-trained classification model has the same model structure as the initial model, but the model parameters may be different. As Figure 7 shown is a schematic structural diagram of the pre-trained classification model provided by some embodiments of the present application. The pre-trained classification model includes a vector conversion module and a classification module. The vector conversion module includes an embedding layer (Embedding) and an encoder layer. Exemplarily, the pre-trained E5 model can be used as the encoder layer. The pre-trained E5 model is a text embedding method trained by contrastive learning and has good generalization ability for texts in different fields, which is beneficial to improving the classification effect of the classification model. The classification module is composed of a fully connected layer (FC), an activation function layer (ReLU), and an output layer (Sigmoid). pooler_output is the output layer of the E5 model. Two fully connected layers are used for classification, and the Sigmoid function is used as the binary classification loss function.

[0113] Before training the classification model, a training data set needs to be prepared. The data set contains multiple request samples and their corresponding type labels. For example, an instruction type request sample corresponds to an instruction type label, for example, it can be 1, and a non-instruction type request sample corresponds to a non-instruction type label, for example, it can be 0. During the training process, a target request sample is selected, the parameters of the vector conversion module are frozen, and only the parameters of the classification module are trained. When the first model loss meets the first stopping condition, the training of the classification module is stopped to obtain a preliminarily trained initial model; then the freezing of the parameters of the vector conversion module is released, and the entire preliminarily trained initial model is adjusted using the full data set or the remaining request samples. When the second model loss meets the second stopping condition, the training is stopped to obtain the pre-trained classification model.

[0114] In this embodiment, the instruction type requests and non-instruction type requests in multiple target requests are determined by a pre-trained classification model. The pre-trained classification model is obtained by training based on an initial model. During the model training process, the parameters of the vector conversion module are first frozen, and the classification module is trained using mini-batch samples. Then, the parameter freezing is lifted, and the complete model is trained. This method is beneficial to improving the training efficiency and classification effect of the model.

[0115] In an exemplary embodiment, a first clustering model is used to cluster instruction type requests into multiple categories, including: performing vector conversion processing on the instruction type requests through the vector conversion module in the first clustering model to obtain request vectors; and performing clustering processing on the request vectors through the clustering algorithm module in the first clustering model to obtain multiple categories.

[0116] Among them, as Figure 8 shown is a schematic structural diagram of the first clustering model provided by some embodiments of the present application. The first clustering model includes a vector conversion module and a clustering algorithm module. The vector conversion module includes an embedding layer and an encoder layer. Exemplarily, a pre-trained RoBERTa model can be used as the encoder layer. The pre-trained RoBERTa model adopts a dynamic masking strategy, which is beneficial to improving the generalization ability of the model. The clustering algorithm module can adopt algorithms such as K-means and DBSCAN.

[0117] The computer device inputs the instruction type requests into the first clustering model, performs vector conversion processing on the input instruction type requests through the vector conversion module to obtain request vectors, and then performs clustering processing on the request vectors through the clustering algorithm module to obtain multiple categories.

[0118] In some embodiments, the second clustering model has the same model structure as the first clustering model, and the model parameters may be different.

[0119] In this embodiment, performing vector conversion processing on the instruction type requests through the vector conversion module in the first clustering model is beneficial to significantly improving the model performance in the processing task of the instruction type requests. Performing clustering processing on the basis of the request vectors through the clustering algorithm module is beneficial to improving the clustering accuracy of the first clustering model, thereby improving the intent recognition accuracy.

[0120] In an exemplary embodiment, the training steps of the first clustering model include: obtaining a plurality of instruction type request samples and the intent labels respectively corresponding to the plurality of instruction type request samples; selecting a target instruction type request sample from the plurality of instruction type request samples, and using an initial clustering model to perform intent classification on the target instruction type request sample to obtain at least one intent category of the target instruction type request sample and the confidence of each intent category; calculating a third model loss based on the confidence of at least one intent category of the target instruction type request sample and the intent label, and adjusting the parameters of the classification module of the initial clustering model based on the third model loss until stopping when the third stopping condition is met, to obtain a preliminarily trained initial clustering model; for the remaining instruction type request samples other than the target instruction type request sample among the plurality of instruction type request samples, using the preliminarily trained initial clustering model to perform intent classification on the remaining instruction type request samples to obtain at least one intent category of the remaining instruction type request samples and the confidence of each intent category; calculating a fourth model loss based on the confidence of at least one intent category of the remaining instruction type request samples and the intent label, and adjusting the parameters of the vector conversion module and the classification module of the preliminarily trained initial clustering model based on the fourth model loss until stopping when the fourth stopping condition is met, to obtain the first clustering model.

[0121] Among them, the training steps of the first clustering model are obtained by training based on an initial clustering model. The initial clustering model has the same model structure as the first clustering model, but may have different model parameters. Refer to Figure 7 , the first clustering model further includes a classification module. The classification module is composed of a fully connected layer (FC), an activation function layer (ReLU), and an output layer (Sigmoid). pooler_output is the output layer of the RoBERTa model. Two fully connected layers are used for classification, and the Sigmoid function is used as the binary classification loss function.

[0122] Before training the classification model, it is necessary to prepare a training data set. The data set contains a plurality of instruction type request samples and their corresponding intent labels. For example, the intent labels include turning on the device, turning off the device, querying the weather, etc.

[0123] During the training process, a target instruction type request sample is selected, the parameters of the vector conversion module are frozen, and only the parameters of the classification module are trained. Stop the training of the classification module when the third model loss meets the third stopping condition to obtain a preliminarily trained initial clustering model; then unfreeze the parameters of the vector conversion module, and use the full data set or the remaining instruction type request samples to adjust the parameters of the entire preliminarily trained initial clustering model, and stop when the fourth model loss meets the fourth stopping condition to obtain the first clustering model. The first clustering model tends to cluster according to similar intents.

[0124] In some embodiments, during the training process of unfreezing the parameters of the vector conversion module, a contrastive learning method can be adopted. Instruction type request samples with similar intents are grouped as positive samples, and instruction type request samples with dissimilar intents are grouped as negative samples. The parameters of the vector conversion module are updated to make the vector distances of instruction type request samples with similar intents closer in the vector space and the vector distances of instruction type request samples with dissimilar intents farther away.

[0125] In some embodiments, the second clustering model can be trained by referring to the training steps of the first clustering model. The training data set used by the second clustering model includes multiple non-instruction type request samples and their respective corresponding entity labels. For example, the entity labels include living room, lamp, etc. The second clustering model tends to cluster according to similar entities.

[0126] When the first clustering model and the second clustering model perform model inference, the classification module can be removed, and the output of the vector conversion module can be directly connected to the clustering algorithm to obtain the clustering result. In this method, the vector conversion module trained according to the intent labels is used for clustering by the clustering algorithm, which is beneficial to improving the clustering accuracy.

[0127] In this embodiment, the first clustering model is trained by the initial clustering model. During the model training process, the parameters of the vector conversion module are first frozen, and the classification module is trained using small batch samples; then the parameter freezing is removed, and the complete model is trained. This method is beneficial to improving the training efficiency and classification effect of the model.

[0128] In an exemplary embodiment, a large language model is used to perform intent recognition on instruction type requests under each category to obtain the intent corresponding to each category, including: sorting multiple categories according to the number of instruction type requests under each category; for the first preset number of categories sorted from more to less in number among the multiple categories sorted in this way, selecting at least one target category from the first preset number of categories; using the large language model to perform intent recognition on the instruction type requests under each target category to obtain the intent corresponding to each target category.

[0129] Among the categories obtained by clustering through the first clustering model, the number of instruction type requests included in each category may be different. The more the number included in the category, the more accurate the intent recognized according to the category. For example, if category 1 includes 100 instruction type requests with similar semantics and category 2 includes 2 instruction type requests with similar semantics, the intent recognized for category 1 can more accurately reflect the intent under this category.

[0130] Such as Figure 9The figure shows a schematic diagram of identifying the intent of instruction type requests by category provided in some embodiments of the present application. The computer device clusters multiple instruction type requests using a first clustering model to obtain M categories, sorts the M categories according to the number of instruction types they contain, selects the top N categories from them, and uses a large language model to identify the intent of the top N categories, obtaining the intent corresponding to each of the top N categories. Then, the intent corresponding to each of the top N categories and the instruction type requests under the top N categories are input into the large language model for intent summarization to obtain the expansion direction of the new business field.

[0131] In some embodiments, categories in which the number of instruction type requests contained in multiple categories exceeds a preset number can also be used as target categories.

[0132] In this embodiment, by selecting at least one target category with a relatively large number from multiple categories, the large language model identifies the intent of the instruction type requests under the target category. Since the number of intents contained in the target category is relatively large, the large language model's identification of the intent of the instruction type requests under the target category can improve the accuracy of the identified intent.

[0133] In an exemplary embodiment, the log analysis method further includes: for the remaining categories other than the target categories among multiple categories, after re-determining the instruction type requests and non-instruction type requests in multiple target requests, integrating the instruction type requests under the remaining categories with the re-determined instruction type requests, and returning to the step of clustering the instruction type requests into multiple categories using the first clustering model and continuing to execute.

[0134] Among them, the remaining categories refer to the categories other than the target categories among multiple categories, that is, the categories with a relatively small number of instruction type requests. Refer to Figure 9 , the remaining categories are categories N + 1 to category M. The remaining categories can be integrated with the re-determined instruction type requests during the next round of log analysis to cluster the integrated instruction type requests using the first clustering model.

[0135] In this embodiment, by using the categories with a relatively small number of instruction type requests as the remaining categories and using them for re-clustering during the next round of log analysis, it is convenient to increase the number of instruction type requests contained in the remaining categories, which is beneficial to improving the accuracy of intent recognition.

[0136] To illustrate the effect of the log analysis method in this solution in detail, the following is an example of the most detailed embodiment:

[0137] The log analysis method provided in the embodiments of the present application can be applied to a variety of application scenarios, for example, voice assistant scenarios, smart home scenarios, etc. As Figure 10The following is a schematic diagram of the overall process of the log analysis method provided by some embodiments of the present application. The computer device obtains the logs to be analyzed at regular time intervals. For example, log analysis is performed once a month. The computer device extracts multiple input requests from the logs to be analyzed and removes duplicate requests. The BERT model is used to perform a preliminary intent classification on the multiple requests to identify the requests with known intents and the target requests with unknown intents among the multiple requests. Then, a pre-trained classification model is used to determine the instruction type requests and non-instruction type requests among the multiple target requests. Among them, the classification model is a semantic-based classification, and there is no type that cannot be covered. Finally, the first clustering model is used to cluster the instruction type requests into multiple categories, and the large language model is used to identify the intents of the instruction type requests under each category to obtain the intents corresponding to each category; the second clustering model is used to cluster the non-instruction type requests into multiple categories, and the large language model is used to determine whether the non-instruction type requests under each category contain entities; in the case where it is impossible to determine whether a category contains entities, the keywords in the non-instruction type requests under the corresponding category are extracted; in the case where it is determined based on the keywords that the corresponding category contains entities, the keywords are used as the entities contained in the non-instruction type requests under the corresponding category.

[0138] Among them, the pre-trained classification model is used to distinguish whether the target request belongs to an instruction type request or a non-instruction type request. Before training the pre-trained classification model, it is necessary to prepare a response data set, which contains instruction type requests and non-instruction type requests, marked as 1 and 0. During training, first use a small batch of data, freeze the E5 model parameters, and only train the classifier; after the loss value stabilizes, unfreeze it and use the full amount of data to train the complete network. After the loss value stabilizes, the training is completed to obtain a binary classification model, and the model outputs the confidence that the current target request is an instruction type request. Next, a threshold Δ (Δ>0.5, the larger the value, the stricter the judgment) is set to perform binary classification on the target request. If it is greater than the threshold, it is considered an instruction type request, otherwise it is a non-instruction type request. To improve the classification efficiency, the batch inference method can be adopted, for example, processing 512 target requests at the same time.

[0139] The model structures of the first clustering model and the second clustering model can be the same, and the model parameters can be different. The first clustering model and the second clustering model are used to classify target requests with the same or similar semantics into one category to reduce the number of calls to the subsequent large language model. Different clustering models can be used for instruction type requests and non-instruction type requests. The first clustering model is more biased towards intent clustering, and the second clustering model is more biased towards entity clustering. The first clustering model and the second clustering model can be pre-trained to ensure their respective clustering tendencies.

[0140] During the training process of the first clustering model or the second clustering model, for instruction type requests, data with clear intentions and intention labels needs to be prepared for multi-classification task training; for non-instruction type requests, data with similar entities and entity labels needs to be prepared for multi-classification (here, the BIO label system is not used to classify tokens, but the entire sentence is classified). During training, first freeze the model parameters of the RoBERTa model and only train the classifier. After the loss converges, perform full-scale parameter training. After training is completed, the first clustering model and the second clustering model will be obtained respectively, and the output is the confidence level for each classification. Through the above training, the model can learn the semantic features of instruction type requests or non-instruction type requests. Next, use the contrastive learning method to combine corpora with similar intentions / entities into positive sample pairs and those with large differences into negative sample pairs, and train and update the model parameters of the RoBERTa model to make the query vectors with similar intentions / entities in the vector space closer in distance and those that are different farther away. During model inference, remove the classifier structure and connect the pooler_output to a clustering algorithm such as K-means or DBSCAN to output the final classification result.

[0141] Among them, for the clustering results of instruction type requests, sort them according to the number of instruction type requests included in each category, and use the top N categories for intention analysis. The remaining categories are used for merging into the re-determined instruction type requests during the next round of log analysis for integration and participation in subsequent clustering. For the top N categories, use a large language model to summarize the intentions and generalize the intentions under this category. Finally, hand over the N intentions and the instruction type requests in their corresponding categories to the large language model for overall summary to obtain the new business area expansion direction.

[0142] For the clustering results of non-instruction type requests, for the categories in which the number of non-instruction type requests included in multiple categories is greater than n, further analysis is carried out. For the categories with a number not greater than n, they are merged into the re-determined non-instruction type requests during the next round of log analysis for integration and participate in subsequent clustering. For the instruction type requests of each category, first use the large language model for the first screening, and with the help of its prior knowledge, screen out the categories that clearly contain entities and those that clearly do not contain entities. For the remaining categories, call the preset knowledge base, and the large language model extracts keywords, returns search results through the search tool, and then the large language model determines whether the non-instruction type requests of this category contain entities based on the search results. Finally, save the entity content to the preset database, which can be used for speech assistant model training or as an external knowledge base to improve the semantic understanding ability of the language assistant. Additionally, optionally, the invalid data in the preset database can be merged into the non-instruction type requests to be clustered next time. When the new non-instruction type requests and the non-instruction type requests marked as invalid data belong to the same category, they can be directly considered as invalid data without being handed over to the large language model for judgment. The advantage of doing this is that it can reduce the time consumption, and the disadvantage is that a small amount of data will be misjudged as invalid data.

[0143] Some embodiments provide a log analysis method, which identifies multiple target requests with unknown intent in the log to be analyzed. Since the requests with known intent can quickly identify the intent and get a response, further analysis of the target requests with unknown intent is beneficial to improving the efficiency of log analysis; instruction type requests are requests with clear intent in the target requests, and non-instruction type requests are requests with unclear intent in the target requests, that is, different types of target requests have different semantic emphases. By identifying instruction type requests and non-instruction type requests in multiple target requests, it is beneficial to guide the clustering model to cluster different types of target requests according to different semantic emphases; a first clustering model is used to cluster the instruction type requests, each category has similar intent, and a large language model is used to identify the intent corresponding to each category, a second clustering model is used to cluster the non-instruction type requests, and entities corresponding to each category are identified based on the large language model and keywords in the non-instruction type requests. This method of classifying and clustering the target requests in the log to be analyzed, and identifying intents and entities according to categories greatly reduces the number of calls to the large language model and the time consumption of log analysis compared to using a large language model to analyze a single target request, which is beneficial to improving the efficiency of log analysis. In addition, the method of using multiple small models to pre-process the logs to be analyzed can reduce the inference time of the large language model. Among them, the classification method of instruction type and non-instruction type can effectively distinguish between intent-first and entity-first requests; the model training method of the clustering model can effectively guide the classification focus of the clustering model; using classification and clustering methods, single requests are merged into classes for analysis and processing, which reduces the number of calls of the large language model by multiples; the importance of logs is distinguished by cluster size, making log analysis more focused. This method can be used to perform large-scale log analysis in a short period of time.

[0144] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0145] Based on the same inventive concept, an embodiment of the present application further provides a log analysis device for implementing the log analysis method involved above. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the log analysis device provided below can refer to the limitations on the log analysis method in the foregoing, and will not be repeated here.

[0146] In an exemplary embodiment, as Figure 11 shown, a log analysis device 1100 is provided, including: a determination module 1120, a first recognition module 1140, and a second recognition module 1160, where:

[0147] The determination module 1120 is configured to determine instruction type requests and non-instruction type requests among a plurality of target requests with unknown intentions in the log to be analyzed.

[0148] The first recognition module 1140 is configured to cluster the instruction type requests into multiple categories by using a first clustering model, and use a large language model to identify the intentions of the instruction type requests under each category, so as to obtain the intentions corresponding to each category.

[0149] The second recognition module 1160 is configured to cluster the non-instruction type requests into multiple categories by using a second clustering model, and use a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where there is a category for which it is impossible to determine whether it contains entities, extract the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined based on the keywords that the corresponding category contains entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding category.

[0150] In one embodiment, the instruction type requests and non-instruction type requests among the multiple target requests are determined by a pre-trained classification model. In terms of the training of the pre-trained classification model, the log analysis device 1100 further includes a first model training module, and the first model training module is configured to: obtain a plurality of request samples and the type labels respectively corresponding to the plurality of request samples; select target request samples from the plurality of request samples, classify the target request samples by using an initial model, and obtain the confidence that the target request samples belong to the instruction type; calculate a first model loss based on the confidence that the target request samples belong to the instruction type and the type labels, and adjust the parameters of the classification module in the initial model based on the first model loss until the first stop condition is met and then stop to obtain the initially trained initial model; for the remaining request samples other than the target request samples among the plurality of request samples, classify the remaining request samples by using the initially trained initial model, and obtain the confidence that the remaining request samples belong to the instruction type; calculate a second model loss based on the confidence that the remaining request samples belong to the instruction type and the type labels, and adjust the parameters of the vector conversion module and the classification module in the initially trained initial model based on the second model loss until the second stop condition is met and then stop to obtain the pre-trained classification model.

[0151] In one embodiment, the instruction type requests are clustered into multiple categories by using a first clustering model. The first identification module 1140 is further configured to: perform vector conversion processing on the instruction type requests by using the vector conversion module in the first clustering model to obtain request vectors; perform clustering processing on the request vectors by using the clustering algorithm module in the first clustering model to obtain multiple categories.

[0152] In one embodiment, in terms of the training of the first clustering model, the log analysis device 1100 further includes a second model training module, and the second model training module is configured to: obtain a plurality of instruction type request samples and the intent labels respectively corresponding to the plurality of instruction type request samples; select a target instruction type request sample from the plurality of instruction type request samples, and perform intent classification on the target instruction type request sample by using an initial clustering model to obtain at least one intent category of the target instruction type request sample and the confidence of each intent category; calculate a third model loss based on the confidence of at least one intent category of the target instruction type request sample and the intent label, and adjust the parameters of the classification module of the initial clustering model based on the third model loss until stopping when the third stopping condition is met, so as to obtain a preliminarily trained initial clustering model; for the remaining instruction type request samples other than the target instruction type request samples in the plurality of instruction type request samples, perform intent classification on the remaining instruction type request samples by using the preliminarily trained initial clustering model to obtain at least one intent category of the remaining instruction type request samples and the confidence of each intent category; calculate a fourth model loss based on the confidence of at least one intent category of the remaining instruction type request samples and the intent label, and adjust the parameters of the vector conversion module and the classification module of the preliminarily trained initial clustering model based on the fourth model loss until stopping when the fourth stopping condition is met, so as to obtain the first clustering model.

[0153] In one embodiment, a large language model is used to perform intent recognition on the instruction type requests under each category to obtain the intent corresponding to each category. The first recognition module 1140 is further configured to: sort the plurality of categories according to the number of instruction type requests under each category; for the first preset number of categories among the plurality of categories sorted from most to least in terms of quantity, select at least one target category from the first preset number of categories; use the large language model to perform intent recognition on the instruction type requests under each target category to obtain the intent corresponding to each target category.

[0154] In one embodiment, the log analysis device 1100 further includes a processing module, and the processing module is configured to: for the remaining categories other than the target categories among the plurality of categories, after re-determining the instruction type requests and non-instruction type requests in the plurality of target requests, integrate the instruction type requests under the remaining categories with the re-determined instruction type requests, and return to the step of clustering the instruction type requests into a plurality of categories by using the first clustering model and continue to execute.

[0155] Some embodiments provide a log analysis device. By identifying multiple target requests with unknown intentions in the logs to be analyzed, since requests with known intentions can have their intentions quickly identified and responses obtained, further analyzing the target requests with unknown intentions is beneficial to improving the log analysis efficiency. Instruction type requests are requests with clear intentions among the target requests, and non-instruction type requests are requests with unclear intentions among the target requests, that is, the semantic focuses of different types of target requests are different. By identifying the instruction type requests and non-instruction type requests among the multiple target requests, it is beneficial to guide the clustering model to cluster different types of target requests according to different semantic focuses. The first clustering model is used to cluster the instruction type requests, each category having similar intentions, and a large language model is used to identify the intentions corresponding to each category. The second clustering model is used to cluster the non-instruction type requests, and entities corresponding to each category are identified based on the large language model and the keywords in the non-instruction type requests. This method of classifying and clustering the target requests in the logs to be analyzed and identifying the intentions and entities according to the categories greatly reduces the number of calls to the large language model and the log analysis time compared to analyzing a single target request using the large language model, which is beneficial to improving the log analysis efficiency.

[0156] Each module in the above log analysis device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form so that the processor can call and execute the operations corresponding to the above respective modules.

[0157] In an exemplary embodiment, a computer device is provided. This computer device can be a server, and its internal structure diagram can be as Figure 12 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the logs to be analyzed, target requests, the intentions of the target requests, and the entities of the target requests. The input / output interface of the computer device is used for the processor to exchange information with external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a log analysis method.

[0158] Those skilled in the art can understand that Figure 12 the structure shown in Figure 12 is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0159] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0160] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0161] In an embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0162] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0163] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0164] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded in the present application.

[0165] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A log analysis method, characterized in that, The method includes: For multiple target requests with unknown intents in the log to be analyzed, determining the instruction type requests and non-instruction type requests among the multiple target requests; Using a first clustering model to cluster the instruction type requests into multiple categories, and using a large language model to identify the intents of the instruction type requests under each category, obtaining the intent corresponding to each category; Using a second clustering model to cluster the non-instruction type requests into multiple categories, and using a large language model to determine whether the non-instruction type requests under each category contain entities; in the case where it is impossible to determine whether a category contains entities, extracting the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined that the corresponding category contains entities based on the keywords, using the keywords as the entities contained in the non-instruction type requests under the corresponding category.

2. The method according to claim 1, wherein The instruction type requests and non-instruction type requests among the multiple target requests are determined by a pre-trained classification model, and the training steps of the pre-trained classification model include: Obtaining multiple request samples and the type labels corresponding to the multiple request samples respectively; Selecting a target request sample from the multiple request samples, classifying the target request sample using an initial model, and obtaining the confidence that the target request sample belongs to the instruction type; Calculating a first model loss based on the confidence that the target request sample belongs to the instruction type and the type label, and adjusting the parameters of the classification module in the initial model based on the first model loss until stopping when a first stop condition is met, obtaining a preliminarily trained initial model; For the remaining request samples among the multiple request samples except the target request sample, classifying the remaining request samples using the preliminarily trained initial model, and obtaining the confidence that the remaining request samples belong to the instruction type; Calculating a second model loss based on the confidence that the remaining request samples belong to the instruction type and the type label, and adjusting the parameters of the vector conversion module and the classification module in the preliminarily trained initial model based on the second model loss until stopping when a second stop condition is met, obtaining a pre-trained classification model.

3. The method according to claim 1, wherein The step of using the first clustering model to cluster the instruction type requests into multiple categories includes: Performing vector conversion processing on the instruction type requests using the vector conversion module in the first clustering model to obtain request vectors; Performing clustering processing on the request vectors using the clustering algorithm module in the first clustering model to obtain multiple categories.

4. The method according to claim 3, characterized in that The training steps of the first clustering model include: Obtaining multiple instruction type request samples and the intent labels corresponding to the multiple instruction type request samples respectively; Selecting a target instruction type request sample from the multiple instruction type request samples, performing intent classification on the target instruction type request sample using an initial clustering model, and obtaining at least one intent category of the target instruction type request sample and the confidence of each intent category; Calculate a third model loss based on the confidence of at least one intention category of the target instruction type request sample and the intention label, and adjust the parameters of the classification module of the initial clustering model based on the third model loss until the third stopping condition is met and then stop, obtaining the initially trained initial clustering model; For the remaining instruction type request samples among the multiple instruction type request samples except the target instruction type request sample, use the initially trained initial clustering model to perform intention classification on the remaining instruction type request samples, obtaining at least one intention category of the remaining instruction type request samples and the confidence of each intention category; Calculate a fourth model loss based on the confidence of at least one intention category of the remaining instruction type request samples and the intention label, and adjust the parameters of the vector conversion module and the classification module of the initially trained initial clustering model based on the fourth model loss until the fourth stopping condition is met and then stop, obtaining the first clustering model.

5. The method according to claim 1, wherein The using the large language model to perform intention recognition on the instruction type requests under each category and obtaining the corresponding intention for each category includes: Sort the multiple categories according to the number of instruction type requests under each category; For the first preset number of categories among the multiple categories sorted from most to least in terms of quantity, select at least one target category from the first preset number of categories; Use the large language model to perform intention recognition on the instruction type requests under each target category, obtaining the corresponding intention for each target category.

6. The method according to claim 5, characterized in that, The method further includes: For the remaining categories among the multiple categories except the target category, after re-determining the instruction type requests and non-instruction type requests in the multiple target requests, integrate the instruction type requests under the remaining categories with the re-determined instruction type requests, and return to the step of clustering the instruction type requests into multiple categories using the first clustering model and continue to execute.

7. A log analysis device, characterized in that, The device includes: A determination module, configured to determine the instruction type requests and non-instruction type requests in multiple target requests with unknown intentions in the log to be analyzed; A first recognition module, configured to cluster the instruction type requests into multiple categories using the first clustering model, and use the large language model to perform intention recognition on the instruction type requests under each category, obtaining the corresponding intention for each category; A second recognition module, configured to cluster the non-instruction type requests into multiple categories using the second clustering model, and use the large language model to determine whether the non-instruction type requests under each category contain entities; in the case where there is a category for which it is impossible to determine whether it contains entities, extract the keywords in the non-instruction type requests under the corresponding category; in the case where it is determined based on the keywords that the corresponding category contains entities, use the keywords as the entities contained in the non-instruction type requests under the corresponding category.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images