Multi-modal operation and maintenance data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a multi-mode operation and maintenance data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to each task request initiated by front-end equipment, generating a task identifier of each task request, the task identifier being a globally unique user experience session identifier; obtaining task operation data of a plurality of modes associated with the task identifier of each task request, wherein the task operation data of each mode is correspondingly acquired from one data source; and with the task identifier of each task request as a main key, aggregating the task operation data of the multiple modals belonging to the same task identifier into the user interaction conversation object corresponding to the task identifier of each task request. According to the technical scheme, the problems of low operation and maintenance data utilization efficiency and low problem positioning efficiency in the prior art can be solved, the scientificity of operation and maintenance decision and the problem troubleshooting efficiency can be improved, and optimization and upgrading of a digital platform and improvement of service quality are promoted.

Description

Technical Field

[0001] This invention relates to the fields of big data, artificial intelligence and IT operations and maintenance management technology, and in particular to a multimodal operations and maintenance data processing method, device, electronic device and storage medium. Background Technology

[0002] As enterprises continue to deepen their digital transformation, digital channel platforms in sectors such as power, finance, and government have accumulated massive amounts of heterogeneous data. Because this data is typically managed separately by different teams—operations, business, and product—in independent systems, it creates a typical data silo phenomenon. Currently, the use of data by these teams is fragmented: operations teams rely solely on technical data to locate system faults, business teams depend on aggregated business data to generate reports, and product teams optimize product experience by sampling user behavior data. This data management and usage model makes it difficult to quickly identify the root cause of problems during cross-team collaboration, hindering the efficient differentiation between technical faults, business logic defects, and front-end design issues. This significantly reduces troubleshooting efficiency and data utilization value, restricting the optimization and upgrading of digital platforms and the improvement of service quality. Summary of the Invention

[0003] This invention provides a multimodal operation and maintenance data processing method to solve the problems of low efficiency in operation and maintenance data utilization and problem location in the prior art.

[0004] According to one aspect of the present invention, a multimodal operation and maintenance data processing method is provided, the method comprising:

[0005] In response to each task request initiated by the front-end device, a task identifier is generated for each task request. The task identifier is a globally unique user experience session identifier.

[0006] Obtain task operation data for multiple modalities associated with the task identifier of each task request. The task operation data for each modality is collected from a data source.

[0007] Using the task identifier of each task request as the primary key, task operation data from multiple modalities belonging to the same task identifier are aggregated into the user interaction dialogue object corresponding to the task identifier of each task request.

[0008] According to another aspect of the present invention, a multimodal operation and maintenance data processing apparatus is provided, the apparatus comprising:

[0009] The identifier generation module is used to generate a task identifier for each task request in response to each task request initiated by the front-end device. The task identifier is a globally unique user experience session identifier.

[0010] The data acquisition module is used to acquire task operation data of multiple modalities associated with the task identifier of each task request. The task operation data of each modality is collected from a data source.

[0011] The data aggregation module is used to aggregate task operation data from multiple modalities belonging to the same task identifier into the user interaction dialogue object corresponding to the task identifier of each task request, using the task identifier of each task request as the primary key.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the multimodal operation and maintenance data processing method according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the multimodal operation and maintenance data processing method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the multimodal operation and maintenance data processing method described in any embodiment of the present invention.

[0018] The technical solution of this invention ensures the traceability and uniqueness of each task request by generating a globally unique user experience session identifier as the task identifier for each task request. Obtaining task operation data from multiple modalities based on the task identifier improves the targeting and completeness of operation and maintenance data collection. Aggregating multiple modal task operation data with the same task identifier as the primary key to the user interaction dialogue object corresponding to the task identifier of each task request enables the association and integration of operation and maintenance data from multiple modalities throughout the entire business request process, thereby providing coherent and complete data support for operation and maintenance analysis and problem localization. Based on the above technical solution, associating and aggregating task operation data from multiple modalities through task identifiers solves the problem of low efficiency in operation and maintenance data utilization and problem localization caused by the scattered analysis of multimodal data in existing technologies. This improves the scientific nature of operation and maintenance decisions and the efficiency of problem investigation, promoting the optimization and upgrading of digital platforms and the improvement of service quality.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a multimodal operation and maintenance data processing method provided in Embodiment 1 of the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of a multimodal operation and maintenance data processing device according to Embodiment 2 of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the multimodal operation and maintenance data processing method of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] Figure 1 This is a flowchart of a multimodal operation and maintenance data processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the operation and maintenance management of digital platforms. The method can be executed by a multimodal operation and maintenance data processing device, which can be implemented in hardware and / or software. This multimodal operation and maintenance data processing device can be configured in any electronic device with network communication capabilities. For example... Figure 1 As shown, the method includes:

[0028] S110. In response to each task request initiated by the front-end device, generate a task identifier for each task request. The task identifier is a globally unique user experience session identifier.

[0029] Here, a task request refers to an operation request issued by the front-end device that needs to be executed or recorded. A task identifier can be a globally unique user experience session identifier generated by the front-end SDK for each task request. This unique task identifier is carried in all front-end events, back-end requests, and recorded business logs triggered by each task request. A user experience session identifier can be a unique identifier that can be jointly identified by the front-end and back-end during a single task request interaction and persists throughout the entire task request execution lifecycle.

[0030] S120. Obtain task operation data of multiple modalities associated with the task identifier of each task request. The task operation data of each modality is collected from a data source.

[0031] Task operation data can refer to the traces of interactive actions and execution status records generated to complete a task request. Multiple modalities refer to various types of task operation data that, under the same task identifier, exhibit heterogeneous characteristics due to different sources, making direct interoperability within the same parsing framework. These data are typically collected and processed independently. Data sources can refer to systems, interfaces, log streams, or sensing devices that provide the raw records of task operation data. For example, data sources may include microservice clusters, database instances, middleware, and endpoint detection probes.

[0032] As an optional but not limited implementation, the task operation data of multiple modalities, including technical performance data, business flow data, and front-end user behavior data, adopt a unified user experience session identifier.

[0033] Specifically, technical performance data refers to backend operation and maintenance data used to reflect backend technical call relationships, running status, response efficiency, and execution stability during the response to and execution of task requests; business flow data refers to business-level operation and maintenance data recorded by the frontend regarding the execution status of key nodes in the business process, the occurrence process of business events, and the configuration of relevant business parameters; and frontend user behavior data refers to frontend operation and maintenance data that triggers various interactive operations, interactive operation trajectories, and related characteristics of the interactive process in frontend interaction scenarios. All task operation data across multiple modalities uses a unified user experience session identifier, ensuring that task operation data from multiple modalities belonging to the same task request can be effectively associated.

[0034] As an optional but not limited implementation, front-end user behavior data includes collecting all interactive operations performed on the front-end page through no-code or code-based tracking techniques to form a front-end interactive operation event sequence. Each event in the front-end interactive operation event sequence includes an event type, event target, timestamp, and user experience session identifier.

[0035] Technical performance data includes complete call chain information collected by transmitting user experience session identifiers between microservice calls through backend devices. The call chain information includes microservice name, microservice interface name, microservice call start and end time, microservice response status code, microservice response time, and user experience session identifier.

[0036] Business transaction data includes business events recorded by embedding business tracking points at key nodes in the business code. Business events include business event type, business parameters, timestamps corresponding to the business event, and user experience session identifiers.

[0037] Without altering core business processes, all interactive actions are captured on the front-end page using no-code or code-based tracking techniques. Each interaction is encapsulated into elements including event type, event target, timestamp, and user experience session identifier. These elements form a continuous sequence of front-end interactive events in chronological order. The event type categorizes the interactive actions occurring on the front-end page, distinguishing different types of front-end interactions. The event target refers to the specific front-end control object that the interactive action points to. Front-end user behavior data can instantly transform previously discrete front-end interactive behaviors into a traceable event sequence.

[0038] User experience session identifiers are passed through between microservice calls, and call chain information is automatically collected every time an interface or service is called. By concatenating the information collected from multiple calls in the order of the calls, a complete backend call chain for a single task request can be obtained, ensuring that the backend operation and maintenance information for the same task request can be continuously and completely saved in a distributed or heterogeneous environment.

[0039] Business event tracking points are embedded at key nodes in the business code to capture and record various business events occurring in the business process. These recorded events integrate event types, parameters, timestamps, and user experience session identifiers. Event types refer to the categorization of specific business behaviors or state changes occurring within the business process; parameters refer to operational information associated with the events that characterizes the specific conditions and / or results of business execution. This business flow data accurately captures core execution information of the business process, ensuring its completeness and relevance.

[0040] S130. Using the task identifier of each task request as the primary key, aggregate the task operation data of multiple modalities belonging to the same task identifier into the user interaction dialogue object corresponding to the task identifier of each task request.

[0041] The primary key refers to the identifier field used to uniquely correspond to a single task request and serve as the user interaction dialog object during the aggregation of task data across multiple modalities. The user interaction dialog object can refer to a context object created when the task identifier is generated, which can persist throughout the task's lifecycle and incrementally aggregate task data from multiple modalities.

[0042] Using the task identifier as the primary key, multimodal task operation data belonging to the same task identifier are aggregated by writing them into the user interaction dialogue object corresponding to the task identifier in temporal, spatial, or semantic order. This allows the various modal operation and maintenance data involved in the same task request to overcome the aggregation barriers caused by multi-source heterogeneity and achieve effective correlation. Each modal data retains its own characteristics within the user interaction dialogue object, but shares the same task identifier as the index entry point for the task request data.

[0043] User interaction dialogue objects can reflect complete operation and maintenance information of a single task request execution process, eliminating the need to repeatedly search different modalities of operation and maintenance information across databases, interfaces, and platforms. This improves the efficiency of operation and maintenance information query, enabling operation and maintenance analysis and problem localization to be completed within the same user interaction dialogue object. It can shorten the time for problem localization or fault repair and enhance the efficiency and accuracy of operation and maintenance decisions.

[0044] Optionally, the user interaction dialogue object corresponding to the task identifier of each task request includes at least the corresponding task identifier, user identity identifier, timestamp of the task request trigger time, task request entry, and task operation data of multiple modalities, wherein the task request entry refers to the interface unit or functional touchpoint on which the task request is triggered.

[0045] As an optional but not limited implementation, the method includes steps A1-A2:

[0046] Step A1: Load the data from the user interaction dialogue object through the data analysis engine, and run the frustration pattern recognition algorithm built into the data analysis engine to detect whether at least one of the preset frustration patterns is triggered during the execution of the task initiated by the front-end device. The preset frustration patterns include repeated attempt mode, search failure mode, and process interruption mode. Repeated attempt mode is the mode in which the same operation is triggered more than a preset number of times for the same functional element within a first preset time period. Search failure mode is the mode in which no search results are clicked or new operations are performed within a second preset time period for performing the search operation. Process interruption mode is the mode in which a process step is entered and the process step is exited directly without entering the next process step after the process step stays for a much longer than a third preset time period.

[0047] A data analysis engine can refer to a computational module that takes user interaction dialogue objects as input and has a built-in frustration pattern recognition algorithm. It can analyze and process the associated data within the user interaction dialogue objects, and then perform frustration pattern recognition. Preset frustration patterns can refer to a pre-defined set of various feature patterns that characterize states where task execution is hindered or goals are not achieved. A frustration pattern recognition algorithm can refer to an algorithm built into the data analysis engine used to detect whether preset frustration patterns occur during task execution.

[0048] The repeated trial mode is a mode in which the same operation is triggered more than a preset number of times on the same functional element within a first preset time period. The number of times the same functional element is triggered can be determined by statistically analyzing the interaction behavior event types in the front-end user behavior data of the user interaction dialogue object. For example, the submit button is triggered more than 3 times consecutively within 30 seconds.

[0049] The "no search results" mode is a mode in which no search results are clicked or any new operations are performed within a second preset time period after the search operation is executed. The mode of clicking any search results or performing any new operations can be determined by judging whether there is new call chain information aggregation in the technical performance data or new business event aggregation in the business flow data in the user interaction dialogue object.

[0050] The process interruption mode is a mode in which a process exits directly without entering the next process step after the time spent in a process step of a multi-step process is far longer than the third preset time. The process can be determined as a multi-step process based on the task request type and the preset multi-step process task set. The time spent in a process step can be determined based on the timestamp information of technical performance data or business flow data in the user interaction dialogue object.

[0051] By combining a data analysis engine with a built-in frustration pattern recognition algorithm, the system can automatically and accurately identify situations where task execution is hindered or goals are not achieved, without requiring manual intervention to detect task execution failures. Preset frustration patterns, including repeated attempts, fruitless searches, and process interruptions, cover typical frustration scenarios in functional operations, search behaviors, and process execution. Furthermore, new frustration patterns can be added based on actual business needs, demonstrating good scalability and adaptability to different business scenarios. Clearly defined pattern characteristics improve the accuracy and reliability of frustration pattern recognition, thereby enhancing the timeliness and specificity of problem localization.

[0052] Step A2: In response to triggering at least one frustration mode, retrieve the corresponding technical performance data and business flow data from the user interaction dialogue object through the task identifier of each task request for fault analysis.

[0053] If at least one preset frustration mode is detected during task execution, the technical performance data and business flow data corresponding to the same task request are retrieved from the user interaction dialogue object that aggregates multimodal data of the same task request, based on the task identifier corresponding to each task request.

[0054] For the retrieved technical performance data, a traversal operation is performed to extract backend operation and maintenance data from each call chain information, which characterizes the system's operating status, response efficiency, and execution stability. The extracted backend operation and maintenance data is then compared with pre-set thresholds. If any backend operation and maintenance data does not meet the corresponding preset threshold, the service or interface pointed to by this call chain information is identified as the cause of the frustration mode, and the backend operation and maintenance data that does not meet the preset threshold is identified as the specific cause of the frustration mode. Similarly, for the retrieved business transaction data, a traversal operation is performed to extract business parameters reflecting the execution status of key nodes from each business event information. The extracted business parameters are then compared with pre-set thresholds. If any business parameter does not meet the corresponding preset threshold, the business event type pointed to by this business event information is identified as the cause of the frustration mode, and the business parameter that does not meet the preset threshold is identified as the specific cause of the frustration mode.

[0055] Optionally, based on the identified cause of the frustration pattern and the specific cause of the frustration pattern, a large language model is invoked to generate a report that can be viewed and analyzed. The large oracle model can be any pre-trained open-source large oracle model.

[0056] For example, it was detected that the payment submit button in multiple user interaction dialogue objects associated with task identifiers was clicked more than 3 times, thus identifying a repeated attempt pattern. Retrieving the corresponding technical performance data and business transaction data, correlation analysis determined that, in the technical performance data, the microservice response time for the call chain information corresponding to the payment creation interface all exceeded 5000ms, and the microservice response status code was 504. The Big Oracle model was invoked to generate a report stating, "Users experienced a frustrating payment process, mainly due to slow response and timeout of the 'payment creation interface'."

[0057] Using task identifiers as the core correlation basis, it can quickly retrieve technical performance data and business flow data directly related to tasks with frustration patterns, ensuring the relevance and correlation of fault analysis data. It can trace from frustration patterns to underlying technical and business logic problems, thereby improving the accuracy and timeliness of fault location.

[0058] As an optional but not limited implementation, the method includes steps B1-B2:

[0059] Step B1: Based on the data in the user interaction dialogue objects associated with multiple task identifiers, calculate in real time whether the target business indicators meet the preset indicator conditions, and trigger the execution of business processing actions related to the target business indicators when the preset indicator conditions are met.

[0060] Target business metrics can refer to pre-defined evaluation indicators used to measure the effectiveness, efficiency, or quality of business operations. Pre-defined indicator conditions can refer to pre-defined judgment rules or threshold requirements for determining whether the target business metrics have met expected standards. Business processing actions can refer to operations or processes that are associated with the target business metrics and are proactively triggered when the pre-defined indicator conditions are met, and can be used to optimize the service quality of the system or digital platform.

[0061] The system continuously acquires user interaction dialogue objects associated with multiple task identifiers of the same task type using a sliding time window. Based on the acquired user interaction dialogue objects, it calculates the target business indicators in real time. By comparing the calculation results with preset indicator conditions, if the calculation results meet the preset indicator conditions, it automatically triggers the execution of the business processing action corresponding to the target business indicator. The size of the sliding event window can be preset according to the time required for task execution and the frequency of task request initiation.

[0062] By embedding real-time data streams into the business metric calculation process, subsequent actions can be automatically triggered immediately after business goals are achieved without waiting for offline reports. This eliminates the delays caused by manual monitoring and intervention, thereby optimizing the service quality of the system or digital platform and improving operational efficiency.

[0063] For example, the system continuously acquires user interaction dialogue objects associated with multiple task identifiers obtained from multiple users triggering order payment tasks within one hour; based on the business parameters contained in the business transaction data of the acquired multiple user interaction dialogue objects, it calculates the payment failure rate of province A using payment method B and the average response time of the payment method B interface. After comparing these with preset indicator conditions, it is determined that the payment failure rate of province A using payment method B is greater than the preset payment failure rate and the average response time of the payment method B interface is greater than the preset average response time. Therefore, it automatically displays a prompt box in real time on the front end to users in province A who are trying to use payment method B: "The current payment method B channel is busy. We recommend that you use payment method C to pay."

[0064] Step B2: Based on the data in the user interaction dialogue objects associated with multiple task identifiers, detect in real time whether the target user's interaction operation meets the preset interaction operation conditions, and trigger the execution of the processing actions related to the target user's interaction operation when the preset interaction operation conditions are met.

[0065] Target user interaction operations refer to the types of interaction operations that are pre-designated in the front-end event sequence and require focused monitoring. Pre-defined interaction operation conditions refer to a set of pre-set rules used to determine whether an interaction operation meets the trigger conditions for a processing action. Processing actions refer to operations or processes associated with target user interaction operations that are proactively triggered and executed when the pre-defined interaction operation conditions are met, and which can be used to optimize the service quality of the system or digital platform.

[0066] The system continuously acquires user interaction dialogue objects associated with multiple task identifiers of the same task type using a sliding time window. Based on the acquired user interaction dialogue objects, it performs real-time counting or timing judgment on the target user's interaction operations. Once the preset interaction operation conditions are met, the system automatically triggers the execution of the processing actions related to the target user's interaction operation. The size of the sliding event window can be preset according to the task execution time and the frequency of task request initiation.

[0067] By capturing key behavioral characteristics of user interactions, personalized responses or risk interventions can be provided without waiting for the session to end, enhancing proactive security measures and thereby optimizing the service quality of systems or digital platforms and improving operational efficiency.

[0068] For example, the system continuously acquires user interaction dialogue objects associated with multiple task identifiers obtained from multiple users triggering electricity price calculation tasks within one day; based on the acquired multiple user interaction dialogue objects, it determines that the electricity price calculator was clicked multiple times in the front-end event sequence, but no further calculation business logic was triggered afterward; therefore, the system automatically pushes a smart assistant to users who clicked the electricity price calculator multiple times but did not continue to trigger the calculation business logic and displays the prompt message "Do you need help interpreting the tiered electricity pricing policy?".

[0069] For example, when user Zhang San clicks the "Pay Now" button on the bill details page, the front-end SDK generates a task identifier UX-Session-ID: ZJ-2025-001 and initializes the user interaction dialogue object {UX-Session-ID: ZJ-2025-001, user_id: user_001, start_time: 2025-01-01T10:00:00Z, entrance: HomePage_BillCard, behavior_sequence: [ ], technical_trace: [ ], business_events:[ ]};

[0070] After clicking the "Pay Now" button, front-end user behavior data {event: click, target: pay_button, id: ZJ-2025-001} is collected and aggregated into the user interaction dialogue object, resulting in {UX-Session-ID: ZJ-2025-001, user_id: user_001, start_time: 2025-01-01T10:00:00Z, entrance:HomePage_BillCard, behavior_sequence: [{event: click, target: pay_button, id:ZJ-2025-001}], technical_trace: [ ], business_events: [ ]};

[0071] In response to the task request, the backend calls the payment interface. The task identifier is passed to the backend via the HTTP header. The backend payment service calls the third-party payment gateway and collects the call chain information {service: payment, api: / create, latency: 6000ms, status: 200, id: ZJ-2025-001}, which is then aggregated into the user interaction dialogue object to obtain {UX-Session-ID: ZJ-2025-001, user_id: user_001, start_time: 2025-01-01T10:00:00Z, entrance: HomePage_BillCard, behavior_sequence: [{event: click, target:pay_button, id: ZJ-2025-001}], technical_trace: [{service: payment, api: / create, latency: 6000ms, status: 200, id: ZJ-2025-001}], business_events:[ ]};

[0072] After the backend completes its task, it reports the business execution status to the frontend, collects the business event {event:payment_created, channel: D, id: ZJ-2025-001}, and aggregates it into the user interaction dialogue object, obtaining {UX-Session-ID: ZJ-2025-001, user_id: user_001, start_time: 2025-01-01T10:00:00Z, entrance: HomePage_BillCard, behavior_sequence: [{event: click, target:pay_button, id: ZJ-2025-001}], technical_trace: [{service: payment, api: / create, latency: 6000ms, status: 200, id: ZJ-2025-001}], business_events:[{event: payment_created, channel: D, id: ZJ-2025-001}]};

[0073] Due to network fluctuations, the payment creation process took 6 seconds. During this time, Zhang San, impatient while waiting, clicked the payment button twice more, thus collecting new front-end user behavior data and aggregating it into the user interaction dialogue object, resulting in {UX-Session-ID: ZJ-2025-001, user_id: user_001, start_time: 2025-01-01T10:00:00Z, entrance: HomePage_BillCard, behavior_sequence: [{event: click, target: pay_button, id: ZJ-2025-001},{event: click, target: pay_button, id: ZJ-2025-001},{event: click, target: pay_button, id: ZJ-2025-001}], technical_trace:[{service: payment, api: / create, latency: 6000ms, status: 200, id: ZJ-2025-001}], business_events: [{event: payment_created, channel: D, id: ZJ-2025-001}]};

[0074] The data analysis engine analyzes user interaction dialogue objects, identifies a "repeated trial pattern" in payment tasks, and determines the high latency of the payment creation service based on correlation analysis. It then calls the Big Prediction model to generate a report and, based on the report's prompts, immediately fixes the high latency issue in the payment creation service.

[0075] Meanwhile, by monitoring the surge in the average time spent on payment method D among user interaction dialogue objects corresponding to multiple task identifiers under a payment task within a time window, a prompt recommending the use of an alternative payment method is automatically pushed to all users who are trying to use payment method D.

[0076] The technical solution of this invention ensures the traceability and uniqueness of each task request by generating a globally unique user experience session identifier as the task identifier for each task request. Obtaining task operation data from multiple modalities based on the task identifier improves the targeting and completeness of operation and maintenance data collection. Aggregating multiple modal task operation data with the same task identifier as the primary key to the user interaction dialogue object corresponding to the task identifier of each task request enables the association and integration of operation and maintenance data from multiple modalities throughout the entire business request process, thereby providing coherent and complete data support for operation and maintenance analysis and problem localization. Based on the above technical solution, associating and aggregating task operation data from multiple modalities through task identifiers solves the problem of low efficiency in operation and maintenance data utilization and problem localization caused by the scattered analysis of multimodal data in existing technologies. This improves the scientific nature of operation and maintenance decisions and the efficiency of problem investigation, promoting the optimization and upgrading of digital platforms and the improvement of service quality.

[0077] Example 2

[0078] Figure 2 This is a schematic diagram of the structure of a multimodal operation and maintenance data processing device provided in Embodiment 2 of the present invention. Figure 2 As shown, the device includes:

[0079] The identifier generation module 210 is used to generate a task identifier for each task request in response to each task request initiated by the front-end device. The task identifier is a globally unique user experience session identifier.

[0080] The data acquisition module 220 is used to acquire task operation data of multiple modalities associated with the task identifier of each task request. The task operation data of each modality is collected from a data source.

[0081] The data aggregation module 230 is used to aggregate task operation data of multiple modalities belonging to the same task identifier to the user interaction dialogue object corresponding to the task identifier of each task request, using the task identifier of each task request as the primary key.

[0082] Based on the above embodiments, optionally, the technical performance data, business flow data, and front-end user behavior data included in the multiple modal task operation data adopt a unified user experience session identifier.

[0083] Based on the above embodiments, optionally, the front-end user behavior data includes collecting all interactive operations performed on the front-end page through no-code or code-based tracking technology to form a front-end interactive operation event sequence. Each event in the front-end interactive operation event sequence includes an event type, an event target, a timestamp, and a user experience session identifier.

[0084] The technical performance data includes complete call chain information collected by transmitting the user experience session identifier between microservice calls through backend devices. The call chain information includes microservice name, microservice interface name, microservice call start and end time, microservice response status code, microservice response time, and user experience session identifier.

[0085] The business flow data includes business events recorded by embedding business tracking points at key nodes in the business code. Each business event includes a business event type, business parameters, a timestamp corresponding to the business event, and a user experience session identifier.

[0086] Based on the above embodiments, optionally, the device includes:

[0087] The pattern recognition module is used to load data from the user interaction dialogue object through the data analysis engine and run the frustration pattern recognition algorithm built into the data analysis engine to detect whether at least one of the preset frustration modes is triggered during the execution of the task initiated by the front-end device. The preset frustration modes include repeated attempt mode, search failure mode, and process interruption mode. The repeated attempt mode is the mode in which the same operation is triggered more than a preset number of times for the same functional element within a first preset time period. The search failure mode is the mode in which no search results are clicked or new operations are performed within a second preset time period for performing the search operation. The process interruption mode is the mode in which a multi-step process is entered and the process step is stayed for a much longer time than a third preset time period, and then the process step is exited directly without entering the next process step.

[0088] The fault analysis module is used to retrieve corresponding technical performance data and business flow data from the user interaction dialogue object in response to the triggering of at least one frustration mode, and to perform fault analysis by using the task identifier of each task request.

[0089] Based on the above embodiments, optionally, the device includes:

[0090] The business processing action module is used to calculate in real time whether the target business indicator meets the preset indicator conditions based on the data in the user interaction dialogue objects associated with multiple task identifiers, and to trigger the execution of the business processing action related to the target business indicator when the preset indicator conditions are met.

[0091] The interaction processing module is used to detect in real time whether the target user's interaction operation meets the preset interaction operation conditions based on the data in the user interaction dialogue object associated with multiple task identifiers, and to trigger the execution of the processing action related to the target user's interaction operation when the preset interaction operation conditions are met.

[0092] The technical solution of this invention ensures the traceability and uniqueness of each task request by generating a globally unique user experience session identifier as the task identifier for each task request. Obtaining task operation data from multiple modalities based on the task identifier improves the targeting and completeness of operation and maintenance data collection. Aggregating multiple modal task operation data with the same task identifier as the primary key to the user interaction dialogue object corresponding to the task identifier of each task request enables the association and integration of operation and maintenance data from multiple modalities throughout the entire business request process, thereby providing coherent and complete data support for operation and maintenance analysis and problem localization. Based on the above technical solution, associating and aggregating task operation data from multiple modalities through task identifiers solves the problem of low efficiency in operation and maintenance data utilization and problem localization caused by the scattered analysis of multimodal data in existing technologies. This improves the scientific nature of operation and maintenance decisions and the efficiency of problem investigation, promoting the optimization and upgrading of digital platforms and the improvement of service quality.

[0093] The multimodal operation and maintenance data processing device provided in the embodiments of the present invention can execute the multimodal operation and maintenance data processing method provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the multimodal operation and maintenance data processing method. For detailed process, please refer to the relevant operations of the multimodal operation and maintenance data processing method in the foregoing embodiments.

[0094] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0095] Example 3

[0096] Figure 3 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0097] like Figure 3As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0098] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as multimodal operational data processing methods.

[0100] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0101] In some embodiments, the multimodal operation and maintenance data processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the multimodal operation and maintenance data processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the multimodal operation and maintenance data processing method by any other suitable means (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0106] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0107] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0108] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0109] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multimodal operation and maintenance data processing method, characterized in that, The method comprises: In response to each task request initiated by the front-end device, a task identification of each task request is generated, which is a globally unique user experience session identification; Obtain the task operation data of multiple modalities associated with the task identification of each task request, and each modal task operation data corresponds to the data collected from a data source; Taking the task identification of each task request as the primary key, the task operation data of multiple modalities belonging to the same task identification is aggregated into the user interaction dialogue object corresponding to the task identification of each task request.

2. The method of claim 1, wherein, The technical performance data, business flow data and front-end user behavior data included in the multiple modal task operation data adopt a unified user experience session identification.

3. The method of claim 2, wherein, The front-end user behavior data includes collecting all interactive operations performed on the front-end page through non-buried point or code buried point technology to form a front-end interactive operation event sequence, and each event in the front-end interactive operation event sequence includes event type, event target, timestamp and user experience session identification; The technical performance data includes complete call chain information collected by the backend device between microservice calls and transmitting the user experience session identification, and the call chain information includes microservice name, microservice interface name, microservice call start and end time, microservice response status code, microservice response time and user experience session identification; The business flow data includes business events recorded by implanting business buried points at key nodes of business code, and the business events include business event type, business parameter, business event corresponding timestamp and user experience session identification.

4. The method of claim 1, wherein, The method comprises: Load the data in the user interaction dialogue object through the data analysis engine, and run the built-in frustration pattern recognition algorithm in the data analysis engine to detect whether at least one of the preset frustration patterns is triggered in the task execution process initiated by the front-end device, the preset frustration patterns include repeated attempt pattern, search fruitless pattern and flow interruption pattern, the repeated attempt pattern is a pattern of triggering the same operation on the same functional element more than a preset number of times within a first preset time period, the search fruitless pattern is a pattern of not clicking any search results or performing new operations within a second preset time period of executing a search operation, and the flow interruption pattern is a pattern of entering a multi-step flow and staying at a flow step for a time far exceeding a third preset time period, and then directly exiting without entering the next flow step; In response to triggering at least one frustration pattern, the corresponding technical performance data and business flow data are retrieved from the user interaction dialogue object through the task identification of each task request for fault analysis.

5. The method of claim 1, wherein, The method comprises: According to the data in the user interaction dialogue object associated with multiple task identifications, whether the target business index meets the preset index condition is calculated in real time, and when the preset index condition is met, a business processing action related to the target business index is triggered to be executed; According to the data in the user interaction dialogue object associated with the task identifier, the target user interaction operation is detected in real time to determine whether the target user interaction operation meets a preset interaction operation condition, and a processing action related to the target user interaction operation is triggered when the preset interaction operation condition is met.

6. A multi-modal operations data processing apparatus, comprising: The device comprises: An identifier generation module is configured to generate a task identifier for each task request initiated by a front-end device, and the task identifier is a globally unique user experience session identifier; A data acquisition module is configured to acquire task operation data of multiple modalities associated with the task identifier of each task request, and each modality of the task operation data is collected from a data source; A data aggregation module is configured to aggregate the task operation data of multiple modalities belonging to the same task identifier into a user interaction dialogue object corresponding to the task identifier of each task request.

7. The apparatus of claim 6, wherein, The technical performance data, business flow data, and front-end user behavior data included in the multiple modalities of the task operation data adopt a unified user experience session identifier.

8. The apparatus of claim 6, wherein, The device comprises: A pattern recognition module is configured to load data in the user interaction dialogue object through a data analysis engine, and run a built-in frustration pattern recognition algorithm in the data analysis engine to detect whether at least one frustration pattern in a preset frustration pattern is triggered in a task execution process initiated by a front-end device, the preset frustration pattern includes a repeated attempt pattern, a search fruitless pattern, and a flow interruption pattern, the repeated attempt pattern is a pattern of triggering more than a preset number of the same operation on the same functional element within a first preset time period, the search fruitless pattern is a pattern of not clicking any search result or performing a new operation within a second preset time period of performing a search operation, and the flow interruption pattern is a pattern of entering a multi-step flow and staying at a flow step for a time period much longer than a third preset time period, and then directly exiting without entering the next flow step; A fault analysis module is configured to retrieve corresponding technical performance data and business flow data from the user interaction dialogue object through the task identifier of each task request in response to triggering at least one frustration pattern.

9. An electronic device, comprising: The electronic device comprises: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the multi-modal operation and maintenance data processing method of any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to perform the multi-modal operation and maintenance data processing method of any one of claims 1-5 when executed.

Images