Business process mining method, apparatus, processor and electronic device

Abstract

The application discloses a kind of business process mining method, device, processor and electronic equipment. It is related to artificial intelligence field, wherein, the method comprises: obtaining the multi-modal data of business process, wherein, business process includes N links, N is the positive integer greater than 1, multi-modal data includes at least two of the following data types: business log, business audio data, business video data;According to multi-modal data, determine the N groups of link data corresponding to N links respectively, wherein, each group of data in N groups of link data is part of the data contained in multi-modal data;According to N groups of link data, determine the business behavior included in business process, and determine the behavior characteristics of business behavior respectively;According to behavior characteristics, determine the behavior to be optimized from business behavior. Through the present application, the technical problem that it is difficult to mine the problem behavior in the process in the process of the flow of business is solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a business process mining method, apparatus, processor, and electronic device. Background Technology

[0002] Currently, banks mainly rely on human experience to make judgments when conducting process analysis. After observing and analyzing business processes through interviews, records, and other methods, the business departments propose process optimization requirements.

[0003] Existing process mining methods mainly rely on manual work by business personnel. During the inquiry process, business personnel are prone to business deviations and omissions, making it difficult to find bottlenecks in process operation. Furthermore, cross-departmental and cross-business surveys are often a huge undertaking, requiring a lot of time and human resources.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a business process mining method, apparatus, processor, and electronic device to at least solve the technical problem of difficulty in mining problematic behaviors in a business process.

[0006] To achieve the above objectives, according to one aspect of the present invention, a business process mining method is provided, comprising: acquiring multimodal data of a business process, wherein the business process includes N stages, where N is a positive integer greater than 1, and the multimodal data includes at least two of the following data types: business logs, business audio data, and business video data; determining N sets of stage data corresponding to the N stages based on the multimodal data, wherein each set of stage data in the N sets of stage data is a portion of the data contained in the multimodal data; determining the business behaviors included in the business process based on the N sets of stage data, and determining the behavioral characteristics of each of the business behaviors; and determining behaviors to be optimized from the business behaviors based on the behavioral characteristics.

[0007] Optionally, determining the business behaviors included in the business process and the behavioral characteristics of each of the business behaviors based on the N sets of process data includes: inputting the N sets of process data into the feature embedding layer of the process mining model, outputting the business behaviors included in each of the N processes by the feature embedding layer, and outputting the behavioral characteristics of each of the business behaviors by the feature embedding layer, wherein the process mining model is a pre-trained neural network model, and the feature embedding layer is a network layer in the process mining model used for feature extraction.

[0008] Optionally, the step of inputting the N sets of process data into the embedding layer of the process mining model and outputting the behavioral features of each business behavior includes at least one of the following steps: inputting audio modal data from the N sets of process data into an audio embedding layer and outputting the audio features of the business behavior, wherein the feature embedding layer includes the audio embedding layer and the behavioral features include the audio features; inputting video modal data from the N sets of process data into a video embedding layer and outputting the video features of the business behavior, wherein the feature embedding layer includes the video embedding layer and the behavioral features include the video features; inputting text modal data from the N sets of process data into a text embedding layer and outputting the text features of the business behavior, wherein the feature embedding layer includes the text embedding layer and the behavioral features include the text features; inputting the N sets of process data into a factor embedding layer and outputting the factor features of the business-specific behaviors included in the business behavior, wherein the feature embedding layer includes the factor embedding layer and the behavioral features include the factor features.

[0009] Optionally, determining the behavior to be optimized from the business behaviors based on the behavioral features includes: inputting the audio features, the video features, the text features, and the factor features into the fully connected layer of the process mining model, and having the fully connected layer output the behavior to be optimized.

[0010] Optionally, inputting the audio modal data from the N sets of process data into the audio embedding layer includes: inputting the business audio data into a speech recognition system and outputting a speech recognition result; inputting the business audio data into a speech emotion classifier and outputting a speech emotion recognition result; inputting the speech recognition result and the speech emotion recognition result into the audio embedding layer, wherein the audio modal data includes the speech recognition result and the speech emotion recognition result.

[0011] Optionally, inputting the video modal data from the N sets of process data into the video embedding layer includes: taking screenshots of the business video data according to preset frame interval parameters to obtain video image data; inputting the video image data into an action recognition model to output action recognition results; and inputting the action recognition results into the video embedding layer, wherein the video modal data includes the action recognition results.

[0012] Optionally, inputting the text modal data from the N sets of process data into the text embedding layer includes: inputting the business log into a natural language processing model to obtain key field information; and inputting the key field information into the text embedding layer, wherein the text modal data includes the key field information.

[0013] To achieve the above objectives, according to another aspect of the present invention, a business process mining apparatus is also provided, comprising: an acquisition module, configured to acquire multimodal data of a business process, wherein the business process includes N stages, N being a positive integer greater than 1, and the multimodal data includes at least two of the following data types: business logs, business audio data, and business video data; a first determination module, configured to determine N sets of stage data corresponding to the N stages respectively, wherein each set of stage data is a portion of the data contained in the multimodal data; a second determination module, configured to determine the business behaviors included in the business process and determine the behavioral characteristics of each of the business behaviors based on the N sets of stage data; and a third determination module, configured to determine the behaviors to be optimized from the business behaviors based on the behavioral characteristics.

[0014] To achieve the above objectives, according to another aspect of the present invention, a processor is also provided, the processor being used to run a program, wherein the program executes any of the above-described business process mining methods during runtime.

[0015] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is executed, the device where the non-volatile storage medium is located is controlled to execute any of the above-described business process mining methods.

[0016] According to another aspect of the present invention, an electronic device is also provided, the electronic device including a memory and a processor, the memory being used to store a program, and the processor being used to run the program stored in the memory, wherein the program executes any of the above-described business process mining methods when it runs.

[0017] In this embodiment of the invention, multimodal data recording business processes is used. Based on the multimodal data, N sets of process data corresponding to N steps are determined, where each of the N sets of process data is a portion of the multimodal data. Based on the N sets of process data, the business behaviors included in the business process are determined, as well as the behavioral characteristics of each business behavior. Based on the behavioral characteristics, behaviors to be optimized are identified from the business behaviors. This achieves the goal of identifying potential behaviors to be optimized in the business process, thereby improving the speed of discovering behaviors to be optimized in the business process. This solves the technical problem of difficulty in discovering problematic behaviors in the business process during process mining. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a computer terminal for implementing a business process mining method is shown.

[0020] Figure 2 This is a flowchart illustrating the business process mining method provided in an embodiment of the present invention;

[0021] Figure 3 This is a structural block diagram of a banking business process mining system provided according to an optional embodiment of the present invention;

[0022] Figure 4 This is a flowchart illustrating the process of a data acquisition device for acquiring multimodal data according to an optional embodiment of the present invention.

[0023] Figure 5 This is a flowchart of the audio and video separation steps provided in an optional embodiment of the present invention;

[0024] Figure 6 This is a flowchart of an audio modal data processing method provided by an optional embodiment of the present invention;

[0025] Figure 7 This is a flowchart of a video modal data processing method provided by an optional embodiment of the present invention;

[0026] Figure 8 This is a flowchart of a text modal data processing method provided by an optional embodiment of the present invention;

[0027] Figure 9 This is a flowchart of the training process of a process mining model provided according to an optional embodiment of the present invention;

[0028] Figure 10 This is a structural block diagram of a business process mining device provided according to an embodiment of the present invention;

[0029] Figure 11 This is a structural block diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0033] Process mining is a technique that extracts knowledge from existing event log data to discover, monitor, and improve actual processes.

[0034] Multimodal means that each source or form of information can be considered a modality, such as text, images, audio, and video. The reason for modal fusion is that different modalities have different ways of being represented, so there are overlapping and complementary phenomena. By properly processing multimodal information, rich feature information can be obtained.

[0035] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0036] According to an embodiment of the present invention, a method embodiment for business process mining is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a business process mining method is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be implemented wholly or partially as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element in the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the business process mining method in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned business process mining method of the application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0040] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0041] The present invention will now be described in conjunction with preferred implementation steps. Figure 2 This is a flowchart illustrating the business process mining method provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the method includes the following steps:

[0042] Step S202: Obtain multimodal data of the business process, wherein the business process includes N steps, where N is a positive integer greater than 1, and the multimodal data includes at least two of the following data types: business logs, business audio data, and business video data.

[0043] It should be noted that the business process can refer to the processes of some business within the bank system. Its multimodal data can include the interaction process between users coming to the bank to conduct business and staff, which can be collected through the bank's system logs, audio recording devices, or on-site cameras. For example, when the business process is a personal loan process, it may include the following four steps: applying for a personal loan, branch pre-loan investigation, center receiving materials, and centralized data entry and scanning.

[0044] Step S204: Based on the multimodal data, determine N sets of process data corresponding to each of the N processes, wherein each set of process data is a portion of the data contained in the multimodal data.

[0045] Distinguishing multimodal data according to business processes can improve the accuracy of multimodal data utilization and avoid comparing multimodal data from different processes when mining problematic behaviors in business processes. This step can improve the accuracy of classifying business behaviors in business processes.

[0046] It should be noted that each set of data can include data from multiple modalities belonging to that stage. For example, for the personal loan application stage, since this stage requires users to go to the site in person to complete the process, the corresponding set of data for this stage can include business logs, audio data, and video data generated in this stage. However, for the centralized data entry and scanning stage, the processing in this stage is more automated, so the corresponding set of data for this stage can only include the business logs generated in this stage.

[0047] Step S206: Based on N sets of process data, determine the business behaviors included in the business process and the behavioral characteristics of each business behavior.

[0048] It should be noted that the business activities included in a business process can be divided into N stages, and N sets of stage data are used to determine the business activities included in each of the N stages. Furthermore, the behavioral characteristics of each business activity can be used to describe basic information such as the type, time, and duration of the business activity.

[0049] Step S208: Based on behavioral characteristics, identify behaviors to be optimized from business behaviors. These behaviors are problem behaviors in the business process, specifically those that cause inefficiencies, excessively long overall processing times, or dissatisfaction among bank users; they can also be referred to as bottlenecks in the business process.

[0050] Through the above steps, multimodal data recording business processes is used to determine N sets of process data corresponding to N steps. Based on the multimodal data, the business behaviors included in the business process and the behavioral characteristics of each business behavior are determined. Then, based on the behavioral characteristics, behaviors to be optimized are identified from the business behaviors. This achieves the goal of identifying behaviors that may be to be optimized in the business process, thereby improving the technical effect of quickly mining behaviors that need to be optimized in the business process. This solves the technical problem of difficulty in mining problematic behaviors in the business process.

[0051] As an optional implementation, in order to identify bottlenecks in business processes, the behavior to be optimized can be mined in the following way: using a process mining model to mine business behaviors and extract the behavioral characteristics of each business behavior, and using a process mining model to determine the behavior to be optimized.

[0052] Optionally, the behavioral characteristics of each business behavior can be determined in the following way: input N sets of process data into the feature embedding layer of the process mining model, output the business behaviors included in each of the N processes, and output the behavioral characteristics of each business behavior, wherein the process mining model is a pre-trained neural network model, and the feature embedding layer is a network layer in the process mining model used for feature extraction.

[0053] It should be noted that the feature embedding layer, also known as the feature embedding layer, is a network layer in a neural network model. It is used to obtain the embedded representation of features based on the input data. The embedded representation of features is the mathematical form of the behavioral features of the business behavior, and the behavioral features can be the embedding vector corresponding to the business behavior.

[0054] As an optional implementation, the behavior to be optimized is determined from business behaviors based on behavioral characteristics. This includes inputting audio features, video features, text features, and factor features into the fully connected layer of the process mining model, and having the fully connected layer output the behavior to be optimized. The fully connected layer acts as a classifier in the neural network, learning appropriate parameter weights through training. This allows the fully connected layer to effectively distinguish between normal and problematic behaviors from business behaviors. Problematic behaviors are those that hinder the smooth operation of the entire business process or those that negatively impact user satisfaction with the business process.

[0055] As an optional embodiment, inputting N sets of process data into the embedding layer of the process mining model and outputting behavioral features of each business behavior includes at least one of the following steps: inputting audio modal data from the N sets of process data into an audio embedding layer and outputting audio features of the business behavior, wherein the feature embedding layer includes an audio embedding layer and the behavioral features include audio features; inputting video modal data from the N sets of process data into a video embedding layer and outputting video features of the business behavior, wherein the feature embedding layer includes a video embedding layer and the behavioral features include video features; inputting text modal data from the N sets of process data into a text embedding layer and outputting text features of the business behavior, wherein the feature embedding layer includes a text embedding layer and the behavioral features include text features; inputting N sets of process data into a factor embedding layer and outputting factor features of the business-specific behaviors included in the business behavior, wherein the feature embedding layer includes a factor embedding layer and the behavioral features include factor features.

[0056] In this optional embodiment, by inputting process data of different modalities from N sets of process data into different network layers in the embedding layer, targeted processing of multimodal data can be achieved. It should be noted that the factor features extracted by the factor embedding layer can be characteristic factors defined by the bank according to its operational strategies and policy requirements. For example, some business processes are mandatory by policy, so even if the business behaviors in such business processes lead to a significant increase in business processing time, the process mining model will not identify any business behaviors in that process as bottlenecks in the business process.

[0057] Figure 3 This is a structural block diagram of a banking business process mining system provided according to an optional embodiment of the present invention, such as... Figure 3 As shown, the banking business process mining system may include the following components: data acquisition device 31, audio / video separation module 32, audio analysis module 33, video analysis module 34, log analysis module 35, model training module 36, and result output module 37. Several modules perform the following functions:

[0058] Data acquisition device 31 is a module used to retrieve workflow logs and related videos of business processing from the banking business processing system.

[0059] The audio / video separation module 32 is used to separate the audio and video of banking transactions obtained from the data acquisition device 31. Common audio / video separation software includes, but is not limited to, iMovie and Adobe Premiere.

[0060] The audio analysis module 33 is used to analyze the audio separated by the audio-video separation module 32.

[0061] The video analysis module 34 is used to analyze the video separated by the audio-video separation module 32.

[0062] The log analysis module 35 is used to analyze the event logs obtained from the data acquisition device 31.

[0063] Model training module 36 is used to build a process mining model based on audio analysis results, video analysis results, and image analysis results.

[0064] The result output module 37 is used to apply the model trained in the model training module 36 to obtain the process mining results.

[0065] For banking operations, the complexity of business processes varies depending on the type of business. Therefore, data collection must capture data from all stages of the process and all relevant business processing systems. Data collection device 31 can provide all the multimodal data for subsequent business process analysis. Figure 4 This is a flowchart illustrating the process of a data acquisition device for acquiring multimodal data according to an optional embodiment of the present invention, such as... Figure 4 As shown, the specific steps are as follows:

[0066] Step S401: Select a time interval, where the time interval is the interval corresponding to the continuous business process.

[0067] Step S402: Obtain the event logs within the specified time interval from the business processing system according to the time interval, that is, obtain the business logs corresponding to the business process. The business processing system can be an electronic system in the bank.

[0068] Step S403: Retrieve the business processing video within the time interval from the database. The database is used by the bank to store multimedia data collected by various sensors and multimedia devices. The business processing video records the dynamic video and sound of the business process.

[0069] Step S404: Store the obtained event logs and business processing videos in the specified directories for easy access during subsequent business process mining.

[0070] After saving the business processing video to the designated directory in step S404, the video can be further processed as follows: the images and audio in the video are separated to obtain two data modalities: business audio data and business video data. This facilitates subsequent operations such as feature extraction. It should be noted that the business video data can be a data modal that only includes image information from the video, excluding the audio information.

[0071] Figure 5 This is a flowchart of the audio and video separation steps provided by an optional embodiment of the present invention, such as... Figure 5 As shown, this step specifically includes:

[0072] Step S501: Set up the environment for using the audio / video separation software.

[0073] Step S502: Install audio / video separation software. Common audio / video separation software includes, but is not limited to, iMovie or Adobe Premiere.

[0074] Step S503: Using the audio and video separation software in step S502, perform audio and video separation operations on the business processing videos in the specified directory in S404.

[0075] Step S504: Save the separated business audio data and business video data to the specified directory in step S404 respectively, and delete the business processing videos in the specified directory. Then, the data stored in the specified directory are all multimodal data for business process mining.

[0076] Optionally, before inputting the multimodal data from the specified directory into the feature embedding layer, the three types of data—business logs, business audio data, and business video data—can be processed separately to help the feature embedding layer more effectively extract business behaviors and their corresponding behavioral features. The following optional embodiments provide a preprocessing implementation method for multimodal data.

[0077] As an optional embodiment, when inputting audio modal data from N sets of process data into the audio embedding layer, the following steps may be included: inputting business audio data into a speech recognition system and outputting speech recognition results; inputting business audio data into a speech emotion classifier and outputting speech emotion recognition results; inputting the speech recognition results and speech emotion recognition results into the audio embedding layer, wherein the audio modal data includes the speech recognition results and the speech emotion recognition results. Figure 6 This is a flowchart of an audio modal data processing method provided by an optional embodiment of the present invention. The above-described processing of service audio data may include the following steps:

[0078] Step S601: Read the business audio data in the specified directory in step S504.

[0079] Step S602: Using speaker separation technology, the speech of relevant personnel involved in the business audio data is separated according to the identity of the personnel to obtain audio data belonging to different personnel.

[0080] Step S603: Load the audio data belonging to different people into a preset speech recognition system (including but not limited to DeepSpeech or Auto Speech Recongnition Tool) for recognition and obtain the speech recognition results.

[0081] Step S604: Construct a classifier based on the audio features of the samples through machine learning (including but not limited to Gaussian Mixture Model (GMM) and Support Vector Machine (SVM)), and output the speech emotion recognition result by the classifier.

[0082] Step S605: Output audio analysis results to the audio embedding layer, wherein the audio analysis results include speech recognition results and speech emotion recognition results.

[0083] As an optional embodiment, inputting video modal data from N sets of process data into the video embedding layer includes: taking screenshots of the business video data according to preset frame interval parameters to obtain video image data; inputting the video image data into an action recognition model to output action recognition results; and inputting the action recognition results into the video embedding layer, wherein the video modal data includes the action recognition results. Figure 7 This is a flowchart of a video modal data processing method according to an optional embodiment of the present invention. The above-described processing of service video data may include the following steps:

[0084] Step S701: Read the business video data in the specified directory in step S404.

[0085] Step S702: Take screenshots of the video in the business video data according to the preset time parameters or frame interval parameters to obtain video image data.

[0086] Step S703: Using motion recognition technology, identify the visual operations performed by personnel involved in the handling of banking business processes (including but not limited to signing, filling in materials, and taking photos) and obtain motion recognition results.

[0087] Step S704: Output the action recognition result.

[0088] As an optional embodiment, inputting text modal data from N sets of process data into a text embedding layer includes: inputting business logs into a natural language processing model to obtain key field information; and inputting the key field information into the text embedding layer, wherein the text modal data includes key field information. Figure 8 This is a flowchart of a text modal data processing method according to an optional embodiment of the present invention. The processing of text modal data may include the following steps:

[0089] Step S801: Read the event log file obtained in S404.

[0090] Step S802: Extract key field information from the event log using natural language processing. Logs typically contain rich process information, including but not limited to time, applicant, approver, step name, status, and notifications. Text features can be extracted from this information, effectively helping us understand the data flow of a single business process.

[0091] Step S803: Output key field information.

[0092] Figure 9This is a flowchart illustrating the training process of a process mining model provided by an optional embodiment of the present invention. It should be noted that the process mining model in the above embodiment and optional embodiment can be trained in the following manner:

[0093] Step S901: Import training sample data. Obtain the audio analysis results, action recognition results, and key field information of the training samples, and establish the correlation between these results according to the "process-step-data" approach. Taking the "personal loan process (pre-loan)" involved in the training samples as an example, the data partitioning method is shown in Table 1:

[0094] Table 1. Personal Loan Process (Pre-Loan) Related Table

[0095]

[0096]

[0097] Step S902: Construct the original neural network model of the process mining model. The original neural network model of this scheme consists of four main embedding layers: audio embedding layer, video embedding layer, text embedding layer, and factor embedding layer. The following is a brief description of each of the four embedding layers.

[0098] The audio embedding layer is responsible for recording the audio data features of business processes, as shown in Table 2. Typically, the dialogue content in the audio corresponds to certain aspects of the banking business process. For example, the customer's answers to questions asked by the teller are usually recorded in the business processing system, creating a correlation with the system logs. Similarly, the customer's responses to instructions given by the teller also correspond to the video. Furthermore, audio contains rich emotional information, which can help the model identify subjective reasons for process delays, expanding the boundaries of process mining.

[0099] Table 2 Audio Feature Table

[0100]

[0101]

[0102] The video embedding layer is responsible for recording the video data features of business steps in the process, as shown in Table 3. There are correlations between actions in the video and the video itself. For example, if a teller asks a customer to fill out a form in the audio, then in the video, the customer will complete a series of actions: "receiving the form," "picking up a pen," "writing," and "returning the form." Therefore, for such a process, the speed of writing is crucial in determining the speed of this business step. By learning these correlations, the process mining model can better identify the key actions that determine the speed of completing a business step.

[0103] Table 3 Video Feature Table

[0104] Feature name Feature meaning Come Sample source Frame_check Number of faces in video Vedio_action The video involves actions Action_check Number of video actions Action_frequency Video motion frequency Action_time Action duration

[0105] The text embedding layer is responsible for recording the text data characteristics of business system logs involved in the process, as shown in Table 4. The business system logs themselves record the workflow within the system. For business steps with audio and video data, the business system logs show a clear correlation with the audio and video data in terms of time and operation. Through the mutual verification of the three, when faced with a highly disguised business transaction, it can effectively identify the steps in the business transaction that were not executed according to the standard process, and can even be broken down to the specific operational steps that were not performed according to the standard.

[0106] Table 4 Text Feature Table

[0107]

[0108]

[0109] The factor embedding layer consists of distinctive factors defined by the bank based on its operational strategies and relevant policies, as shown in Table 5. For example, if certain business processes are mandated by the policies of relevant institutions, the process mining model will not identify them as bottlenecks in the process, even if such processes significantly increase the time required to complete the business.

[0110] Table 5. Characteristic Table of Featured Factors

[0111] Feature name Feature meaning VIP_ornot Is the applicant a VIP? Urgent_service Is this an expedited service? Handling_priority Processing priority Government_policy Is this a requirement of the relevant authorities? ... ...

[0112] Step S903: Model training.

[0113] (1) Set the number of iterations for model training, the number of datasets, and the learning rate.

[0114] (2) Read the data in S901 and divide it into training set and test set according to a certain ratio.

[0115] (3) Load the original neural network model constructed in step S902 for training, and after completing a specified number of iterations, obtain the process mining model.

[0116] Step S904: Evaluate the model. If the model accuracy reaches the evaluation baseline preset by the business personnel, save the model file of the mining model for this process.

[0117] This invention overcomes the shortcomings of traditional process mining research, such as large scale, susceptibility to errors, and high resource consumption. It provides a method for mining banking business processes based on multimodal fusion. The purpose of this invention is to analyze logs, videos, and audio generated by the operation of banking business workflows, and to integrate technologies such as speech recognition, action recognition, and natural language processing to achieve the fusion of multiple modalities, including video, audio, and text. By verifying the information between modalities, problems in the process can be identified, reducing business deviations caused by human experience and improving the efficiency of banking process operation.

[0118] Specifically, the user information (including, but not limited to, user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this application are all information and data that have been authorized by the user or have been fully authorized.

[0119] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the business process mining method according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0121] According to embodiments of the present invention, a business process mining apparatus for implementing the above-described business process mining method is also provided. Figure 10 This is a structural block diagram of a business process mining device provided according to an embodiment of the present invention, such as... Figure 10 As shown, the business process mining device includes: an acquisition module 1002, a first determination module 1004, a second determination module 1006, and a third determination module 1008. The business process mining device will be described below.

[0122] The acquisition module 1002 is used to acquire multimodal data of a business process, wherein the business process includes N steps, where N is a positive integer greater than 1, and the multimodal data includes at least two of the following data types: business logs, business audio data, and business video data.

[0123] The first determining module 1004 is connected to the above-mentioned obtaining module 1002 and is used to determine N sets of process data corresponding to the N processes respectively based on the multimodal data, wherein each set of data in the N sets of process data is a portion of the data contained in the multimodal data;

[0124] The second determining module 1006 is connected to the first determining module 1004 and is used to determine the business behaviors included in the business process and the behavioral characteristics of each of the business behaviors based on the N sets of process data.

[0125] The third determining module 1008, connected to the second determining module 1006, is used to determine the behavior to be optimized from the business behavior based on the behavioral characteristics.

[0126] It should be noted that the acquisition module 1002, the first determination module 1004, the second determination module 1006, and the third determination module 1008 correspond to steps S202 to S208 in the embodiments. Multiple modules implement the same instances and application scenarios as their corresponding steps, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in the embodiments.

[0127] The business process mining device includes a processor and a memory. The aforementioned acquisition module 1002, first determination module 1004, second determination module 1006 and third determination module 1008 are all stored in the memory as program units. The processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0128] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can improve the speed at which optimization-needed behaviors are identified in the business process.

[0129] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0130] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned business process mining method.

[0131] This invention provides a processor for running a program, wherein the program executes the above-described business process mining method during runtime.

[0132] Figure 11 This is a structural block diagram of an electronic device provided according to an embodiment of the present invention, such as... Figure 11 As shown, this embodiment of the invention provides an electronic device 11. The device includes a processor, a memory, and a program stored in the memory and executable on the processor. The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the business process mining method and apparatus in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned business process mining method. The memory may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0133] When the processor executes the program, it performs the following steps: Acquire multimodal data of the business process, where the business process includes N stages, N being a positive integer greater than 1. The multimodal data includes at least two of the following data types: business logs, business audio data, and business video data. Based on the multimodal data, determine N sets of stage data corresponding to each of the N stages, where each set of stage data is a portion of the multimodal data. Based on the N sets of stage data, determine the business behaviors included in the business process and the behavioral characteristics of each business behavior. Based on the behavioral characteristics, determine the behaviors to be optimized from the business behaviors. The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0134] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: acquiring multimodal data of a business process, wherein the business process includes N stages, where N is a positive integer greater than 1, and the multimodal data includes at least two of the following data types: business logs, business audio data, and business video data; determining N sets of stage data corresponding to each of the N stages based on the multimodal data, wherein each set of stage data is a portion of the multimodal data; determining the business behaviors included in the business process and determining the behavioral characteristics of each business behavior based on the N sets of stage data; and determining the behaviors to be optimized from the business behaviors based on the behavioral characteristics.

[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0140] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0141] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0142] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0143] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A business process mining method characterized by, The method comprises: acquiring multi-modal data of a business process, wherein the business process comprises N links, N is a positive integer greater than 1, and the multi-modal data comprises at least two of the following data types: business logs, business audio data, and business video data; determining N sets of link data corresponding to the N links respectively according to the multi-modal data, wherein each set of data in the N sets of link data is part of the multi-modal data; determining business behaviors included in the business process and respective behavior features of the business behaviors according to the N sets of link data; determining a to-be-optimized behavior from the business behaviors according to the behavior features. The determination of the business behaviors included in the business process and the respective behavior features of the business behaviors according to the N sets of link data comprises: inputting the N sets of link data into a feature embedding layer of a process mining model, outputting the business behaviors included in the N links respectively by the feature embedding layer, and outputting the respective behavior features of the business behaviors by the feature embedding layer, wherein the process mining model is a pre-trained neural network model, and the feature embedding layer is a network layer in the process mining model for feature extraction.

2. The method of claim 1, wherein, The inputting of the audio modal data in the N sets of link data into the audio embedding layer comprises: inputting the business audio data into a speech recognition system to output a speech recognition result; inputting the business audio data into a speech emotion classifier to output a speech emotion recognition result; inputting the speech recognition result and the speech emotion recognition result into the audio embedding layer, wherein the audio modal data comprises the speech recognition result and the speech emotion recognition result. ​ ​ 3. The method of claim 2, wherein, ​ ​ 4. The method of claim 2, wherein, ​ ​ ​ ​ 5. The method of claim 2, wherein, The video modality data in the N groups of link data is input into a video embedding layer, including: According to the preset frame interval parameter, the service video data is screenshot to obtain video image data; The video image data is input into an action recognition model to output an action recognition result; The action recognition result is input into the video embedding layer, wherein the video modality data includes the action recognition result.

6. The method of claim 2, wherein, The text modality data in the N groups of link data is input into a text embedding layer, including: The service log is input into a natural language processing model to obtain key field information; The key field information is input into the text embedding layer, wherein the text modality data includes the key field information.

7. A business process mining apparatus characterized by comprising: Comprising: An acquisition module is configured to acquire multi-modal data of a business process, wherein the business process includes N links, N is a positive integer greater than 1, and the multi-modal data includes at least two of the following data types: service log, service audio data, and service video data; A first determination module is configured to determine N groups of link data corresponding to the N links according to the multi-modal data, wherein each group of data in the N groups of link data is part of the multi-modal data; A second determination module is configured to determine business behaviors included in the business process according to the N groups of link data, and determine respective behavior characteristics of the business behaviors; A third determination module is configured to determine a to-be-optimized behavior from the business behaviors according to the behavior characteristics. The second determination module is further configured to input the N groups of link data into a feature embedding layer of a process mining model, output business behaviors included in the N links by the feature embedding layer, and output respective behavior characteristics of the business behaviors by the feature embedding layer, wherein the process mining model is a pre-trained neural network model, and the feature embedding layer is a network layer in the process mining model for feature extraction.

8. A processor, comprising: The processor is configured to run a program, wherein the program performs the business process mining method of any one of claims 1 to 6 when running.

9. An electronic device, comprising: One or more processors and a memory are included, and the memory is configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the business process mining method of any one of claims 1 to 6.

Images