Personalized operation guide intelligent generation method based on multi-modal behavior path chain

Abstract

The invention discloses a personalized operation guide intelligent generation method based on a multi-modal behavior path chain, and relates to the technical field of software, the method comprises the following steps: defining an interaction object, and setting an ID and a basic attribute of the interaction object; the user triggers an interaction object and uploads interaction record data; performing user path modeling by using the interaction record data, and inputting user path modeling data into an AI large model to analyze a user operation intention; and generating an operation guide by combining the interaction record data of the user, the user path modeling data and the user operation intention. According to the method, interaction event nodes are defined, user operation behaviors are recorded, identity information is associated, a user path chain database is constructed, an AI algorithm is adopted to analyze user path chain data to visualize the operation guide, and dynamic optimization of the operation guide is achieved. The technical problems that a traditional static guide depends on manual writing, when a product has a personalized page generation function, the user portrait cannot be dynamically adapted, and operation traceability visualization is insufficient are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software, in particular to a personalized operation guide intelligent generation method based on a multi-modal behavior path chain. BACKGROUND

[0002] With the expansion of business, a product will often integrate more and more functions or modules, which will lead to user-oriented pages having too many interactive points or operation processes being too long. In order to provide users with a better user experience, it is usually necessary to configure a guide or a tutorial for the product. The traditional operation guide relies on manual writing, and when the product has a personalized page generation function, it cannot dynamically adapt to the user portrait, and the conventional user behavior analysis system lacks the association of UI features and operation paths. SUMMARY

[0003] The present application aims to provide a personalized operation guide intelligent generation method based on a multi-modal behavior path chain, which solves the problem that the operation guide relies on manual writing in the prior art and cannot dynamically adapt to the user portrait when the product has a personalized page generation function.

[0004] The present application solves the above problems through the following technical solutions:

[0005] A personalized operation guide intelligent generation method based on a multi-modal behavior path chain, comprising:

[0006] Step S1, defining an interactive object and setting the ID and basic attributes of the interactive object;

[0007] Step S2, the user triggering the interactive object and uploading interactive record data;

[0008] Step S3, using the interactive record data to model the user path, inputting the user path modeling data into an AI large model to analyze the user operation intention;

[0009] Step S4, generating an operation guide in combination with the user's interactive record data, user path modeling data and user operation intention.

[0010] The present application defines an interactive event node, records user operation behavior and associates identity information, constructs a user path chain database, and uses an AI algorithm to analyze user path chain data to visualize an operation guide, thereby achieving dynamic optimization of the operation guide. The technical problems of the traditional static guide relying on manual writing, the inability to dynamically adapt to the user portrait when the product has a personalized page generation function, and the lack of operation traceability visualization are solved.

[0011] Further, the step S1 specifically comprises:

[0012] Set the ID and basic attributes of the interactive object of the user operation behavior, the basic attributes including interaction type, component type, interaction influence, interaction environment, whether it is a starting event of an event chain, whether it is an ending event of an event chain, and the UI container ID of the interactive object.

[0013] Further, the step S2 specifically includes:

[0014] Step S21, when the user accesses the product, start the interactive event chain record queue generated by the user-side system, the interactive event chain record queue is provided with a queue ID, the queue ID including an application name, an application version number, a user ID, and a time stamp; the queue maximum length and the event chain uploading opportunity can also be set, and the expiration time interval (not set to never expire);

[0015] Step S22, when the user interacts with the interactive object in the operation interface, record the interactive object triggered by the user in the interactive event chain record queue according to the triggering order, and record the ID of the interactive object;

[0016] Step S23, upload the user's interaction record to the server according to the pre-set uploading opportunity. The default uploading opportunity is after the product is started, because it can record the complete operation of the user to the maximum extent. Preferably, whether there is an interaction record queue saved after previous use is found in the cache after the product is started. If it exists, parse the queue ID to obtain the time stamp, convert it into time and compare it with the current time, if it exceeds the set expiration time interval, only delete it without uploading, if it does not exceed the expiration time interval, upload it, and after the uploading is successful, delete it. In addition to this, the uploading opportunity can also be set as page switching, when the user can complete a complete business interaction process through an interactive event chain in a page, the uploading opportunity of the record queue can be set as page switching.

[0017] Further, the step S3 of using the interaction record data to model the user path specifically includes:

[0018] Step S31, build a visual feature pool, the visual feature pool containing the visual features of the interactive objects of all versions of the product, and sub-data pools are created in the visual feature pool according to the application name, for subsequent positioning and searching according to the application name;

[0019] Step S32, obtain the visual features of the interactive objects from the visual feature pool in combination with the interaction record data.

[0020] Further, the step S32 specifically includes:

[0021] Step A1, run the application in the tool or virtual machine, the rendered interactive object UI after the application runs carries the encoded ID, find the interactive object UI entity using the ID of the interactive object UI, get the container ID of the interactive object UI set in its attributes, and find the container containing the interactive object using the obtained container ID;

[0022] Step A2, extract the interactive object and place it in a visual highlight container, replace the previous interactive object with the visual highlight container object, render the obtained container UI, automatically generate visual feature screenshots based on the display range of the container, upload the visual feature screenshots to a specified location and save the visual feature screenshot access address;

[0023] Step A3, extract the basic attributes of the interactive object, add the visual feature screenshot access address to the basic attributes, generate a data object using the basic attributes, use the product version number as the key key, and the data object as the value value, save the key and value as a key-value pair in the corresponding product interactive object group of the visual feature pool. In this way, an interactive object not only has an ID and basic attributes, but also has a screenshot that can intuitively express its visual features.

[0024] Further, the step S3 of inputting the user path modeling data into the AI large model to analyze the user operation intention specifically includes:

[0025] Step B1, parse the user interaction event chain queue ID to get the application name, application version number and user ID;

[0026] Step B2, find the corresponding sub-visual feature pool of the application using the obtained application name;

[0027] Step B3, find the corresponding visual feature group in the found sub-visual feature pool using the interactive object ID saved in the interactive event chain queue; confirm whether the visual feature group exists data, if the data exists, use the application version number to find the data object of the interactive object corresponding version, the data object contains the UI visual feature screenshot access address, if the group does not exist data or the group does not exist, return an empty object;

[0028] Step B4, repeat step B3 to query the basic attributes and visual feature screenshot access address of all interactive objects recorded in the interactive event chain queue in turn, if the user uploaded interactive record data carries the basic attributes, use it to replace the default attribute value, and then fill it in the position of the interactive object in the queue;

[0029] Step B5, analyze the data in the interactive event chain queue, filter complete interactive event chains by analyzing the start and end markers of each interactive object, and eliminate invalid interactive records;

[0030] Step B6, assemble the user's behavior description using the attribute values of the interactive object, analyze the user's operation intention in combination with the user behavior description paragraph.

[0031] Further, the assembling of the user's behavior description using the attribute values of the interactive object specifically comprises:

[0032] Step B61, concatenate the interactive environment, interactive type, component type, and interactive impact of the interactive object in order to obtain a behavior description sentence;

[0033] Step B62, repeat step B61 to obtain all behavior description sentences contained in the interactive event chain; associate the behavior description sentences with the interactive objects in the interactive event chain, and according to the behavior description sentences, eliminate the same interactive objects in the interactive event chain;

[0034] Step B63, add the step order in front of all the obtained behavior description sentences to assemble a user behavior description paragraph; combine the UI visual features screenshots of the interactive objects into a picture group according to the step order, and hand over to the AI large model for image reasoning, and analyze the user operation intention in combination with the user behavior description paragraph.

[0035] Further, the step S4 specifically comprises:

[0036] Step S41, find the original data for generating the operation guide: find the user label in the third-party personalized system using the user ID, save the user label, application name, application version number, user behavior description paragraph text, user operation intention text, and interactive event chain data as the original data for generating the operation guide;

[0037] Step S42, build an operation guide data pool to save the dynamically generated operation guide, build different sub-data pools according to different application names, and then group them according to version numbers;

[0038] Step S43, generate the operation guide using the original data of the operation guide and save it to the corresponding group of the operation guide data pool.

[0039] Further, the generating of the operation guide using the original data of the operation guide specifically comprises:

[0040] Step C1, find the corresponding sub-data pool using the application name, if not found, create a new sub-data pool; find whether the group exists using the application version number, if not exist, create a new group; find all operation guide data in the group, if no data exist, directly generate the guide; if the operation guide data exists in the group before generation, compare the similarity of the user operation intention to determine whether a new operation guide needs to be generated;

[0041] Step C2, summarize the operation intention text as a guide title using an AI large model, place the user operation intention as an operation guide abstract at the front, and then take out the user behavior description and UI feature screenshot address in the order of the interaction event chain for content filling, generating an html file placed on the file storage server; the access link of the operation guide file is associated with the original data of the generated operation guide and saved in the group;

[0042] Step C3, combine the user operation intention text with the new user operation intention text from an operation guide original data in the group and give it to the AI large model to analyze the similarity of the operation intention described by the two paragraphs, and analyze the similarity of the operation guide saved in the group according to this step;

[0043] Step C4, after completing the similarity analysis of the operation intention, find out whether there is an operation guide with too high similarity, if there is, then take out the user behavior description paragraph text respectively and combine it and give it to the AI large model to analyze whether the steps described by the two texts exist a containing relationship, if there is a containing relationship, then generate a new operation guide according to step C2, otherwise update the personalized label data and update the number of times of submitting the corresponding personalized label; if there is no operation guide with too high similarity, then generate a new operation guide according to step C2.

[0044] Further, it further includes step S6: constructing an operation guide management center for managing data in the operation guide data pool, and pushing operation guides to users according to user personalized labels.

[0045] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0046] (1) The present application defines the interaction event node, records the user operation behavior and associates the identity information, constructs the user path chain database; uses the AI algorithm to analyze the UI features and behavior semantics of the event nodes in the path chain, generates a visual operation guide; aggregates multiple user behavior data through path containing relationship, realizes dynamic optimization and classified recommendation of the guide. It solves the technical problems that the traditional static guide cannot adapt to the personalized needs of users, when the product has a personalized page generation function, it cannot dynamically adapt to the user portrait and the operation traceability visualization is insufficient, and can be applied to the internet operation scene.

[0047] (2) The present application can summarize multiple different operation guides into a specific operation guide with more behavior data by AI calculating the containing relationship of the user path. The operation guides with different labels are classified, the matched operation guide is recommended for new users, and the user can also search the operation guide through the label or behavior description.

[0048] (3) The application improves the generation efficiency of the operation guide, reduces the error rate of user operation, improves the satisfaction of users, and reduces the possibility of user complaints. The user operation steps have visual annotations and have the function of visualizing operation traceability. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of the present application;

[0050] Figure 2 is a flowchart of the present application for constructing a visual feature pool;

[0051] Figure 3 is a flowchart of the present application for collecting user interaction data;

[0052] Figure 4 is a flowchart of the present application for generating operation guide raw data by using an AI large model to analyze user operation intention;

[0053] Figure 5 is a flowchart of the present application for generating an operation guide;

[0054] Figure 6 is a flowchart of the present application for pushing an operation guide to a user according to a personalized label. DETAILED DESCRIPTION

[0055] The present application will be further described in detail below in conjunction with embodiments, but the embodiments of the present application are not limited thereto.

[0056] Example 1:

[0057] In conjunction with the accompanying Figure 1 , a personalized operation guide intelligent generation method based on a multi-modal behavior path chain includes:

[0058] Step S1, define an interactive object and set the ID and basic attributes of the interactive object:

[0059] Generally, the interactive events of Internet B-end systems and C-end applications are mainly composed of basic events triggered by user specific operation behaviors, such as mouse click events, keyboard input events, touch events, etc. The entity object related to user operation behavior refers to a basic interactive unit (such as a button, an input box, etc.) or a complex interactive structure (such as a container containing multiple basic interactive units, such as a single selection component, a multiple selection component, a paging component, etc.), and the entity object is also called an interactive object. The interactive object has a unique ID (action_id) and interactive attributes.

[0060] The basic attributes of the interactive object include: action type (action_type), component type (component_type), whether it is the start event of the event chain (only_start), whether it is the end event of the event chain (only_end), interaction effects (effects), interaction context (context), and the UI container ID (ui_container) of the interactive object. The setting of the basic attributes helps to adapt to the recording of the operation behavior of the user in various scenarios.

[0061] When the user accesses the product, the interactive event chain recording queue (action_chain) is started at the first time after the product is started, and the queue ID (action_chain_id) is set. The queue ID is generated by the user side system and is composed of the application name (app_name), the application version number (version), the user ID (user_id), and the time stamp (timestamp) to ensure uniqueness. In addition, the maximum length of the queue (action_chain_max_length) and the event chain upload timing (upload_timing) can be set, and the expiration time interval (expiration_nterval) is not set to expire forever.

[0062] Step S2, the user triggers the interactive object, and uploads the interactive record data:

[0063] As shown in Figure 3 When the user interacts with the interactive object (basic interactive unit or complex interactive structure) in the operation interface, the basic interactive unit or complex interactive structure triggered by the user is recorded in the interactive event chain recording queue according to the triggering order, and the ID of the interactive object is recorded. Generally, the interactive object has pre-set basic attributes. In most cases, simple interaction behavior can be described using pre-set basic attributes, and the user side only records the interactive object ID to reduce performance consumption. The user side can analyze the user behavior branch to reset the basic attributes and record them together with the interactive object ID.

[0064] The user's interaction record is uploaded to the server for analysis according to the preset upload timing. The default upload timing is after the product is started, because the complete operation of the user can be recorded to the maximum extent. After the product is started, whether there is a previously used and saved interaction event chain record queue is found in the cache. If it exists, the queue ID is parsed to obtain the timestamp, which is converted into time and compared with the current time. If the set expiration time interval is exceeded, only the record queue is deleted and not uploaded. If the expiration time interval is not exceeded, the record queue is uploaded. After the uploading is successful, the record queue is deleted. In addition to the above, the upload timing can also be set as page switching. When the user is in a page, a complete business interaction process can be completed through an interaction event chain. The upload timing of the record queue can be set as page switching.

[0065] Step S3, user path modeling is performed using the interaction record data, and the user operation intention is analyzed using an AI large model:

[0066] Step S31, before user path modeling, the visual features of the interaction object are first obtained, and a visual feature pool of the interaction object is created and maintained. All application interaction visual features are uniformly saved in the visual feature pool. All visual features of the interaction objects in the visual feature pool are created in a sub-data pool according to the product application name, so as to facilitate subsequent application name positioning and searching.

[0067] As shown in Figure 2 , the visual feature acquisition method of the interaction object is as follows:

[0068] Step S311: run the application in a tool or a virtual machine. The rendered interaction UI after the application is run carries an encoded ID. The ID of the interaction object is used to find the UI entity of the interaction object, and the interaction object UI container ID (ui_container) set in the attribute is obtained. Generally, the container of the interaction object is the parent or higher element of the interaction element. The container ID obtained is used to find the container (context_container) containing the interaction object.

[0069] Step S312: the position of the interaction object UI is located through step 1, the interaction object is extracted and placed in a visual highlight container (highlight_container), the visual highlight container object is used to replace the previous interaction object, the container object UI obtained in step S311 is rendered, a screenshot is automatically generated based on the display range of the container, and the screenshot is uploaded to a specified location to save the access address of the obtained picture.

[0070] Step S313: Group all the interactive objects according to the ID of the interactive object, which is convenient for subsequent quick search. Extract the basic attributes of the interactive object, add the screenshot address to the basic attributes, and generate a data object using the basic attributes. Take the product version number (version) as the key, and the data object as the value. Save the key and value as a key-value pair in the interactive object group of the corresponding product in the visual feature pool. By this step, an interactive object not only has an ID and basic attributes, but also has a screenshot that can intuitively express its visual features.

[0071] Generally, the visual features of the interactive object should be obtained on the service side. In addition, the visual features of the interactive objects in the visual feature pool should contain the visual features of the latest version of the product. The visual features of the interactive object do not need to be updated too frequently. Only the new visual features of the interactive object need to be added after each new version is released, which can reduce the cost of investment.

[0072] A visual feature maintenance center is established to integrate all the interactive objects of each version of the product and visually display them. If the automatically generated visual feature screenshot of a certain interactive object does not meet the requirements of the administrator, the administrator can manually modify the visual feature screenshot.

[0073] Step S32, after collecting the user's interaction record, the application uploads the data, uses the uploaded data to model the user path, and uses the AI large model to analyze the user operation intention. As shown in Figure 4 The processing steps are as follows:

[0074] Step S321: The interactive event chain queue ID carries the necessary information. First, parse the user interactive event chain queue ID (action_chain_id) to obtain the application name, application version number, and user ID.

[0075] Step S322: Use the obtained application name to find the corresponding sub-visual feature pool of the application, which can narrow the matching range and improve the search efficiency.

[0076] Step S323: Use the interactive object ID saved in the interactive event chain queue to find the corresponding visual feature group in the sub-visual feature pool found in step S322.

[0077] Step S324: Confirm whether there is data in the group found in step S323. If there is data, use the application version number to find the data object of the corresponding version of the interactive object (the data object of the interactive object in the visual feature pool contains the UI visual screenshot access address). If there is no data in the group or the group does not exist, return an empty object.

[0078] Step S325: According to the above steps S323, S324, the basic attributes and visual feature screenshot addresses of all the interactive objects recorded in the interactive event chain queue are queried in turn. If the user uploaded interactive record data carries basic attributes, the default attribute value is replaced with it (the application can judge the user's complex interactive behavior and record reasonable interactive description), and then filled into the position of the interactive object in the queue (convenient for subsequent processing). The processed data is then used to analyze the user behavior path.

[0079] Step S326: Analyze the data in the interactive event queue, analyze the start and end markers of each interactive object to filter complete interactive event chains, and eliminate invalid interactive records (interactive objects not included in complete interactive event chains are considered invalid records). Generally, a complete interactive event chain contains at least two interactive objects, and as the first interactive object has a start event marker (that is, only_start is true), the last interactive object has an end event marker (only_end is true). Interactive event chains can be nested, that is, a part of the interactive record in many interactive event chains can form a shorter interactive event chain. Generally, an interactive object can be both the end of an interactive event chain and the beginning of another interactive event chain. There is only a containing relationship between interactive event chains, and two mutually intersecting interactive event chains will be considered as a large interactive event chain containing a smaller interactive event chain.

[0080] Step S327: After obtaining the complete interactive event chain, the user's behavior description needs to be assembled using the attribute values of the interactive objects. Generally, part of the attribute values of the interactive objects (interaction type, component type, interaction impact, etc.) are set to Chinese (large language models can handle natural language, and using Chinese also facilitates the generation of subsequent Chinese operation guides). The interactive environment, interaction type, component type, and interaction impact are concatenated in order to obtain a behavior description sentence. Repeat this step to obtain all behavior description sentences contained in the interactive event chain, and associate the behavior description sentences with the interactive objects in the interactive event chain. According to the behavior description sentences, the same interactive objects in the interactive event chain are removed (the user repeatedly operates a certain interactive object to generate a large number of identical records, but this interactive behavior only needs to be performed once in the process, so it can be simplified and combined into one).

[0081] Step S328: Add the step sequence (e.g., first step, second step, etc., which helps the AI large model identify that this is an operation flow) in front of all the obtained behavior description sentences, and assemble them into a user behavior description paragraph (action_textarea, which can be analyzed by the large model at one time to ensure the integrity of the operation and improve the accuracy of the large model analysis). According to the step sequence, the UI visual features of the interactive object are combined into a picture group, which is given to the AI large model for image reasoning (according to the UI visual features of the user operation, the operation steps are traced back. The reasoning result is generally used to analyze the user behavior description paragraph for the large language model to analyze the user operation intention).

[0082] Step S4, generate operation guide combining user interaction record data, user path modeling data and user operation intention, as follows:

[0083] Step S41, find the original data for generating operation guide: use ID to find user's label in the individualization system (convenient for adding individualization label after generating operation guide). Save user label, application name, application version number, user behavior description paragraph text, user operation intention text, and interactive event chain data as original data for generating operation guide.

[0084] Step S42, as shown in Figure 5 , build an operation guide data pool to save dynamically generated operation guide, build different sub-data pools according to different application names, and then group them according to version number (to improve the efficiency of the system to find operation guide). Use the above obtained original data of operation guide to generate operation guide, as follows:

[0085] Step S421: use application name to find corresponding sub-data pool, if not found, create new sub-data pool. Use application version number to find if the group exists, if not, create new group. Find all operation guide data in the group, if no data exists, directly generate guide. If operation guide data exists in the group before generation, compare the similarity of user operation intention to determine whether to generate new operation guide (comparing the similarity of user operation intention can avoid generating repeated operation guide).

[0086] Step S422: summarize the operation intention text as a guide title using an AI large model, place the user operation intention as an operation guide abstract at the front, and then fill in the content with the user behavior description and UI feature screenshot address in the order of the interaction event chain to generate an html file on the file storage server. Save the access link of the operation guide file in the group after associating it with the original data of the generated operation guide (saving the original data of the generated operation guide helps subsequent similarity analysis and comparison to determine whether to generate a new operation guide before generating the operation guide).

[0087] Step S423: combine the user operation intention text from one operation guide original data in the group with the new user operation intention text and give it to the AI large model to analyze the similarity of the operation intention described by the two paragraphs. Follow this step to analyze the similarity of the operation guide saved in the group.

[0088] Step S424: after completing the similarity analysis of the operation intention, check if there is an operation guide with too high similarity. If there is, then take out the user behavior description paragraph text separately and combine it to give it to the AI large model to analyze whether the two texts described contain a relationship (there is a case where a longer operation process contains multiple short processes, in which case the description paragraphs of the operation process have a relatively obvious containing relationship). If there is a containing relationship, generate a new operation guide according to step S422, otherwise update the personalized label data and update the number of times the corresponding personalized label is submitted (two operation processes with extremely high similarity do not have an obvious containing relationship, which means that the operation processes are extremely likely to be the same. In this case, there is no need to generate a new operation guide, only the number of personalized label submission times needs to be updated). If there is no operation guide with too high similarity, a new operation guide is also generated according to step S422.

[0089] Preferably, it also includes:

[0090] Step S5, build an operation guide management center to manage the data in the operation guide data pool. The administrator can view, delete, audit and publish the operation guide on it, and the application on the user side can request to obtain the operation guide after it is published after being audited.

[0091] Step S6, push the operation guide to the user according to the personalized label;

[0092] For example Figure 6As shown, after the application on the user side is started, the operation guide is requested from the operation guide data pool with parameters such as user ID, application name, and application version number. First, the corresponding operation guide data list is searched according to the application name and version number, and the user's personalized label is queried using the user ID. Since the original data contains data with personalized labels when the operation guide is saved, the user's label is used for matching, and the operation guide with the same label and the most label submissions is given priority (the more times personalized labels are submitted, the more popular this operation process is in this label group, and the higher the total number of label submissions, the more frequent this operation process is, and the higher the accuracy will be when it is pushed to the user). For new users or users for whom personalized labels have not been found, the operation guide with the most total number of label submissions will be pushed first. After the user side obtains the return data of the request, it requests the operation guide link address to obtain the operation guide source file for rendering and viewing.

[0093] Example 2:

[0094] Taking the operation management backend of a certain product as an example, the implementation steps of the intelligent generation method of personalized operation guides based on multimodal behavior path chains are as follows:

[0095] Step1: In the process of building the operation management background, according to the interaction prototype design and define the user's interaction objects, and set the ID and basic attributes of the interaction objects. From the product entrance, such as a page menu entry on the side, set a unique ID (action_id), set the basic attributes (action_type: "click", component_type: "menu button", only_start: true, only_end: false, effects: "open a certain page", context: "under a certain management module", ui_container: "container_id"). For list operations in the page, such as the edit button, set the action_id and basic attributes (action_type: "click", component_type: "edit button", only_start: false, only_end: false, effects: "open a certain edit page", context: "in the operation column of a certain data", ui_container: "container_id"), for the form filling interaction object, also set the basic attributes (action_type: "click", component_type: "input box", only_start: false, only_end: false, effects: "input a certain content", context: "in the form", ui_container: "container_id"), for the save button basic attribute setting (action_type: "click", component_type: "save button", only_start: false, only_end: true, effects: "save a certain data", context: "in a certain page", ui_container: "container_id").

[0096] Step2: Use the tool to simulate the background operation, use the pre-set action_id to find the node node of the page, generate the visual feature screenshot of the interaction object UI, save the related data in the visual feature pool.

[0097] Step3: The user has interacted with the interaction object, and the id of the corresponding interaction object is recorded according to the user's interaction sequence, and then the user interaction record data is uploaded to the server for analysis.

[0098] Step4: Use the interaction record data to model, for example, a user edits data on a certain page and saves the submission process, the behavior path chain can be parsed as follows:

[0099] 1) Click the menu button under a certain management module to open a certain page.

[0100] 2) Click the edit button in the operation bar of a certain data to open the edit page.

[0101] 3) Click the input box in the form to input content.

[0102] 4) Click the selection box in the form to select the desired content.

[0103] 5) …

[0104] 6) Click the save button on a certain page to save data.

[0105] The description paragraph of the user operation is obtained.

[0106] Step 5: Send the description paragraph of the user operation and the related UI visual feature screenshot data to the AI large model for image reasoning. Image reasoning can use Ultralytics YOLO, and paragraph text analysis can call deepseek for analysis. After image reasoning and text analysis, the user operation intention can be obtained, such as editing a certain data.

[0107] Step 6: Use the data obtained in the above steps to generate an operation guide HTML file, save it on the server, and save the access link of the operation guide in the operation guide data pool.

[0108] Step 7: Another user accesses, matches the user's personalized label for recommendation, and preferentially recommends the operation guide with the same label and the most frequent occurrence.

[0109] Although the present application is described herein with reference to the explanatory embodiments of the present application, the above-described embodiments are only the preferred embodiments of the present application, and the embodiments of the present application are not limited by the above-described embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in the present application.

Claims

1. A method for intelligently generating personalized operation guides based on a multimodal behavior path chain, characterized in that: include: Step S1: define the interactive object and set the ID and basic attributes of the interactive object; Step S2: The user triggers the interaction object and uploads the interaction record data; Step S3: Use the interaction record data to model the user path, and input the user path modeling data into the AI ​​big model to analyze the user's operation intention; Step S4: Generate an operation guide based on the user's interaction record data, user path modeling data, and user operation intention.

2. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 1, characterized in that: The step S1 specifically includes: setting the ID and basic attributes of the interactive object of the user operation behavior, the basic attributes including interaction type, component type, interaction impact, interaction environment, whether it is an event chain start event, whether it is an event chain end event and the UI container ID of the interactive object.

3. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 2, characterized in that: The step S2 specifically includes: Step S21: When a user accesses a product, an interaction event chain record queue generated by the user-side system is opened. The interaction event chain record queue is set with a queue ID, which includes the application name, application version number, user ID, and timestamp; Step S22: When the user interacts with the interactive objects in the operation interface, the interactive objects triggered by the user are recorded in the interactive event chain record queue in the order of triggering, and the IDs of the interactive objects are recorded; Step S23: Upload the user's interaction record to the server according to the preset upload timing.

4. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 3, characterized in that: The user path modeling using the interaction record data in step S3 specifically includes: Step S31: Build a visual feature pool, which contains visual features of interactive objects of all versions of all products. In the visual feature pool, create sub-data pools according to application names. Step S32: Acquire visual features of the interactive object from the visual feature pool in combination with the interaction record data.

5. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 4, characterized in that: The step S32 specifically includes: Step A1: Run the application in a tool or virtual machine. The interactive object UI rendered after the application runs carries the encoded ID. Use the interactive object UI ID to find the interactive object UI entity, obtain the interactive object UI container ID set in its properties, and use the obtained container ID to find the container containing the interactive object. Step A2: Extract the interactive object and place it in a visually highlighted container. Use the visually highlighted container object to replace the previous interactive object. Render the UI of the obtained container. Automatically generate a visual feature screenshot based on the display range of the container. Upload the visual feature screenshot to a specified location and save the visual feature screenshot access address. Step A3: Extract the basic attributes of the interactive object, add the visual feature screenshot access address to the basic attributes, use the basic attributes to generate a data object, use the product version number as the key, and the data object as the value. Save the key and value as a key-value pair in the visual feature pool to group the interactive objects of the corresponding product.

6. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 5, characterized in that: In step S3, the user path modeling data is input into the AI ​​big model to analyze the user operation intention, which specifically includes: Step B1: Parse the user interaction event chain queue ID to obtain the application name, application version number, and user ID; Step B2: Use the obtained application name to find the sub-visual feature pool corresponding to the application; Step B3: Use the interactive object ID saved in the interactive event chain queue to find the corresponding visual feature group in the found sub-visual feature pool; confirm whether there is data in the visual feature group. If there is data, use the application version number to find the data object of the corresponding version of the interactive object. The data object contains the access address of the UI visual feature screenshot. If there is no data in the group or the group does not exist, return an empty object; Step B4: Repeat step B3 to retrieve the basic attributes and visual feature screenshot access addresses of all interactive objects recorded in the interactive event chain queue. If the interaction record data uploaded by the user contains basic attributes, replace the default attribute values ​​with them and then fill them in the position of the interactive object in the queue; Step B5: Analyze the data in the interaction event chain queue, filter the complete interaction event chain by analyzing the start and end identifiers of each interaction object, and eliminate invalid interaction records; Step B6: Assemble the user's behavior description using the attribute values ​​of the interactive objects, and analyze the user's operation intention in combination with the user's behavior description paragraphs.

7. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 6, characterized in that: The specific description of assembling the user's behavior using the attribute values ​​of the interactive objects is: Step B61: Concatenate the four attribute values ​​of the interactive object, namely, the interactive environment, interaction type, component type, and interaction impact, in order to obtain a behavior description statement; Step B62: Repeat step B61 to obtain all behavior description statements contained in the interaction event chain; associate the behavior description statements with the interaction objects in the interaction event chain, and eliminate the same interaction objects in the interaction event chain based on the behavior description statements; Step B63: Add a step sequence to the front of all the obtained behavior description statements to assemble them into a user behavior description paragraph; combine the UI visual feature screenshots of the interactive objects into an image group according to the step sequence, and submit them to the AI ​​large model for image reasoning. Combined with the user behavior description paragraph analysis, the user's operation intention is obtained.

8. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 7, characterized in that: The step S4 specifically includes: Step S41: Searching for raw data for generating the operation guide: Using the user ID, searching for the user tag in a third-party personalization system, and saving the user tag, application name, application version number, user behavior description paragraph text, user operation intention text, and interaction event chain data as the raw data for generating the operation guide; Step S42: construct an operation guide data pool to store the dynamically generated operation guides, construct different sub-data pools according to different application names, and then group them according to version numbers; Step S43: Generate an operation guide using the original data of the operation guide, and save it to the corresponding group in the operation guide data pool.

9. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 8, characterized in that: The generating of the operation guide using the original data of the operation guide specifically includes: Step C1: Use the application name to find the corresponding sub-data pool. If not found, create a new sub-data pool. Use the application version number to find out whether the group exists. If not, create a new group. Find all operation guide data in the group. If no data exists, directly generate the guide. If the operation guide data already exists in the group before generation, compare the similarity of the user's operation intention and determine whether to generate a new operation guide. Step C2: Use the AI ​​model to summarize the action intent text as the guide title, place the user action intent at the front of the guide summary, and then extract the user behavior description and UI feature screenshot URLs in the order of the interaction event chain to fill in the content. Generate an HTML file and store it on the file storage server. Associate the access link of the guide file with the original data used to generate the guide and save it in a group. Step C3: Extract the user operation intention text from the original data of an operation guide in the group, combine it with the new user operation intention text, and submit it to the AI ​​large model to analyze the similarity of the operation intentions described by the two paragraphs. Follow this step to perform similarity analysis with the operation guides saved in the group in turn; Step C4: After completing the similarity analysis of the operation intentions, check whether there are any operation guides with high similarity. If so, extract the user behavior description paragraph text combination separately and submit it to the AI ​​large model to analyze whether there is an inclusion relationship between the steps described in the two paragraphs. If there is an inclusion relationship, generate a new operation guide according to step C2. Otherwise, update the personalized label data and the number of corresponding personalized label submissions. If there is no operation guide with high similarity, generate a new operation guide according to step C2.

10. The method for intelligently generating personalized operation guides based on multimodal behavior path chains according to claim 9, characterized in that: The method further includes step S6: constructing an operation guide management center for managing the data in the operation guide data pool and pushing operation guides to users according to user tags.

Images