A large model-based information processing system

Abstract

This application provides an information processing system based on a large model, comprising: acquiring information to be processed; determining target operation information corresponding to a target operation object based on the information to be processed, wherein the target operation information is used to characterize the operation category of the target operation object.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically to an information processing system based on a large model. Background Technology

[0002] With the development of smart devices, more and more smart devices are entering people's daily lives. However, many of these smart devices are buttonless or have only a few buttons, and users typically operate them through virtual interfaces.

[0003] In existing technologies, users operate devices manually, which cannot meet people's needs for intelligent operation of smart devices. Summary of the Invention

[0004] This application provides an information processing system.

[0005] In a first aspect, this application provides a method comprising:

[0006] Obtain information to be processed;

[0007] Based on the information to be processed, determine the target operation information corresponding to the target operation object. The target operation information is used to characterize the operation category of the target operation object.

[0008] Secondly, this application also provides a system comprising:

[0009] The acquisition module is used to acquire information to be processed.

[0010] The determination module is used to determine the target operation information corresponding to the target operation object based on the information to be processed. The target operation information is used to characterize the operation category of the target operation object.

[0011] Thirdly, this application also provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in any of the methods.

[0012] Fourthly, this application also provides a computer-readable storage medium on which a computer program is stored, the computer program being executed by a processor to implement the steps in any of the methods. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of a scenario for the information processing system provided in the embodiments of this application;

[0015] Figure 2 This is a schematic flowchart of one embodiment of the information processing method in this application;

[0016] Figure 3 This is a schematic diagram of an embodiment of the information processing method in this application.

[0017] Figure 4 This is a schematic diagram of a functional module of the information processing system in an embodiment of this application;

[0018] Figure 5 This is a schematic diagram of the structure of the terminal device in the embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0021] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. Furthermore, it is understood that in the specific embodiments of this application, user information, user data, and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0022] To enable any person skilled in the art to implement and use this application, the following description is provided. In this description, details are set forth for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0023] This application provides an information processing method, system, device, and storage medium, which are described in detail below.

[0024] Please see Figure 1 , Figure 1 This is a schematic diagram of a scenario for an information processing system provided in an embodiment of this application. The information processing system may include a terminal device 100. For example... Figure 1 The terminal device 100 can obtain user instruction information to execute the information processing method in this application.

[0025] In this embodiment of the application, the terminal device 100 may include, but is not limited to, desktop computers, portable computers, network servers, handheld computers (Personal Digital Assistants, PDAs), tablet computers, wireless terminal devices, embedded devices, virtual reality devices, etc.

[0026] It should be noted that, Figure 1 The schematic diagram of the information processing system shown is merely an example. The information processing system and scenario described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of interactive systems and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0027] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the information processing method in this application. The information processing method may include the following steps 201-202:

[0028] 201. Obtain information to be processed.

[0029] In this embodiment, the information to be processed can be any type of instruction information, such as voice control instructions given by the user, or control instructions through specific body language, including a body posture, gesture, etc. This embodiment does not limit the information to be processed.

[0030] The acquisition method for different types of information to be processed can be selected according to the actual situation. For example, different input devices can be set, such as a microphone to receive the user's voice control commands, and a camera to acquire the user's body language control commands. The specific input devices can be set according to the actual situation, and this application embodiment does not limit the acquisition method.

[0031] 202. Based on the information to be processed, determine the target operation information corresponding to the target operation object. The target operation information is used to characterize the operation category of the target operation object.

[0032] In this embodiment, the information to be processed can be data of a single modality (such as text data, image data, audio data, or other interactive data), or it can be a combination of multiple different modalities (such as information that includes both text and image data, or information that includes text, images, and audio data). The target operation object can be a smart device. Different smart devices have different specific operation methods, and it can also be the interface to be operated on (such as a UI interface) of the smart device. The target operation information corresponding to the target operation object can be the corresponding interface interaction control instructions (including page turning, swiping, clicking, etc.), that is, information used to instruct the computer device on how to control the UI interface. The operation categories include page turning, swiping, clicking, etc., of the target operation information. When the user-issued information to be processed is obtained, taking voice information as an example, the semantic information in the information to be processed can be analyzed to determine the user's target intent. The method of analyzing semantic information can be determined in any way, and this embodiment does not limit it.

[0033] Preferably, assuming the user's intention is to open the music software and play music, the operating system can identify the smart devices connected to its network, thereby determining which smart devices are capable of playing music. If multiple smart devices capable of playing music exist on the network, the system selects the user's most frequently used smart device for playback, or selects the smart device closest to the user based on the location information provided by the user's voice. Specific embodiments of this application do not limit this approach.

[0034] Therefore, in this embodiment, the target operation information is the specific operation obtained by analyzing the information to be processed, such as music playback as described above. It should be noted that in this embodiment, a relevant recognition model can be set for the terminal device and trained so that the recognition model can determine the specific smart device that matches the user's needs from the target operation interface based on the semantic information in the user's information to be processed, and then execute the user's information to be processed. The specific training scheme can be obtained through any training method, and this embodiment does not limit it.

[0035] Furthermore, the target of the operation can also be a specific UI interface. Let's assume the user's intention is still to open a music app and play music. The operating system can then use a network connection to identify the UI interface of a smart device capable of playing music, and use image recognition to search for the same interactive area as the music app. Once the music app is found, it can be opened. Then, image recognition can be used to locate the play button area within the music app, and the user can interact with the play button to begin playing music.

[0036] At this point, the target operation information is the specific operation obtained by analyzing the information to be processed, such as music playback as described above. Based on this, the recognition model can not only identify smart devices capable of playing music, but also identify the interactive areas on the corresponding UI interface of a specific smart device. The specific training scheme can be obtained through any training method, and this application embodiment does not limit it.

[0037] The information processing method provided in this application can acquire user's pending information, determine the user's target intent based on the pending information, and determine the target operation information of the target operation object based on the user's target intent, so as to execute the user's instructions. This application eliminates the need for manual operation of the corresponding device by the user.

[0038] To better implement the embodiments of this application, in one embodiment, the target operation information corresponding to the target operation object is determined based on the information to be processed, including:

[0039] Based on the information to be processed, at least one target matching information is determined. The target matching information is used to characterize the matching relationship between the target operation object and each operation information. Based on each target matching information, the target operation information corresponding to the target operation object is determined.

[0040] The above embodiments provide a method for UI interface recognition, enabling the terminal device to identify interactive areas in the UI interface that match the user's needs in real time based on the user's pending information. To improve the accuracy of the terminal device in matching the user's pending information with functional areas, this application also provides an implementation method.

[0041] Preferably, the model can learn the functions of each functional area in the target user interface. For example, the model can interact with each functional area in the target user interface according to instructions or in a random manner to obtain the corresponding operation results for each functional area, thereby enabling the model to learn the real functions of each functional area. Specifically, the model can be trained to recognize the icons or button shapes of each functional area in the target user interface, enabling the model to identify each functional area. Then, the model interacts with each identified functional area to learn the execution results after the interaction. Therefore, in this embodiment, the target matching information can be understood as the simulated execution results generated by the model when learning the functions of each functional area.

[0042] It should be noted that in reality, there may be some similar functional areas in the UI interface. For example, a user may download two music playback apps. Therefore, during the learning process, the simulated interaction results for the two music apps will be similar. Furthermore, if the user's pending information is to play song A, the target operation interface will determine, based on historical simulation results, that the simulated operation results for both music apps match the user's pending information, thus obtaining two target matching information. Since the model has learned which specific functional area each model simulation result is based on, the model may deduce at least one target matching information. Finally, based on the user's pending information, the model selects one target matching information from at least one target matching information to determine the final operation, i.e., which music app to use for music playback, such as by randomly selecting one; this embodiment does not limit this method.

[0043] In practice, music apps don't contain all searchable songs. For example, due to copyright issues, some songs may only exist in the libraries of certain music apps. Therefore, the model can search within music apps based on the user's requested information to see if the specified song exists. If it does, it plays it. If not, it searches in another music app until a song matching the user's command is found and played. Furthermore, music apps typically have a search bar with the word "search," allowing the model to locate the specific search bar using text-based retrieval, thus achieving the purpose of music search.

[0044] Of course, this application embodiment only uses a music app as an example, and other methods are also included. For example, a user wants to set a memo, such as: Please remind me to send a gift to so-and-so on XX month XX day. In this case, the model can interact with the corresponding functional area of ​​the calendar app on the smart device based on the user's pending information, thereby opening the calendar app and setting a reminder alarm or text message to send a gift to so-and-so within the user-specified timeframe. In this case, the functional area of ​​the calendar app is the target operation object. In summary, this application embodiment does not limit the specific pending information of the user, nor does it limit the specific execution process.

[0045] To better implement the embodiments of this application, in one embodiment of this application, at least one target matching information is determined based on the information to be processed, including:

[0046] Feature extraction is performed on the information to be processed to obtain feature information; the feature information is compared with each piece of information to be matched to obtain the similarity information corresponding to each piece of information to be matched; based on the similarity information, the target matching information is determined from each piece of information to be matched.

[0047] This application provides a scheme for determining target matching information by comparing user-defined information to be processed with simulation results. Specifically, whenever the model interacts and learns based on various functional areas in the UI, the execution results are vectorized. Then, the acquired user-defined information to be processed is also vectorized. Finally, when determining target matching information that matches the information to be processed, the similarity between the two vectors is calculated.

[0048] The similarity between the two vectors can be calculated using methods such as cosine similarity or Euclidean distance, and this application embodiment does not limit the method. Furthermore, the target similarity threshold in this application embodiment can also be set according to actual conditions, such as 95% or 90%, and this application embodiment does not limit the threshold. When the similarity between two vectors exceeds the target similarity threshold, the corresponding simulation result information is determined as the target matching information.

[0049] Preferably, when receiving the user's pending information, the text conversion function can be used to convert the voice information into task text T, and the k closest simulation result information S={S_1,S_2,…,S_k} can be retrieved from the APP learning memory M through text vectorization and vector retrieval.

[0050] To better implement the embodiments of this application, in one embodiment, the target operation information corresponding to the target operation object is determined based on the target matching information, including:

[0051] Based on the object information corresponding to the target operation object and the matching information of each target, determine the associated operation object information that matches the information to be processed; based on the information to be processed and the associated operation object information, determine the target operation information corresponding to the target operation object.

[0052] In this embodiment, the target operation object can be a UI interface, and the target matching information can be the execution information of the simulation results based on different interactive areas in various UI interfaces. Therefore, to improve the accuracy of execution, in this embodiment, when identifying the target operation object, it can be matched with the target matching information based on the object information of the target operation object itself. The object information of the target operation object can be the position, color, style, outline, font, or description information on buttons of various interactive areas in the current UI interface. The model can then convert this information into text information, such as HTML text information. The effect is that the model has a better recognition effect for text information. At this point, the text information can be associated with various target matching information to determine the best target matching information, which is the associated operation object information.

[0053] Preferably, five types of HTML tags can be used (the element categories and attributes of the UI interface are shown in Table 1), namely... <button>、 <checkbox> 、 <scroller>, <input> and These represent elements that can be clicked, checked, swiped, edited, and viewed in any other way. Each element category contains the following attributes: ID (the order in which the element appears in the GUI tree), label (descriptive content describing the element's function), onclick (a hint about the UI state that will be accessed by clicking this button or checking / unchecking this checkbox), text (the text on the element), direction (the scrolling direction, including up / down / left / right), checked (whether the checkbox is checked), and value (the text entered into the text box).

[0054] Information from scrollable sections is crucial for decision-making, especially when the target button is located on a scrollable portion of the screen that is not yet visible. Therefore, to provide comprehensive information, components of the scrollable portion of the screen need to be included in the current UI state. To achieve this, all scrollable components are automatically scrolled first, and information about visible UI elements is recorded. This information is then provided to the model for decision-making. This approach has two advantages. First, it prevents the model from making blind choices when it cannot see all the information on the screen. Second, it eliminates the need for the model to provide scrolling instructions, reducing the frequency of model calls and lowering related computational overhead.

[0055] category property Actions / Events <button> < / button> ID, label, onclick, text click <checkbox> < / checkbox> ID, checked, label, onclick, text check / uncheck <scroller> < / scroller> ID, scroll direction, label, text scroll <direction> < / direction> <input> ID, label, text, value input <text> < / text> ID, label, text

[0056] Table 1

[0057] Furthermore, since the information to be processed includes specific semantic information, the model can determine how to operate the current UI interface based on the information to be processed and the associated operation object information, thereby obtaining the corresponding target operation information. Therefore, this method can improve the model's recognition accuracy.

[0058] To better implement the embodiments of this application, in one embodiment, the target operation information corresponding to the target operation object is determined based on the information to be processed and the associated operation object information, including:

[0059] Based on the information to be processed and the information of the associated operation object, generate the first request information; input the first request information into the target processing model, and output the target operation information corresponding to the target operation object through the target processing model.

[0060] The above embodiments provide a scheme for determining target operation information based on a model. However, to further improve the accuracy of model execution, this application also provides a scheme for generating request information.

[0061] Specifically, in this embodiment, the first request information may include prompt information. This prompt information is used to prompt the model to perform relevant target operations. For example, the attribute information of each interactive area of ​​the current UI interface, such as appearance and position, can be vectorized to form simplified HTML text, denoted as UI_current, and the UI operation history queue is denoted as History. UI_current records the vectors corresponding to the attribute information of each functional area. Simultaneously, the UI operation history queue records the execution status of each functional area; if execution is completed, a completion flag is set, thus achieving the effect of executing all target matching information. In practical application scenarios, completing a user's task only requires a few UI elements. Therefore, the most relevant UI information is selectively included in the prompt. Through mature text vectorization and vector retrieval technologies, the k most similar simulated tasks in the simulation results can be found. The text of the simulated task is added to the prompt word generator, and a task plan is generated through a large language model. After privacy filtering, it is sent to the large language model to generate specific operation steps. Finally, the GUI task is completed after confirmation by the task executor and the user. The task executor contains APIs for performing specific operations. Specific UI operations are completed by calling the APIs, such as... Figure 3 As shown.

[0062] The current prompt word is generated using the PromptGenerator, which can be referenced in formula (1):

[0063] Prompt=PromptGenerator(T, UI_current, History)......(1)

[0064] In formula (1), T is the information to be processed, UI_current is the vector corresponding to the HTLM text attribute information of the current UI interface, History is the execution status information in the UI operation history queue described above, and Prompt is the generated prompt feature information.

[0065] After obtaining the prompt feature information, the model can perform operations based on the prompt feature information, that is:

[0066] Action = LLM(Prompt)……(2)

[0067] In formula (2), Action is the execution action information, and Prompt is the prompt feature information in formula (1) above.

[0068] Next, the current action is sent to the task executor to update the UI. If a dangerous operation (such as deletion) is encountered, user confirmation is required; for example, if partial file deletion is required, the user should be prompted to confirm. If the execution is complete, the current action is inserted into the history queue; then, the model is queried to see if the task is completed, and the task status flag Tfinis is updated. If the task is completed, Tfinis = True.

[0069] To better implement the embodiments of this application, in one embodiment, the associated operation object information matching the information to be processed is determined based on the object information corresponding to the target operation object and the target matching information, including:

[0070] Based on the target matching information, the first operation object information is determined; based on the first operation object information, the second operation object information is determined; based on the object information corresponding to the target operation object, the second operation object information is filtered to obtain the first associated operation object information that matches the information to be processed.

[0071] As can be seen from the above embodiments, since target matching information can include multiple items, this application also provides a specific execution scheme.

[0072] Specifically, in the above embodiments, the target matching information includes information related to the simulation results. Therefore, the simulation UIs required by each simulation result can be summarized to obtain the first operation object information, denoted as... Next, after determining the required UI, we can identify the corresponding UI call function or element, and obtain the second operation object information, denoted as...

[0073] Then, iterate through Guide.UI, and check each UI element in turn. current If it appears in Guide.UI, update the UI. current The `.Element.hint` parameter is set to `Guide.UI.Element.Function`, thus completing the filtering process. In other words, the purpose of these steps is to first determine which specific styles and types of UI are required by similar simulated tasks in executing the information to be processed, and then compare each UI element in the current UI with the UI of the simulated tasks to determine which UI elements in the simulated tasks are similar to the current UI. This determines which specific UI elements in the current interface are needed when executing the information to be processed. Based on the method of the embodiments of this application, the accuracy of the model in executing user information to be processed can be greatly improved.

[0074] To better implement the embodiments of this application, in one embodiment of this application, after determining the target operation information corresponding to the target operation object based on the information to be processed, the method further includes:

[0075] Execute the target operation information to obtain the first execution result information; verify the first execution result information to determine whether the execution of the target operation information has been completed.

[0076] In the above embodiments, specific target operation information can be determined through a model. Therefore, execution can be performed according to the target operation information. After execution, a corresponding execution result, i.e., the first execution result information, will be generated. For example, music plays successfully, or the cooling mode is turned on successfully. However, to further ensure successful execution, a corresponding prompt box can be generated after execution, displaying whether the execution was correct. If the user selects correct execution, the verification is completed; otherwise, the verification fails. Of course, the embodiments of this application may also include other verification methods, such as model self-checking, etc., and the specific embodiments of this application are not limited thereto.

[0077] To better implement the embodiments of this application, in one embodiment, verifying the first execution result information and determining whether the execution target operation information has been completed includes:

[0078] If the first execution result information meets the first condition, then the execution target operation information is determined to be completed; if the first execution result information does not meet the first condition, then the execution target operation information is determined to be incomplete.

[0079] The above embodiments provide a scheme for verification through user feedback. However, this method increases the user's operational costs. Therefore, this application also provides an automatic detection scheme.

[0080] For example, in practice, multiple target matching information may exist. Therefore, when filtering and executing based on multiple target matching information, restrictions are needed to avoid unlimited execution. Specifically, when filtering UI based on target matching information, filtering can be done sequentially from highest to lowest similarity. In other words, if after multiple filtering attempts, a suitable UI still cannot be found, and the similarity of subsequent target matching information begins to decrease, then there is no need to continue filtering. Therefore, this approach also avoids an overly lengthy execution process that leads to low efficiency.

[0081] Therefore, in this embodiment of the application, during each trial execution according to the target matching information, the execution count is recorded once after each acquisition of the target result information. Simultaneously, the total execution count is recorded, such as step. current =step current +1. When, step current If, after exceeding the threshold number of attempts, no UI matching the user's pending information is found, execution stops. Alternatively, in step... current Only when the number of attempts has not exceeded the threshold and the corresponding UI has been found will the verification pass.

[0082] In other words, if the execution result of the pending information is not the target result, determine to add one more execution count to obtain the total number of executions; if the total number of executions is less than the target number threshold, then continue to determine the first functional area that matches the pending information based on the functional attribute information corresponding to each second functional area; execute the pending information based on the execution action information and the second matching first functional area; if the execution result of the pending information is not the target result, continue to execute the step of determining to add one more execution count to obtain the total number of executions until the execution result of the pending information is the target result or the total number of executions is greater than or equal to the target number threshold; then determine the second matching first functional area as the first functional area.

[0083] To better implement the embodiments of this application, in one embodiment of this application, if the first execution result information does not meet the first condition, after determining that the target operation information has not been completed, the following steps are included:

[0084] The second operation object information is further filtered based on the object information corresponding to the target operation object to obtain the second associated operation object information that matches the information to be processed; the second request information is generated based on the second associated operation object information and the information to be processed; the target operation information is executed based on the second request information to obtain the second execution result information.

[0085] The above embodiments provide a scheme for confirming operation information based on request information. However, there is a possibility that the verification might fail after execution based on the execution result information. Therefore, to avoid errors during execution, simulation results corresponding to target matching information with low similarity can be selected and executed again using the above implementation method to obtain a second execution result information. This allows for further verification of whether the execution was completed, maximizing the execution of various possible schemes and avoiding omissions.

[0086] Based on this, after executing the target operation information based on the second request information and obtaining the second execution result information, the method further includes:

[0087] Verify the second execution result information until an execution result information that satisfies the first condition is obtained; if an execution result information that satisfies the first condition is obtained, determine that the execution process of the pending information is complete.

[0088] After obtaining the second execution result information, verification can continue. If the second execution result information also fails to be verified, the operation is repeated until a solution that has been successfully verified is obtained, or if a solution that has been successfully verified is never obtained, the execution process ends, and the user is informed of the execution failure.

[0089] To better implement the embodiments of this application, in one embodiment of this application, before obtaining the information to be processed, the method further includes:

[0090] Identify each functional area in the target operation interface; determine the attribute information of each functional area; execute each functional area and obtain the execution result information of each functional area; record the execution result information and corresponding attribute information of each functional area to obtain target memory information, which is used to determine the first functional area in the target operation interface that matches the information to be processed.

[0091] This application also provides a scheme for how a model learns the execution results corresponding to each functional area.

[0092] Preferably, the model can search for application UI relationships by randomly clicking buttons on the target operation interface on the screen and recording the results in the UI transition graph memory. Next, the model can traverse all UI elements in the UI transition graph and summarize the tasks they can perform. This mainly involves establishing connections between various possible user operations and GUI interface response actions and generating an LLM (Application Memory Model) of the app, constituting the model's memory specific to the target operation interface.

[0093] To better implement the information processing method in the embodiments of this application, an information processing system is also provided in the embodiments of this application, such as... Figure 4 As shown, system 300 includes:

[0094] The acquisition module 301 is used to acquire information to be processed;

[0095] The determination module 302 is used to determine the target operation information corresponding to the target operation object based on the information to be processed. The target operation information is used to characterize the operation category of the target operation object.

[0096] Optionally, the determining module 302 determines the target operation information corresponding to the target operation object based on the information to be processed, including:

[0097] Based on the information to be processed, at least one target matching information is determined. The target matching information is used to characterize the matching relationship between the target operation object and each operation information.

[0098] Based on the target matching information, determine the target operation information corresponding to the target operation object;

[0099] Optionally, the determining module 302 determines at least one target matching information based on the information to be processed, including:

[0100] The information to be processed is subjected to feature extraction to obtain feature information;

[0101] The feature information is compared with each piece of information to be matched to obtain the similarity information corresponding to each piece of information to be matched.

[0102] Based on the similarity information, the target matching information is determined from each piece of information to be matched.

[0103] Optionally, the determining module 302 determines the target operation information corresponding to the target operation object based on the target matching information, including:

[0104] Based on the object information corresponding to the target operation object and the matching information of each target, determine the associated operation object information that matches the information to be processed;

[0105] Based on the information to be processed and the information of the associated operation objects, the target operation information corresponding to the target operation object is determined;

[0106] Optionally, the determining module 302 determines the target operation information corresponding to the target operation object based on the information to be processed and the associated operation object information, including:

[0107] Based on the information to be processed and the information of the associated operation object, generate the first request information;

[0108] The first request information is input into the target processing model, and the target processing model outputs the target operation information corresponding to the target operation object.

[0109] Optionally, the determining module 302 determines the associated operation object information that matches the information to be processed based on the object information corresponding to the target operation object and the target matching information, including:

[0110] Based on the matching information of each target, the information of the first operation object is determined;

[0111] Based on the information of the first operation object, the information of the second operation object is determined;

[0112] The second operation object information is filtered and processed based on the object information corresponding to the target operation object to obtain the first associated operation object information that matches the information to be processed.

[0113] Optionally, after determining the target operation information corresponding to the target operation object based on the information to be processed, the determining module 302 further includes:

[0114] Execute the target operation information and obtain the first execution result information;

[0115] Verify the first execution result information to determine whether the target operation information has been executed.

[0116] Optionally, module 302 verifies the first execution result information to determine whether the target operation information has been completed, including:

[0117] If the first execution result information meets the first condition, then the execution target operation information is determined to be completed;

[0118] If the first execution result information does not meet the first condition, then it is determined that the target operation information has not been executed.

[0119] Optionally, after determining that the first execution result information does not meet the first condition, the module 302 further includes:

[0120] The second operation object information is further filtered based on the object information corresponding to the target operation object to obtain the second associated operation object information that matches the information to be processed.

[0121] Based on the second associated operation object information and the information to be processed, generate the second request information;

[0122] Based on the second request information, the target operation information is executed to obtain the second execution result information;

[0123] Optionally, after determining the second execution result information based on the second request information, the determining module 302 further includes:

[0124] Verify the second execution result information until the execution result information that satisfies the first condition is obtained;

[0125] If the execution result information that meets the first condition is obtained, the execution flow for the pending information is determined.

[0126] This application also provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of any of the information processing methods in this application. This terminal device integrates any of the information processing methods provided in this application, such as... Figure 5 As shown, it illustrates a structural schematic diagram of the terminal device involved in the embodiments of this application. Specifically:

[0127] The terminal device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 5 The terminal device structure shown does not constitute a limitation on the terminal device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0128] The processor 401 is the control center of the terminal device. It connects various parts of the terminal device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the terminal device. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and application programs, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into the processor 401.

[0129] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0130] The terminal device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0131] The terminal device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0132] Although not shown, the terminal device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the terminal device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, such as:

[0133] Obtain information to be processed;

[0134] Based on the information to be processed, determine the target operation information corresponding to the target operation object. The target operation information is used to characterize the operation category of the target operation object.

[0135] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the information processing methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0137] Obtain information to be processed;

[0138] Based on the information to be processed, determine the target operation information corresponding to the target operation object. The target operation information is used to characterize the operation category of the target operation object.

[0139] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0140] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0141] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0142] The above provides a detailed description of an information processing method and system provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.< / scroller> < / checkbox> < / button>

Claims

1. A method, characterized in that, The method includes: Obtain information to be processed; Based on the information to be processed, target operation information corresponding to the target operation object is determined, and the target operation information is used to characterize the operation category of the target operation object.

2. The method according to claim 1, characterized in that, The step of determining the target operation information corresponding to the target operation object based on the information to be processed includes: Based on the information to be processed, at least one target matching information is determined, and the target matching information is used to characterize the matching relationship between the target operation object and each operation information; Based on the target matching information, the target operation information corresponding to the target operation object is determined.

3. The method according to claim 2, characterized in that, The step of determining at least one target matching information based on the information to be processed includes: The information to be processed is subjected to feature extraction to obtain feature information; The feature information is compared with each piece of information to be matched to obtain the similarity information corresponding to each piece of information to be matched. Based on the similarity information, target matching information is determined from the information to be matched.

4. The method according to claim 2, characterized in that, The step of determining the target operation information corresponding to the target operation object based on each of the target matching information includes: Based on the object information corresponding to the target operation object and each target matching information, the associated operation object information that matches the information to be processed is determined; Based on the information to be processed and the associated operation object information, the target operation information corresponding to the target operation object is determined.

5. The method according to claim 4, characterized in that, The step of determining the target operation information corresponding to the target operation object based on the information to be processed and the associated operation object information includes: Based on the information to be processed and the associated operation object information, a first request information is generated; The first request information is input into the target processing model, and the target processing model outputs the target operation information corresponding to the target operation object.

6. The method according to claim 5, characterized in that, The step of determining the associated operation object information that matches the information to be processed based on the object information corresponding to the target operation object and each of the target matching information includes: Based on the target matching information, the information of the first operation object is determined; Based on the first operation object information, the second operation object information is determined; The second operation object information is filtered based on the object information corresponding to the target operation object to obtain the first associated operation object information that matches the information to be processed.

7. The method according to claim 6, characterized in that, After determining the target operation information corresponding to the target operation object based on the information to be processed, the method further includes: Execute the target operation information to obtain the first execution result information; Verify the first execution result information to determine whether the target operation information has been executed.

8. The method according to claim 7, characterized in that, The step of verifying the first execution result information and determining whether the target operation information has been executed successfully includes: If the first execution result information satisfies the first condition, then it is determined that the target operation information has been executed. If the first execution result information does not meet the first condition, then it is determined that the execution of the target operation information has not been completed.

9. The method according to claim 8, characterized in that, If the first execution result information does not meet the first condition, and it is determined that the execution of the target operation information has not been completed, the method further includes: Based on the object information corresponding to the target operation object, the second operation object information is filtered again to obtain the second associated operation object information that matches the information to be processed. Based on the second associated operation object information and the information to be processed, a second request information is generated; Based on the second request information, the target operation information is executed to obtain the second execution result information.

10. The method according to claim 9, characterized in that, After executing the target operation information based on the second request information to obtain the second execution result information, the method further includes: Verify the second execution result information until an execution result information that satisfies the first condition is obtained; If the execution result information that satisfies the first condition is obtained, the execution flow of the information to be processed is determined to be completed.

11. A system, characterized in that, The system includes: The acquisition module is used to acquire information to be processed. The determination module is used to determine the target operation information corresponding to the target operation object based on the information to be processed, wherein the target operation information is used to characterize the operation category of the target operation object; Optionally, the determining module determines the target operation information corresponding to the target operation object based on the information to be processed, including: Based on the information to be processed, at least one target matching information is determined, and the target matching information is used to characterize the matching relationship between the target operation object and each operation information; Based on the target matching information, determine the target operation information corresponding to the target operation object; Optionally, the determining module determines at least one target matching information based on the information to be processed, including: The information to be processed is subjected to feature extraction to obtain feature information; The feature information is compared with each piece of information to be matched to obtain the similarity information corresponding to each piece of information to be matched. Based on the similarity information, target matching information is determined from the information to be matched. Optionally, the determining module determines the target operation information corresponding to the target operation object based on each of the target matching information, including: Based on the object information corresponding to the target operation object and each target matching information, the associated operation object information that matches the information to be processed is determined; Based on the information to be processed and the associated operation object information, the target operation information corresponding to the target operation object is determined; Optionally, the determining module determines the target operation information corresponding to the target operation object based on the information to be processed and the associated operation object information, including: Based on the information to be processed and the associated operation object information, a first request information is generated; The first request information is input into the target processing model, and the target processing model outputs the target operation information corresponding to the target operation object. Optionally, the determining module determines the associated operation object information that matches the information to be processed based on the object information corresponding to the target operation object and each of the target matching information, including: Based on the target matching information, the information of the first operation object is determined; Based on the first operation object information, the second operation object information is determined; The second operation object information is filtered based on the object information corresponding to the target operation object to obtain the first associated operation object information that matches the information to be processed; Optionally, after determining the target operation information corresponding to the target operation object based on the information to be processed, the determining module further includes: Execute the target operation information to obtain the first execution result information; Verify the first execution result information to determine whether the target operation information has been executed successfully; Optionally, the determining module verifies the first execution result information and determines whether the target operation information has been executed, including: If the first execution result information satisfies the first condition, then it is determined that the target operation information has been executed. If the first execution result information does not meet the first condition, then it is determined that the execution of the target operation information has not been completed; Optionally, if the determining module determines that the target operation information has not been completed if the first execution result information does not meet the first condition, it further includes: Based on the object information corresponding to the target operation object, the second operation object information is filtered again to obtain the second associated operation object information that matches the information to be processed. Based on the second associated operation object information and the information to be processed, a second request information is generated; Based on the second request information, the target operation information is executed to obtain the second execution result information; Optionally, after the determining module executes the target operation information based on the second request information to obtain the second execution result information, it further includes: Verify the second execution result information until an execution result information that satisfies the first condition is obtained; If the execution result information that satisfies the first condition is obtained, the execution flow of the information to be processed is determined to be completed.

12. A terminal device, characterized in that, The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to perform the steps of the method according to any one of claims 1 to 10.

Images