An agent, a processing method, device and equipment of a project

Abstract

The embodiment of the specification discloses an intelligent agent, a processing method, device and equipment of a project, which comprises the following steps: receiving project requirement information input by a user for a target project; performing intention analysis processing on the target project based on the project requirement information, and determining logical description information of the target project; converting the logical description information of the target project into an execution script corresponding to the project requirement information through a preset domain-specific language based on the logical description information of the target project, and performing compilation processing on the execution script corresponding to the project requirement information to obtain a corresponding device control instruction sequence; based on the device control instruction sequence, sending corresponding operation commands to terminal equipment corresponding to the target project in sequence, and obtaining a result of the terminal equipment executing the operation commands corresponding to the device control instruction sequence, and determining an execution result of the target project based on the obtained result of executing the operation commands.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to an intelligent agent, a method for processing projects, an apparatus, and a device. Background Technology

[0002] Currently, many automated execution and processing mechanisms encounter numerous practical difficulties and pain points when deployed in mobile applications. For example, the practical difficulties and pain points encountered by automated inspection and RPA technology when deployed in mobile applications are that, in the usual system, device control program sequences can be built through drag-and-drop and low-code process orchestration. Specifically, visual component modules are pre-set, and users rely on dragging and dropping lines with the mouse and manually entering various attributes (such as element ID, timeout time, coordinates, etc.) in the sidebar to build a complete device control program sequence.

[0003] However, the above methods are highly dependent on the proficiency of personnel. When faced with complex tasks that require dozens of steps, the node diagram becomes very chaotic, the learning cost for newcomers is too high, and the response to sudden changes in requirements is extremely slow. Therefore, there is a need to provide a project automation processing method with a better interaction and generation mode, so as to achieve a higher component generation accuracy, reduce resource consumption and costs, and improve the overall process construction efficiency. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a project automation processing method with a better interaction and generation mode, thereby achieving a higher component generation accuracy, reducing resource consumption and costs, and improving the overall process construction efficiency.

[0005] To achieve the above technical solution, the embodiments in this specification are implemented as follows: This specification provides a method for processing a project, comprising: receiving project requirement information for a target project input by a user; performing intent parsing processing on the target project based on the project requirement information to determine logical description information for executing the target project; converting the logical description information for executing the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and compiling the execution script to obtain a corresponding device control instruction sequence; sequentially sending corresponding operation commands to a terminal device corresponding to the target project based on the device control instruction sequence, obtaining the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determining the execution result of the target project based on the obtained result of executing the operation commands.

[0006] This specification provides an intelligent agent, comprising an interaction layer, an intelligent agent processing layer, a DSL and compilation layer, an execution and device control layer, and an observation and data closed-loop layer. The interaction layer is configured to receive project requirement information for a target project input by a user, and to display the processing results of each layer within the intelligent agent. The intelligent agent processing layer is configured to perform intent parsing processing on the target project based on the project requirement information to determine the logical description information for executing the target project. The DSL and compilation layer is configured to convert the logical description information for executing the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and to compile the execution script to obtain a corresponding device control instruction sequence. The execution and device control layer is configured to send corresponding operation commands sequentially to the terminal device corresponding to the target project based on the device control instruction sequence. The observation and data closed-loop layer is configured to obtain the result of the terminal device executing the operation command corresponding to the device control instruction sequence, and determine the execution result of the target project based on the obtained result of the execution operation command.

[0007] This specification provides a project processing apparatus, comprising: a requirement receiving module for receiving project requirement information for a target project input by a user; an intent parsing module for performing intent parsing processing on the target project based on the project requirement information to determine logical description information for executing the target project; a conversion and compilation module for converting the logical description information for executing the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and compiling the execution script to obtain a corresponding device control instruction sequence; and a result determination module for sequentially sending corresponding operation commands to a terminal device corresponding to the target project based on the device control instruction sequence, obtaining the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determining the execution result of the target project based on the obtained result of the execution operation commands.

[0008] This specification provides an embodiment of a project processing device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein when the executable instructions are executed, the processor: receives project requirement information for a target project input by a user; performs intent parsing processing on the target project based on the project requirement information to determine logical description information for executing the target project; converts the logical description information for executing the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and compiles the execution script corresponding to the project requirement information to obtain a corresponding device control instruction sequence; and, based on the device control instruction sequence, sequentially sends corresponding operation commands to a terminal device corresponding to the target project, obtains the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determines the execution result of the target project based on the obtained result of executing the operation commands.

[0009] This specification also provides a storage medium for storing computer-executable instructions. When executed by a processor, the executable instructions implement the following process: receiving project requirement information for a target project input by a user; performing intent parsing processing on the target project based on the project requirement information to determine logical description information for executing the target project; converting the logical description information for executing the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and compiling the execution script to obtain a corresponding device control instruction sequence; sending corresponding operation commands to a terminal device corresponding to the target project sequentially based on the device control instruction sequence, obtaining the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determining the execution result of the target project based on the obtained result of executing the operation commands.

[0010] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the following process: receiving project requirement information for a target project input by a user; based on the project requirement information, performing intent parsing processing on the execution of the target project to determine logical description information for the execution of the target project; based on the logical description information for the execution of the target project, converting the logical description information for the execution of the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and compiling the execution script corresponding to the project requirement information to obtain a corresponding device control instruction sequence; based on the device control instruction sequence, sequentially sending corresponding operation commands to a terminal device corresponding to the target project, obtaining the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determining the execution result of the target project based on the obtained result of the execution operation commands. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the structure of a processing system for one project in this specification; Figure 2 This is a schematic diagram illustrating the processing procedure of one project in this specification; Figure 3 This is a schematic diagram illustrating the processing procedure for another item in this specification; Figure 4 This is a schematic diagram illustrating the processing procedure for yet another item in this specification; Figure 5 This is a schematic diagram illustrating the processing procedure for yet another item in this specification; Figure 6 This is a schematic diagram illustrating the processing procedure for yet another item in this specification; Figure 7 This is a schematic diagram of the structure of an intelligent agent as described in this specification; Figure 8 This is a schematic diagram of the structure of another intelligent agent described in this specification; Figure 9 This is a schematic diagram illustrating the processing procedure of an agent-based project as described in this specification; Figure 10 This is a schematic diagram of a processing device for one of the items described in this specification; Figure 11 This is a schematic diagram of a processing device for one of the items described in this specification. Detailed Implementation

[0012] This specification provides an embodiment of a method, apparatus, and device for processing intelligent agents and projects.

[0013] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0014] This specification provides an automated execution mechanism for complex projects. Many current automated execution and processing mechanisms encounter numerous practical difficulties and pain points when implemented in mobile applications. For example, automated inspection and RPA technologies face challenges when deployed in mobile applications. In typical systems, device control program sequences can be built through drag-and-drop, low-code workflow orchestration. Specifically, visual component modules are pre-set, and users drag and drop lines with the mouse and manually input various attributes (such as element ID, timeout, coordinates, etc.) in the sidebar to construct a complete device control program sequence. However, this method is highly dependent on user proficiency; when encountering complex tasks requiring dozens of steps, the node diagram becomes extremely chaotic, the learning cost for newcomers is too high, and the response to sudden changes in requirements is extremely slow. Alternatively, device control program sequences can be generated using RPA's natural language. Specifically, generative large models are used to convert natural language into general automated workflow code, such as the "Text-to-Flow" capability provided by RPA platforms. However, its instruction set is mostly limited to the Web. While requesting or desktop-level window operations is possible, this method lacks a domain-specific language and device control command system tailored for mobile testing scenarios. It is helpless against mobile-specific issues such as nested layers within real device applications, gesture swiping, and pop-up interference, and forced integration often leads to frequent crashes. Alternatively, a complete device control program sequence can be built using a mobile automation framework combined with manual coding. Specifically, relying on mature underlying frameworks such as Appium, UIAutomator2, Airtest, or Maestro, test development engineers manually write Python or Java test scripts or directly record device operation logs using auxiliary tools and then replay them. However, this method has a high coding development threshold, completely excluding business users. If... Recording and playback mode suffers from extremely poor script reusability. Once the interface undergoes even minor changes (such as button repositioning), the script becomes unusable immediately, resulting in high maintenance costs. Alternatively, a complete device control program sequence can be constructed by exploring end-to-end multimodal "black boxes." Specifically, interpretable intermediate states (such as program code) are weakened or directly eliminated. After receiving the task objective, the current page screenshot is analyzed directly through visual processing, and the screen coordinates for the next click or swipe are predicted for fully automatic pathfinding. However, due to the lack of a defined expression layer, the execution process is like a "black box." Not only is the result highly uninterpretable and difficult to audit, but when an error is discovered, it is impossible to accurately trace back to which link deviated, making it extremely difficult to scale up for enterprise-level precision testing tasks.

[0015] In summary, although a visual drag-and-drop "canvas designer" has been introduced to replace handwritten code, the surge in business demands and the increasing complexity of application scenarios have revealed serious lag and limitations in this model. Taking automated inspection as an example, the first issue is the high learning and maintenance threshold: According to internal feedback data from the network-wide inspection platform, a standard automated inspection process often consists of dozens or even hundreds of nodes (instructions) with independent functions. This requires operators to be familiar with the function of each tool in the component library and its input and output specifications. When faced with a new project or task, people without a programming background usually need to spend a lot of time flipping through manuals, watching video tutorials, or even undergoing professional training before they can barely complete the above operations. In drag-and-drop arrangement, the process of locating the required components and associating data flows alone accounts for more than 60% of the total time spent. Secondly, there is the extremely low efficiency caused by repeated configuration in complex scenarios: In real mobile inspection domains (such as multi-layered nested mini-programs, marketing campaign pages with many random pop-ups, etc.), linear process logic alone is no longer sufficient to meet the requirements. Users must tediously drag and drop conditional branch nodes, loop nodes, and various fault tolerance and retry configurations in the designer, and manually fill in a lot of complex component input and output parameters (such as element selector paths, waiting times, timeout thresholds, etc.). This repetitive operation is not only tedious, but also very easy to cause configuration errors due to human negligence, which greatly reduces the agility of the requirement to go live.

[0016] Given the aforementioned significant pain points, there is an urgent need to introduce a revolutionary interaction and generation model, namely, to reshape the entire process orchestration system through natural language understanding, aiming to achieve the goal of "speaking requirements, generating processes." The embodiments in this specification aim to completely break down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes": FlowAgent intelligently takes over the underlying details, eliminating the need for users to worry about variable passing and exception handling. Users simply need to send requirement information using everyday language. After rigorous testing and algorithm refinement, the current intelligent system has achieved a high component generation accuracy rate. This not only significantly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process setup efficiency, compressing the originally lengthy training and setup cycle to a very low level, thus giving the automation platform high vitality and ease of use. Specific processing details can be found in the following embodiments.

[0017] The project processing methods provided in one or more embodiments of this specification are applicable to the implementation environment of project processing. (Refer to...) Figure 1 The implementation environment includes at least: Client 100 and server 200. Furthermore, server 200 may include intelligent agents, large models, and related algorithms. The intelligent agents may include various algorithms, large models, and different network models (such as neural network models, BERT, etc.). Client 100 can run on terminal devices, which can be mobile phones, personal computers, tablets, e-book readers, wearable devices, devices that interact with information based on AR (Augmented Reality) and VR (Virtual Reality), and laptop computers, etc. Client 100 can be installed on terminal devices. Client 100 can be an application, a browser, or a subroutine embedded in an application, etc.

[0018] Server 200 can run on a server, which can be one or more servers, a server cluster consisting of several servers, or a cloud server of a cloud computing platform. Server 200 can be installed on the server. Server 200 can be an application or a subroutine embedded in an application. Intelligent agents, large models, and related algorithms can be integrated into server 200. Alternatively, server 200 can call any one or more of intelligent agents, large models, and related algorithms to perform corresponding operations.

[0019] In addition, it may include a database 300, which may be set in the server on which the server 200 runs, or outside the server on which the server 200 runs. The database 300 may store project requirement information for different projects, various types of components and other related information.

[0020] In this implementation environment, server 200 can receive project requirement information for the target project input by the user. Then, based on the project requirement information, it can perform intent parsing processing on the target project to determine the logical description information of the target project. After that, based on the logical description information of the target project, it can convert the component logical description information into an execution script corresponding to the project requirement information using a preset domain-specific language. The execution script corresponding to the project requirement information is then compiled to obtain the corresponding device control command sequence. Finally, based on the device control command sequence, the corresponding operation commands are sent to the terminal device where the client 100 corresponding to the target project is located, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution operation commands.

[0021] like Figure 2As shown in the embodiments of this specification, a method for processing a project is provided. The execution subject of this method can be a terminal device or a server, etc. The terminal device can be a mobile terminal device such as a mobile phone or tablet computer, or a computer device such as a laptop or desktop computer, or an IoT device (specifically, a smartwatch, in-vehicle device, etc.). The server can be a single server or a server cluster composed of multiple servers, etc. The server can be a backend server in fields such as finance or online shopping, or a backend server of an application, etc. This embodiment uses a server as the execution subject for detailed description. For the case where the execution subject is a terminal device, please refer to the following server case processing, which will not be repeated here. The method may specifically include the following steps: In step S202, the project requirement information for the target project is received from the user.

[0022] Users can be any user who needs to execute the target project. For example, a user might be someone who needs to inspect risky pages, meaning they need to automatically click, favorite, and browse the right-hand list of a page to verify whether the logic of specified risky components is functioning correctly. Specific settings can be configured according to actual needs. The target project can be any project requiring various simulation tests, such as white-box or black-box testing of terminal devices, hardware testing of a device, or inspection and exploration of specified risky pages (e.g., fraudulent pages, phishing pages). Specific settings can be configured according to actual needs. Project requirement information can be information about the requirements that need to be met to execute the target project, such as "Open the App and wait 1 second; if a pop-up appears, close it," or "Open the mini-program; if a pop-up appears, try to close it, then click the first item in the list and take a screenshot." In practical applications, project requirement information can be one or multiple. For multiple different project requirement information, they can be unrelated or have a specified relationship, such as an execution sequence. Specific settings can be configured according to actual needs.

[0023] In implementation, corresponding applications and pages can be pre-defined. When the target project needs to be executed, the user can launch the application, which can retrieve page data from the homepage. The user can retrieve a specified page through the application based on the actual situation. This page can include elements (buttons, hyperlinks, etc.) used to drive the execution of the target project. This page may include input boxes for project requirements information. The user can enter the project requirements information for the target project in these input boxes. After entering the information, the user can click the "OK" button on the page. At this point, the application can retrieve the data entered in the input boxes and use this data as the project requirements information.

[0024] In step S204, based on the above project requirement information, intent parsing processing is performed on the target project to determine the logical description information of the target project.

[0025] Among them, logical description information can be a verbal description of the user's complex project requirements, accurately mapped to a description of reasonable executable logic. Logical description information can translate the user's verbal description into a logical description that conforms to strict format specifications. Logical description information can then be used to further write programs to generate a sequence of device control instructions.

[0026] In practice, complex project requirement information input by users in colloquial language is often difficult for servers or terminal devices to understand or further process. Therefore, semantic understanding and semantic recognition can be performed on the above project requirement information to determine the semantic information to be expressed by the project requirement information. Then, based on the semantic information to be expressed by the project requirement information, the task can be broken down and each part can be preprocessed in a standardized manner to plan reasonable logical execution steps. Then, based on the planned reasonable logical execution steps, logical description information of the target project can be generated.

[0027] In practical applications, the above processing can be implemented through a pre-trained network model (such as a convolutional neural network model or a recurrent neural network model). That is, the pre-trained network model performs semantic understanding and semantic recognition on the project requirement information to determine the semantic information to be expressed by the project requirement information. Then, the pre-trained network model (which can be different from or the same as the above network model) can decompose the task according to the semantic information to be expressed by the project requirement information, and perform standardized preprocessing on each decomposed part to plan a reasonable logical execution step.

[0028] Furthermore, the above processing can be implemented using a pre-trained large model (such as a large language model or a specified generative large model). That is, the pre-trained large model performs semantic understanding and semantic recognition on the project requirement information to determine the semantic information to be expressed by the project requirement information. Then, the pre-trained large model (which may be different from or the same as the above large model) can decompose the task according to the semantic information to be expressed by the project requirement information, and perform standardized preprocessing on each decomposed part to plan a reasonable logical execution step.

[0029] In addition, the above processing can also be implemented through specified algorithms and / or rules. That is, the project requirement information is semantically understood and semantically recognized by pre-set algorithms and / or rules to determine the semantic information to be expressed by the project requirement information. Then, the task can be decomposed according to the semantic information to be expressed by the project requirement information by pre-set algorithms and / or rules (which may be different from the above algorithms and / or rules). Each decomposed part is preprocessed in a standardized manner to plan reasonable logical execution steps, etc. The specific steps can be set according to the actual situation.

[0030] In step S206, based on the logical description information of the target project, the logical description information of the target project is converted into an execution script corresponding to the project requirement information using a preset domain-specific language, and the execution script corresponding to the project requirement information is compiled to obtain the corresponding device control instruction sequence.

[0031] Domain-Specific Languages ​​(DSLs) are highly structured, standardized languages ​​specifically designed for mobile device control. They transform complex page operations and logical controls into standardized syntax, acting as a bridge between natural language and low-level execution. DSLs focus on a specific application domain, unlike general-purpose cross-domain languages ​​(GPLs). DSLs are used only in specific domains. DSLs include external DSLs and internal DSLs. External DSLs differ from the primary language used by the application system, typically employing custom syntax. The host application's code uses a text parsing mechanism to parse scripts written in external DSLs, such as regular expressions, SQL, AWK, and Struts configuration files. Internal DSLs use a specific syntax of a general-purpose language. Scripts written in internal DSLs are valid programs, but they have a specific style and only utilize a subset of the language's features to handle a small aspect of the system. A language workbench can be a dedicated IDE used to define and build DSLs. Specifically, the language workbench not only determines the language structure of the DSL but also serves as the editing environment for writing DSL scripts. The final script tightly integrates the editing environment and the language itself. The device control instruction sequence can support advanced logic structures such as sequential, parallel, loop, and conditional branching, giving the automation solution Turing complete characteristics. The device control instruction sequence can be constructed from device control instructions in a pre-defined device control instruction set. The device control instruction set can be a collection of device control instructions formed on the canvas according to a preset language specification, as well as related information such as the pointing relationship between device control instructions, execution order, and condition judgment.

[0032] In implementation, since the logical description information of the target project is an abstract concept, and controlling the terminal device cannot be achieved using abstract concepts, it is necessary to convert the abstract concept into concrete device control instructions or program code. Domain-Specific Languages ​​(DSLs) can serve as a tool for this conversion. Specifically, DSLs can be used to parse the logical description information of the target project into strongly formatted DSL language code. The DSL can act as a blueprint for the workflow automation script of the target project. Based on the logical description information of the target project, and using a pre-defined DSL language and a pre-set set of device control instructions, each step in the logical description information is converted into program code corresponding to the project requirements. The resulting program code can be sorted according to the order of the steps in the logical description information to obtain a program code sequence. An execution script can be generated based on this sequence, and this execution script can be optimized and adjusted according to actual conditions.

[0033] In practical applications, this execution script can be presented on three levels: at the bottom layer, it can be presented as program code in a DSL language; at the product logic level, it can be presented as a directed node graph; and at the interaction level, it can be presented as draggable components (which may be related components composed of device control commands). Because a mobile-specific DSL specification has been established, the output of the large model is strictly limited to the device control command set with a built-in whitelist. Illegal and destructive commands can be intercepted at the compilation layer, thus eliminating the risk of disordered operation caused by the illusion of a large model and achieving accurate generation. Based on data interaction and display, the component generation accuracy has stabilized at a high level.

[0034] To directly control terminal devices, the execution scripts corresponding to the project requirements can be compiled using a parser or compiler. This transforms the high-level language DSL into an unambiguous "device control instruction set," forming a device control instruction sequence. The device control instructions in this sequence can be used to control various operations of the terminal device. The compiled device control instruction sequence contains a set of underlying device operation sequences with high-concurrency processing, timeout polling, and exception-tolerant degradation distribution capabilities.

[0035] In step S208, based on the device control command sequence, corresponding operation commands are sent to the terminal device corresponding to the target project in sequence, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution of the operation commands.

[0036] In implementation, operation commands corresponding to the device control commands can be sent sequentially to the terminal device corresponding to the target project based on the order of the device control command sequence. Upon receiving the corresponding operation command, the terminal device can execute the command and return the execution result to the server. The server can then retrieve this result. Based on this method, the server can obtain the execution results of the operation commands corresponding to each device control command in the device control command sequence returned by the terminal device. By summarizing these results, the final execution result of the target project can be determined.

[0037] This specification provides a method for processing a project. It involves receiving project requirement information from a user for a target project. Based on this requirement information, intent parsing is performed to determine the logical description information of the target project. Then, based on this logical description information, a preset domain-specific language is used to convert it into an execution script corresponding to the project requirement information. This execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project. The results of the terminal device executing the operation commands are obtained, and the execution of the target project is determined based on the obtained results. As a result, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0038] In practical applications, the specific processing method of step S204 can vary. The following provides another optional processing method, which may specifically include the processing of steps S2042 and S2044. Based on this, in the above... Figure 2 Based on this, the specific steps included in this method can be as follows: Figure 3 As shown.

[0039] In step S2042, based on the above project requirement information, the target project is decomposed to obtain the logical execution process of executing the target project. Based on the logical execution process of executing the target project, the logical execution process is converted into component logical description information and the reasoning chain corresponding to the component logical description information in accordance with the preset format specification through a preset large model.

[0040] The logical execution process of the target project can be entirely described by language, such as "open webpage A, click button B on webpage A, and save a screenshot." The component logical description information can be a logical execution process formed by language description combined with the component's form. Based on the above example, the corresponding component logical description information could be "execute component k1 to open webpage A, trigger component k2 through a click operation to click button B on webpage A, and call component k3 to save a screenshot," etc. The reasoning chain corresponding to this component logical description information can be a reasoning chain obtained through a large model. The chain of thought (CoT) can be a series of intermediate reasoning steps required to obtain the model's output, forming a reasoning chain that explains the basis (or reasoning) for the model's ability to identify this type of risk. The large model can be constructed using BERT or large language models, such as the Qwen3-4B-Instruct large model or GPT-4, depending on the specific circumstances. Large models can be used for understanding and translating natural language, transforming colloquial expressions into component logic descriptions that conform to strict formatting specifications. Large models can be pre-trained models obtained directly from a specified database, or they can be architectures built using specified algorithms and / or machine learning networks, then trained with a large amount of sample data to obtain a trained large model. Alternatively, a pre-trained large model can be obtained, and a certain amount of sample data can be acquired to fine-tune the pre-trained large model, resulting in a fine-tuned large model. The specific details can be configured according to the actual situation.

[0041] In implementation, an orchestration mechanism can be pre-set. Upon obtaining the user's project requirements information, the large model is not immediately invoked. Instead, the orchestration mechanism uses a series of engineering techniques (such as context population and historical dialogue memory retrieval) to decompose and standardize the project requirements information into tasks. Through intent parsing and context completion, reasonable logical execution steps can be planned, resulting in the logical execution process for the target project. This logical execution process for the target project will be more complete and comprehensive than the project requirements information provided.

[0042] A large, finely tuned model can serve as the core foundation. This model can be used to understand and translate the natural language of the logical execution process of the target project. Based on specially trained domain feature data, the colloquial expression of the logical execution process of the target project can be translated into a component logical description that conforms to strict format specifications, presented in the form of language descriptions combined with components. At the same time, the reasoning thought chain obtained from the component logical description information through the large model can also be obtained.

[0043] In step S2044, the logical description information of the target project is determined based on the component logical description information.

[0044] In implementation, the component logic description information obtained above can be directly used as the logic description information of the target project. Alternatively, the component logic description information obtained above can be further supplemented and improved, and the final result can be used as the logic description information of the target project. Or, other relevant auxiliary information can be obtained, and the component logic description information obtained above can be adjusted through the auxiliary information, and the final result can be used as the logic description information of the target project. The specific settings can be made according to the actual situation.

[0045] Based on the above-mentioned reasoning thought chain, the reasoning thought chain can also be processed as follows: display the reasoning thought chain corresponding to the above component logical description information.

[0046] In practice, to ensure users are aware of the current processing status, execution status bubbles of the reasoning thought chain corresponding to the above component logic description information can be displayed.

[0047] In practical applications, the specific processing method for converting the logical description information of the target project into an execution script corresponding to the project requirement information using a preset domain-specific language in step S206 can vary. One optional processing method is provided below, which may specifically include the processing steps S2062 and S2064. Based on this, in the above... Figure 3 Based on this, the specific steps included in this method can be as follows: Figure 4 As shown.

[0048] In step S2062, based on the component logic description information corresponding to the logic description information of the execution target project, a corresponding execution logic tree is constructed using a preset domain-specific language.

[0049] In implementation, the processing of the execution script corresponding to the above-mentioned project requirement information generated by the preset domain-specific language serves as the mainstay of the entire execution process. It can accurately transform the user's intent into a definite DSL logic tree. That is, through the domain-specific language DSL, the component logic description information corresponding to the logical description information of the execution target project can be accurately transformed into a definite DSL logic tree (i.e., execution logic tree). For example, the user's project requirement information can be "If a 'Claim Coupon' button appears at the bottom of the page, click it; otherwise, swipe up twice." Through the domain-specific language DSL, it will be accurately translated into a DSL program fragment containing a "conditional branch node" and two corresponding "operation instruction nodes" (such as click and / or swipe). In the above way, multiple different fragments can be accurately transformed into definite DSL program fragments, and multiple different DSL program fragments can be spliced ​​together to obtain a complete execution logic tree.

[0050] In step S2064, an execution script corresponding to the above-mentioned project requirement information is generated based on the execution logic tree and the logical description information of the execution target project.

[0051] In implementation, based on the logical description information of the target project and combined with the execution logic tree, the execution script corresponding to the above project requirement information can be further generated. The above processing method relies on static syntax structure and strict component attribute constraints, and has the characteristics of strong interpretability and extremely high execution success rate. It is the preferred solution for current driver automatic detection to execute accurately according to the standard process.

[0052] In practical applications, such as Figure 5 As shown, the Domain-Specific Language (DSL) can be configured with a whitelist mechanism and error prevention checks. After the execution script is generated, DSL error prevention checks can be performed. If the syntax and type checks in the execution script pass, a security and compliance check can be performed to identify any dangerous operations or permissions. If the check passes, subsequent compilation can proceed. If the check fails, automatic error correction or clarification can be performed, allowing for the addition of relevant parameters and rewriting of related information to regenerate the execution script.

[0053] In practical applications, a multimodal large model can also be set up to perform visual detection and coordinate suggestion. For details, please refer to the following content: Obtaining images of multi-level pages through a multimodal large model.

[0054] Multimodal large models can be constructed using BERT or large language models, depending on the specific circumstances. These models can be used for understanding and processing text, images, audio, and video, and for acquiring relevant data. A multimodal large model can be a pre-trained model obtained directly from a specified database, or its architecture can be built using a specified algorithm and / or machine learning network, and then trained with a large amount of multimodal sample data. Alternatively, a pre-trained large model can be obtained, and then a certain amount of multimodal sample data can be acquired to fine-tune the pre-trained model, resulting in a fine-tuned multimodal large model. The specific details can be determined based on the actual situation.

[0055] In implementation, such as Figure 6 As shown, it is not only possible to process text data by parsing it, but also to integrate the perception capabilities of multimodal large models based on computer vision. Specifically, machine vision mechanisms of multimodal large models can be inserted at various stages of the entire process. At this time, images of various pages can be obtained through multimodal large models, including images of multi-level pages during the process of controlling the terminal device to execute specified operation commands, images of different components during the process of generating component logic description information, images of directed node graphs corresponding to component logic description information, and images of the canvas where the directed node graphs corresponding to component logic description information are located, etc.

[0056] Based on the above, the specific processing method in step S2042, which converts the logical execution process of the target project into component logical description information conforming to the preset format specification through a preset large model, can be varied. Here is another optional processing method, which may include the following: based on the logical execution process of the target project and the obtained multi-level page images, the above logical execution process is converted into component logical description information conforming to the preset format specification through a preset large model.

[0057] In implementation, such as Figure 6 As shown, a corresponding prompt can be generated based on the logical execution process of the target project and the acquired images. This prompt can be input into the large model, guiding it to convert the logical execution process into component logic description information conforming to a preset format. In this way, with the assistance of the acquired images, more accurate component logic description information conforming to the preset format can be generated.

[0058] In practical applications, the relevant large models can also be fine-tuned using the data obtained above. For details, please refer to steps A2 and A4 below.

[0059] In step A2, abnormal data is extracted from the images of the acquired multi-level pages and / or the results of the acquired execution operation commands, and the abnormal data corresponding to the abnormal data is obtained.

[0060] In practice, the above processing can yield a large amount of data, including images obtained through a multimodal large model and the results of executed operation commands. Abnormal data can be extracted from this large amount of data and can be used as anomaly cases. At the same time, non-abnormal data corresponding to the abnormal data (i.e., correct data corresponding to the abnormal data) can be obtained and can be used as correct cases.

[0061] In step A4, the abnormal data and the corresponding non-abnormal data are stored in a preset database. The abnormal data and the corresponding non-abnormal data stored in this database are used to fine-tune the preset large model.

[0062] In implementation, such as Figure 5 As shown, the above-mentioned abnormal cases and correct cases can be stored in the database as data pairs. Subsequently, the preset large model can be fine-tuned by using the abnormal data (i.e., abnormal cases) and the corresponding non-abnormal data (i.e., correct cases) stored in the database, so as to continuously optimize the large model and keep it in a better state.

[0063] In practical applications, the visual perception capabilities of multimodal large models can be used to autonomously determine potentially high-value areas, thereby completing unmanned "black box" pathfinding. For details, please refer to steps B2 and B4 below.

[0064] In step B2, images of the page during the execution of the operation command corresponding to the device control instruction sequence by the terminal device are obtained through a multimodal large model, and key areas in the page are determined based on the obtained page images.

[0065] In implementation, based on the above, multimodal large-scale model perception capabilities based on computer vision can be integrated. When encountering abnormal pages or entirely new interfaces or pages that cannot be described by preset rules or methods, the system can switch to a semi-autonomous exploration state. Then, the real-time transmitted device screenshots (i.e., page images) can be analyzed to autonomously determine the next "potential high-value hotspot," thereby completing "black-box" pathfinding without human intervention. Specifically, the multimodal large-scale model acquires images of pages during the execution of operation commands corresponding to the device control command sequence by the terminal device. These pages can be a single specified page, multiple different pages, or multi-level pages. The acquired page images can be analyzed to determine key areas (i.e., potential high-value hotspots, such as the upper right corner, right edge, and lower right corner of the page). The determination of key areas can be achieved in various ways. For example, the image can be used to identify areas that are constantly changing (such as areas with dynamically changing images or videos), or areas with pop-ups. These areas can be identified as key areas, and the specific settings can be configured according to the actual situation.

[0066] In step B4, additional operation commands are generated for the key areas of the above page using a multimodal large model, and the additional operation commands are sent to the terminal device to obtain the results of the terminal device executing the additional operation commands.

[0067] In implementation, key areas of the aforementioned page can be input into a multimodal large model. Additionally, images of the acquired page can also be input into the multimodal large model. Through the multimodal large model, corresponding additional operation commands are generated for the key areas of the page. These additional operation commands can be specific to actions such as clicking or swiping, and can be set according to actual needs. Then, the additional operation commands can be sent to the terminal device. The terminal device can perform corresponding automated operations on the key areas of the page according to the additional operation commands, and can record the operation behavior, the results of executing the additional operation commands, page state changes after executing the additional operation commands, logs of whether the operation was successful or not, and the final data captured, etc., in real time. The results of executing the additional operation commands can be returned to the server, as can the recorded relevant data. The server can obtain the results of the terminal device executing the additional operation commands.

[0068] Based on the processing in steps B2 and B4 above, there are various ways to determine the execution result of the target project in step 208 based on the result of the obtained execution operation command. Here is another optional processing method, which may include the following: determining the execution result of the target project based on the result of the obtained execution operation command and the result of the execution of the additional operation command.

[0069] In implementation, the results of the executed operation commands and the results of the executed additional operation commands can be combined for a comprehensive judgment to determine the execution result of the target project. By combining the results of the executed operation commands and the results of the executed additional operation commands, the risks present on the page can be more accurately determined, and abnormal pages, entirely new interfaces, or pages that cannot be described by preset rules or methods can be handled more flexibly.

[0070] In practical applications, the device control instruction sequence includes one or more device control instructions from different dimensions, such as logical flow control, multi-dimensional element search, gesture and data input, state awareness, exception handling, and variable control. Among them, the device control instructions in the logical flow control dimension include one or more of the following: sequential execution instructions, parallel concurrent task execution instructions, conditional judgment instructions, and loop iteration instructions. The device control instructions in the multi-dimensional element search dimension include one or more of the following: instructions for locating the control hierarchy structure using a preset positioning tool, and instructions for locking interactive objects using recognized and displayed text. The device control instructions in the gesture and data input dimension include one or more of the following: click, long press, directional multi-level swipe, text box content input instructions, and focus wake-up instructions. The device control instructions in the state awareness dimension include one or more of the following: explicit wait instructions, implicit element loading wait instructions, and image capture instructions for compliance and evidence preservation. The device control instructions in the exception handling dimension include one or more of the following: instructions corresponding to the preset interruption tolerance mechanism for interference, exception self-healing instructions, and retry scheduling instructions. The device control instructions in the variable control dimension include one or more of the following: cross-component variable declaration and reference instructions, and cross-step variable declaration and reference instructions.

[0071] In implementation, to ensure that the generated DSL language device control command sequence can cover all the complex and tedious tasks in the real world without any blind spots, the DSL specification pre-defines an extremely rich variety of atomic capability types. These capabilities not only include simple clicks, but also cover multiple dimensions such as logic processing, including logic flow control, multi-dimensional element search, gesture and data entry, status awareness, exception handling, and variable control.

[0072] The device control instructions in the logical flow control dimension include one or more of the following: sequential execution instructions, parallel concurrent task execution instructions, conditional judgment instructions, and loop iteration instructions, thereby ensuring the Turing completeness of the process.

[0073] Device control commands for multi-dimensional element search include one or more of the following: commands that locate the control hierarchy using preset positioning tools (such as XPath), and commands that use methods to recognize displayed text (such as using OCR to recognize screen text) to lock onto interactive objects. For example... Figure 5As shown, the device control commands for multi-dimensional element search can search for specified elements (such as buttons, hyperlinks, input boxes, etc.) on the page while the terminal device is executing corresponding operation commands. Based on the specified elements found, corresponding operation commands (such as clicking or swiping) can be executed subsequently.

[0074] In addition, the generated DSL language device control instruction sequence can also have exception handling capabilities. That is, when encountering various system-level interferences (such as pop-up interruptions), it has a preset interruption tolerance mechanism and the ability to actively eliminate pop-up interruptions and continue the process. Therefore, device control instructions in the exception handling dimension can be set. Device control instructions in the exception handling dimension include one or more of the following: instructions corresponding to the preset interruption tolerance mechanism for interference, exception self-healing instructions, and retry scheduling instructions.

[0075] Device control instructions in the variable control dimension include one or more of the following: cross-component variable declaration and reference instructions, and cross-step variable declaration and reference instructions. Device control instructions in the variable control dimension can ensure the safe flow of dynamic data.

[0076] In practical applications, the target items are the page-down items of the preset application or the inspection items of the risk page.

[0077] The above processing methods can be applied to complex and varied tasks with extremely deep chains, such as including but not limited to: App (or application program or mini-program, etc.) page exploration (such as simulating real users to delve into the application level by level to explore hidden errors, etc.), risk page inspection (such as automatically clicking, favorited, browsing the right-hand list of the page and verifying whether the logic of potentially risky components is working properly, etc.), and exploration, etc. The specific settings can be set according to the actual situation.

[0078] This specification provides a method for processing a project. It involves receiving project requirement information from a user for a target project. Based on this requirement information, intent parsing is performed to determine the logical description information of the target project. Then, based on this logical description information, a preset domain-specific language is used to convert it into an execution script corresponding to the project requirement information. This execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project. The results of the terminal device executing the operation commands are obtained, and the execution of the target project is determined based on the obtained results. As a result, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0079] The above describes the project processing method provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide an intelligent agent, such as... Figure 7 As shown.

[0080] The intelligent agent 700 may include an interaction layer 701, an intelligent agent processing layer 702, a DSL and compilation layer 703, an execution and device control layer 704, and an observation and data closed-loop layer 705, wherein: The interaction layer 701 primarily relies on the FlowAgent dialog box within the SIPA platform's process designer. The process designer (i.e., the canvas, node, and device control command orchestration system) serves as the core interface for user interaction with automation capabilities. Typically, users need to manually navigate through a vast component library, drag and drop "building blocks," and construct complex upstream and downstream logic between nodes. However, in the embodiments described in this specification, the process designer introduces dynamic interaction between global control and canvas nodes, making node states (such as "generating" and "pending confirmation") highly visible and dynamically responsive. Furthermore, the process designer can integrate an intelligent assistant module, enabling precise configuration of underlying canvas node-level attributes, including selector positioning strategies, input parameters, and exception handling, without user intervention. The FlowAgent (intelligent dialogue assistant window) dialog box, acting as a "Copilot" within the designer, exists as a floating dialog window (e.g., integrated into the control bar entry). It completely takes over the previously extremely tedious task of arranging device control commands. Users only need to input natural language in the dialog box, such as "Open the App and wait 1 second; close the pop-up if there is one," and FlowAgent can interact with the user through progress bubbles (e.g., "Understanding Intent," "Generate Device Control Command," "Configure Attributes") and generate the corresponding canvas node logic with one click. Based on the above architecture, this interaction layer can handle the user's project requirement information and display the execution status bubble. Because FlowAgent has built an interaction layer that directly maps and automatically derives all node attribute input parameters based on natural language intent understanding (reasoning thought chain), the learning curve is lowered and configuration steps are reduced. Non-technical operations personnel can simply "speak" the process through natural dialogue, effectively saving time spent learning various component descriptions and improving the overall efficiency of process setup.

[0081] The intelligent agent processing layer 702 can decompose the user's project requirement information and understand the intention to execute the target project, thereby planning a reasonable logical execution process. Various large models and algorithms can be set in the intelligent agent processing layer 702.

[0082] The DSL and compiler layer 703 serve as a bridge between abstract concepts and concrete programs. The DSL language not only acts as a blueprint for process automation scripts, but more importantly, it allows for the setting of whitelist mechanisms and error prevention checks. After compilation, the high-level language is transformed into an unambiguous "device control instruction set".

[0083] The execution and device control layer 704 can receive and schedule the above instruction set through a preset execution engine, and send operation commands to specific terminal devices in the real device pool. It can accurately execute complex gestures (such as multi-finger swipes) and control search tasks, and strictly handle timeout and retry logic.

[0084] The observation and data closed-loop layer 705 can collect information such as page state changes after the terminal device executes operation commands, logs of operation success or failure, and the final captured data. On the one hand, it can provide visualized feedback to the user; on the other hand, it can collect abnormal and correct cases to continuously optimize the large model. By creating a comprehensive observation system from front-end command generation to back-end execution data, correct and error-correcting examples are accumulated and incorporated into the training data closed-loop flywheel project, thereby achieving continuous evolution. The feedback data can be used for post-training optimization and iteration, enabling the agent's understanding and translation capabilities in handling complex combined scenarios to spiral upwards in real-world applications.

[0085] Based on the above architecture, in the embodiments of this specification, the intelligent agent can implement the specific processing of the above-mentioned project processing method through its various layers, as follows: Interaction layer 701 is configured to receive user input of project requirement information for the target project, and to display the processing results of each layer in the agent. The intelligent agent processing layer 702 is configured to perform intent parsing processing on the execution target project based on the above-mentioned project requirement information, and determine the logical description information of the execution target project. The DSL and compilation layer 703 are configured to convert the logical description information of the execution target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and then compile the execution script corresponding to the project requirement information to obtain the corresponding device control instruction sequence. The execution and device control layer 704 is configured to send corresponding operation commands to the terminal device corresponding to the target project in sequence based on the device control command sequence. The observation and data closed-loop layer 705 is configured to acquire the results of the operation commands corresponding to the sequence of device control instructions executed by the terminal device, and determine the execution result of the target project based on the acquired results of the execution operation commands.

[0086] The specific processing steps for the intelligent agents composed of the above layers to implement the processing methods of the above projects can be found in the aforementioned related content, and will not be repeated here.

[0087] This specification provides an intelligent agent that receives project requirement information for a target project input by a user. Based on this requirement information, it performs intent parsing to determine the logical description information of the target project. Then, based on this logical description information, it converts the logical description information into an execution script corresponding to the project requirement information using a preset domain-specific language. The execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this device control command sequence, it sequentially sends corresponding operation commands to the terminal device corresponding to the target project and obtains the results of the terminal device executing the operation commands. The execution result of the target project is determined based on the obtained results of the execution operation commands. Thus, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes." The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes." The intelligent agent takes over the underlying details, and users do not need to worry about variable transmission and anomaly capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only significantly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0088] In practical applications, such as Figure 8 As shown, the aforementioned agent processing layer 702 also includes an agent orchestration layer 702-1 and a large model layer 702-2, wherein: After obtaining the user's project requirement information, the intelligent agent orchestration layer 702-1 does not immediately call the large model. Instead, it uses a series of engineering methods (such as context filling, historical dialogue memory retrieval, etc.) to decompose and standardize the project requirement information into tasks. In this way, through intent parsing and context completion, reasonable logical execution steps can be planned.

[0089] The large model layer 702-2 is based on a designated large model that has been fine-tuned. This layer undertakes the most challenging natural language understanding and translation work. Based on specially trained domain feature data, it translates colloquial expressions into component logic descriptions that conform to strict format specifications.

[0090] Specifically, the intelligent agent orchestration layer 702-1 is configured to decompose the target project based on the above-mentioned project requirement information to obtain the logical execution process for executing the target project.

[0091] The large model layer 702-2 is configured to perform the logical execution process based on the target project. It uses a pre-defined large model to convert this logical execution process into component logical description information and corresponding reasoning chains that conform to a pre-defined format. Based on the component logical description information, the logical description information of the target project is determined.

[0092] Correspondingly, the interaction layer 701 is configured to display the reasoning chain corresponding to the logical description information of the display component.

[0093] In practical applications, the DSL and compiler layer 703 can be configured with a dual-track driving mode: a standard DSL generation mode and an intelligent probing mode. In the standard DSL generation mode, the DSL and compiler layer 703 are configured to construct the corresponding execution logic tree using a preset domain-specific language, based on the component logic description information corresponding to the logical description information of the execution target project. Based on the execution logic tree and the logical description information of the execution target project, the execution script corresponding to the aforementioned project requirements is generated. By implementing a clear "standard DSL generation" bridging mode, even with intelligent inference technology, the final determined execution actions are still visualized as a readable and correctable list of steps, ensuring extremely high auditability and interpretability. All process actions are traceable and visible, and the system can accurately locate specific instruction nodes where faults occur (such as failure to click due to abnormal obstruction), greatly facilitating backtracking and troubleshooting.

[0094] like Figure 9 As shown, in intelligent exploration mode, the system can integrate multimodal large-scale model perception capabilities based on computer vision. When encountering abnormal pages or entirely new interfaces or pages that cannot be described by preset rules or methods, it can switch to a semi-autonomous exploration state. Then, it can analyze the real-time transmitted device screenshots and autonomously determine the next "potential high-value hotspot," thereby completing "black-box" pathfinding without human intervention. Specifically, the DSL and compilation layer 703 are configured to acquire images of pages during the execution of operation commands corresponding to the device control instruction sequence by the multimodal large-scale model in the large-scale model layer, and determine the key areas in the page based on the acquired page images; generate additional operation commands for the key areas in the page through the multimodal large-scale model in the large-scale model layer, send the additional operation commands to the terminal device, and obtain the results of the terminal device executing the additional operation commands.

[0095] Correspondingly, the observation and data closed-loop layer 705 is configured to determine the execution result of the target project based on the results of the acquired execution operation commands and the results of the execution of additional operation commands.

[0096] like Figure 9As shown, the real device pool can be a large cluster composed of various real-world mobile phone models and other terminal devices, providing a physical carrier for task distribution. The instruction adapter can act as a driver, smoothing out the differences between different terminal devices and the underlying system, such as openApp, click, and swipe. Device drivers can be used to drive the normal operation of terminal devices, specifically including ADB, UIAutomator, and Appium. The DSL specification in the 703 compiler layer includes constraints on nodes, parameters, and control flow. The control flow can support advanced logic structures such as sequential, parallel, loop, and conditional branching, thus giving the automation solution Turing complete characteristics. Furthermore, a complete control flow (conditional branching) and built-in timeout waiting, multi-path automatic retry, and pop-up window recognition and exception capture modules can be injected into the DSL specification, thereby improving execution stability and resilience. When terminal devices encounter occasional interference, they no longer directly crash and report errors; the system can implement self-healing strategies to successfully achieve the final capture goal, ensuring a high completion rate for massive unattended inspection tasks.

[0097] In practical applications, a multimodal large model can be set in the large model layer 702-2, and the large model layer 702-2 is configured to obtain images of multi-level pages through the multimodal large model.

[0098] Correspondingly, the large model layer 702-2 is configured to convert the logical execution process of the target project and the images of the multi-level pages obtained into component logical description information that conforms to the preset format specifications through the preset large model.

[0099] In practical applications, based on the above, the observation and data closed-loop layer 705 is configured to extract abnormal data from the images of the acquired multi-level pages and / or the results of the acquired execution commands, and to obtain the corresponding anomaly-free data. The abnormal data and the corresponding anomaly-free data are stored in a preset database, and the stored data is used to fine-tune a preset large model. This can provide an instruction-level fine-tuning model aligned with domain feature data, generating an abstract logical node structure. Combined with the dynamic search and recalculation capabilities of the execution engine control elements, this improves the accuracy and consistency of process generation, significantly reduces the time spent manually patching execution scripts, greatly enhances reusability, makes the logic more robust, and reduces human maintenance costs.

[0100] In practical applications, the target items are the page-down items of the preset application or the inspection items of the risk page.

[0101] The specific processing procedures for each of the above layers can be found in the aforementioned content, and will not be repeated here.

[0102] This specification provides an intelligent agent that receives project requirement information for a target project input by a user. Based on this requirement information, it performs intent parsing to determine the logical description information of the target project. Then, based on this logical description information, it converts the logical description information into an execution script corresponding to the project requirement information using a preset domain-specific language. The execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this device control command sequence, it sequentially sends corresponding operation commands to the terminal device corresponding to the target project and obtains the results of the terminal device executing the operation commands. The execution result of the target project is determined based on the obtained results of the execution operation commands. Thus, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes." The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes." The intelligent agent takes over the underlying details, and users do not need to worry about variable transmission and anomaly capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only significantly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0103] The following describes in detail a project processing method provided by the embodiments of this specification, using a specific application scenario. The target project is the inspection of risk pages, and the project requirement information can be inspection requirement information. This method can be applied to a full-network inspection platform, which can be an automated platform managing various rendering devices including mobile phones, browsers, and mini-program containers. This technical solution can provide a seamlessly integrated exploration engine to support its core objectives of achieving multi-level dynamic exploration and in-depth page content collection.

[0104] This specification provides a method for processing a project. The execution subject of this method can be a terminal device or a server, such as a mobile terminal device like a mobile phone or tablet, a computer device like a laptop or desktop computer, or an IoT device (specifically, a smartwatch, in-vehicle device, etc.). The server can be a single server or a server cluster composed of multiple servers. The server can be a backend server in fields such as finance or online shopping, or a backend server for a specific application. This embodiment uses a server as the execution subject for detailed explanation. For the case where the execution subject is a terminal device, please refer to the following section on server-side processing, which will not be repeated here. Refer to the above... Figure 5 , Figure 6 and Figure 9 The method may specifically include the following steps: In step C02, the system receives the inspection request information for the risk page, input by the user.

[0105] In step C04, based on the inspection requirement information, the inspection of the risk page is decomposed into tasks to obtain the logical execution process for performing the inspection of the risk page.

[0106] In step C06, based on the logical execution process of the inspection of the execution risk page, the logical execution process is converted into component logical description information and the reasoning thought chain corresponding to the component logical description information in accordance with the preset format specification through the preset large model.

[0107] In step C08, the reasoning chain corresponding to the component logic description information is displayed.

[0108] In step C10, based on the component logic description information, the corresponding execution logic tree is constructed using a preset domain-specific language.

[0109] In step C12, based on the execution logic tree and component logic description information, the execution script corresponding to the inspection requirement information is generated.

[0110] In step C14, the execution script is validated. If the validation fails, automatic error correction or clarification can be performed to supplement relevant parameters and rewrite relevant information, thereby regenerating the execution script. In step C16, if the verification passes, a security and compliance check is performed on the executed script.

[0111] In step C18, if the inspection passes, the execution script corresponding to the inspection requirement information is compiled to obtain the corresponding equipment control instruction sequence.

[0112] In step C20, based on the equipment control instruction sequence, corresponding operation commands are sent sequentially to the terminal devices corresponding to the inspection on the risk page, and the results of the terminal devices executing the operation commands corresponding to the equipment control instruction sequence are obtained.

[0113] In step C22, images of the page during the execution of the operation command corresponding to the device control instruction sequence by the terminal device are obtained through a multimodal large model, and key areas in the page are determined based on the obtained page images.

[0114] In step C24, additional operation commands are generated for key areas of the page using a multimodal large model, and the additional operation commands are sent to the terminal device to obtain the results of the terminal device executing the additional operation commands.

[0115] In step C26, the execution result of the risk page inspection is determined based on the results of the obtained execution operation command and the results of the execution of the additional operation command.

[0116] This specification provides a method for processing a project. It involves receiving project requirement information from a user for a target project. Based on this requirement information, intent parsing is performed to determine the logical description information of the target project. Then, based on this logical description information, a preset domain-specific language is used to convert it into an execution script corresponding to the project requirement information. This execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project. The results of the terminal device executing the operation commands are obtained, and the execution of the target project is determined based on the obtained results. As a result, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0117] Based on the same idea, this specification also provides a project processing apparatus, such as... Figure 10 As shown.

[0118] The processing unit for this project includes: a demand receiving module 1001, an intent parsing module 1002, a conversion and compilation module 1003, and a result determination module 1004, wherein: The requirement receiving module 1001 receives project requirement information for the target project input by the user; The intent parsing module 1002 performs intent parsing processing on the target project based on the project requirement information to determine the logical description information of the target project. The conversion and compilation module 1003, based on the logical description information of the target project, converts the logical description information of the target project into an execution script corresponding to the project requirement information through a preset domain-specific language, and compiles the execution script corresponding to the project requirement information to obtain the corresponding device control instruction sequence. The result determination module 1004, based on the device control instruction sequence, sequentially sends corresponding operation commands to the terminal device corresponding to the target project, obtains the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determines the execution result of the target project based on the obtained result of executing the operation commands.

[0119] In this embodiment of the specification, the intent parsing module 1002 includes: The logic information determination unit, based on the project requirement information, performs project decomposition processing on the target project to obtain the logical execution process of executing the target project, and based on the logical execution process of executing the target project, converts the logical execution process into component logic description information conforming to the preset format specification and the reasoning thought chain corresponding to the component logic description information through a preset large model; The logic description unit determines the logic description information for executing the target project based on the component logic description information; The device further includes: The display module shows the reasoning chain corresponding to the logical description information of the component.

[0120] In the embodiments of this specification, the conversion and compilation module 1003 includes: The logic tree construction unit constructs a corresponding execution logic tree based on the component logic description information corresponding to the logic description information of the target project, using a preset domain-specific language. The script generation unit generates an execution script corresponding to the project requirement information based on the execution logic tree and the logical description information for executing the target project.

[0121] In the embodiments described in this specification, the device further includes: The image acquisition module acquires images from multiple levels of pages using a multimodal large model; The logic information determination unit, based on the logical execution process of the target project and the images of the multi-level pages obtained, converts the logical execution process into component logic description information that conforms to a preset format specification through a preset large model.

[0122] In the embodiments described in this specification, the device further includes: The data acquisition module extracts abnormal data from the images of the acquired multi-level pages and / or the results of the acquired execution operation commands, and obtains the abnormal data corresponding to the abnormal data. The data processing module stores the abnormal data and the corresponding normal data in a preset database. The abnormal data and the corresponding normal data stored in the database are used to fine-tune the preset large model.

[0123] In the embodiments described in this specification, the device further includes: The image processing module acquires images of the page during the execution of the operation command corresponding to the device control instruction sequence by the terminal device through a multimodal large model, and determines the key areas in the page based on the acquired page images; The additional operation command processing module generates additional operation commands for key areas in the page through a multimodal large model, sends the additional operation commands to the terminal device, and obtains the result of the terminal device executing the additional operation commands. The result determination module 1004 determines the execution result of the target project based on the result of the acquired execution operation command and the result of executing the additional operation command.

[0124] In the embodiments of this specification, the device control instruction sequence includes one or more device control instructions of the following different dimensions: logical flow control, multi-dimensional element search, gesture and data input, state awareness, exception handling, and variable control. Specifically, the device control instructions of the logical flow control dimension include one or more of the following: sequential execution instructions, parallel concurrent task execution instructions, conditional judgment instructions, and loop iteration instructions. The device control instructions of the multi-dimensional element search dimension include one or more of the following: instructions for locating the control hierarchy structure using a preset positioning tool, and instructions for locking interactive objects using recognized and displayed text. The device control instructions of the gesture and data input dimension include one or more of the following: click, long press, directional multi-level swipe, text box content input instructions, and focus wake-up instructions. The device control instructions of the state awareness dimension include one or more of the following: explicit wait instructions, implicit element loading wait instructions, and image capture instructions for compliance and evidence preservation. The device control instructions of the exception handling dimension include one or more of the following: instructions corresponding to a preset interruption tolerance mechanism for interference, exception self-healing instructions, and retry scheduling instructions. The device control instructions of the variable control dimension include one or more of the following: cross-component variable declaration and reference instructions, and cross-step variable declaration and reference instructions.

[0125] In the embodiments of this specification, the target item is a page-down navigation item of a preset application or an inspection item of a risk page.

[0126] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more embodiments of this specification, the functions of each module or unit can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative; the division of each module and unit is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or modules can be combined or integrated into another system, or some features can be ignored or not executed, etc.

[0127] This specification provides a project processing device that receives project requirement information for a target project input by a user. Based on this requirement information, it performs intent parsing to determine the logical description information of the target project. Then, based on this logical description information, it converts the logical description information into an execution script corresponding to the project requirement information using a preset domain-specific language. The execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution of the target project is determined based on the obtained results of the execution operation commands. As a result, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0128] The above describes the project processing apparatus provided in the embodiments of this specification. Based on the same concept, the embodiments of this specification also provide a project processing device, such as... Figure 11 As shown.

[0129] The processing equipment for the project can be a terminal device or a server, as described in the above embodiments.

[0130] The processing equipment of the project can vary considerably due to differences in configuration or performance, and may include a communication interface 1102, a user interface 1104, a processor 1106, and a data storage 1108. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1110. The communication interface 1102 enables the processing equipment 1100 of the project to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1102 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1102 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1102 may also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1102 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide-area wireless interfaces.

[0131] User interface 1104 includes receiving user input and providing output to the user. Therefore, user interface 1104 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1104 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1104 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, the processing device 1100 may support remote access from other devices via communication interface 1102 or another physical interface (not shown). User interface 1104 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1104 can also be configured as a display device for rendering or displaying text fragments.

[0132] Processor 1106 may include one or more general-purpose processors and / or special-purpose processors.

[0133] Data storage 1108 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 1106. Data storage 1108 may include removable and non-removable components.

[0134] Processor 1106 is capable of executing program instructions 1118 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 1108 to perform the various functions described herein. Data storage 1108 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by the processing device 1100 of the project, enable the processing device 1100 of the project to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1118 by processor 1106 may result in processor 1106 using data 1112.

[0135] For example, program instructions 1118 may include an operating system 1122 (e.g., an operating system kernel, device drivers, and / or other modules) installed on the project's processing device 1100, and one or more applications 1120 (e.g., a browser, social application, or game application). Similarly, data 1112 may include operating system data 1116 and application data 1114. Operating system data 1116 is primarily accessible to the operating system 1122, while application data 1114 is primarily accessible to one or more applications 1120. Application data 1114 may reside in a file system that is visible or hidden from the user on the project's processing device 1100.

[0136] Application 1120 can communicate with operating system 1112 through one or more application programming interfaces (APIs). These APIs help application 1120 read and / or write application data 1114, transmit or receive information via communication interface 1102, receive or display information on user interface 1104, etc.

[0137] In some terminology, application 1120 may be simply referred to as "app". Furthermore, application 1120 can be downloaded to the project's processing device 1100 via one or more online app stores or app markets. However, the application can also be installed on the project's processing device 1100 in other ways, such as through a web browser or a physical interface on the project's processing device 1100 (e.g., a USB port).

[0138] Specifically, in this embodiment, the processing device 1100 of the project includes a data storage 1108 and one or more program instructions 1118, wherein one or more program instructions 1118 are stored in the data storage 1108, and one or more program instructions 1118 are configured to be executed by one or more processors. The one or more program instructions include computer-executable instructions for performing the following: Receive project requirement information from the user for the target project; Based on the project requirement information, intent parsing processing is performed on the target project to determine the logical description information for executing the target project; Based on the logical description information of the target project, the logical description information of the target project is converted into an execution script corresponding to the project requirement information through a preset domain-specific language, and the execution script corresponding to the project requirement information is compiled to obtain the corresponding device control instruction sequence. Based on the device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution of the operation commands.

[0139] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the processing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0140] This specification provides a project processing device that receives project requirement information input by a user for a target project. Based on this requirement information, it performs intent parsing to determine the logical description information of the target project. Then, based on this logical description information, it converts the logical description information into an execution script corresponding to the project requirement information using a preset domain-specific language. The execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution of the target project is determined based on the obtained results of the execution operation commands. As a result, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0141] Furthermore, based on the above Figures 1 to 6 This specification also provides a storage medium for storing computer-executable instruction information in one or more embodiments. In one specific embodiment, the storage medium may be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, it can realize the following process: Receive project requirement information from the user for the target project; Based on the project requirement information, intent parsing processing is performed on the target project to determine the logical description information for executing the target project; Based on the logical description information of the target project, the logical description information of the target project is converted into an execution script corresponding to the project requirement information through a preset domain-specific language, and the execution script corresponding to the project requirement information is compiled to obtain the corresponding device control instruction sequence. Based on the device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution of the operation commands.

[0142] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described storage medium embodiment is basically similar to the method embodiment, so the description is relatively simple; relevant parts can be referred to the description of the method embodiment.

[0143] This specification provides a storage medium that receives project requirement information for a target project input by a user. Based on this requirement information, it performs intent parsing to determine the logical description information of the target project. Then, based on this logical description information, it converts the logical description information into an execution script corresponding to the project requirement information using a preset domain-specific language. The execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution operation commands. As a result, the embodiments of this specification introduce a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of the embodiments of this specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the efficiency of overall process construction, compressing the originally lengthy training and construction cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0144] Furthermore, based on the above Figures 1 to 6 This specification also provides one or more embodiments of a computer program product, including a computer program, which, when executed by a processor, can perform the following processes: Receive project requirement information from the user for the target project; Based on the project requirement information, intent parsing processing is performed on the target project to determine the logical description information for executing the target project; Based on the logical description information of the target project, the logical description information of the target project is converted into an execution script corresponding to the project requirement information through a preset domain-specific language, and the execution script corresponding to the project requirement information is compiled to obtain the corresponding device control instruction sequence. Based on the device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution of the operation commands.

[0145] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described embodiment of a computer program product is relatively simple in description because it is fundamentally similar to the method embodiment; relevant parts can be referred to the description of the method embodiment.

[0146] This specification provides a computer program product that receives project requirement information for a target project from user input. Based on this requirement information, it performs intent parsing to determine the logical description information of the target project. Then, using a preset domain-specific language, it converts the logical description information into an execution script corresponding to the project requirement information. This execution script is then compiled to obtain a corresponding device control command sequence. Finally, based on this sequence, it sequentially sends corresponding operation commands to the terminal device corresponding to the target project and obtains the results of the terminal device executing the operation commands. The execution of the target project is then determined based on the obtained results of the execution operation commands. As a result, this embodiment of the specification introduces a revolutionary interaction and generation mode, namely, reshaping the entire process orchestration system through natural language understanding capabilities, aiming to achieve the goal of "speaking requirements and generating processes". The technical solution of this embodiment of the specification completely breaks down the technical barriers between "business-language requirements" and "precise configuration of underlying component attributes". The intelligent agent takes over the underlying details, and users do not need to care about variable transmission and exception capture. They only need to send project requirement information using everyday communication language. Based on the large model, corresponding algorithms and rules, a high component generation accuracy rate is achieved. This not only greatly reduces the cost of manual verification and error checking, but more importantly, it greatly improves the overall process building efficiency, compressing the originally lengthy training and building cycle to a very low level, thereby giving the automation platform high vitality and ease of use.

[0147] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous. Moreover, although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps and does not represent the only execution order. Therefore, when method steps are involved in the claims, adjustments to the order of those steps, or parallelism between steps, are also within the scope of protection of the claims.

[0148] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0149] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0150] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0151] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0152] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] The embodiments described herein are illustrated with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0156] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0157] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0158] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0159] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical or equivalent elements in the process, method, article, or apparatus that includes said element. Furthermore, "a," "an," and "the" are not specifically singular and may include plural forms. Ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish objects. For example, "first server" and "second server" usually refer to two servers, described as "first server" and "second server" to differentiate them; however, sometimes these two servers may be the same server. Moreover, in this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can be indirect receiving and sending (i.e., receiving and sending indirectly through one or more entities). Similarly, in this specification, unless otherwise stated, the relationships between structures can be direct or indirect.

[0160] Furthermore, the specific terms used in this specification to describe embodiments, such as "an embodiment," "one embodiment," or "some embodiments," refer to a particular feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. Moreover, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples, without contradiction.

[0161] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0163] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0164] The above description is merely an embodiment of this specification and is not intended to limit this document. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims in this document.

Claims

1. A method for processing a project, the method comprising: Receive project requirement information from the user for the target project; Based on the project requirement information, intent parsing processing is performed on the target project to determine the logical description information for executing the target project; Based on the logical description information of the target project, the logical description information of the target project is converted into an execution script corresponding to the project requirement information through a preset domain-specific language, and the execution script corresponding to the project requirement information is compiled to obtain the corresponding device control instruction sequence. Based on the device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution of the operation commands.

2. The method according to claim 1, wherein the step of performing intent parsing processing on the target project based on the project requirement information to determine the logical description information for executing the target project includes: Based on the project requirements information, the target project is decomposed to obtain the logical execution process of executing the target project. Based on the logical execution process of executing the target project, the logical execution process is converted into component logical description information and the reasoning chain corresponding to the component logical description information in accordance with the preset format specifications through a preset large model. Based on the component logic description information, determine the logic description information for executing the target project; The method further includes: Display the reasoning chain corresponding to the logical description information of the component.

3. The method according to claim 2, wherein converting the logical description information of the target project into an execution script corresponding to the project requirement information using a preset domain-specific language based on the logical description information of the target project includes: Based on the component logic description information corresponding to the logic description information of the target project, a corresponding execution logic tree is constructed using a preset domain-specific language; Based on the execution logic tree and the logical description information for executing the target project, an execution script corresponding to the project requirement information is generated.

4. The method according to claim 2, further comprising: Obtain images of multi-level pages using a multimodal large model; The logical execution process based on the execution of the target project is converted into component logical description information conforming to a preset format specification through a preset large model, including: Based on the logical execution process of the target project and the images of the multi-level pages obtained, the logical execution process is converted into component logical description information that conforms to the preset format specification through a preset large model.

5. The method according to claim 4, further comprising: Extract the data with anomalies from the images of the multi-level pages and / or the results of the executed operation commands, and obtain the data without anomalies corresponding to the data with anomalies; The abnormal data and the corresponding normal data are stored in a preset database. The abnormal data and the corresponding normal data stored in the database are used to fine-tune the preset large model.

6. The method according to claim 1, further comprising: The image of the page during the execution of the operation command corresponding to the device control instruction sequence by the terminal device is obtained through a multimodal large model, and the key areas in the page are determined based on the obtained page image. The multimodal large model generates additional operation commands for key areas in the page, sends the additional operation commands to the terminal device, and obtains the result of the terminal device executing the additional operation commands. Determining the execution result of the target project based on the obtained execution operation command includes: Based on the results of the acquired execution operation commands and the results of executing the additional operation commands, the execution result of the target project is determined.

7. The method according to claim 1, wherein the device control instruction sequence includes one or more device control instructions of different dimensions: logic flow control, multi-dimensional element search, gesture and data input, status perception, exception handling, and variable control, wherein, Device control instructions in the logic flow control dimension include one or more of the following: sequential execution instructions, parallel concurrent task execution instructions, conditional judgment instructions, and loop iteration instructions. Device control instructions in the multi-dimensional element search dimension include one or more of the following: instructions for locating the control hierarchy structure using a preset positioning tool, and instructions for locking interactive objects using recognized and displayed text. Device control instructions in the gesture and data input dimension include one or more of the following: click, long press, directional multi-level swipe, text box content input instructions, and focus wake-up instructions. Device control instructions in the state awareness dimension include one or more of the following: explicit wait instructions, implicit element loading wait instructions, and image capture instructions for compliance and evidence preservation. Device control instructions in the exception handling dimension include one or more of the following: instructions corresponding to the preset interruption tolerance mechanism for interference, exception self-healing instructions, and retry scheduling instructions. Device control instructions in the variable control dimension include one or more of the following: cross-component variable declaration and reference instructions, and cross-step variable declaration and reference instructions.

8. The method according to claim 1, wherein the target project is a page-down project of a preset application or an inspection project of a risk page.

9. An intelligent agent, the intelligent agent comprising an interaction layer, an intelligent agent processing layer, a DSL and compilation layer, an execution and device control layer, and an observation and data closed-loop layer, wherein: The interaction layer is configured to receive project requirement information for the target project input by the user, and to display the processing results of each layer in the intelligent agent. The intelligent agent processing layer is configured to perform intent parsing processing on the target project based on the project requirement information, and determine the logical description information of the target project. The DSL and compilation layer are configured to convert the logical description information of the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, based on the logical description information of the target project, and to compile the execution script corresponding to the project requirement information to obtain a corresponding device control instruction sequence. The execution and device control layer is configured to send corresponding operation commands to the terminal device corresponding to the target project in sequence based on the device control instruction sequence. The observation and data closed-loop layer is configured to obtain the result of the operation command corresponding to the sequence of device control instructions executed by the terminal device, and determine the execution result of the target project based on the obtained result of the execution operation command.

10. The agent according to claim 9, wherein the agent processing layer further comprises an agent orchestration layer and a large model layer, wherein: The intelligent agent orchestration layer is configured to decompose the target project based on the project requirement information to obtain the logical execution process for executing the target project. The large model layer is configured to convert the logical execution process of the target project into component logical description information and the reasoning chain corresponding to the component logical description information in accordance with the preset format specifications through the preset large model; and determine the logical description information of the target project based on the component logical description information. Accordingly, the interaction layer is configured to display the reasoning chain corresponding to the component's logical description information.

11. A processing apparatus for a project, the apparatus comprising: The requirement receiving module receives project requirement information for the target project from the user. The intent parsing module performs intent parsing processing on the target project based on the project requirement information to determine the logical description information for executing the target project; The conversion and compilation module, based on the logical description information of the target project, converts the logical description information of the target project into an execution script corresponding to the project requirement information using a preset domain-specific language, and compiles the execution script corresponding to the project requirement information to obtain the corresponding device control instruction sequence; The result determination module, based on the device control instruction sequence, sequentially sends corresponding operation commands to the terminal device corresponding to the target project, obtains the result of the terminal device executing the operation commands corresponding to the device control instruction sequence, and determines the execution result of the target project based on the obtained result of executing the operation commands.

12. A project processing device, the project processing device comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to: Receive project requirement information from the user for the target project; Based on the project requirement information, intent parsing processing is performed on the target project to determine the logical description information for executing the target project; Based on the logical description information of the target project, the logical description information of the target project is converted into an execution script corresponding to the project requirement information through a preset domain-specific language, and the execution script corresponding to the project requirement information is compiled to obtain the corresponding device control instruction sequence. Based on the device control command sequence, corresponding operation commands are sent sequentially to the terminal device corresponding to the target project, and the results of the terminal device executing the operation commands corresponding to the device control command sequence are obtained. The execution result of the target project is determined based on the obtained results of the execution of the operation commands.

Images