Task procedural management method based on screen analysis

Abstract

The invention provides a task procedural management method based on screen analysis, which comprises the following steps: a task management end sends task release information to a task execution end, the task release information carries a task stage division rule and index information of each stage, and the task execution end deploys a screen acquisition module and a task execution tool integration module; the task execution end collects screen content and task execution tool operation data and displays a record button in a task execution tool; in response to the trigger operation of the record button, the task execution end generates a stage achievement package, the stage achievement package comprises the current screenshot, task execution tool version data and an operation log, and the stage achievement package is uploaded to a processing center; and the processing center analyzes and processes the stage achievement package, generates visual progress information and a stage completion state, and pushes the visual progress information and the stage completion state to the task management end. According to the invention, real-time monitoring, accurate feedback and efficient cooperation of the whole task execution process can be realized.

Description

Technical Field

[0001] This invention relates to the field of information management technology, and in particular to a task process management method based on screen analysis. Background Technology

[0002] In the field of modern task management and execution, efficient collaboration and real-time monitoring are considered crucial pillars for improving work efficiency and quality. With the increasing complexity of tasks, ensuring transparency and timely feedback during execution has become a critical challenge for many industries. Research and practice in this area directly impact the smoothness of team collaboration and the achievement of task objectives, holding undeniable value for enterprise operations and project management. However, current task management and execution methods often have significant shortcomings. Many methods lack effective monitoring of interim results during task execution, making it difficult for managers to grasp the true progress of tasks. Simultaneously, feedback on problems encountered during execution is often delayed, and information asymmetry between different stakeholders prevents problems from being identified and resolved in a timely manner. This situation is particularly prominent in multi-party collaborative scenarios, severely impacting overall task efficiency.

[0003] A deeper technical challenge lies in the core factor of "real-time" execution. Due to the lack of immediate recording and feedback mechanisms for the execution process, the progress of each stage of the task is difficult to accurately capture and present. For example, in a design project requiring multi-person collaboration, after the executors complete a portion of the work, the manager cannot immediately know the content and quality of the specific deliverables, nor can they promptly identify potential deviations or errors in the execution. This lack of real-time capability not only leads to a delay in the manager's assessment of the task status but may also cause greater risks due to the accumulation of problems, further increasing communication costs in collaboration.

[0004] Therefore, how to achieve real-time recording and visualization of phased results throughout the entire task execution process, and ensure that problems can be quickly discovered and resolved, has become a key issue that urgently needs to be addressed in the field of task management and execution. Summary of the Invention

[0005] This invention provides a task process management method based on screen analysis, which aims to achieve real-time monitoring, accurate feedback and efficient collaboration of the entire task execution process, thereby improving the intelligence level and execution efficiency of task management.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides a task process management method based on screen analysis, applied to a task process management system. The task process management system includes a task management terminal and a task execution terminal communicatively connected to the task management terminal. The method includes:

[0008] The task management terminal sends task release information to the task execution terminal. This information includes task phase division rules and indicator information for each phase. The task execution terminal deploys a screen capture module and a task execution tool integration module. The task execution terminal captures screen content and task execution tool operation data, and displays a record button within the task execution tool. In response to the record button's activation, the task execution terminal generates a phase result package containing a current screenshot, task execution tool version data, and operation logs, and uploads the package to the processing center. The processing center analyzes and processes the phase result package, generating visual progress information and phase completion status, and pushes this information to the task management terminal. When the task execution terminal detects a problem on the screen, it automatically generates a question ticket containing a screenshot of the problem and contextual data, and pushes it to the task management terminal. The task management terminal adds annotations to the screenshot of the question ticket using an annotation tool and sends this information back to the task execution terminal.

[0009] Furthermore, the task execution terminal collects screen content and task execution tool operation data, including: the screen capture module captures screen pixel data and interface elements in real time, reduces data volume through compression processing, and performs fuzzy filtering on sensitive areas before local caching; the task execution tool integration module records file operations, code editing, and debugging commands, forming an operation log and associating it with the screen pixel data; the screen content and task execution tool operation data are fused into a continuous data stream, supporting real-time transmission to the processing center; the record button is displayed in the task execution tool interface and bound to a hook function to capture the screen content at the moment of triggering.

[0010] Furthermore, in response to the triggering operation of the record button, the task execution terminal generates a stage result package, including: the task execution tool integration module saves the current version data and generates a version identifier; the screen capture module captures a screenshot at the triggering moment and extracts key operation trajectories; the operation log is packaged with the screenshot and the task execution tool version data to form the stage result package, and a stage number is added based on the task stage division rules; the stage result package is uploaded to the processing center through an encrypted channel and can be stored after permission verification.

[0011] Furthermore, the processing center analyzes and processes the phased results package, including: using image recognition to determine the interface elements and operation trajectories in the screenshot; using a clustering algorithm to divide the current phase according to the operation log and task phase division rules and calculate the completion rate of each phase indicator information; generating the visualized progress information including progress bars, node status and quality data, and pushing it to the task management terminal through a real-time protocol; if the phase completion rate is lower than a preset threshold, an anomaly is marked and an alert is pushed to the task management terminal.

[0012] Furthermore, when the task execution terminal detects problem information on the screen, it automatically generates a question ticket, including: the screen capture module identifies error pop-ups or abnormal interface elements, captures error text and surrounding screen areas; the task execution tool integration module associates the current operation log and task stage division rules to generate context data; the question ticket encapsulates the problem screen screenshot, error text and context data, and pushes it to the task management terminal with a task identifier; the task management terminal receives the question ticket and displays it on the interface.

[0013] Furthermore, the task management terminal adds annotation information to the screenshot of the question work order using an annotation tool, including: the annotation tool loads the question screenshot, supports circling the question area and adding text descriptions; the annotation information is associated with the question work order, forming feedback data that is sent to the task execution terminal in real time through the communication module; the task execution terminal receives the annotation information and displays it overlaid on the current screen content; the feedback data is recorded in the operation log, supporting inclusion in subsequent stage result packages.

[0014] Furthermore, the processing center analyzes and processes the phased results package to generate visualized progress information, including: the visualized progress information integrates the phase completion status, indicator information of each phase, and operation trajectory charts; the task management terminal receives the pushed visualized progress information and renders it as a dashboard, supporting the viewing of screen operation replays; if no operation is detected for a long time, an abnormal status is generated based on the screen content and associated with the visualized progress information.

[0015] Furthermore, the task execution terminal deploys a screen capture module and a task execution tool integration module, including: the screen capture module is bound to the task execution tool after initialization, providing a data synchronization interface; the task execution tool integration module embeds the record button and hook function, supporting automatic saving of version data; the task release information activation module is associated with each other to form a data collection link.

[0016] The technical solutions provided by the embodiments of the present invention have the following beneficial effects:

[0017] This invention discloses an intelligent task processing method based on task management and execution collaboration. Through seamless collaboration between the task management end and the task execution end, it solves problems in business scenarios such as insufficient monitoring of stage results, delayed problem feedback, and cross-end information asymmetry during task execution. This invention integrates screen capture and tool operation data recording functions on the task execution end. Combined with a record button, it triggers the generation of stage result packages, which are then uploaded to the processing center for image recognition and progress analysis. Visualized progress information is generated and pushed to the management end in real time, ensuring transparency in task stage division and indicator completion. Simultaneously, for screen anomalies during execution, this invention automatically generates and pushes question tickets. The management end provides rapid feedback and guidance through annotation tools, forming a closed-loop mechanism from problem discovery to resolution. Ultimately, this invention achieves real-time monitoring, accurate feedback, and efficient collaboration throughout the entire task execution process, improving the intelligence level and execution efficiency of task management and providing reliable support for complex task scenarios. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of a task process management method based on screen analysis provided in an embodiment of the present invention.

[0019] Figure 2 This is another schematic diagram of a task process management method based on screen analysis provided in an embodiment of the present invention.

[0020] Figure 3 This is another schematic diagram of a task process management method based on screen analysis provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0022] like Figure 1-3 This invention provides a task process management method based on screen analysis, applied to a task process management system. The task process management system includes a task management terminal and a task execution terminal communicatively connected to the task management terminal. Specifically, the method may include:

[0023] S1, the task management terminal sends task release information to the task execution terminal. The task release information carries task phase division rules and indicator information for each phase. The task execution terminal is equipped with a screen capture module and a task execution tool integration module.

[0024] The task management terminal sends task release information to the task execution terminal. This information includes task phase division rules and indicator information for each phase. Upon receiving the information, the task execution terminal installs a screen capture module and a task execution tool integration module to capture screen operation data and operation records within the tool, generating an initial operation dataset. The screen capture module captures screen operations in real time, extracting interface changes and operation trajectories from the initial operation dataset to form an operation behavior sequence. This sequence records the time and specific content of each operation. The task execution tool integration module matches the operation behavior sequence with the operation logs within the tool, generating phase node markers for the matched data. These markers correspond to the division criteria in the task phase division rules, forming phased operation records. These phased operation records are further processed, comparing and verifying them against the indicator information for each phase. If a discrepancy is found between the operation record and the indicator information, an error message is generated. This error message assists the task execution terminal in adjusting its operation behavior to ensure compliance with the requirements in the task release information.

[0025] In one embodiment, the task management terminal, acting as a platform used by the requester, sends task release information to the task execution terminal, i.e., the capability provider's device. This information includes task phase division rules, such as division by functional modules, and indicator information for each phase, such as code quality scoring standards. After receiving the information, the task execution terminal installs a screen capture module and a task execution tool integration module. The former is a software component used to capture screen images and operations in real time, while the latter is a plugin embedded in the IDE environment used to record code editing logs. This generates an initial operation dataset, i.e., a preliminary set of screen data and logs. This approach ensures a transparent connection from task release to execution, improving collaboration efficiency because it allows the requester to preset rules and the execution terminal to respond immediately, forming the basis for real-time feedback.

[0026] For example, the screen capture module captures screen operations on the task execution end in real time through system services. It extracts interface changes such as window switching and operation trajectories such as mouse movement paths from the initial operation dataset to form an operation behavior sequence. This sequence records the time point and specific content of each operation, such as the time and location of clicking the save button. This transforms abstract operations into a quantifiable data stream, which is beneficial for subsequent analysis because it captures the dynamic details of the execution process, avoids the lag of manual reporting, and thus improves the visibility of task progress. It also supports multi-dimensional monitoring from multiple perspectives, such as time series and behavioral patterns.

[0027] In one embodiment, the task execution tool integration module matches the sequence of operation behaviors with the operation logs within the tool. For example, it matches the code submission actions on the screen with the submission records in the IDE logs. It generates stage node markers for the matched data. These markers correspond to the basis in the task stage division rules, such as the end mark of the coding stage, forming staged operation records. This integration can achieve fine process division. Its beneficial effect is that it bridges the gap between screen data and tool data, ensures the accuracy of stage markers, and supports each other from multiple directions, such as data fusion and rule application, to jointly improve the standardization of task management.

[0028] For example, the phased operation records are further processed, and the operation records are compared and verified in conjunction with the indicator information of each phase, such as the test pass rate. If any discrepancies are found, such as code quality being lower than the standard, an exception message is generated. This information is used to assist the task execution end in adjusting its behavior to ensure that it meets the requirements of the task release information. This can bring about the beneficial effect of dynamic quality control because it forms a closed-loop feedback from recording to verification. From the side, such as the exception detection and adjustment mechanism, they support each other to optimize task execution. It is also closely linked to the installation of modules after the task release information is sent to support the overall goal of phase rules and indicators.

[0029] S2, the task execution terminal collects screen content and task execution tool operation data, and displays a record button in the task execution tool.

[0030] S3, in response to the user's trigger operation on the record button, the task execution terminal generates a phase result package, which includes the current screenshot, task execution tool version data and operation log, and uploads the phase result package to the processing center.

[0031] A record button is embedded in the task execution interface. When the user triggers this button, the task execution terminal immediately captures the current screen image data, forming an initial screen snapshot. Simultaneously, it extracts the current version's runtime data and historical operation records from the task execution tool, forming a preliminary data set. For this preliminary data set, the task execution terminal compresses the initial screen snapshot to reduce storage space usage, and formats and organizes the runtime data and historical operation records to ensure data structure consistency, generating a phased result package containing three parts. After the phased result package is generated, the task execution terminal transmits it to the processing center through a pre-established encrypted channel, performing segmented verification during transmission to ensure transmission integrity. Upon receiving the phased result package, the processing center performs decryption and preliminary storage classification of the data content, archiving the screen snapshot, runtime data, and operation records separately for subsequent on-demand access, completing the phased result package upload process.

[0032] A record button is embedded in the task execution interface. When the user triggers the button, the task execution end immediately captures the image data of the current screen to form an initial screen snapshot. At the same time, it extracts the current version's running data and historical operation records from the task execution tool to form a preliminary data set.

[0033] Specifically, this embedding method can be achieved by adding a custom control in the integrated development environment. This control is connected to the operating system's screen capture interface. Once triggered, it calls the relevant function to obtain pixel-level image data and pulls runtime data and operation records such as code commit history from the tool's version control system, thereby ensuring data real-time performance and integrity. This approach can bring the beneficial effect of quickly responding to user operations and avoiding data loss.

[0034] For the initial dataset, the task execution end compresses the initial screen snapshot to reduce storage space usage, and formats and organizes the running data and historical operation records to ensure data structure consistency, generating a phase result package containing three parts.

[0035] In one possible implementation, compression is performed using lossless compression methods such as PNG format conversion, and pixel optimization of image data is performed to reduce file size. Formatting involves converting the running data into a JSON structure and adding timestamp tags to the operation records. This process helps improve the efficiency of subsequent data parsing and brings the beneficial effect of reducing transmission bandwidth consumption, supporting efficient cloud interaction.

[0036] After the phase deliverables package is generated, the task execution end transmits the deliverables package to the processing center through a pre-established encrypted channel, and performs segmented verification on the data during the transmission process to ensure the integrity of the transmission.

[0037] For example, the encrypted channel can be built on the TLS protocol, and the entire result packet can be protected by applying the AES encryption algorithm. The segmented verification divides the data into fixed-size blocks and calculates the hash value of each block, which is then verified one by one at the receiving end. This mechanism not only prevents data tampering, but also brings the beneficial effect of enhanced security, especially in multi-user collaborative environments where trust can be maintained.

[0038] After receiving the phase result package at the processing center, the decryption operation is performed and the data content is initially stored and classified. Screen snapshots, running data and operation records are archived separately for later access as needed, thus completing the upload process of the phase result package.

[0039] In one embodiment, the decryption operation uses the corresponding key to restore the data, and then the screen snapshot is stored in the image library, the running data is put into the version repository, and the operation record is put into the log table through the database index. This classified storage facilitates quick retrieval by the requester and brings the beneficial effect of improving audit efficiency and ensuring the traceability of the task process.

[0040] S4, the processing center analyzes and processes the phase result package, generates visual progress information and phase completion status, and pushes it to the task management terminal.

[0041] From the capability provider's operating environment, screen operation data and code editing content are collected in real time via an embedded lightweight plugin. This data includes changes in interface elements, operation trajectories, and code version update records, forming a raw operation dataset. For this raw operation dataset, pre-established image recognition rules are used to parse the screen operation data, extracting key action nodes, such as code submission or debugging execution, and combining this with code version update records to generate a phased results package. The key action nodes and code version information in the phased results package are structured and transformed into visual progress information using preset progress mapping rules. Simultaneously, the phase completion status is determined based on the completion status of the key action nodes. The visual progress information and phase completion status are transmitted to the task management terminal via a data push interface, ensuring that the task management terminal can obtain the capability provider's work progress and phased results in real time.

[0042] In one embodiment, screen operation data and code editing content are collected in real time from the capability provider's operating environment through an embedded lightweight plugin. This effectively captures dynamic changes. For example, the plugin runs as a background service, monitoring mouse clicks and keyboard input in the integrated development environment, thereby forming a raw operation dataset. This collection method ensures data integrity, provides a reliable foundation for subsequent analysis, and further enhances the transparency of the task process.

[0043] Specifically, a lightweight plugin is a software extension that does not consume too many system resources. It intercepts screen events through hook functions, such as the transition of interface elements from static to interactive states, and the recording of operation trajectories such as drag paths. At the same time, it captures code version update records, that is, the difference logs generated each time a save or commit is made. This can bring about the effect of real-time data accumulation, so that task execution no longer depends on manual reports, but automatically generates traceable original operation datasets, which is beneficial for the requesting party to understand the progress in a timely manner.

[0044] In one embodiment, the original operation dataset is parsed using pre-established image recognition rules to extract key action nodes. For example, the rules are based on optical character recognition technology to match specific patterns on the screen, such as the icon change when the code submission button is clicked, or the error message that pops up in the debug run window. Combined with code version update records, a phase result package is generated. This parsing process involves a predefined set of rules, such as classifying operation trajectories into valid action sequences, thereby forming a result package containing screenshots and logs. The beneficial effect of doing so is to standardize the process, avoid missing important nodes, and further support refined management.

[0045] Specifically, image recognition rules are preset pattern matching criteria. For example, code submission actions are identified by comparing pixels, that is, comparing the difference areas of the screen images before and after. If the differences are concentrated in the submission area, they are marked as key nodes. Then, they are integrated with code version update records, such as attaching update logs to node data to generate stage result packages. This helps to improve the accuracy of quality assessment because it provides multi-dimensional data support.

[0046] In one embodiment, the key action nodes and code version information in the phase deliverables package are structured and converted into visual progress information through preset progress mapping rules. This allows for simultaneous determination of the phase completion status. For example, the mapping rule is defined as converting the completion percentage of nodes into a progress bar display. If three out of five key nodes are completed, the progress is 60%. The phase completion is determined based on the node status, such as all nodes being successfully debugged and run. This provides intuitive feedback, allowing the client to quickly grasp the overall situation.

[0047] Specifically, structured organization refers to organizing scattered data into a hierarchical structure, such as arranging nodes in chronological order, marking change points in code version information, and then applying progress mapping rules, that is, a correspondence table that maps the number of nodes to percentage values. At the same time, the completion status of a stage is judged by checking whether the nodes meet preset conditions, such as the absence of error logs. This improves collaboration efficiency because visualized information reduces misunderstandings.

[0048] In one embodiment, visual progress information and stage completion status are transmitted to the task management terminal through a data push interface, which can ensure real-time acquisition of work progress. For example, the interface uses the WebSocket protocol to realize instant push. When the progress is updated, a packet containing progress bar data and status label is automatically sent. The beneficial effect of doing so is to enhance interactivity and support the immediate intervention of the requesting party.

[0049] Specifically, the data push interface is a standardized channel, such as an HTTP-based API endpoint. During transmission, it encrypts and visualizes progress information, i.e., data represented graphically, as well as stage completion status indicators such as "completed" or "in progress." This ensures that the task management end, such as a web dashboard, refreshes and displays the capability provider's work progress and stage results in real time. This helps advance the overall task because it bridges the gap between the execution end and the management end.

[0050] S5, when the task execution terminal detects problem information on the screen, it automatically generates a question ticket. The question ticket contains a screenshot of the problem screen and context data, and is pushed to the task management terminal. The task management terminal adds annotation information to the screenshot of the question ticket using an annotation tool and feeds it back to the task execution terminal.

[0051] When detecting screen content on the task execution end, screen image data and operation context information are captured in real time. If a problem prompt or abnormal pop-up is detected on the screen, the current screen image is saved as the first image data, and the relevant operation log is extracted as the first context record. Based on the first image data and the first context record, the task execution end automatically generates a query ticket file containing both parts and pushes this query ticket file to the task management end through a pre-established communication channel, awaiting a response from the management end. Upon receiving the query ticket file, the task management end parses the first image data and adds guiding marks to it using a preset annotation tool, forming the second image data, along with corresponding text descriptions as feedback records. The response file containing the second image data and feedback records is then sent back to the task execution end through the communication channel. The task execution end parses the response file and displays the marked content in the second image data to guide subsequent problem resolution.

[0052] When detecting screen content at the task execution end, screen image data and operation context information are captured in real time. If a problem prompt or abnormal pop-up is detected on the screen, the current screen image is saved as the first image data, and the relevant operation log is extracted as the first context record.

[0053] Specifically, this detection is achieved through a plugin integrated into the development environment that monitors screen pixel changes and text recognition. When a specific pattern, such as a code error message, is detected, a capture is triggered, which helps to pinpoint the problem instantly and prevent subsequent operations from overwriting critical details.

[0054] In one embodiment, when the capability provider encounters a debugging failure pop-up during the coding process, the plugin immediately saves the image of the pop-up area as the first image data, and simultaneously records the keyboard input logs from the previous few minutes as the first context record. This provides a complete problem background, facilitating subsequent analysis and rapid response, and improving collaboration efficiency.

[0055] For the first image data and the first context record, the task execution end automatically generates a question ticket file containing these two parts, and pushes the question ticket file to the task management end through a pre-established communication channel, waiting for the management end to respond and process it.

[0056] For example, when generating a work order, the system embeds the first image data into the file body and converts the first context record into a structured text attachment. This method ensures that the data is transmitted completely without losing details, which helps the requester to accurately understand the origin of the problem.

[0057] In one embodiment, if the first context record shows repeated modifications to the same line of code, the ticket file will automatically mark this pattern as a potential loop error and push it through an encrypted instant messaging channel. This not only accelerates the feedback loop but also enhances data security and supports seamless remote collaboration.

[0058] After receiving the question ticket file in the task management terminal, the first image data is parsed, and guidance marks are added to the image data using the preset annotation tool to form the second image data. At the same time, the corresponding text description is attached as a feedback record.

[0059] Specifically, the annotation tool is a graphical interface-based editor that allows users to highlight problem areas with arrows or circles, creating a second image data. This process helps to intuitively convey the solution and avoids ambiguity in textual descriptions.

[0060] In one embodiment, the requester views the first image data in the work order, discovers the missing interface call, and uses an annotation tool to draw the correction path on the image and add text descriptions such as adding specific function calls. The resulting second image data and feedback records can directly guide the capability provider to make modifications, improving the accuracy and speed of problem solving.

[0061] The response file, containing the second image data and feedback records, is sent back to the task execution end via the communication channel. The task execution end parses the response file and displays the marked content in the second image data to guide the solution of subsequent problems.

[0062] For example, the response file is compressed during transmission to reduce latency, and the task execution end parses it and displays the markers on the screen. This helps the capability provider to immediately apply the guidance and achieve closed-loop management.

[0063] In one embodiment, if the feedback log includes suggestions for incremental modifications, the executor will compare and display the second image data with the original screen, highlighting the changed areas. This approach not only enhances understanding but also promotes the continuous progress of the task, ensuring smooth collaboration and quality control.

[0064] S21, the task execution terminal collects screen content and task execution tool operation data, including real-time capture of screen pixel data and file operations and code editing records in the task execution tool, and caches them locally after compression and sensitive information filtering.

[0065] The task execution client uses a built-in screen capture tool to acquire screen pixel content in real time, and simultaneously extracts file operation records and code editing records from the task execution tool to form a raw operation dataset. The raw operation dataset contains changes in the screen display and specific operational details within the tool, completely recording the dynamic process of task execution. For the raw operation dataset, the task execution client uses preset compression rules to reduce the data volume of the screen pixel content and operation records, generating a compressed operation dataset. The compressed operation dataset reduces storage space usage while maintaining the integrity of key information, facilitating subsequent processing. Based on the compressed operation dataset, the task execution client uses preset sensitive information filtering rules to identify and block content involving privacy or irrelevant tasks, forming a filtered operation dataset. The filtered operation dataset ensures data security and complies with the compliance requirements of task execution. The filtered operation dataset is stored in the task execution client's local cache space, with access permissions set to restrict unauthorized viewing or modification. The local cache space periodically verifies the stored content to ensure data integrity, providing reliable screen content and operation data support for subsequent interactions between the task execution client and the requesting party.

[0066] In one possible implementation, the screen capture tool built into the task execution end can capture the screen pixel content in real time, such as capturing changes in the code editing interface during development tasks. This ensures that dynamic updates of interface elements are captured, thereby providing complete basic data for subsequent data processing.

[0067] It's important to note that such capture tools are typically implemented using APIs provided by the operating system. For example, they might use hook functions to monitor screen refresh events, capturing pixel data while simultaneously extracting file opening and saving records from task execution tools such as IDEs, forming a raw operation dataset. This dataset includes error message pop-ups on the screen and details of code insertion and deletion within the tool, providing a complete record of the dynamic process of task execution. This helps improve the accuracy of data collection because real-time capture avoids human error and ensures that all operational traces are included in the analysis.

[0068] In one possible implementation, when reducing the data volume of the original operation dataset, the preset compression rules can adopt lossless compression methods, such as applying pixel difference-based compression to the screen pixel content and merging and storing the unchanging parts in consecutive frames. In this way, the compressed operation dataset can significantly reduce the data volume while maintaining the integrity of key information, which is beneficial to reducing the transmission burden and accelerating the subsequent filtering process.

[0069] Specifically, this compression rule involves comparing screen content frame by frame, identifying static areas and recording only the changed parts, and merging duplicate file operation logs for operation records, thereby preparing a more efficient data format for local caching. The purpose of this processing is to optimize the use of storage resources and avoid data bloat from affecting device performance.

[0070] In one possible implementation, the compressed operation dataset is processed using pre-defined sensitive information filtering rules. For example, the rules could be defined as scanning the text content in the dataset, identifying and obfuscating information such as personal information or non-task-related window content, thus forming a filtered operation dataset. This filtering rule is often implemented by matching sensitive keywords using regular expressions and applying masking techniques at the pixel level to block irrelevant areas. This helps maintain data privacy because it prevents potential information leaks, ensures that only task-related screen content and operation data are retained, complies with compliance requirements, and enhances system trust.

[0071] In one possible implementation, when storing the filtered operation dataset in a local cache space, role-based access permissions can be set, such as allowing only internal modules of the task execution end to read it. At the same time, the local cache space confirms that the data has not been tampered with through periodic hash verification. This helps to ensure data integrity and provides reliable support for the interaction between the task execution end and the requesting party.

[0072] Specifically, this storage mechanism involves encrypting and saving the dataset in blocks, and comparing the current hash value with the initial value during verification. If they do not match, an alarm is triggered, thereby ensuring the stability of screen content and operation data during the interaction process and avoiding collaboration interruptions due to data corruption.

[0073] S22, the record button is associated with the hook function of the task execution tool. When triggered, it automatically saves the current version data, captures the screen content, and associates it with the operation log to form the number identifier of the stage result package.

[0074] A record button is embedded in the task execution tool and associated with a pre-defined hook function. When the button is triggered, the hook function immediately captures the current task execution environment data, including the current version of the code and the screen display content, forming a preliminary data snapshot. Based on this preliminary snapshot, the hook function further calls the logging module to bind the operation records during task execution to the data snapshot, ensuring that the timestamp and details of each operation are completely saved, generating a set of operation records with timestamps. Key information is extracted from the operation record set, and a unique stage result identifier is generated for each set of data snapshots and operation records according to a pre-defined numbering rule. This information is packaged into an encrypted result package and stored in a designated location for subsequent access. Access control is set for the generated stage result packages to ensure that only authorized users can access the corresponding numbered identifier content. Access logs are also recorded to track the usage of the result packages, ensuring the integrity and security of the task execution data, closely adhering to the goal of generating stage result package number identifiers upon triggering the record button.

[0075] In one possible implementation, task execution tools such as integrated development environments can embed a record button as a plugin. This button is associated with a hook function to capture data. This ensures that task nodes are recorded in real time, avoids delays caused by manual intervention, and helps improve the continuity of process management.

[0076] Specifically, when a developer triggers a button while coding the user login module, the hook function captures the current code version, such as function definitions and variable declarations, as well as screen content, such as the lines of code displayed in the editor window, thus forming a preliminary data snapshot. This helps to quickly trace back the execution status during subsequent audits and provides a more reliable task tracing effect.

[0077] For example, processing the initial data snapshot involves binding the logging module, such as adding operation records like timestamps of typing specific codes and details of content modifications to the snapshot, generating a collection with time stamps. This binding can enhance data integrity, allowing a clear view of the evolution from initial editing to final saving when reviewing tasks, thereby improving the efficiency of problem diagnosis in collaboration.

[0078] In one possible implementation, key information is extracted from the set of operation records, and a unique identifier is generated by applying a preset numbering rule. For example, the code version in the snapshot and the operation sequence in the log are combined into a stage result identifier such as S001, and packaged into an encrypted result package and stored in the cloud. This not only ensures data security, but also facilitates access by the requesting party as needed, and improves the convenience of sharing task results.

[0079] For example, access control can be set for the generated stage deliverable packages. For instance, a role-based access mechanism can be used to ensure that only the project manager can access the numbered content, while access logs such as time and user ID are recorded. This helps to track usage, prevent unauthorized disclosure, and closely aligns with the goal of forming the stage deliverable package number, thereby enhancing the overall security and traceability of task execution.

[0080] In one possible implementation, this embedding and association mechanism can capture data layer by layer from button triggers in real-world development scenarios, such as building web applications. This ensures seamless integration of snapshots and logs. For example, when a developer debugs code and encounters an error, the screen content captured by the hook function includes the error message window, while the timestamp bound to the log records the modifications made before the error. The extracted identifiers make the output package an auditable unit, and access control further protects these details from misuse, which is beneficial for maintaining task privacy and promoting efficient collaboration.

[0081] For example, further examining the various aspects of this process, such as in mobile app development, the snapshot formed after a button is triggered may include the current view of the interface design tool. Key information extracted after binding logs, such as the number of times UI elements were adjusted, can be packaged and stored by number. This can support auditing requirements from a security perspective. At the same time, the logs recorded by access control can be used to analyze access patterns, thus mutually reinforcing the reliability and effectiveness of the entire task management.

[0082] One possible implementation considers the integration of different tools, such as embedding buttons in Visual Studio, capturing code version data from hook functions to create snapshots of screen content, generating a collection of operation records bound to calling modules, and then extracting and packaging the results package with access control. This ensures the continuity between steps in terms of implementation methodology, which is beneficial for achieving real-time monitoring and improving the accuracy of quality assessment.

[0083] For example, taking another approach, in the Eclipse environment, when the record button is triggered, the hook function captures environmental data to form a snapshot, binds the log, extracts information to generate an identifiable result package, and storage and permission settings ensure security. These aspects together support the standardized management of the task process, bringing greater process transparency and collaboration efficiency.

[0084] S31, the processing center analyzes and processes the phased results package, including extracting interface elements and operation trajectories using image recognition, dividing the task into phases and calculating the completion rate of indicators through clustering algorithms, and generating progress charts and early warning information.

[0085] Screen recordings and operation logs are retrieved from the capability provider's operation records. Image recognition technology is used to extract interface elements and operation trajectories from the recordings, forming an initial operation data set. This data is then organized into an ordered operation sequence according to time. The ordered operation sequence is grouped, categorizing adjacent operation trajectories and interface element changes with similar characteristics into the same task stage, resulting in stage division results. Completion data for each stage is calculated based on preset indicator rules. Completion data for each stage is extracted from the stage division results and combined with time information from the operation sequence to generate a visual progress chart. Corresponding warning messages are generated for stages with completion rates below a preset threshold. The generated progress charts and warning messages are integrated into an analysis report of the stage outcome package, which is pushed to the requesting party via cloud storage, ensuring that the processing center's analysis of the stage outcome package fully presents the progress and potential problems of each task stage.

[0086] When obtaining screen recording clips and operation logs from the capability provider's operation records, this data can be collected first through a local caching mechanism. For example, during development tasks, when the capability provider uses IDE tools to write code, the system automatically records the changes in the editing window and the mouse click path that appear on the screen. This ensures the real-time and completeness of data collection, which is beneficial to the accurate basis of subsequent analysis.

[0087] In one possible implementation, image recognition technology is used to extract interface elements and operation trajectories from video clips to form an initial operation data set. For example, the text input in the code editing area and the click trajectory of the debug button can be identified. These elements are labeled as timestamp sequences, which helps to capture the dynamic details of task execution, avoid missing key operations, and thus improve the transparency of process management.

[0088] For example, after organizing these data into an ordered sequence of operations in chronological order, the data can be grouped for processing within the ordered sequence of operations. For instance, continuous code writing actions can be categorized as the coding phase, while repeated attempts at debugging and running can be categorized as the testing phase. This grouping helps to clearly define the task flow, reduce manual intervention, and improve efficiency.

[0089] In one possible implementation, adjacent operation trajectories and interface element changes with similar characteristics in the operation sequence are classified into the same task stage to form stage division results. For example, similar characteristics include mouse hover actions in the same area or error message changes in interface colors. These results are helpful in quantifying task progress and avoiding subjective judgment bias.

[0090] For example, the completion rate data for each stage is calculated according to preset indicator rules. For instance, the rule is defined as a high completion rate when the number of lines of code reaches the expected proportion. This calculation is beneficial for objectively evaluating quality and timely identifying lagging stages.

[0091] In one possible implementation, completion data for each stage is extracted from the stage division results and combined with time information in the operation sequence to generate a visual progress chart. For example, the chart shows that the coding stage is represented by blue bars indicating 80% completion and the testing stage by red bars indicating 50% completion. This helps the client to intuitively monitor the overall progress and facilitates rapid decision-making.

[0092] For example, the system can generate corresponding warning information for stages where the completion rate is lower than a preset threshold. For instance, when the completion rate of a test stage is lower than 60%, the system can issue a notification describing potential debugging bottlenecks. Such warnings are beneficial for early intervention and reduce the risk of task delays.

[0093] In one possible implementation, the generated progress charts and early warning information are integrated into an analysis report of the phase deliverables package, which is then pushed to the requesting party via a cloud storage path. For example, the report is packaged in PDF format with charts and text descriptions, and the requesting party can view it immediately after it is pushed, which is beneficial for achieving seamless remote collaboration.

[0094] For example, ensuring that the processing center's analysis and processing of the phased deliverables can fully present the progress and potential problems of the task phase, such as in practical applications, where the requester can identify the efficiency of the coding phase and the inefficiency of the testing phase through the report, thereby adjusting resource allocation, such a complete presentation is beneficial to improving the overall quality and efficiency of task collaboration.

[0095] S32, the visualized progress information includes stage progress bars, key node status and quality assessment data, which are pushed to the task management terminal via a real-time protocol for viewing and interaction.

[0096] The system captures screen content in real-time from the capability provider's interface, obtaining operation trajectories and code editing records. Image recognition is used to determine if key actions, such as code commits or file saves, have occurred. The time points and related operational content of these key actions are then compiled into preliminary operation sequence data. This preliminary operation sequence data is divided into stages based on pre-established stage division rules, categorizing the data into different task stages, such as requirements analysis or code implementation. Corresponding progress bar data and key node status information are generated for each stage. The segmented stage progress bar data and key node status information are integrated with code quality assessment data, extracting compliance information from code editing records to form comprehensive quality assessment data. This data is then compiled into visualized progress information. This visualized progress information is pushed to the task management terminal via a real-time communication protocol, displaying the stage progress bar, key node status, and quality assessment data on the task management terminal interface, ensuring that the management terminal can view and interact with the visualized progress information in real time.

[0097] In one possible implementation, the process of capturing screen content in real time from the capability provider's user interface can be accomplished using a lightweight capture tool installed on the terminal device. This tool periodically captures screen images and mouse and keyboard input logs to obtain operation trajectories and code editing records.

[0098] For example, when a capability provider enters code in an integrated development environment, the tool records the coordinate changes of each key press and cursor movement. It uses image recognition to determine whether key actions have occurred, such as identifying the moment the code submission button is clicked or the moment the file save dialog box appears. The tool then organizes the time points of these key actions and related operations into preliminary operation sequence data. This ensures that every subtle change in task execution is captured, which is beneficial for subsequent accurate analysis and tracking, and improves the transparency of process management.

[0099] For example, when dividing the initial operation sequence data into stages, pre-established stage division rules can be used. These rules can be defined based on operation type and time sequence. For instance, continuous code viewing and modification operations can be classified as the coding implementation stage, while debugging and error correction can be classified as the testing and debugging stage. Corresponding progress bar data and key node status information are generated for each stage. This division helps to transform messy operation data into a structured task flow, which is beneficial for the requester to quickly understand the task progress, avoids the delay and subjective bias of traditional manual reporting, and thus improves collaboration efficiency.

[0100] In one possible implementation, the process of integrating the segmented progress bar data and key node status information with code quality assessment data involves extracting compliance information from code editing records, such as checking whether the code conforms to preset naming conventions or avoids common syntax errors, forming comprehensive quality assessment data, and organizing this data into visualized progress information. This integration can provide multi-dimensional task insights, which is beneficial for early detection of potential problems, such as rework that may be caused by poor code quality, thereby ensuring the overall quality and timeliness of the task.

[0101] For example, the compiled and visualized progress information is pushed to the task management terminal via a real-time communication protocol. The task management terminal interface displays the stage progress bar, key node status, and quality assessment data, ensuring that the management terminal can view and interact with the visualized progress information in real time. For example, the requester can click on the progress bar to view detailed operation logs or add feedback comments directly on the interface. This push and interaction mechanism enables remote real-time supervision, which is beneficial for quickly responding to questions during execution, promoting the timeliness of problem solving, and ultimately supporting the viewing and interaction functions of the task management terminal.

[0102] S41, the task execution terminal detects problem information on the screen, including identifying error pop-ups or abnormal operations, generates the question work order by capturing error text and surrounding screen area, and pushes it in real time.

[0103] During screen operation, the task execution terminal continuously monitors interface changes, capturing pop-up content or operational anomalies, and obtaining error text and surrounding image areas on the screen to form a preliminary problem data package. Key fields of the error text are extracted from this preliminary problem data package and combined with interface elements from the surrounding image area to generate a structured query ticket, ensuring the problem description includes specific error information and contextual images. For the generated structured query ticket, the ticket content is instantly pushed to the corresponding receiving end through a pre-established communication channel, ensuring problem information is transmitted in the shortest possible time. Simultaneously with pushing the structured query ticket, the original record of the problem data package is saved for subsequent traceability and verification, ensuring that the problem information detected by the task execution terminal is completely archived to support the query ticket processing flow.

[0104] In one embodiment, the task execution end continuously monitors screen operations through a plugin installed in the integrated development environment. This monitoring relies on the screen capture interface provided by the operating system to detect interface changes. For example, when an error pop-up appears during code debugging, the plugin immediately identifies the pop-up's boundaries and captures its text content, such as "Syntax error: Undefined variable," while simultaneously capturing the surrounding image area, including code lines and cursor positions, forming a preliminary problem data packet. This approach helps to detect anomalies early, thereby accelerating problem localization and avoiding collaboration bottlenecks caused by task delays. Specifically, this preliminary problem data packet is directly used as input for subsequent extraction processes. For example, when developing web applications, if an operational anomaly causes the browser console to display a network connection failure, the data packet will contain the error text "Connection timed out" and a screenshot of the surrounding network configuration. This processing ensures data integrity and provides a reliable basis for error tickets, improving the overall transparency and response speed of task execution. In one embodiment, extracting key fields from the initial problem data packet's error text can be achieved using optical character recognition (OCR) technology. This involves identifying error codes and descriptive phrases within the text, and then combining these with surrounding interface elements such as button icons or menu bars to generate a structured error ticket. The problem description integrates the specific error message "memory overflow" with contextual images such as stack trace screenshots. This generation process makes the ticket more readable and targeted, helping stakeholders quickly understand the essence of the problem and providing precise guidance, thus reducing communication misunderstandings. For example, in a mobile application testing scenario, if the data packet captures a crash pop-up in the simulator, the generated ticket after extracting key fields will include the error text "null pointer exception" and a device status image of the surrounding area. This approach offers technical benefits such as improved problem-solving efficiency because the structured format facilitates tracking and analysis, supporting the immediacy of subsequent push notifications. Specifically, for the generated structured query tickets, a pre-established communication channel, such as a WebSocket-based real-time messaging service, is used for instant push notifications. For example, the ticket content is encoded in JSON format and sent to the requester's dashboard via the channel, ensuring that the issue information is transmitted within seconds. This push mechanism achieves zero-latency feedback, allowing the requester to respond immediately and maintain smooth task progress, avoiding the lag issues of traditional email methods. In one embodiment, while pushing the structured query ticket, the original record of the issue data packet is saved to a cloud storage service for subsequent traceability and verification. For example, original screenshots and text logs including timestamps are saved. This not only ensures that the issue information detected by the task execution end is completely archived, but also supports the query ticket processing flow, such as audit verification, which is beneficial for long-term quality control and collaborative optimization.For example, when reviewing historical tasks, the saved records can reproduce the original scene, such as the moment when a specific error pop-up appeared, thereby verifying the accuracy of the work order processing. This method enhances the reliability and traceability of the system, ultimately supporting the technical objective of "S41, the task execution end detects problem information on the screen, including identifying error pop-ups or abnormal operations, generating the question work order by capturing error text and surrounding screen areas, and pushing it out in real time."

[0105] S42, after receiving the question work order, the task management terminal uses the screen annotation tool to circle the problem area, add text descriptions, and feeds back the annotation results to the task execution terminal for display on the screen in real time through the communication module.

[0106] When the task management terminal receives a query ticket from the task execution terminal, it retrieves the screenshot data and error description information contained in the ticket. Using a built-in image processing tool, it divides the screenshot into regions to identify potentially problematic interface areas. For these regions, the task management terminal loads a screen annotation tool, allowing users to draw markers and add text descriptions within designated areas, creating an annotated image. The annotated image and text descriptions are encoded via a communication module to generate a real-time feedback data packet, which is sent to the task execution terminal, ensuring the packet can be decoded and displayed. Upon receiving the feedback data packet, the task execution terminal uses an interface rendering component to overlay the annotated image and text descriptions onto the current screen, providing a clear and intuitive presentation of the problem area annotations and descriptions in the query ticket.

[0107] For example, when the task management terminal receives a query ticket from the task execution terminal, it first retrieves the screenshot data and error description information from the ticket. This retrieval process is achieved by parsing the structured format of the ticket; for example, the ticket may contain a JPEG screenshot file and a JSON error text file, thus ensuring data integrity. Next, the built-in image processing tool is used to divide the screenshot into regions. This tool, based on a pixel-level segmentation method, divides the screenshot into multiple rectangular areas, such as identifying the code editing area, error pop-up area, and menu bar area. This segmentation helps to quickly locate the core of the problem and improves the accuracy of feedback because it avoids the inefficiency of manually searching the entire screenshot.

[0108] In one possible implementation, for the divided interface areas, the task management terminal loads a screen annotation tool. This tool is a Canvas-based drawing component that allows users to draw markers such as circles or arrows within a specified area and add text descriptions. For example, drawing a red circle in the code editing area to mark a missing variable declaration, while simultaneously entering the text "Type checking needs to be added here," forms the annotated image content. The purpose of this approach is to visualize the problem, making subsequent feedback easier to understand. The beneficial effects are reduced misunderstandings and faster problem-solving.

[0109] For example, the labeled image content and text description are encoded through a communication module that uses the WebSocket protocol for real-time transmission. First, the image and text are merged into a PNG file, then Base64 encoding is applied to generate a data packet, which is sent to the task execution end to ensure that the data packet can be decoded and displayed at the receiving end. This encoding process ensures the security and integrity of data during network transmission, helps prevent information loss or tampering, and thus maintains the reliability of collaboration.

[0110] In one possible implementation, after receiving the feedback data packet, the task execution end uses a UI rendering component to overlay the annotated image content and text description onto the current screen. This rendering component utilizes HTML5 overlay technology to display the markings on top of the existing screen. For example, a semi-transparent layer pops up on the developer's IDE interface to display circled annotations and text prompts, providing a clear presentation of the question area annotations and descriptions on the work order. The purpose of this overlay display is to provide immediate visual feedback. The beneficial effect is that it allows the executor to immediately see the problem and make corrections, thereby closing the entire interaction process and improving task collaboration efficiency.

[0111] For example, from another perspective, in practical applications, if a question ticket involves a debugging failure scenario, the task management terminal can further analyze the error description information when obtaining screenshots, such as identifying the keyword "NullPointerException". Then, in the area division, the stack trace area is highlighted first. This is connected with the loading of the annotation tool, because specific guidance text can be added to this area during annotation, such as "check variable initialization". This multi-faceted support ensures the logical coherence from data acquisition to final display, which is beneficial for the rapid diagnosis of complex problems.

[0112] In one possible implementation, considering privacy protection, the communication module can embed an encryption layer when encoding data packets, using the AES algorithm to encrypt sensitive screenshots, and only decrypt and display them at the task execution end. This complements the overlay function of the rendering component, because the annotated content can only be securely presented after decryption. The purpose of this design is to balance real-time feedback and data security. The beneficial effects are to enhance user trust and meet compliance requirements, while also supporting the robustness of the overall system.

[0113] For example, extending to multi-user collaboration scenarios, when the task management terminal processes multiple work orders, it can divide the regions in parallel and generate data packets. For instance, it can process screenshots from two execution terminals simultaneously, one labeled as having a missing interface and the other as having a logical error. These screenshots are then sent in batches through the communication module, ensuring that each task execution terminal sees its own overlay display in real time. This example of mutual support demonstrates the system's scalability and is beneficial for improving efficiency in large-scale task management.

[0114] S211, the screen acquisition module is responsible for pixel data capture and compression, and the task execution tool integration module provides button interface and version saving function. The two are synchronously associated to form a data link.

[0115] The screen capture module captures screen pixel data in real time using a lightweight plugin. The captured pixel data undergoes preliminary compression to form a compressed screen data stream for subsequent transmission and parsing. This compressed screen data stream is transmitted to the task execution tool integration module. This module records the current operation state through a preset button interface and associates the screen data stream with code version information, forming a preliminary data link packet. This preliminary data link packet further utilizes the version saving function of the task execution tool integration module to synchronously store the screen data stream and code version information, ensuring the integrity and traceability of the data link. The stored data link packet undergoes an internal verification mechanism to confirm the matching degree between the screen data stream and the version information. For data link packets with a matching degree meeting a preset threshold, final synchronization and association are completed, forming a complete data link that can be used for subsequent calls.

[0116] For example, in the process of the screen capture module capturing screen pixel data in real time through a lightweight plugin, the lightweight plugin is an embedded software component that is installed in the operating system of the capability provider and can continuously monitor screen changes without significantly affecting system performance.

[0117] Specifically, this plugin utilizes the operating system's API to access the graphics buffer, thereby capturing pixel data for each frame. For example, when the capability provider is editing code using an IDE, the plugin captures screen images several times per second, ensuring the dynamic process of code editing is captured. This provides the advantage of strong real-time performance, as it allows subsequent modules to immediately obtain fresh data, avoiding inaccurate task monitoring due to latency. The captured pixel data undergoes initial compression processing by applying lossless compression algorithms, such as variants of the PNG format, to reduce the size of the original pixel data, for example, from several megabytes to several hundred kilobytes. This facilitates network transmission without losing critical details, such as the clarity of code lines and error messages. This compression improves data transmission efficiency, ensuring rapid delivery to the next module even in bandwidth-constrained environments, forming a foundation for efficient data links.

[0118] In one possible implementation, the compressed screen data stream is transmitted to the task execution tool integration module. This module records the current operation status through a preset button interface. This preset button interface is a user interface element integrated into the IDE toolbar. When the capability provider clicks it, a recording event is triggered. For example, clicking the button during the coding phase will automatically capture the current timestamp and operation type, such as "code save." The screen data stream is then associated and bound with code version information to form a preliminary data link package. This association is achieved through metadata tags, such as adding a commit ID tag from the code repository to the data stream, thus linking the screenshot to a specific code version. The benefits of this approach include enhanced data context relevance, facilitating understanding of the operation background when reviewing data, avoiding misunderstandings caused by isolated data, and providing a solid foundation for subsequent storage.

[0119] For example, when the initial data link package is further processed through the version saving function of the task execution tool integration module, the version saving function is a built-in storage mechanism that synchronously stores the screen data stream and code version information in a cloud database, such as using a distributed file system like HDFS to store the package files, ensuring consistency across multiple devices. This synchronous storage ensures data integrity and traceability because it creates a time-series record chain. For instance, if a problem is discovered later, it's possible to trace back to a specific package to view the screen state at that time, thereby improving the reliability of task collaboration and directly supporting the next verification step.

[0120] In one possible implementation, when the stored data link packet verifies the matching degree between the screen data stream and version information through an internal verification mechanism, this internal verification mechanism is a hash function-based verification process. It calculates the hash value of the data stream and compares it with the hash value of the code version. If they match, the consistency is confirmed. For example, in the matching degree calculation, a similarity score such as cosine similarity is used for quantification. If the score is higher than a preset threshold, such as 0.9, it is considered valid. The beneficial effect of this mechanism is to prevent data tampering or transmission errors and ensure the reliability of the final synchronization association. For data link packets whose matching degree meets the preset threshold, the final synchronization association is completed, forming a complete data link that can be called subsequently. This allows the requester to call the link data through the dashboard, realizing real-time closed-loop processing of the task process. For example, when feedback interface documentation comments are missing, they can be directly supplemented and modified based on the link packet, thereby improving the overall collaboration efficiency.

[0121] S221, the phase result package is encrypted and stored in the processing center, supports downloading according to permissions, and verifies its integrity by matching with the task phase rules.

[0122] The system retrieves interim results data from the capability provider's operation logs, integrating screen recording clips, code versions, and operation logs into an initial results set. This set is then structured and organized into a unified format data packet suitable for encryption. This unified format data packet is protected using a pre-defined encryption mechanism and stored in the processing center's cloud database, ensuring unauthorized access during storage. In the processing center's cloud database, access control rules are configured for the encrypted data packets. Access permissions are determined based on the task participant's identity information, and this permission information is associated with the task stage rules to verify the data packet's integrity. When a task participant requests to download the encrypted data packet, the corresponding permission information and task stage rules are retrieved from the cloud database, and the request is matched and verified. If the conditions are met, download is allowed, ensuring the secure distribution of the interim results package and aligning with the objective.

[0123] In one embodiment, interim results data are obtained from the operation records of the capability provider. For example, screen recordings are used as visual evidence, code versions are used as development products, and operation logs are used as behavior tracking records. These data are integrated to form an initial results set. This integration can unify different types of information sources and is beneficial to the consistency of subsequent processing. Then, the set is structured and organized, for example, by sorting the video recordings by timestamps, marking the change points of the code versions, and classifying the logs by event types, to form a unified format data packet that can be encrypted. This can bring about the standardization effect of data processing and improve the reliability of overall collaboration.

[0124] The standardized data packets are protected using a pre-defined encryption mechanism, such as symmetric encryption methods like Advanced Encryption Standard (AES) to encrypt the data packet content block by block, ensuring the security of transmission and storage. The processed encrypted data packets are then stored in the cloud database of the processing center. This storage method prevents data leakage, helps maintain task privacy, and ensures that data is not accessed without authorization during storage. This provides long-term data protection and supports the sustainable management of tasks.

[0125] In the cloud database of the processing center, access control rules are configured for encrypted data packets. For example, access permissions are determined based on the identity information of task participants, such as role tags. Requesters are set to read-write and providers to read-only. This configuration refines access control and helps prevent abuse. Then, the permission information is associated with task stage rules, such as matching stage completion flags to verify the integrity of data packets. This automates the verification process and improves the accuracy and trustworthiness of the task flow.

[0126] When a task participant requests to download an encrypted data packet, the corresponding permission information and task stage rules are extracted from the cloud database to match and verify the request. For example, the system checks whether the requester's identity meets the permission rules and whether the data packet matches the current task stage. If the conditions are met, the download is allowed. This verification ensures that only authorized parties can obtain the data, which is beneficial for secure collaboration in the task and ensures that the secure distribution of stage result packages is consistent with the goal. This approach improves the overall system reliability and supports efficient task execution.

[0127] S411, the question work order carries a task identifier and stage information to ensure that it is associated with the current task execution process, so as to realize problem location and closed-loop feedback.

[0128] Real-time screen data, including interface elements and operation trajectories, is acquired from the capability provider's user interface. Key action information is extracted using image recognition to form a preliminary operation record set for subsequent problem association and localization. This preliminary operation record set is then categorized and organized based on task identifiers and stage information, binding the set to the current task's execution stage to generate an operation record package containing task identifiers and stage markers, ensuring the preservation of contextual information at the time of the problem. Based on this operation record package, if error messages or abnormal operations are detected, a query ticket is automatically generated. This query ticket embeds task identifiers and stage markers, ensuring the ticket is associated with a specific stage of the task execution process for easy problem tracing. The generated query ticket is pushed to the requester via instant messaging, while the task identifier and stage information are saved to the database, forming a closed-loop record for problem localization and feedback, ensuring continuous tracking of the correlation between the query ticket and the task execution process.

[0129] In one possible implementation, the process of acquiring real-time screen data from the capability provider's user interface can be achieved through a lightweight plugin installed in the integrated development environment. This plugin captures screen images and mouse and keyboard events at fixed intervals, thereby collecting interface elements such as button positions and text content, as well as operation trajectories such as drag paths and click sequences. This acquisition method helps to capture dynamic changes during task execution in a timely manner, thus providing a basic data source for subsequent extraction of key action information, ensuring the integrity of operation records, and improving the accuracy of problem localization.

[0130] For example, during the coding phase, when the capability provider edits the code, the plugin records the entire process from opening the file to saving the changes. This not only preserves contextual details but also allows for the analysis of potential error sources through this data, resulting in a more efficient feedback mechanism.

[0131] In one possible implementation, the specific method for extracting key action information through image recognition to form a preliminary operation record set is to use optical character recognition technology to process the screen image. First, the captured image is converted into a grayscale image, and then edge detection is applied to identify the boundaries of interface elements. Then, it is matched with predefined action templates, such as the click of the code submission button, thereby generating a record set containing timestamps and action types. This extraction process ensures the structure of the data, which is beneficial to the smooth progress of subsequent classification and organization.

[0132] For example, during debugging, if an error pop-up is detected, the system will automatically mark the action and associate it with relevant trajectory data. This helps to preserve the immediate state of the problem and thus supports more precise stage binding.

[0133] In one possible implementation, the specific method for classifying and organizing the initial operation log set by combining task identifiers and stage information is to first read the unique identifier of the current task, such as the task number, and the preset stage division, such as requirements analysis or test debugging, from the task management system. Then, each action item in the log set is mapped and matched with this information to generate an operation log package. Each package contains an encrypted task identifier and stage marker. This binding method ensures that context information is preserved, which is beneficial for rapid tracing when problems occur.

[0134] For example, when a capability provider encounters a code debugging failure at a specific stage, the logging package will embed the marker for that stage, so that the subsequent question tickets will directly point to the root cause of the problem, avoiding delays caused by isolated information.

[0135] In one possible implementation, the process of automatically generating a question ticket if an error message or abnormal operation is detected involves monitoring keywords such as error codes or abnormal logs in the operation log package. If a match is found, the ticket template is populated, embedding the task identifier and stage marker to ensure that the ticket is associated with a specific stage of the task execution process, facilitating problem tracing. This generation mechanism is beneficial for achieving continuous closed-loop feedback.

[0136] For example, when frequent erroneous inputs are detected during the code submission phase, the system will create a work order with screenshots and logs attached. This not only helps stakeholders quickly understand the problem, but also guides targeted solutions by linking information from different phases, thus enhancing collaboration efficiency.

[0137] In one possible implementation, the specific method for pushing the generated question tickets to the requester via instant messaging tools is to use an application programming interface to package the ticket data and send it to the chat platform, while saving the task identifier and stage information to a cloud database, forming a closed-loop record of problem location and feedback. This combination of pushing and saving ensures that the relevance can be continuously tracked, which is beneficial to the transparent management and efficient advancement of the entire task process.

[0138] For example, after receiving a work order, the requester can directly view the relevant operation record package based on the embedded stage markers, thereby providing remote guidance. This supports the timeliness of problem resolution from multiple perspectives, such as reducing communication delays and improving quality control.

[0139] The above description of the embodiments is only for the purpose of helping to understand the technical solutions and core ideas of this application; those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A task process management method based on screen analysis, applied to a task process management system, the task process management system comprising a task management end and a task execution end in communication connection with the task management end, and characterized in that, The method comprises: The task management end sends task publishing information to the task execution end, wherein the task publishing information carries task stage division rules and stage index information, and the task execution end deploys a screen capture module and a task execution tool integration module; The task execution end captures screen content and task execution tool operation data, and displays a record button in the task execution tool; In response to a triggering operation of the record button, the task execution end generates a stage achievement package, which contains a current screenshot, task execution tool version data and operation logs, and uploads the stage achievement package to a processing center; The processing center analyzes and processes the stage achievement package to generate visual progress information and a stage completion state, and pushes them to the task management end; When the task execution end detects problem information in the screen, it automatically generates a question ticket, which contains a problem screenshot and context data, and pushes it to the task management end. The task management end adds annotation information to the screenshot of the question ticket using an annotation tool and feeds back to the task execution end.

2. The method of claim 1, wherein, The task execution end captures screen content and task execution tool operation data, comprising: The screen capture module captures screen pixel data and interface elements in real time, reduces data volume through compression processing, and locally caches after blurring sensitive areas; The task execution tool integration module records file operations, code editing and debugging commands to form operation logs and associate them with screen pixel data; The screen content and task execution tool operation data are fused into continuous data streams, which support real-time transmission to the processing center; The record button is displayed in the task execution tool interface and is bound to a hook function to capture the screen content at the triggering moment.

3. The method of claim 1, wherein, In response to the triggering operation of the record button, the task execution end generates a stage achievement package, comprising: The task execution tool integration module saves the current version data and generates a version identifier; The screen capture module intercepts a screenshot at the triggering moment and extracts key operation tracks; The operation logs, the screenshot and the task execution tool version data are packaged to form the stage achievement package, and a stage number is added based on the task stage division rules; The stage achievement package is uploaded to the processing center through an encrypted channel and is stored after permission verification.

4. The method of claim 1, wherein, The processing center analyzes and processes the stage achievement package, comprising: Determine the interface elements and operation tracks in the screenshot using image recognition; Divide the current stage and calculate the completion degree of each stage index information according to the operation logs and the task stage division rules through clustering algorithm; Generate the visual progress information, which contains a progress bar, node state and quality data, and push it to the task management end through real-time protocol; If the stage completion degree is lower than the preset threshold, mark it as abnormal and push a warning to the task management end.

5. The method of claim 1, wherein, When the task execution end detects problem information in the screen, it automatically generates a question ticket, comprising: The screen capture module identifies error pop-up windows or abnormal interface elements, captures error text and surrounding screen areas; The task execution tool integration module associates the current operation log and the task stage division rule generation context data; The question work order encapsulates the question screenshot, error text and context data, and carries the task identification to push to the task management end; The task management end displays the question work order in the interface after receiving the question work order.

6. The method of claim 1, wherein, The task management end adds annotation information on the screenshot of the question work order through an annotation tool, including: The annotation tool loads the question screenshot, supports annotating the problem area and adding text explanation; The annotation information is associated with the question work order to form feedback data, which is sent to the task execution end in real time through a communication module; The task execution end receives the annotation information and superimposes it on the current screen content; The feedback data is recorded in the operation log, and the subsequent stage achievement package is included.

7. The method of claim 1, wherein, The processing center analyzes and processes the stage achievement package to generate visual progress information, including: The visual progress information fuses the stage completion status, stage index information and operation trajectory chart; The task management end receives the pushed visual progress information and renders it as a dashboard to support screen operation playback; If no operation is detected for a long time, an abnormal state is generated according to the screen content and is associated with the visual progress information.

8. The method of claim 1, wherein, The task execution end deploys a screen capture module and a task execution tool integration module, including: The screen capture module binds the task execution tool after initialization to provide a data synchronization interface; The task execution tool integration module embeds the record button and the hook function to support automatic saving of version data; The task publishing information activates the inter-module association to form a data acquisition link.

Images