663 results about "Screen capture" patented technology

The invention provides a multi-dimensional data authority management and privacy protection method for an electric power information network, and the method comprises the steps: dynamically selecting the type of a watermark according to the leakage risk level of a screenshot event, and enabling the watermark type to comprise a simple character watermark, a character and image mixed watermark, an invisible watermark, and a two-dimensional code watermark; user screenshot content factors are added in a screenshot event distribution curve of user historical data to judge the risk level of a user, the risk level of a user group is updated, and when the screenshot frequency, time interval, screenshot mode, distribution mode and curve volatility of the user are all normal and screenshot content only aims at sensitive data screenshot, the risk level is improved; a screen capture auditing and monitoring mechanism is established, full-life-cycle recording and tracking are carried out on screen capture behaviors of a user, and screen capture events, watermark embedding conditions and abnormal behavior information in a system are monitored in real time through a visual instrument panel and a report.

Machine learning techniques are leveraged to provide personalized assistance on a computing device. In some configurations a timeline of a user's interactions with the computing device is generated. For example, screenshots and audio streams may be saved as entries in the timeline. Context—the state of the computing device when the entry is created, such as which documents and websites are open—is also stored. Entries in the timeline are processed by a model to generate embedding vectors. The timeline may be searched by finding the embedding vector that is closest to an embedding vector derived from a search query. The user may select a query result, causing the associated context to be restored. For example, if the query is “show me all documents related to my upcoming trip to Japan”, the query result may open documents and websites that were open when booking a flight to Japan.

The present disclosure provides techniques and solutions for automatically correcting software tests. When a test failure is detected, it is determined whether a screenshot or code associated with a second version of a software artifact includes a user interface element that has a semantically equivalent identifier to a user interface element in a screenshot or code associated with a first version of the software artifact. Identifying a semantically equivalent identifier can include determining a section of the user interface, in the screenshot of the code, of the first version of the software artifact where the user interface is located and searching the corresponding section of the user interface of the second version of the software artifact. A definition of the software test can be updated to reference the semantically equivalent user interface element.

Media, methods, and systems are provided for screen sharing. A user may initiate a video recording. A recording preview interface may be displayed for the user to preview the video recording before beginning the recording. The recording preview interface may comprise a live video preview of the user and of a window or application being shared. When the user selects a window or application for previewing that may cause an infinity mirror effect to occur, a screenshot of the window or application may be generated. The screenshot may be rendered over the live video preview of the window or application to obscure the infinity mirror effect. When the user initiates recording, the screenshot may be removed to expose the live video. The screenshot may be cached and retrieved if the user navigates away and then returns to the preview of the window or application causing the infinity mirror effect.

A management node local to a customer site of a private communications network stores a model. The model is a compact version of a visual language model remote from the customer site. A first screen shot of a dashboard of telemetry data measured from the private communications network is accessed. A prompt is formulated comprising the first screen shot and information to adapt the model to the private communications network via few shot learning. The prompt is submitted to the model. An output is received from the model comprising textual information about anomalies or trends depicted in the first screen shot. The output is checked against data from a statistical model of the telemetry data, the statistical model being independent of the model. In response to the check being successful, an action is triggered to manage the private communications network according to the output.

One variation of a method includes: accessing a test statement defining a target outcome for a target webpage; capturing a screenshot of the target webpage depicting a set of target content; accessing a set of webpage code defined for the target webpage; transforming the set of webpage code into a sequence of contextual tags, corresponding to the set of target content depicted in the screenshot, to generate a textual representation of the target webpage; generating a prompt including the test statement, the screenshot, and the textual representation; accessing a language model configured to generate responses to test statements based on visual and textual content extracted from corresponding prompts; based on the language model and the prompt, generating a textual response to the test statement representing occurrence of the target outcome at the target webpage; and serving the textual response to a user associated with the organization.

The invention discloses a screen shooting traceability method, device and equipment and a readable storage medium, which are applied to the field of computer software, and comprise the following steps: encoding watermark information to obtain a watermark information sequence; embedding the watermark information sequence into the image through an R channel, a G channel and a B channel of RGB (Red, Green, Blue) to obtain an image with a watermark; and acquiring the shot image with the watermark image, and tracing to obtain watermark information based on the shot image. According to the method, the watermark is generated by using the RGB channel of the single pixel image, so that the visual concealment of the watermark image is improved, the difficulty in perceiving of human eye recognition is ensured, and the visual quality of a user is not influenced; the watermark embedding position is not influenced by an original image carrier, and watermark information can be hidden in any area of the image; and the watermark information recovery success rate is high.

The invention discloses a watermark imperceptible embedding and recovering method based on a deep learning network structure. Firstly, a mask guide watermark embedding scheme is designed, a mask generation module is constructed by using a residual dense feature extraction module and an attention mask generation module, and a watermark is adaptively guided to be embedded into an image texture rich area so as to improve the invisibility of the watermark. And secondly, constructing a watermark decoding network based on comparative learning, and by comparing a loss function, taking the decoding features of the same watermark image under different noise conditions as positive samples and taking the decoding features of different watermark images as negative samples so as to enhance the consistency of the decoding features, thereby improving the robustness of the watermark. The deep learning network structure can improve the robustness of the watermark in a real screen shooting scene, and has a huge application value in copyright protection and traceability tracking.

The disclosed techniques provide improved methods of operating system (OS) search. Users are enabled to search for documents, emails, presentations, content they entered into a web form, meetings they participated in, and other interactions they had with their computing device. To accomplish this, screenshots are periodically captured and indexed. Machine learning models are used to infer embeddings for visual elements of the screenshots and / or text extracted from the screenshots. A full text index of a relational database may also be populated with text extracted from the screenshots. The embeddings and full text index may then be used to retrieve screenshots in response to a user history query. For example, screenshots of embeddings within a defined distance of an embedding of the user history query may be selected. Query results from different embedding indices and relational databases may be ordered by applying different weights to different kinds of search scores.

A system and method includes connecting an interface device to a port of a vehicle to be in communication with an electrical system of the vehicle for monitoring the vehicle, and running a selected one of multiple available calibration and / or repair programs via the interface device to generate a calibration / repair data file of the vehicle, where the data / log file is generated in one of a plurality of possible native file formats. A report generation tool acquires screen captures and / or screen recordings to document the data / information displayed during the steps of a calibration / repair procedure. An evaluation tool extracts calibration / repair data from the data / log file, where the evaluation tool is configured to extract calibration / repair diagnostic data from data / log files in each of the plurality of possible native file formats, and outputs the calibration / repair diagnostic data to a database in a common format from which detailed calibration / repair diagnostic reports are generated.

Large language models (LLMs) and visual-language models (VLMs) are able to provide robust results based on specified formatting and organization. Although LLMs and VLMs are designed to receive natural language input, users often lack the skill, knowledge, or patience to utilize LLMs and VLMs to their full potential. By leveraging screen understanding, AI prompts (or “pills”) may automatically be generated for artificial-intelligence (AI) assistance and query resolution in a VLM / LLM environment. Using an image encoder, a current screenshot is processed into an image embedding and compared to text embeddings representing screenshot activities. By identifying the text embedding having the closest similarity to the image embedding, a screen activity being performed by the user may be determined. Suggested AI prompts (or “pills”) may then be generated in real-time to assist the user in performing the screen activity.

Systems, tools and methods to allow for the deterministic replay of nondeterministic events and testing of interactions with systems and interfaces. The system may record or retrieve user sessions corresponding to a first version of an application or user interface and replay the events of the user session with a second version of the application or user interface. The system may test updates to code by comparing screenshots or outputs generated by the second version against those generated by the first version to identify unintended differences or errors. The sessions used to test the updates may also be generated by a script or by a machine learning model trained on previous user sessions.

An implementation method supporting a Kiosk mode and remote control, comprising: a main process of a TMS client establishing a connection with a TMS main server; when a remote control instruction is received, enabling an Android device in a Kiosk mode to temporarily quit the Kiosk mode; starting a remote control interface of the TMS client, setting the remote control interface to be transparent, setting a screen capture permission of the remote control interface, and enabling the Android device to enter the Kiosk mode; the remote control interface being connected to a remote control server and broadcasting connection information; the main process acquiring login information from detected connection success information and uploading the login information to the TMS main server; the TMS main server popping up a login dialog box of the remote control interface by means of a browser and automatically filling in the login information; and when login confirmation information triggered by a user is received, capturing the screen of the Android device and displaying same on the remote control server, so that the server end starts to synchronously display the screen with the Android device and performs operations. According to the present invention, a device in a Kiosk mode can be operated, and the use is convenient.

Disclosed are techniques for synthesizing large amounts of human-computer interaction data that is representative of real-world user data. An automated screenshot capture engine may cause an automated agent to use an application or a website in a manner designed to mimic real-world human-computer interaction. Screenshots are captured to record how a user might interact with the application. Metadata, such as window location and size, may be obtained for each screenshot. Screenshots and corresponding metadata may be automatically annotated with a large language model to indicate the context of the application and / or computer system when the screenshot was captured. Data created in this way may be used to validate AI-based software application features or to train (or retrain) a machine learning model that predicts human-computer interactions. Automated synthesis of training data significantly increases the scale of data that can be obtained for training while also reducing computing and financial costs.

Various systems and methods for contextual capture and processing of screen capture data are disclosed. An example method for screen capture data processing in a computing device may include: determining an active screen of the computing device based on a user interaction event; identifying a graphics rendering event associated with a software application presented in the active screen; identifying screen capture data in a buffer (e.g., of graphics processing circuitry such as a GPU) that corresponds to the graphics rendering event; and communicating a contextual screen update event via an application programming interface (e.g., a an API operated by a GPU driver). The receipt of this contextual screen update event can be used by an AI engine to control whether to perform contextual processing on particular frames of the screen capture data.

The invention provides an anti-screen-shooting robust text image watermarking method fusing edge attention gating and multi-scale cavity convolution, which comprises the following steps: step 1, designing a message processor for preprocessing a secret message, adding an expansion network ExpandNet and a channel attention SENet in the message processor, and carrying out edge attention gating and multi-scale cavity convolution on the expansion network ExpandNet and the channel attention SENet; inputting a secret message in a binary form and outputting a message feature map; step 2, constructing a text watermark framework based on edge attention gating and multi-scale cavity convolution fusion; adding a functional module in an encoder of the end-to-end watermark framework; 3, designing a noise layer, and performing simulation attack on the model in the watermark model training process; and step 4, designing an optimization mode of the loss function. According to the method, the robustness and imperceptibility of the watermark are remarkably improved.

A network security system that classifies webpages and uses the classification of the webpages to enforce relevant security policies is disclosed. To classify a webpage, a screenshot of at least a portion of the webpage is captured. An embedding engine generates a subject image embedding of the screenshot. The subject image embedding is classified by an image classifier that includes an index of training image embeddings each having a label of their classification and an associated approximate nearest neighbors model trained to identify the label of the closest training image embedding to the subject image embedding and a score representing their similarity. The webpage is classified based at least in part on the score and the label from the image classifier. The network security system applies security policies to requests from client devices that identify the webpage as its destination based at least in part on the webpage classification.

The invention discloses a method and equipment for realizing mouse and virtual key control based on an inertial measurement unit, and a computer readable storage medium. The method comprises the following steps: acquiring original three-dimensional motion data of the inertial measurement unit to generate a standard motion feature flow; forming screen key layout information according to the screenshot of the current active application program; determining an inferred running state of the current active application program to obtain an effective virtual key set containing function annotations; determining a current effective interaction mode according to the updated human-computer interaction situation information, the updated current application program inference operation state and a user input instruction; and selecting a mapping algorithm to process the standardized motion feature flow according to the currently effective interaction mode to generate at least one of a mouse cursor movement instruction, a mouse key instruction and a virtual key operation instruction. The method has the advantage that the operation efficiency and the user experience of IMU-based mouse control in multiple scenes are improved.

The invention provides a game real-time intelligent translation method and device based on interactive reinforcement learning. The method comprises the following steps: determining a corresponding screen capture mode according to current game picture information and user operation habit information; dynamically sensing the current game picture according to the screen capture mode; performing text recognition on the image data in the user attention area by using a context-enhanced image recognition model, and obtaining a recognition text according to a context knowledge base exclusive to the game scene and user operation historical information; inputting the recognition text, the corresponding image data and user operation historical information into a multi-modal fusion translation model to generate a target language text; and based on the corresponding relationship between the recognition text and the target language text, rendering the target language text on the original foreign language region in an augmented reality mode. According to the method, real-time, automatic and immersive translation of the game picture content is realized, a large number of continuous texts can be processed, and the translation result better conforms to a game scene.

The invention discloses a vision enhancement robust digital dark watermarking method based on deep learning, a storage medium and equipment, and belongs to the technical field of digital image processing and information security. According to the method, a visual enhancement module comprising a self-adaptive watermark region adjustment module and a frequency enhancement module is constructed, and a multi-region local loss training mechanism and a screen shooting simulation module are combined, so that the visual quality of a watermark image is improved, and meanwhile, the robustness of the watermark image for resisting physical attacks such as screen shooting is enhanced. The specific processing flow comprises the following steps: acquiring an original image and watermark information; embedding the watermark information into the image by using an encoder, wherein an embedded region is optimized by a self-adaptive region guide matrix generated based on the image edge and gray information; in the model training process, a noise layer simulating screen shooting physical distortion is introduced, and the area weight is dynamically adjusted according to residual error distribution in the later stage of training so as to focus and optimize the image quality; watermark information is extracted from an image which may suffer from an attack by using a decoder.

The invention relates to the technical field of network information security, in particular to a network information security monitoring method based on artificial intelligence, which comprises the following steps: step 1, interface information acquisition: monitoring a user interface in real time and capturing all information on the interface, including a desktop, an application window and document content, capturing a snapshot of the user interface at high frequency through a screen capture tool, extracting a video stream from the user interface and separating an audio stream, and extracting text information from an image by using an optical character recognition technology; according to the scheme, the character, image and sound information of the user interface is collected in real time, advanced artificial intelligence algorithms such as a convolutional neural network and potential semantic analysis are used for deep risk analysis, prohibited information grades are accurately divided, flexible and diverse processing modes such as pop-up window shielding are adopted according to the grades, and the prohibited information classification efficiency is improved. The network information security is comprehensively, accurately and effectively monitored, the defects of the traditional technology are overcome, and a powerful guarantee is provided for the health and stability of the network environment.

The invention discloses an automatic filling method and system based on YOLO and RPA, and the method comprises the steps: collecting a form interface image through screen capture, inputting the image into a YOLO model for processing, and recognizing form elements to generate bounding box information; reading data from an external data source, matching a corresponding relationship between the data and fields according to a preset mapping rule, verifying a data format, and obtaining a data set passing verification; using RPA to simulate user operation, inputting data passing verification into a form field, and formatting to obtain a filled form interface; checking fields required to be filled and data formats, and determining the validity of the form; simulating and clicking a submission button to complete form operation, and recording a submission result; and monitoring potential errors in the whole process, performing classified processing, recording detailed log information, and obtaining an optimized filling record. According to the method, the automation degree and accuracy of form processing are remarkably improved, the manual operation burden is reduced, the business process efficiency is optimized, and the method is suitable for complex form filling scenes.

The present application relates to an anti-screenshot method, device, computer equipment and storage medium. The method comprises: monitoring whether the target method of the target interface is called, and if the target method is monitored, obtaining the original window attribute data in the target method; modifying the original window attribute data according to the target data to obtain the target window attribute data. Since the target data is used to set the target interface as an interface that prohibits screenshots, the target interface is displayed according to the target window attribute data, and the anti-screenshot effect of all window objects is reinforced to improve the reinforcement quality, thereby preventing malicious programs from taking screenshots and completely preventing malicious programs from taking screenshots, thereby avoiding information leakage.

A method for automatic detection and remediation of security posture in web-applications using large vision models is fulfilled in the ongoing description by (a) initiating a headless browser as an agent to access an administrative section of a web-application, (b) enabling a pre-trained large vision model to navigate through a web user-interface of the web-application using a state transition graph, (c) determining subsequent navigation actions of the navigated web-user interface using screenshots of the navigated web-user interface with the large vision model, (d) detecting and analyzing a final state of navigation sequence of the administrative section to extract security attributes, (e) monitoring and collecting data associated with security posture of the web-application based on the security attributes, and (f) initiating automated corrective actions through a security posture remediation module upon identifying a security issue in the web-application.

The invention discloses an intelligent task management method based on OCR (optical character recognition) and AI (artificial intelligence) technologies, which comprises the following steps: S1, information source identification: a user intercepts a screenshot of a task message through a shortcut key, and analyzes the screenshot content through an image identification technology to judge which communication tool the message source is; s2, OCR (Optical Character Recognition) and task analysis: identifying text content in the screenshot through an OCR technology, and analyzing key information of a task in combination with an AI (Artificial Intelligence) natural language processing technology; s3, attachment identification and path positioning: automatically positioning a storage path of a task attachment in a user computer through an OCR (Optical Character Recognition) technology and a file type identification function; and S4, automatic task recording: associating the analyzed task information with the identified attachment, and then automatically inputting the task information into a task management system. Compared with the tedious process that a traditional task management tool needs to manually input, sort and manage task information, the operation steps of the user are greatly reduced, and the overall efficiency of task management is improved.

Disclosed herein are systems and methods for translating and verifying text in a variety of different languages for the same software application. When the text in the application changes. embodiments of the disclosure may include translating and / or verifying only the text that has changed. The system may compare the new screenshot(s) with previously accepted screenshot(s) to locate the text that has changed. The text that has not changed (since the last accepted translation) may not be translated and / or verified once accepted by a translator. The system may highlight the text that has changed so that the translator may focus only on the relevant portions of the user interface and not have to search for the text that has changed. For rejected translations. the system may repeat the process. translating and / or verifying only the rejected text (instead of translating / verifying all text again).

The invention discloses a tracing method, device and equipment based on a digital certificate and a hidden watermark, and belongs to the technical field of information security. Wherein secret-related files are distributed through an SSLVPN or a terminal security tool; the secret-related file is subjected to hidden watermark processing, the hidden watermark processing is based on opencv time domain or frequency domain watermark addition or extraction, and hidden watermark information comprises signature information of each node in a propagation link; when the file is leaked, multi-level tracing is carried out by extracting signature information in the hidden watermark, and a leakage source and a propagation link are determined; and accessing the confidential file through an SSLVPN (Secure Socket Layer Virtual Private Network) or a terminal security tool, and meanwhile, preventing file downloading and screen capture operations. The invention provides a traceability method, device and equipment based on a digital certificate and a hidden watermark, which not only can protect a document from illegal access and leakage, but also can realize multi-level traceability through signature information in the digital watermark when leakage occurs, so that a leakage source and a propagation link are accurately tracked, and technical support is provided for afterward responsibility investigation.

The invention relates to the technical field of data backup and storage, and discloses a universal chatting record backup and storage method and system.The universal chatting record backup and storage system comprises a multi-source data acquisition unit, an information storage unit, an operation recording unit and a supplementary acquisition unit. One-stop efficient acquisition is realized, and the data processing efficiency is improved; classified storage and structured processing are carried out according to data characteristics, original data storage and efficient query analysis are considered, and data structure changes are flexibly coped with; the operation recording unit completely records the whole operation process by using a screen recording and Hash algorithm, ensures that the operation record is tamper-proof, and provides authenticity guarantee for the chat record as evidence in scenes such as commercial disputes and lawsuit; the supplementary acquisition unit breaks through the limitation of a data interface by means of technologies such as screenshot and optical character recognition, realizes complete acquisition of chat records of a complex communication platform, and enhances the universality and adaptability of the system.

Provided is a system and method for dynamically adjusting a descriptive document with instructions for a user interface based on configuration settings and software version and / or release for a particular user and document data stored in a technical data object. The process ensures that the content of the descriptive document matches what a user will see on a screen when viewing the user interface. In one example, the method may include retrieving user interface configuration settings generating a simulated user interface that includes a plurality of user interface elements and formatting the user interface elements based on the user interface configuration settings, composing a window with text from a document and screenshot images of one or more formatted user interface elements from the simulated user interface corresponding to the text, and rendering the window via a display of a computing device.

A system and method automatically evaluate participant online testing and automatically provide feedback regarding test results of a participant. Automatic evaluation includes processing video frames, audio, timestamped screenshots, and initial test results of the participant answering multiple test questions and transcribing audio. The system and method utilize optical character recognition on the screenshots to generate a set of text for each screenshot and compare the set of text from each timestamped screenshot to determine which screenshots are associated with each test question. The system and method further determine time periods for each question, segment the transcribed audio, the screenshots, and the video frames by question and select a screenshot and a video frame for each test question from the segmented screenshots and video frames. The system and method utilize a first AI tool and a second AI tool to determine whether the participant demonstrated mastery for each test question.