Screen capture processing method and apparatus, electronic device, medium, and program product

Abstract

Embodiments of the present disclosure disclose screenshot processing methods, devices, electronic devices, media and program products. A specific embodiment of the method comprises: in response to listening to real-time screenshot processing for a target device, obtaining a real-time device state corresponding to the target device; constructing an initial device instance for the target device; injecting the real-time device state into the initial device instance to obtain a device instance; and performing screenshot processing on the device instance to obtain a real-time device screenshot. This embodiment is related to image screenshots, and can quickly and accurately obtain a real-time device screenshot with less network bandwidth consumption.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to screenshot processing methods, apparatus, electronic devices, media, and program products. Background Technology

[0002] Currently, screenshot processing is increasingly used in various scenarios, enabling tools to quickly capture screen content and supporting region selection, full-screen, or window screenshots. It provides editing functions such as annotation, cropping, and mosaicking, and can save in multiple formats, with one-click sharing to social media platforms or documents, improving efficiency and convenience. For screenshots of target devices, the target device directly executes the screenshot of the displayed interface. It receives the corresponding interface screenshot from the target device.

[0003] However, when using the above method, the following technical problems often arise: Existing screenshots for display interfaces are often high-definition images. Transmitting high-definition images in a weak network environment presents problems such as difficulty in uploading and high network bandwidth consumption. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure provide screenshot processing methods, apparatuses, electronic devices, computer-readable media, and program products to address the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a screenshot processing method, including: in response to listening to real-time screenshot processing for a target device, obtaining the real-time device state corresponding to the target device; constructing an initial device instance for the target device; injecting the real-time device state into the initial device instance to obtain a device instance; and performing screenshot processing for the device instance to obtain a real-time device screenshot.

[0007] Optionally, the aforementioned real-time device status includes: device environment parameter information; and the aforementioned construction of an initial device instance for the aforementioned target device includes: constructing an initial device instance for the aforementioned target device based on the aforementioned device environment parameter information.

[0008] Optionally, the above real-time screenshot processing involves taking a screenshot of the target page in the target application. The real-time device state includes business data and page rendering information. Injecting the real-time device state into the initial device instance to obtain a device instance includes: starting the application instance corresponding to the target application in the initial device instance; intercepting network requests corresponding to the application instance; and injecting the business data into the application instance as a real network request. Based on the page rendering information, the following processing steps are performed: navigating the application instance with the injected network request to the corresponding target page to obtain the target device instance displaying the target page; and scrolling the target page displayed by the target device instance to the corresponding target progress to obtain the device instance.

[0009] Optionally, the above-mentioned real-time device status is generated through the following steps: in response to the detection of a real-time screenshot of the target device, the real-time initial device metadata is obtained; the real-time initial device metadata is de-identified to obtain de-identified data; the de-identified data is packaged to obtain the real-time device status.

[0010] Optionally, after responding to the detection of real-time screenshot processing for the target device and obtaining the real-time device status corresponding to the target device, the method further includes: determining the screenshot intent for performing the real-time screenshot processing; and filtering each piece of data in the real-time device status according to the screenshot intent to obtain filtered data as the real-time device status.

[0011] Optionally, the above method further includes: obtaining a screenshot text description for real-time screenshot processing; generating screenshot response content based on the screenshot text description, the real-time screenshot image, and the real-time device status; and sending the screenshot response content to the target device.

[0012] Secondly, some embodiments of this disclosure provide a screenshot processing apparatus, including: an acquisition unit configured to acquire a real-time device state corresponding to the target device in response to listening to real-time screenshot processing for a target device; a construction unit configured to construct an initial device instance for the target device; a state injection unit configured to inject the real-time device state into the initial device instance to obtain a device instance; and an execution unit configured to perform screenshot processing for the device instance to obtain a real-time device screenshot.

[0013] Optionally, the aforementioned real-time device status includes: device environment parameter information; and the construction unit can be configured to: construct an initial device instance for the aforementioned target device based on the aforementioned device environment parameter information.

[0014] Optionally, the above real-time screenshot processing involves taking a screenshot of the target page in the target application. The real-time device state includes business data and page rendering information. The state injection unit can be configured to: launch the application instance corresponding to the target application in the initial device instance; intercept the network request corresponding to the application instance and inject the business data into the application instance as a real network request; and perform the following processing steps based on the page rendering information: navigate the application instance that has injected the network request to the corresponding target page to obtain the target device instance displaying the target page; and scroll the target page displayed by the target device instance to the corresponding target progress to obtain the device instance.

[0015] Optionally, the apparatus further includes: determining the screenshot intent for performing real-time screenshot processing; filtering each piece of data in the real-time device status according to the screenshot intent to obtain filtered data as the real-time device status.

[0016] Optionally, the apparatus further includes: acquiring a screenshot text description for real-time screenshot processing; generating screenshot response content based on the screenshot text description, the real-time screenshot image, and the real-time device status; and sending the screenshot response content to the target device.

[0017] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0018] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0019] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0020] The above embodiments of this disclosure have the following beneficial effects: Through the screenshot processing model method of some embodiments of this disclosure, real-time device screenshots can be obtained quickly and accurately with minimal network bandwidth consumption. Specifically, the reason why related device screenshots cannot be obtained efficiently and quickly is that existing screenshots of display interfaces are often high-definition images. Transmitting high-definition images in a weak network environment presents difficulties in uploading and consumes high network bandwidth. Based on this, the screenshot processing method of some embodiments of this disclosure first, in response to the detection of real-time screenshot processing for the target device, obtains the real-time device state corresponding to the target device, so as to facilitate the construction of a mirror device instance for the target device based on the real-time device state. Then, an initial device instance for the target device is constructed to facilitate the subsequent construction of a mirror instance for the target device. Furthermore, the real-time device state is injected into the initial device instance to obtain a device instance. Here, by injecting the real-time device state, a device instance with the same display content as the target device can be obtained. Moreover, by obtaining the real-time device state, the high-definition screenshot transmission can avoid consuming a lot of network bandwidth; only the information content corresponding to the real-time device state needs to be transmitted, and accurate acquisition of device screenshots can still be achieved in a weak network environment. Finally, screenshot processing is performed on the aforementioned device instance to obtain an accurate real-time device screenshot. In summary, addressing the issue of high-definition screenshot transmission consuming significant network bandwidth in weak network environments, constructing a real-time device instance for the target device allows for accurate and efficient reproduction of the target device's current content. This enables accurate and efficient acquisition of device screenshots even when transmitting limited content in weak network environments. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario of a screenshot processing method according to some embodiments of the present disclosure; Figure 2 This is a flowchart of some embodiments of the screenshot processing method according to this disclosure; Figure 3 These are flowcharts of other embodiments of the screenshot processing method according to this disclosure; Figure 4 These are schematic diagrams illustrating the structure of some embodiments of the screenshot processing apparatus according to this disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0023] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0024] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0025] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0026] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0027] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0028] Before performing any of the operations involving the collection, storage, or use of user personal information (such as real-time device status) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0029] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] Figure 1 This is a schematic diagram of an application scenario of a screenshot processing method according to some embodiments of the present disclosure.

[0031] exist Figure 1In this application scenario, firstly, in response to the detection of real-time screenshot processing for target device 102, electronic device 101 can obtain the real-time device status 103 corresponding to the target device 102. In this application scenario, the real-time device status 103 can be "Device Model: Model A**1, Device Display Interface Description: Clicking component B on interface A results in an error message". Then, electronic device 101 can construct an initial device instance 104 for the target device 102. Furthermore, electronic device 101 can inject the real-time device status 103 into the initial device instance 104 to obtain device instance 105. Finally, electronic device 101 can perform screenshot processing on device instance 105 to obtain a real-time device screenshot 106.

[0032] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0033] It should be understood that Figure 1 The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0034] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a screenshot processing method according to the present disclosure. This screenshot processing method includes the following steps: Step 201: In response to the detection of real-time screenshot processing for the target device, obtain the real-time device status corresponding to the target device.

[0035] In some embodiments, in response to detecting real-time screenshot processing of a target device, the execution entity of the screenshot processing method (e.g., Figure 1The electronic device 101 shown can obtain the real-time device status of the target device via a wired or wireless connection. The target device can be the device to be screenshotted. Screenshotting here refers to taking a screenshot of the content displayed on the target device's interface. The target device can be a terminal device. For example, the target device can be a terminal used by a target user. The target user is the user who instructs the screenshot process. In practice, real-time screenshotting can be taking a screenshot of the currently displayed interface content of the target device. In specific practices, real-time screenshotting of the target device can be performed in various scenarios. For example, real-time screenshotting scenarios can be, but are not limited to, the following: mobile application development and testing scenarios, user feedback scenarios, customer service scenarios, and screenshot-based Q&A scenarios. The real-time device status can be the device status data of the target device at the current screenshot processing time. Here, the real-time device status is used to reproduce the real-time displayed content of the target device on the device instance, facilitating the accuracy of subsequent screenshots. For example, the real-time device status can include, but is not limited to, at least one of the following: device model, and interface metadata corresponding to the device's display interface. Interface metadata can be descriptive information about the interface content displayed on the device's display screen.

[0036] Here, the real-time device status of the target device is obtained based on the cloud-based reconstruction system, which is part of the execution entity. The cloud-based reconstruction system can be a system that builds device instances based on cloud servers. Specifically, the real-time device status can be collected and obtained by the target device. After collecting the real-time device status, the target device sends it to the cloud-based reconstruction system for the construction of the device instance.

[0037] In some optional implementations of certain embodiments, the above-mentioned real-time device state is generated through the following steps: The first step is to obtain the initial real-time device metadata in response to the detection of real-time screenshot processing on the target device. This initial device metadata can be all the metadata from the background processes of the target device during the real-time screenshot processing. In practice, the initial device metadata can be in text format.

[0038] In practice, when the target device's corresponding backend module continuously listens for screenshot signals (e.g., screenshot notifications or screenshot broadcasts), the target device immediately triggers various device metadata at that time. In practice, the initial device metadata includes device data corresponding to the real-time device state. Compared to the real-time device state, the initial device metadata includes more sensitive content. Specifically, sensitive content can be user privacy information. For example, user privacy information may include: bank card numbers, ID cards, and personal chat logs.

[0039] In other words, by removing sensitive content and converting the format of the initial device metadata, the real-time device status can be obtained.

[0040] The second step is to perform desensitization processing on the aforementioned real-time initial device metadata to obtain desensitized data. This desensitization process can involve removing sensitive content from the real-time device metadata.

[0041] As an example, regular expression matching is performed on the collected initial device metadata to automatically mask sensitive fields (such as mobile phone numbers and card numbers) (e.g., replace them with "***"), resulting in de-identified data.

[0042] It should be noted that by performing de-identification processing, privacy violations during the screenshot process can be effectively avoided, and the risk of sensitive information leakage can be prevented.

[0043] The third step is to package the above-mentioned de-identified data to obtain the real-time device status.

[0044] As an example, the aforementioned execution entity can convert the anonymized data into massive amounts of JSON format data to obtain the real-time device status.

[0045] In some optional implementations of certain embodiments, after step 201, the steps further include: The first step is to determine the screenshot intent for performing real-time screenshot processing. This intent can be the target device's intention to take a screenshot, or the purpose of performing screenshot processing on the target device. For example, the screenshot intent could be the intent to provide user feedback on an anomaly.

[0046] As an example, the screenshot text description input during real-time screenshot processing is obtained. This screenshot text description can be input on the target device to explain the purpose of the screenshot processing. Then, the screenshot text description is input into the intent recognition model to obtain the screenshot intent.

[0047] The second step is to filter the data in the real-time device status according to the screenshot intent, obtaining filtered data as the real-time device status. In practice, different data in the real-time device status can be filtered out for subsequent re-rendering based on different screenshot intents. That is, different screenshot intents correspond to different real-time device statuses, which in turn correspond to device instances displaying different interface content, capturing screenshots of different content.

[0048] As an example, the aforementioned execution entity can determine the necessary data corresponding to the screenshot intent through a relational query. The relational relationship represents the correspondence between the screenshot intent and the necessary data. Then, based on the necessary data, each piece of data in the aforementioned real-time device status is filtered to obtain the filtered data, which serves as the real-time device status.

[0049] Here, by setting up data filtering based on screenshot intent, the interface rendering during subsequent device instance image construction can focus on data related to the screenshot intent, retaining only the data relevant to that intent. This allows for a rapid response to screenshot intents while ensuring that no key image content is missing from the screenshot.

[0050] Step 202: Construct an initial device instance for the aforementioned target device.

[0051] In some embodiments, the execution entity may construct an initial device instance for the target device. This initial device instance may be a device instance that has not yet undergone state synchronization, constructed based on the device specifications corresponding to the target device.

[0052] As an example, the aforementioned execution entity can schedule a physical machine or simulator instance from the cloud real machine device pool whose configuration is closest to that of the target device as the initial device instance.

[0053] In some optional implementations of certain embodiments, the aforementioned real-time device status includes: device environment parameter information. This device environment parameter information can be the parameter content of various environmental parameters under the device's operating environment. In practice, these environmental parameters may include: device model, operating system version, screen resolution, current system font size (Dynamic Type), and dark / light mode status.

[0054] Optionally, the aforementioned execution entity may, based on the aforementioned device environment parameter information, schedule a physical machine or simulator instance from the cloud real machine device pool whose configuration is closest to that of the target device, as the initial device instance.

[0055] Step 203: Inject the real-time device status into the initial device instance to obtain the device instance.

[0056] In some embodiments, the execution entity can inject the real-time device state into the initial device instance to obtain a device instance. The content displayed by the device instance is generated based on the currently displayed interface content of the target device. In practice, the purpose of constructing the device instance here is to recreate the display content of the target device during real-time screenshot processing, in order to obtain accurate screenshots.

[0057] As an example, the aforementioned execution entity can inject real-time device status into the initial device instance to drive the initial device instance to display the same real-time interface content as the target device.

[0058] In some optional implementations of certain embodiments, the above real-time screenshot processing involves taking a screenshot of a target page within a target application. The real-time device state includes business data and page rendering information. The target application is the application opened by the target device when the screenshot processing is performed. The target page can be a specific application page within the target application. The target page can be the page displayed by the target device when the screenshot processing is performed. The business data can be business-related data at the time of screenshot execution. In practice, business data may include: the JSON data model (Model) upon which the current page rendering depends, list data sources, current memory data, and API response data. Page rendering information may be rendering content related to the rendering process of the target page. In practice, page rendering information may include: UI state and resource fingerprint. In practice, UI state may include: the current page route (ViewController / Activity class name), pop-up hierarchy, scroll offset (ContentOffset) of ScrollView / TableView, and focus state of input boxes. Resource fingerprint may include: the URL of an image loaded on the page, its loading status (success / failure / loading), and scroll position. The resource fingerprint here can characterize the real-time interactive state of the target device.

[0059] Optionally, the aforementioned execution entity can inject the aforementioned real-time device state into the aforementioned initial device instance to obtain a device instance, including the following steps: The first step is to launch the application instance corresponding to the target application in the initial device instance described above. An application instance can be a specific running instance of an application or a real-world application scenario, used to demonstrate the specific application methods and effects of a technology, method, or system.

[0060] The second step is to intercept the network requests corresponding to the aforementioned application instances and inject the aforementioned business data into the aforementioned application instances as real network requests.

[0061] In practice, network requests from the app can be intercepted using Hook technology or debug bridges (ADB / WDA), and "business data" can be directly injected to replace the real network requests, ensuring that the data is completely consistent with the user's side.

[0062] Third, based on the page rendering information above, perform the following processing steps: Sub-step 1: Navigate the application instance that received the network request to the corresponding target page to obtain the target device instance that displays the target page.

[0063] In practice, based on the UI status, the application instance that has been injected with the network request can be navigated to the corresponding target page, thus obtaining the target device instance that displays the target page.

[0064] Sub-step 2 involves scrolling the target page displayed by the aforementioned target device instance to the corresponding target progress, thereby obtaining the device instance. The target progress can be the scrolling progress of the target page during real-time screenshot processing of the target device.

[0065] In practice, based on resource fingerprints, automated scripts can be used to scroll the target page displayed by the above target device instance to the corresponding target progress, thereby obtaining the device instance.

[0066] Step 204: Perform screenshot processing on the above device instance to obtain a real-time device screenshot.

[0067] In some embodiments, the aforementioned execution entity may perform screenshot processing on the aforementioned device instance to obtain a real-time device screenshot.

[0068] It should be noted that the screenshot processing for the device instance is performed only after the rendering of the interface content corresponding to the device instance has stabilized (for example, after listening for the quiescent signal of the Layer tree).

[0069] As an example, the screenshot command corresponding to the cloud reconstruction system can be called to perform screenshot processing for the above device instance and obtain a real-time device screenshot.

[0070] The above embodiments of this disclosure have the following beneficial effects: Through the screenshot processing model method of some embodiments of this disclosure, real-time device screenshots can be obtained quickly and accurately with minimal network bandwidth consumption. Specifically, the reason why related device screenshots cannot be obtained efficiently and quickly is that existing screenshots of display interfaces are often high-definition images. Transmitting high-definition images in a weak network environment presents difficulties in uploading and consumes high network bandwidth. Based on this, the screenshot processing method of some embodiments of this disclosure first, in response to the detection of real-time screenshot processing for the target device, obtains the real-time device state corresponding to the target device, so as to facilitate the construction of a mirror device instance for the target device based on the real-time device state. Then, an initial device instance for the target device is constructed to facilitate the subsequent construction of a mirror instance for the target device. Furthermore, the real-time device state is injected into the initial device instance to obtain a device instance. Here, by injecting the real-time device state, a device instance with the same display content as the target device can be obtained. Moreover, by obtaining the real-time device state, the high-definition screenshot transmission can avoid consuming a lot of network bandwidth; only the information content corresponding to the real-time device state needs to be transmitted, and accurate acquisition of device screenshots can still be achieved in a weak network environment. Finally, screenshot processing is performed on the aforementioned device instance to obtain an accurate real-time device screenshot. In summary, addressing the issue of high-definition screenshot transmission consuming significant network bandwidth in weak network environments, constructing a real-time device instance for the target device allows for accurate and efficient reproduction of the target device's current content. This enables accurate and efficient acquisition of device screenshots even when transmitting limited content in weak network environments.

[0071] Further reference Figure 3 The diagram illustrates a flow 300 of another embodiment of the screenshot processing method according to the present disclosure. This screenshot processing method includes the following steps: Step 301: In response to the detection of real-time screenshot processing for the target device, obtain the real-time device status corresponding to the target device.

[0072] Step 302: Construct an initial device instance for the aforementioned target device.

[0073] Step 303: Inject the real-time device status into the initial device instance to obtain the device instance.

[0074] Step 304: Perform screenshot processing on the above device instance to obtain a real-time device screenshot.

[0075] In some embodiments, the specific implementation of steps 301-304 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-204 in the corresponding embodiments will not be repeated here.

[0076] Step 305: Obtain the screenshot text description for real-time screenshot processing.

[0077] In some embodiments, the executing entity (e.g. Figure 1 The electronic device 101 shown can acquire a screenshot text description for real-time screenshot processing. The screenshot text description can be the purpose of performing the real-time screenshot processing. For example, the screenshot text description could be "provide error handling methods for the screenshot content".

[0078] In practice, the screenshot text description can be entered or selected in the corresponding page area of ​​the target device.

[0079] Step 305: Generate screenshot response content based on the screenshot text description, the real-time screenshot image, and the real-time device status.

[0080] In some embodiments, the executing entity may generate screenshot response content based on the screenshot text description, the real-time screenshot image, and the real-time device status. The screenshot response content may be a response addressing the purpose of the screenshot as described in the screenshot text description. For example, if the screenshot text description is "provide error handling methods for the screenshot content," the corresponding screenshot response content may be "the actual error handling method for the screenshot content."

[0081] As an example, the aforementioned executing entity can generate screenshot response content based on the screenshot text description, the aforementioned real-time screenshot image, and the aforementioned real-time device status, using the deployed large language model that supports intelligent responses.

[0082] Step 306: Send the above screenshot reply to the above target device.

[0083] In some embodiments, the aforementioned executing entity may send the screenshot response content to the aforementioned target device.

[0084] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 The screenshot processing method in some corresponding embodiments, in process 300, is based on the screenshot text description, the real-time screenshot image, and the real-time device status. It includes not only the visual interface content (i.e., the screenshot content) but also the underlying data content corresponding to the interface, which can accurately generate screenshot response content.

[0085] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a screenshot processing apparatus, which are similar to... Figure 2Corresponding to the method embodiments shown, this screenshot processing device can be specifically applied to various electronic devices.

[0086] like Figure 4 As shown, a screenshot processing device 400 includes: an acquisition unit 401, a construction unit 402, a state injection unit 403, and an execution unit 404. The acquisition unit 401 is configured to acquire the real-time device state corresponding to the target device in response to detecting real-time screenshot processing for a target device; the construction unit 402 is configured to construct an initial device instance for the target device; the state injection unit 403 is configured to inject the real-time device state into the initial device instance to obtain the device instance; and the execution unit 404 is configured to perform screenshot processing for the device instance to obtain a real-time device screenshot.

[0087] In some optional implementations of some embodiments, the real-time device state mentioned above includes: device environment parameter information; and the construction unit 402 can be further configured to: construct an initial device instance for the target device based on the device environment parameter information mentioned above.

[0088] In some optional implementations of certain embodiments, the above-mentioned real-time screenshot processing is to take a screenshot of the target page in the target application. The real-time device state includes: business data and page rendering information. The state injection unit 403 can be further configured to: start the application instance corresponding to the target application in the initial device instance; intercept the network request corresponding to the application instance, and inject the business data into the application instance as a real network request; and perform the following processing steps according to the page rendering information: perform page navigation on the application instance with injected network requests to navigate to the corresponding target page to obtain the target device instance displaying the target page; and scroll the target page displayed by the target device instance to the corresponding target progress to obtain the device instance.

[0089] In some optional implementations of certain embodiments, the apparatus 400 further includes a determining unit and a filtering unit (not shown in the figure). The determining unit can be configured to determine the screenshot intent for performing real-time screenshot processing. The filtering unit can be configured to filter the data in the real-time device state according to the screenshot intent, obtaining filtered data as the real-time device state.

[0090] In some optional implementations of certain embodiments, the apparatus 400 further includes a description acquisition unit, a generation unit, and a sending unit (not shown in the figure). The description acquisition unit can be configured to acquire a screenshot text description for real-time screenshot processing. The generation unit can be configured to generate screenshot response content based on the screenshot text description, the real-time screenshot image, and the real-time device status. The sending unit can be configured to send the screenshot response content to the target device.

[0091] It is understandable that the units described in the screenshot processing device 400 are related to the reference. Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the screenshot processing device 400 and the units contained therein, and will not be repeated here.

[0092] The following is for reference. Figure 5 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of electronic device 101)500. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0093] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage device 508 into a random access memory 503. The random access memory 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0094] Typically, the following devices can be connected to the input / output interface 505: input devices 506 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5Each box shown can represent a device or multiple devices as needed.

[0095] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a read-only memory 502. When the computer program is executed by the processing device 501, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0096] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0097] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0098] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to detecting real-time screenshot processing for a target device, obtain the real-time device state corresponding to the target device; construct an initial device instance for the target device; inject the real-time device state into the initial device instance to obtain a device instance; and perform screenshot processing for the device instance to obtain a real-time device screenshot.

[0099] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0101] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a construction unit, a state injection unit, and an execution unit. The names of these units do not necessarily limit the unit itself; for example, a construction unit may also be described as "a unit for constructing an initial device instance for the aforementioned target device."

[0102] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0103] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the screenshot processing methods described above.

[0104] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A screenshot processing method, comprising: In response to the detection of real-time screenshot processing of the target device, the real-time device status corresponding to the target device is obtained; Construct an initial device instance for the target device; The real-time device state is injected into the initial device instance to obtain a device instance; Perform screenshot processing on the device instance to obtain a real-time device screenshot.

2. The method according to claim 1, wherein, The real-time device status includes: device environmental parameter information; and The construction of the initial device instance for the target device includes: Based on the device environment parameter information, an initial device instance is constructed for the target device.

3. The method according to claim 1, wherein, The real-time screenshot processing involves taking screenshots of the target page within the target application. The real-time device status includes: business data and page rendering information; and The step of injecting the real-time device state into the initial device instance to obtain the device instance includes: Launch the application instance corresponding to the target application in the initial device instance; Intercept the network requests corresponding to the application instance and inject the business data into the application instance as a real network request; Based on the page rendering information, perform the following processing steps: The application instance that receives the injected network request is navigated to the corresponding target page, and the target device instance that displays the target page is obtained. Scroll the target page displayed by the target device instance to the corresponding target progress to obtain the device instance.

4. The method according to claim 1, wherein, The real-time device status is generated through the following steps: In response to the detection of real-time screenshot processing of the target device, obtain real-time initial device metadata; The real-time initial device metadata is de-identified to obtain de-identified data; The de-identified data is packaged to obtain the real-time device status.

5. The method according to claim 1, wherein, After responding to the detection of a real-time screenshot of the target device and obtaining the real-time device status corresponding to the target device, the method further includes: Determine the screenshot intent for performing real-time screenshot processing; Based on the intent of the screenshot, the data in the real-time device status are filtered to obtain filtered data, which is used as the real-time device status.

6. The method according to claim 1, wherein, The method further includes: Get the screenshot text description for real-time screenshot processing; Based on the screenshot text description, the real-time screenshot image, and the real-time device status, generate the screenshot response content; The screenshot response will be sent to the target device.

7. A screenshot processing device, comprising: The acquisition unit is configured to acquire the real-time device status corresponding to the target device in response to listening for real-time screenshot processing of the target device. The construction unit is configured to construct an initial device instance for the target device; The state injection unit is configured to inject the real-time device state into the initial device instance to obtain a device instance; The execution unit is configured to perform screenshot processing on the device instance to obtain a real-time device screenshot.

8. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images