An image acquisition method and device for screen sharing

Abstract

Embodiments of the present specification provide an image acquisition method and device for screen sharing, the method comprising: in a screen sharing process, before acquiring a current frame image for a current shared screen, obtaining current filtering window information, the current filtering window information comprising information of a window that needs to be filtered currently; based on the current filtering window information, determining a target acquisition scheme corresponding to performance information matching a preset screening condition from a target acquisition scheme list, wherein the target acquisition scheme list comprises a plurality of acquisition schemes corresponding to different performance information; and based on the target acquisition scheme, acquiring the current frame image.

Description

Technical Field

[0001] This specification relates to the field of data sharing technology, and in particular to an image acquisition method and apparatus for screen sharing. Background Technology

[0002] Currently, screen sharing software often provides user-interactive options for setting capture schemes, such as "whether to ignore (filter) windows (e.g., filter the screen sharing software's toolbar window)" and "whether to enable hardware acceleration," and informs users of the effects of selecting these options. Accordingly, before triggering a screen sharing request using the screen sharing software, users can first select the appropriate settings within the software's user interface to activate the corresponding function, i.e., set the capture scheme for the screen sharing process. Then, during screen sharing, the software uses this capture scheme to acquire each frame (image) of the shared screen for screen sharing.

[0003] As can be seen from the above process, current screen sharing software determines the capture scheme based on the user's settings before the screen sharing process, and then captures each frame of the screen to be shared according to the capture scheme to perform screen sharing. However, this capture scheme may cause resource waste in some scenarios.

[0004] Therefore, how to provide a solution for image (screen) acquisition during screen sharing that reduces resource waste has become an urgent problem to be solved. Summary of the Invention

[0005] This specification provides one or more embodiments of an image acquisition method and apparatus for screen sharing, which aims to reduce resource consumption during screen sharing and improve user experience.

[0006] According to the first aspect, an image acquisition method for screen sharing is provided, comprising:

[0007] During screen sharing, before capturing the current frame image for the currently shared screen, the current filtering window information is obtained, which includes information about the window that needs to be filtered.

[0008] Based on the current filtering window information, target acquisition schemes that match the preset filtering conditions with the corresponding performance information are determined from the target acquisition scheme list. The target acquisition scheme list includes multiple acquisition schemes with different performance information.

[0009] Based on the target acquisition scheme, the current frame image is acquired.

[0010] According to a second aspect, an image acquisition device for screen sharing is provided, comprising:

[0011] The first acquisition module is configured to acquire current filter window information during screen sharing, before acquiring the current frame image for the currently shared screen. The current filter window information includes information about the window that needs to be filtered.

[0012] The first determining module is configured to determine, based on the current filtering window information, a target acquisition scheme whose performance information matches the preset filtering conditions from the target acquisition scheme list, wherein the target acquisition scheme list includes multiple acquisition schemes with different performance information.

[0013] The first acquisition module is configured to acquire the current frame image based on the target acquisition scheme.

[0014] According to a third aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0015] According to a fourth aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in the first aspect.

[0016] According to the method and apparatus provided in the embodiments of this specification, during screen sharing, before capturing the current frame image (each frame image) for the currently shared screen, current filtering window information, including information about the windows that need to be filtered, is obtained. Then, based on the current filtering window information, a target acquisition scheme whose performance information matches preset filtering conditions is determined from a list of target acquisition schemes containing multiple acquisition schemes with different performance information. The effect corresponding to this target acquisition scheme satisfies the effect corresponding to the current filtering window information. Finally, the current frame image is captured based on the target acquisition scheme. In the above process, the acquisition scheme is not fixed during screen sharing. Before capturing each frame image for the currently shared screen, the latest information about the windows that need to be filtered (i.e., current filtering window information) needs to be obtained. Based on this current filtering window information, a target acquisition scheme that satisfies the display effect corresponding to the current filtering window information and whose performance information matches the preset filtering conditions is promptly determined from the list of target acquisition schemes. The image is then captured based on this scheme. This approach can, to some extent, avoid resource waste caused by using a fixed acquisition scheme in some situations, and to a certain extent, reduce resource consumption during screen sharing, thereby improving the user experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the implementation framework of one embodiment disclosed in this specification;

[0019] Figure 2 A schematic flowchart of an image acquisition method for screen sharing provided in an embodiment;

[0020] Figure 3A , Figure 3B and Figure 3C These are schematic diagrams of the interface displayed in the window of the current shared screen provided in the embodiments;

[0021] Figure 4 A schematic block diagram of an image acquisition device for screen sharing provided in an embodiment;

[0022] Figure 5 This is a schematic block diagram of an electronic device provided for an embodiment. Detailed Implementation

[0023] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0026] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] This specification discloses an image acquisition method and apparatus for screen sharing. The application scenarios and technical concepts of the method are first introduced below:

[0029] As mentioned earlier, current screen sharing software determines the capture scheme based on the user's settings before the screen sharing process, and then captures each frame of the screen to be shared according to the capture scheme. However, this capture scheme may cause resource waste in some scenarios.

[0030] For example, in one exemplary scenario, the user-configured capture scheme includes settings for filtering windows. However, during actual screen sharing, the windows to be filtered may not be displayed (e.g., completely transparent, not open) or may be completely obscured by other windows (whether they require filtering or not) (i.e., the windows to be filtered are invisible). In such cases, a capture scheme that does not filter these undisplayed or completely obscured windows can be used. Its display effect is the same as the capture scheme that includes settings for filtering windows, but it saves more resources. Correspondingly, the current fixed capture scheme set by the user in screen sharing suffers from resource waste.

[0031] In another exemplary scenario, the user-configured capture scheme includes settings for capturing multiple windows simultaneously. However, during actual screen sharing, it's possible that only one window (the window to be shared) may be visible. For example, it might be necessary to capture both the video conferencing interface and an annotation layer (which overlays the video conferencing interface) simultaneously, but if the annotation layer doesn't contain annotation information, it would be completely transparent. For such cases, a capture scheme that captures only one window can be used, achieving the same display effect as capturing multiple windows, but with more resources. Correspondingly, current screen sharing processes using fixed user-configured capture schemes suffer from resource waste.

[0032] In view of the above, this specification provides an image acquisition method for screen sharing, which automatically switches the underlying acquisition scheme when the content to be shared (each frame of the shared screen) meets the requirements of a better-performing acquisition scheme during the screen sharing process, thereby reducing resource consumption during the screen sharing process and improving user experience.

[0033] Figure 1This diagram illustrates an implementation scenario according to an embodiment disclosed in this specification. In this implementation scenario, multiple clients participating in screen sharing are exemplarily shown. These clients include a client that needs to share its screen, referred to as screen sharing client A, and at least one client that needs to receive the shared screen, referred to as screen receiving client B. Screen sharing client A and at least one screen receiving client B can be connected via a network (server).

[0034] In one exemplary scenario, the multiple clients participating in screen sharing can be multiple clients participating in a video conference. In yet another exemplary scenario, the multiple clients participating in screen sharing can be multiple clients participating in an online teaching scenario, wherein screen sharing client A can be a teacher's user client in the online teaching scenario, at least one screen receiving client B can be a student's user client in the online teaching scenario, and so on.

[0035] The designated storage space corresponding to screen sharing client A can pre-store a list of collection schemes, including multiple collection schemes with different performance information, such as... Figure 1 As shown, this includes acquisition scheme 1, acquisition scheme 2, ..., acquisition scheme N. Understandably, performance information characterizes the performance (or quality) of the corresponding acquisition scheme. In one scenario, the acquisition schemes in the acquisition scheme list can be sorted sequentially from best to worst (highest to lowest) according to their corresponding performance information, or sequentially from worst to best (lowest to highest) according to their corresponding performance information, or randomly.

[0036] During screen sharing, before capturing each frame of the currently shared screen (e.g., the current frame), screen sharing client A obtains current filtering window information, including information about the windows that need to be filtered. In one implementation, the current filtering window information may include window identifier information of the windows that need to be filtered.

[0037] Then, based on the current filtering window information, screen-sharing client A determines the target acquisition scheme whose performance information matches the preset filtering conditions from the acquisition scheme list. These preset filtering conditions may include conditions indicating optimal performance, or conditions indicating that the performance information of the acquisition scheme is superior to that of the acquisition scheme corresponding to the previous frame image. Accordingly, the target acquisition scheme matching the preset filtering conditions is either the target acquisition scheme with the optimal performance, or the target acquisition scheme whose performance information is superior to that of the acquisition scheme corresponding to the previous frame image. In other words, by using preset filtering conditions, a relatively superior acquisition scheme can be selected for the current frame image as its target acquisition scheme.

[0038] Understandably, the display effect corresponding to the target acquisition scheme satisfies the display effect corresponding to the current filter window information (i.e., the display effect corresponding to the target acquisition scheme is the same as the display effect corresponding to the current filter window information). Here, the display effect corresponding to the target acquisition scheme can refer to the display effect (i.e., the shared content) of the image acquired by the currently shared screen based on the target acquisition scheme. The display effect corresponding to the current filter window information can refer to the display effect (including other windows besides the window to be filtered as indicated by the current filter window information) of the image acquired by the currently shared screen based on the current filter window information.

[0039] Next, screen sharing client A captures the current frame image based on the target acquisition scheme. Then, on one hand, screen sharing client A sends the current frame image to at least one screen receiving client B via the network (server). At least one screen receiving client B receives the current frame image and displays it. On the other hand, screen sharing client A uses the next frame image as the new current frame image to be captured, and for this new current frame image, executes the user screen sharing image acquisition process provided in the embodiments of this specification, thus cyclically capturing images of the currently shared screen for screen sharing.

[0040] In the above process, during screen sharing, the acquisition scheme is not fixed. Before acquiring each frame of image for the currently shared screen, it is necessary to obtain the latest information of the window to be filtered (i.e., the current filter window information). Based on this current filter window information, the target acquisition scheme that meets the display effect corresponding to the current filter window information and matches the preset filtering conditions is promptly determined from the list of target acquisition schemes. Based on this, the image is acquired. To a certain extent, this can avoid the resource waste caused by using a fixed acquisition scheme in some cases, and to a certain extent, reduce the resource consumption during screen sharing and improve the user experience.

[0041] The image acquisition method for screen sharing provided in this specification will be described in detail below with reference to specific embodiments.

[0042] Figure 2A flowchart of an image acquisition method for screen sharing according to one embodiment of this specification is shown. This method can be implemented using a target program, which can exist as a standalone client or as a plugin for existing client software with screen sharing functionality. The target program can be installed on an electronic device, which can be any device, equipment, platform, device cluster, etc., with computing and processing capabilities. The following describes the image acquisition process for screen sharing provided in the embodiments of this specification using the target program as a client (hereinafter referred to as the target client). Figure 2 As shown, the method includes the following steps S210-S230:

[0043] In step S210, during screen sharing, before capturing the current frame image for the currently shared screen, current filtering window information is obtained. This current filtering window information includes information about the windows that need to be filtered. In one implementation, a window can refer to an interface provided by the client for interaction with the user, and a client can provide at least one window.

[0044] In one exemplary scenario, the target client can be one of multiple clients participating in a video conference, in which the target client needs to share its screen (i.e., the currently shared screen) with other clients participating in the video conference. In yet another exemplary scenario, the target client can be one of multiple clients participating in an online teaching scenario, where the target client can be a teacher's client in the online teaching scenario, and can share its screen (i.e., the currently shared screen) with student clients (other clients) participating in the online teaching scenario, and so on.

[0045] During screen sharing, before capturing the current frame image of the shared screen, the target client obtains the current filtering window information, which may include information about the windows that need to be filtered. The current shared screen may refer to the screen of the electronic device on which the target client is located.

[0046] Understandably, the current frame image can refer to the image to be acquired for the currently shared screen. For example, when the screen sharing process begins (before acquiring an image for the currently shared screen), the first frame image to be acquired for the currently shared screen is determined to be the current frame image. Before acquiring this first frame image (i.e., the current frame image), the filter window information at this time (before acquiring the first frame image) is acquired as the current filter window information (also called the current filter window information corresponding to the first frame image). After acquiring the first frame image, the second frame image to be acquired for the currently shared screen is determined to be the current frame image. Before acquiring this second frame image (i.e., the current frame image), the filter window information at this time (before acquiring the second frame image) is acquired as the current filter window information (also called the current filter window information corresponding to the second frame image). And so on. Before acquiring each frame image for the currently shared screen, the filter window information before acquiring that frame image is acquired is acquired as the corresponding current filter window information.

[0047] To enhance user experience, the target client offers users the ability to customize their data collection scheme. Correspondingly, the target client provides a user interface displaying various options for setting the collection scheme, such as: whether to filter windows (e.g., specifically whether to filter windows of type xx or yy), whether to enable hardware acceleration, and whether to collect data from multiple windows. Users can select and configure these options to determine the appropriate collection scheme, achieving the desired sharing effect (or display effect).

[0048] Users can set corresponding window filtering information before or during the screen sharing process. This window filtering information can include: information about the windows that the user wants to filter, such as: filtering all toolbar windows of the target client; not filtering any windows (of the target client or all clients on the target client's electronic device); filtering certain or certain toolbar windows of the target client; filtering certain or certain types of windows of other clients on the target client's electronic device, etc.

[0049] Furthermore, considering that during screen sharing, the following situations may occur: the user has set windows to be filtered, but the windows to be filtered are not visible in the currently shared screen (i.e., they are not displayed in the current shared screen). In this case, the windows to be filtered may be completely transparent, completely obscured by other windows, or not open. For example, the windows that the user sets to be filtered include window 1 (e.g., a annotation layer, which is transparent if the annotation information is not included) and window 2, such as... Figure 3A As shown, window 1 is in a transparent state and is invisible. Window 2 (represented by a dashed box) is completely covered (obscured) by window 3 (represented by a solid box) and is also invisible.

[0050] In this situation, although the user has set up window filtering, in practice, it is not necessary to filter out windows that are not visible when capturing images. Accordingly, in one embodiment, step S210 may include: determining the current filtering window information based on the window filtering information set by the user for the screen sharing process and the state information of each window in the current shared screen. The state information of each window in the current shared screen can indicate whether the corresponding window is visible, such as whether the corresponding window is transparent, and whether it is covered (obscured) by other windows. Each window in the current shared screen may refer to windows that are open.

[0051] Specifically, in one implementation, step S210 may include: incorporating the information of windows that need to be filtered, as represented by the window filtering information, and whose corresponding state information indicates a visible state, into the current filtering window information. For example, the user sets the windows to be filtered to include window A and window B (window filtering information). The state information corresponding to window A indicates that window A is completely covered (occluded) by window C and is in an invisible state. The state information corresponding to window B indicates that it is in a visible state. Accordingly, window B is included in the current filtering window information, while window A is not included in the current filtering window information.

[0052] In one implementation, the windows whose corresponding state information represents a visible state include: windows that are not transparent and are not completely covered by other windows.

[0053] In another implementation, the user can directly use the target client's default settings without setting filter window information for the screen sharing process. In one case, if the target client's default settings include settings to filter some or all of the target client's windows, then step S210 may include: determining the current filter window information based on the status information of each window in the currently shared screen.

[0054] In another implementation, if the user does not set window filtering information for the screen sharing process, and the target client does not filter windows by default, the obtained current filtered window information can be empty (i.e., it does not include the window identification information of the window).

[0055] In one scenario, the current filtering window information can exist in the form of a list, which may include the window identifier information of the windows that need to be filtered.

[0056] After determining the current filtering window information, in step S220, the target client, based on the current filtering window information, determines the target acquisition scheme whose performance information matches the preset filtering conditions from the target acquisition scheme list. This target acquisition scheme list includes multiple acquisition schemes with different performance information. In one implementation, the preset filtering conditions may include conditions for optimal performance, or conditions for performance information that is superior to the acquisition scheme corresponding to the previous frame image. Accordingly, the target acquisition scheme matching the preset filtering conditions is either the target acquisition scheme with optimal performance, or the target acquisition scheme whose performance information is superior to the acquisition scheme corresponding to the previous frame image. In other words, by using preset filtering conditions, a relatively superior acquisition scheme can be selected for the current frame image as its target acquisition scheme.

[0057] In one implementation, the data acquisition scheme may include at least one of the following: whether to filter windows, whether to enable hardware acceleration, and whether to acquire multiple windows. In another implementation, the data acquisition scheme may also include: whether to filter windows of type xx or yy, and whether to acquire windows of type zz, etc.

[0058] Understandably, different acquisition scheme settings result in varying resource consumption on the target client's electronic device when acquiring images, leading to different performance (information) levels. Specifically, with other functional settings identical, an acquisition scheme without window filtering performs better than one requiring window filtering; similarly, an acquisition scheme capturing only one window performs better than one capturing multiple windows. Furthermore, an acquisition scheme requiring fewer filtered windows performs better than one requiring more filtered windows; and an acquisition scheme with fewer captured windows performs better than one with more captured windows.

[0059] Without enabling hardware acceleration, the best performance is generally achieved by setting the acquisition scheme to not filter windows and acquire data from only one window.

[0060] Once the data acquisition scheme is determined, its corresponding performance is determined, and the performance information that can characterize the performance of the corresponding data acquisition scheme (or its performance level) is determined.

[0061] In one implementation, the designated storage space of the target client can store a list of acquisition schemes (which can be called the initial acquisition scheme list), including multiple acquisition schemes corresponding to different performance information. These multiple acquisition schemes can be pre-arranged and combined based on various functions supported by the target client.

[0062] In one scenario, the initial collection scheme list can be directly determined as the target collection scheme list. In another scenario, after determining the user's settings for this screen sharing process, collection schemes that do not meet the display effects corresponding to those settings can be removed from the initial collection scheme list to obtain the target collection scheme list. This reduces the number of collection schemes in the target list and decreases the search complexity. The settings may include window filtering information (and / or window collection information mentioned later), where window filtering information includes information about the windows the user expects to filter, and window collection information includes information about the windows the user expects to collect.

[0063] For example, a user sets not to filter windows (window filtering information) and collects multiple windows (window collection information); accordingly, the collection schemes with filtered windows in the initial collection scheme list can be removed to obtain a target collection scheme list that includes multiple collection schemes with unfiltered windows.

[0064] In one scenario, the collection schemes in the collection scheme list can be sorted sequentially from best to worst according to their corresponding performance information, or sequentially from worst to best according to their corresponding performance information, or randomly.

[0065] Subsequently, after determining the current filter window information, the target client can select from the list of target acquisition schemes which meets the display effect corresponding to the current filter window information and whose corresponding performance information matches the preset filtering conditions.

[0066] Understandably, in one implementation, the preset filtering conditions may include the condition of optimal performance information. Accordingly, the target client can determine the optimal collection scheme from the list of target collection schemes that satisfies the display effect corresponding to the current filter window information and has the optimal performance information, and use it as the target collection scheme.

[0067] In another implementation, the preset filtering conditions may include conditions that the corresponding performance information is better than the performance information of the previous frame image. Accordingly, in one case, if the current frame image is not the first frame image, and the performance information corresponding to the acquisition scheme of the previous frame image is not the optimal acquisition scheme, at least one first acquisition scheme that satisfies the display effect corresponding to the current filtering window information and whose corresponding performance information is better than the performance information of the acquisition scheme corresponding to the previous frame image can be determined from the list of target acquisition schemes. Then, one first acquisition scheme can be randomly selected from the determined at least one first acquisition scheme as the target acquisition scheme.

[0068] In another scenario, if the current frame is not the first frame, and the acquisition scheme of the previous frame is the best performing acquisition scheme, then the acquisition scheme of that previous frame can be directly identified as the target acquisition scheme. In yet another scenario, if the current frame is the first frame, then the acquisition scheme that best meets the display requirements of the current filter window and has the best corresponding performance information can be directly identified as the target acquisition scheme from the target acquisition scheme list.

[0069] The following example, using preset screening conditions including the condition of optimal corresponding performance information, illustrates the process of determining the target acquisition scheme in the embodiments of this specification.

[0070] Specifically, in one embodiment, step S220 may include the following steps 11-12: In step 11, if the current frame image is not the first frame image in the screen sharing process, obtain the historical filtering window information corresponding to the previous frame image of the current frame image. In step 12, if the current filtering window information is consistent with the historical filtering window information, then determine the acquisition scheme corresponding to the previous frame image as the target acquisition scheme.

[0071] In this implementation, if the current frame image is determined to be a non-first frame image in the screen sharing process, it can be determined that the target client has already captured an image for the currently shared screen based on one of the capture schemes in the target capture scheme list, i.e., the previous frame image of the current frame image. Accordingly, the target client can obtain the historical filtering window information corresponding to the previous frame image of the current frame image, i.e., the filtering window information obtained by the target client before capturing that previous frame image. It is then determined whether the current filtering window information is consistent with the historical filtering window information.

[0072] If the current filter window information is determined to be consistent with the historical filter window information, then the acquisition scheme corresponding to the previous frame image is the acquisition scheme that can meet the display effect corresponding to the current filter window information and has the best corresponding performance information. Accordingly, the acquisition scheme corresponding to the previous frame image is determined as the target acquisition scheme.

[0073] If the current filter window information is found to be inconsistent with the historical filter window information, it is necessary to re-determine from the target acquisition scheme list the target acquisition scheme that satisfies the display effect corresponding to the current filter window information and has the best corresponding performance information.

[0074] To facilitate the identification of the optimal target acquisition scheme from the list that best meets the display requirements of the current filter window, the schemes in this list can be sorted in descending order of performance (from best to worst), or in ascending order of performance. Each acquisition scheme is assigned a serial number, which can be incremented in ascending order based on its position in the target acquisition scheme list. For example, the acquisition scheme at the beginning of the list has a serial number of 1, and subsequent acquisition schemes have serial numbers incremented by 1.

[0075] In another embodiment, when the target acquisition schemes are sorted in order of performance information from high to low (from best to worst) in the target acquisition scheme list, step S220 may also include the following step 13: In step 13, if the current filter window information is inconsistent with the historical filter window information and the current filter window information is empty, the acquisition scheme at the top of the target acquisition scheme list is determined as the target acquisition scheme.

[0076] In this implementation, if the current filter window information differs from the historical filter window information, the acquisition scheme corresponding to the previous frame image can be determined. For the current frame image, this may not necessarily be the optimal acquisition scheme that satisfies the display effect of the current filter window information and has the best performance. The acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information, indicating that acquisition schemes ranked higher in the list have better performance. Furthermore, based on the sequence number of the acquisition scheme corresponding to the previous frame image, it can be determined whether the acquisition scheme corresponding to that previous frame image is the optimal (i.e., the one with the best corresponding performance information) acquisition scheme in the target acquisition scheme list.

[0077] If the current filter window information is empty, it can be determined that there is no window that needs to be filtered in the current shared screen. In this case, it is necessary to switch from the current acquisition scheme that needs to filter windows to the acquisition scheme that does not filter windows. However, if the default is to acquire one window (or the current acquisition window information mentioned later indicates that there is only one window that needs to be acquired), the acquisition scheme that is first in the target acquisition scheme list can be directly determined as the target acquisition scheme.

[0078] If we consider the current acquisition window information mentioned later, and the current filter window information is empty, and the current acquisition window information indicates that there are multiple windows to be acquired, then we need to select the first acquisition scheme that sets no filter window and sets multiple acquisition windows from the target acquisition scheme list in top-to-bottom order, and use it as the target acquisition scheme.

[0079] In another embodiment, when the acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information; step S220 may further include the following step 14: In step 14, if the current filter window information is inconsistent with the historical filter window information and the historical filter window information is empty, in the target acquisition scheme list, starting from the next acquisition scheme corresponding to the previous frame image, each acquisition scheme is traversed sequentially until an acquisition scheme that satisfies the display effect corresponding to the current filter window information is determined as the target acquisition scheme.

[0080] In this implementation, if the current filtering window information is inconsistent with the historical filtering window information, and the historical filtering window information is empty, then the current filtering window information is determined to be not empty, meaning it includes window identifier information for at least one window. Accordingly, it is necessary to switch from a capture scheme that does not filter windows to a capture scheme that requires filtering windows. In the target capture scheme list, the capture schemes that do not filter windows are listed before the capture schemes that require filtering windows. Therefore, the target capture scheme list can be iterated sequentially, starting from the capture scheme following the one corresponding to the previous frame image, until a capture scheme that satisfies the display effect corresponding to the current filtering window information is determined, and this is taken as the target capture scheme.

[0081] For example: If the default is to capture one window (or the current capture window information mentioned later indicates that there is only one window that needs to be captured), and the target client only provides the function of setting not to filter windows and filtering windows, the first capture scheme that sets the window to be filtered and captures one window can be determined from the target capture scheme list, starting from the capture scheme after the capture scheme corresponding to the previous frame image (that is, the capture scheme that satisfies the display effect corresponding to the current filter window information), and used as the target capture scheme.

[0082] For example, if the current acquisition window information mentioned later indicates that there are multiple windows that need to be acquired, and the target client only provides the function of setting not to filter windows and filtering windows, then the first acquisition scheme that sets the window to be filtered and acquires multiple windows can be determined from the target acquisition scheme list, starting from the acquisition scheme after the acquisition scheme corresponding to the previous frame image, and used as the target acquisition scheme.

[0083] In another embodiment, when the acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information, if the current filter window information is inconsistent with the historical filter window information, and both the historical and current filter window information are not empty (or both are empty), it can be determined whether the acquisition scheme corresponding to the previous frame image is the best-performing acquisition scheme in the target acquisition scheme list based on the sequence number of the acquisition scheme corresponding to the previous frame image. If it is determined based on the sequence number of the acquisition scheme corresponding to the previous frame image that there is an even better-performing acquisition scheme, the target client can traverse each acquisition scheme in the target acquisition scheme list from top to bottom until it determines the acquisition scheme that satisfies the display effect corresponding to the current filter window information, and uses it as the target acquisition scheme.

[0084] In another embodiment, when the acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information; step S220 may also include the following step 21: in step 21, if the current frame image is the first frame image in the screen sharing process and the current filter window information is empty, the acquisition scheme at the top of the target acquisition scheme list is determined as the target acquisition scheme.

[0085] In this implementation, if the current frame image is the first frame image in the screen sharing process and the current filter window information is empty, a window is captured by default (or the current capture window information mentioned later indicates that there is only one window to be captured). The capture scheme at the top of the target capture scheme list is the capture scheme with the highest performance (superior) and meets the display effect corresponding to the current filter window information (no filter window and only one capture window). Accordingly, the capture scheme at the top of the target capture scheme list can be directly determined as the target capture scheme.

[0086] In another embodiment, when the acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information, step S220 may further include the following step 22: In step 22, if the current frame image is the first frame image in the screen sharing process and the current filter window information is not empty, the acquisition schemes in the target acquisition scheme list are traversed sequentially until an acquisition scheme that satisfies the display effect corresponding to the current filter window information is determined as the target acquisition scheme.

[0087] In this implementation, if the current frame image is the first frame image in the screen sharing process, and the current filter window information is not empty (i.e., it includes window identifier information of at least one window), then in the target acquisition scheme list, each acquisition scheme in the target acquisition scheme list is traversed sequentially from top to bottom until an acquisition scheme that satisfies the display effect corresponding to the current filter window information is determined as the target acquisition scheme.

[0088] For example: if the default is to collect one window (or the current collection window information mentioned later indicates that there is only one window that needs to be collected), and the target client only provides the functions of not filtering windows and filtering windows, then the collection schemes in the target collection scheme list are traversed in order from top to bottom, and the first collection scheme that sets a filtering window and collects one window is determined as the target collection scheme.

[0089] For example, if the current collection window information mentioned later indicates that there are multiple windows that need to be collected, and the target client only provides the functions of not filtering windows and filtering windows, then the collection schemes in the target collection scheme list are traversed in order from top to bottom, and the first collection scheme that sets a filtering window and collects multiple windows is determined as the target collection scheme.

[0090] Considering that users can set corresponding window filtering information before or during the screen sharing process, they can also set corresponding window capture information, which may include, for example, capturing one window or capturing multiple windows (or capturing X windows).

[0091] Furthermore, considering that during screen sharing, the following situation may occur: the user sets up to capture multiple windows, but only one of these windows is visible in the currently shared screen (i.e., one window is in a visible state). For example, the user sets up to capture multiple windows, window 4 and window 5, where window 5 is an annotation layer, and window 5 overlaps window 4. Figure 3B As shown, when window 5 does not contain annotation information, it is transparent and invisible. In this case, although the user has set up to capture multiple windows, using a more efficient method of capturing only one window can achieve the desired display effect. It is possible to capture only window 4 (i.e., capture the content displayed within it). For example... Figure 3C As shown, when window 5 includes annotation information, window 5 is in a non-transparent state and is in a visible state. At this time, windows 4 and 5 can be captured (that is, the content displayed in window 4 and the content displayed in window 5 can be captured).

[0092] Based on the above considerations, when the user sets corresponding window capture information for the screen sharing process, before capturing the current frame image of the currently shared screen, the current capture window information can be determined based on the window capture information and the state information of each window in the currently shared screen. This current capture window information includes information representing the window that needs to be captured (e.g., window identifier information).

[0093] Specifically, the information of the window whose status information is currently visible and needs to be collected can be included in the current collection window information. That is, the current collection window information includes the information of the window whose status information is currently visible and needs to be collected.

[0094] After obtaining the aforementioned current acquisition window information for the screen sharing process, based on the current filter window information and the aforementioned current acquisition window information, a target acquisition scheme is determined from the target acquisition scheme list. The target acquisition scheme is the acquisition scheme that satisfies the display effect corresponding to the current filter window information and the current acquisition window information and has the best corresponding performance information (step S220).

[0095] For example, the target acquisition schemes are sorted in descending order of performance. The target acquisition scheme list includes: the first acquisition scheme (no filtering supported, acquires only one window); the second acquisition scheme (no filtering supported, acquires multiple windows); the third acquisition scheme (supports filtering, acquires only one window); and the fourth acquisition scheme (supports filtering, acquires multiple windows). If the current window acquisition information indicates that windows a, b, and c need to be acquired (i.e., windows a, b, and c are all visible), and the current filter window list is empty, then the second acquisition scheme satisfies the display effect corresponding to the current filter window information and the current acquisition window information. Therefore, the second acquisition scheme is determined as the target acquisition scheme.

[0096] After the target client determines the target acquisition scheme, in step S230, it acquires the current frame image based on the target acquisition scheme. In this step, the target client acquires the current frame image of the currently shared screen based on the acquisition method set in the target acquisition scheme. In one implementation, if the current filter window information is empty, the current frame image can be acquired by directly taking a screenshot of the current shared screen. In another implementation, if the current filter window information is not empty, the current frame image can be acquired by taking a screenshot of the current shared screen after filtering all windows included in the current filter window information.

[0097] In one embodiment, after the target client acquires the current frame image, it sends the current frame image to the screen receivers corresponding to the screen sharing process, so that each screen receiver receives and displays the current frame image. Simultaneously, the target client uses the next frame image as the new current frame image to be acquired, and executes the user screen sharing image acquisition process provided in this embodiment for the new current frame image, thus cyclically acquiring images of the currently shared screen for screen sharing.

[0098] In this embodiment, the acquisition scheme is not fixed during screen sharing. Before acquiring each frame of image for the currently shared screen, it is necessary to obtain the latest information of the window to be filtered (i.e., the current filtering window information). Based on the current filtering window information, the target acquisition scheme that meets the display effect corresponding to the current filtering window information and matches the preset filtering conditions (e.g., optimal) is determined from the target acquisition scheme list (multiple acquisition schemes with different performance information). The image is then acquired based on the target acquisition scheme. To a certain extent, this can avoid the resource waste caused by a fixed acquisition scheme in some cases, and reduce the resource consumption during screen sharing to a certain extent, thereby improving the user experience.

[0099] In one embodiment, after receiving a screen sharing request, the target client responds to the screen sharing request by initializing the first acquisition scheme in the target acquisition scheme list. During the initialization process, initialization may fail. In this case, the first acquisition scheme in the target acquisition scheme list can be initialized multiple times. After multiple unsuccessful initialization attempts, the next acquisition scheme in the target acquisition scheme list can be re-initialized, and the acquisition scheme that failed initialization can be removed from the target acquisition scheme list. The sequence number of each acquisition scheme in the new target acquisition scheme list can then be determined.

[0100] In another embodiment, during screen sharing, when the target client attempts to capture an image for the currently shared screen based on the determined target acquisition scheme, acquisition failures may occur. If the number of consecutive failures to capture an image based on the target acquisition scheme exceeds a preset number, the image is directly captured based on the next acquisition scheme in the target acquisition scheme list. The target acquisition scheme is then removed from the target acquisition scheme list, and the sequence number of each acquisition scheme in the new target acquisition scheme list is determined.

[0101] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0102] Corresponding to the above method embodiments, this specification provides an image acquisition device 400 for screen sharing, the schematic block diagram of which is shown below. Figure 4 As shown, it includes:

[0103] The first acquisition module 410 is configured to acquire current filter window information before acquiring the current frame image for the currently shared screen during the screen sharing process. The current filter window information includes information about the window that needs to be filtered.

[0104] The first determining module 420 is configured to determine, based on the current filtering window information, a target acquisition scheme whose corresponding performance information matches the preset filtering conditions from the target acquisition scheme list, wherein the target acquisition scheme list includes multiple acquisition schemes corresponding to different performance information.

[0105] The first acquisition module 430 is configured to acquire the current frame image based on the target acquisition scheme.

[0106] In one optional implementation, the target acquisition scheme that matches the preset filtering conditions includes: the target acquisition scheme with the best corresponding performance information, or the target acquisition scheme with performance information that is better than the acquisition scheme corresponding to the previous frame image of the current frame image.

[0107] In one optional implementation, the first acquisition module 410 is specifically configured to determine the current filter window information based on the window filtering information set by the user for the screen sharing process and the status information of each window in the current shared screen.

[0108] In one optional implementation, the first acquisition module 410 is specifically configured to include the information of the windows that need to be filtered, as represented by the window filtering information, and whose corresponding state information represents a visible state, into the current filtering window information.

[0109] In one alternative implementation, the windows whose corresponding state information is characterized as visible include: windows that are not transparent and are not completely covered by other windows.

[0110] In one alternative implementation, the first determining module 420 includes:

[0111] The first acquisition unit (not shown in the figure) is configured to acquire the historical filtering window information corresponding to the previous frame image of the current frame image if the current frame image is a non-first frame image in the screen sharing process.

[0112] The first determining unit (not shown in the figure) is configured to determine the acquisition scheme corresponding to the previous frame image as the target acquisition scheme if the current filtering window information is consistent with the historical filtering window information.

[0113] In one alternative implementation, the target acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information;

[0114] The first determining module 420 further includes a second determining unit (not shown in the figure), configured to determine the first acquisition scheme in the target acquisition scheme list as the target acquisition scheme if the current filter window information is inconsistent with the historical filter window information and the current filter window information is empty.

[0115] In one optional implementation, the first determining module 420 further includes:

[0116] The third determining unit (not shown in the figure) is configured to, if the current filtering window information is inconsistent with the historical filtering window information and the historical filtering window information is empty, in the target acquisition scheme list, starting from the next acquisition scheme corresponding to the acquisition scheme of the previous frame image, sequentially traverse each acquisition scheme until an acquisition scheme that satisfies the display effect corresponding to the current filtering window information is determined as the target acquisition scheme.

[0117] In one alternative implementation, the target acquisition schemes in the target acquisition scheme list are sorted in descending order of performance information;

[0118] The first determining module 420 includes:

[0119] The fourth determining unit (not shown in the figure) is configured to determine the first acquisition scheme in the target acquisition scheme list as the target acquisition scheme if the current frame image is the first frame image in the screen sharing process and the current filter window information is empty.

[0120] In one alternative embodiment, the first determining module 420 further includes:

[0121] The fifth determining unit (not shown in the figure) is configured to, if the current frame image is the first frame image in the screen sharing process and the current filter window information is not empty, sequentially traverse the acquisition schemes in the target acquisition scheme list until an acquisition scheme that satisfies the display effect corresponding to the current filter window information is determined as the target acquisition scheme.

[0122] In one optional implementation, the first determining module 420 is specifically configured to acquire current acquisition window information for the screen sharing process;

[0123] Based on the current filter window information and the current acquisition window information, the target acquisition scheme is determined from the list of target acquisition schemes. The target acquisition scheme is the acquisition scheme that satisfies the display effect corresponding to the current filter window information and the current acquisition window information and has the best corresponding performance information.

[0124] In one optional implementation, the information collected by the current collection window includes information representing the window that needs to be collected and whose corresponding state information is represented as visible.

[0125] In one alternative embodiment, the device further includes:

[0126] The sending module (not shown in the figure) is configured to send the current frame image to the screen receiver corresponding to the screen sharing process for screen sharing.

[0127] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0128] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device 500 suitable for implementing the embodiments of this specification. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments described in this specification.

[0129] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0130] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0131] In particular, according to embodiments of this specification, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this specification include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of embodiments of this specification.

[0132] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the image acquisition method for screen sharing provided in this specification.

[0133] It should be noted that the computer-readable medium described in the embodiments of this specification may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the embodiments of this specification, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the embodiments of this specification, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (Radio Frequency), etc., or any suitable combination thereof.

[0134] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: during screen sharing, before acquiring the current frame image for the currently shared screen, acquire current filter window information, the current filter window information including information about the windows currently to be filtered; based on the current filter window information, determine, from a list of target acquisition schemes, a target acquisition scheme whose corresponding performance information matches preset filtering conditions, wherein the list of target acquisition schemes includes multiple acquisition schemes corresponding to different performance information; and acquire the current frame image based on the target acquisition scheme.

[0135] Computer program code for performing the operations of the embodiments of this specification can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for storage media and computing devices are basically similar to the method embodiments, so they are described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0137] Those skilled in the art will recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0138] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image acquisition method for screen sharing, comprising: During screen sharing, before capturing the current frame image for the currently shared screen, the current filter window information is determined based on the window filtering information set by the user for the screen sharing process and the state information of each window in the currently shared screen. The current filter window information includes the information of the window that needs to be filtered, and the state information corresponding to the window in the current filter window information is represented as the visible state. Based on the current filtering window information, target acquisition schemes that match the preset filtering conditions with the corresponding performance information are determined from the target acquisition scheme list. The target acquisition scheme list includes multiple acquisition schemes with different performance information. Based on the target acquisition scheme, the current frame image is acquired.

2. The method as described in claim 1, wherein, The target acquisition scheme that matches the preset filtering conditions includes: the target acquisition scheme with the best corresponding performance information, or the target acquisition scheme with performance information that is better than the acquisition scheme corresponding to the previous frame image of the current frame image.

3. The method as described in claim 2, wherein, Determining the current filtering window information includes: The information of windows that need to be filtered, as represented by the window filtering information, and whose corresponding state information represents a visible state, are included in the current filtering window information.

4. The method of claim 3, wherein, The windows whose corresponding state information is represented as visible include: windows that are not transparent and are not completely covered by other windows.

5. The method of claim 1, wherein, The process of determining the target acquisition scheme that matches the corresponding performance information with the preset screening conditions includes: If the current frame image is not the first frame image in the screen sharing process, obtain the historical filtering window information corresponding to the previous frame image of the current frame image; If the current filtering window information is consistent with the historical filtering window information, then the acquisition scheme corresponding to the previous frame image is determined to be the target acquisition scheme.

6. The method of claim 5, wherein, The target acquisition schemes in the list are sorted in descending order of performance information; The method further includes: if the current filter window information is inconsistent with the historical filter window information, and the current filter window information is empty, the first-ranked acquisition scheme in the target acquisition scheme list is determined as the target acquisition scheme.

7. The method of claim 6, further comprising: If the current filter window information is inconsistent with the historical filter window information, and the historical filter window information is not empty, in the target acquisition scheme list, starting from the next acquisition scheme corresponding to the acquisition scheme of the previous frame image, each acquisition scheme is traversed sequentially until an acquisition scheme that satisfies the display effect corresponding to the current filter window information is determined as the target acquisition scheme.

8. The method of claim 1, wherein, The target acquisition schemes in the list are sorted in descending order of performance information; The process of determining the target acquisition scheme that matches the corresponding performance information with the preset screening conditions includes: If the current frame image is the first frame image in the screen sharing process, and the current filter window information is empty, the acquisition scheme that is first in the target acquisition scheme list is determined as the target acquisition scheme.

9. The method of claim 8, further comprising: If the current frame image is the first frame image in the screen sharing process, and the current filter window information is not empty, the acquisition schemes in the target acquisition scheme list are traversed sequentially until an acquisition scheme that satisfies the display effect corresponding to the current filter window information is determined, and this scheme is taken as the target acquisition scheme.

10. The method of claim 1, wherein, The process of determining the target acquisition scheme that matches the corresponding performance information with the preset screening conditions includes: Obtain the current acquisition window information for the screen sharing process; Based on the current filter window information and the current acquisition window information, the target acquisition scheme is determined from the list of target acquisition schemes. The target acquisition scheme is the acquisition scheme that satisfies the display effect corresponding to the current filter window information and the current acquisition window information and has the best corresponding performance information.

11. The method of claim 10, wherein, The current acquisition window information includes information representing the window that needs to be acquired and whose corresponding status information is represented as visible.

12. The method according to any one of claims 1-11, further comprising: The current frame image is sent to the screen receiver corresponding to the screen sharing process to perform screen sharing.

13. An image acquisition device for screen sharing, comprising: The first acquisition module is configured to, during the screen sharing process, before acquiring the current frame image for the currently shared screen, determine the current filter window information based on the window filtering information set by the user for the screen sharing process and the state information of each window in the currently shared screen. The current filter window information includes the information of the window that needs to be filtered, and the state information corresponding to the window in the current filter window information is characterized as the visible state. The first determining module is configured to determine, based on the current filtering window information, a target acquisition scheme whose performance information matches the preset filtering conditions from the target acquisition scheme list, wherein the target acquisition scheme list includes multiple acquisition schemes with different performance information. The first acquisition module is configured to acquire the current frame image based on the target acquisition scheme.

14. An electronic device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-12.

Images