Screen projection control method and device, electronic equipment and storage medium

Abstract

The invention provides a screen projection control method and device, electronic equipment and a storage medium. The projection screen control method is applied to a projection screen receiving end, and when the projection screen receiving end is in a projection screen state, a projection screen display type is determined based on hierarchical region information of a projection screen window. And when it is determined that the screen projection display type is multi-path simultaneous display and the decoder load is greater than a preset threshold, performing decoding resource allocation on each path of screen projection window, and indicating the screen projection sending end of each path to adjust the screen projection video stream according to the decoding resource allocated to each path of screen projection window. Therefore, the projection screen windows which are displayed at the same time are dynamically adjusted, system resources are saved, consumption of excessive system resources is avoided, and meanwhile user experience is not affected. The problem that in the prior art, when multi-path screen projection is carried out, excessive system resources are consumed easily, an equipment system is unstable, and user experience is affected is solved.

Description

Technical Field

[0001] The present application relates to the field of screen projection technology, and in particular to a screen projection control method, device, electronic device and storage medium. Background Art

[0002] With the development of technology, the application of screen projection technology is becoming more and more popular. Users can project screens between different terminal devices and share the displayed content on the screen. For example, you can project videos on your mobile phone to a smart TV for playback, so that users can watch videos on their mobile phones on the smart TV.

[0003] In the prior art, when performing multi-channel screen projection, no matter what state each screen projection window is in, the screen will be projected according to fixed parameters, continuously consuming system resources, which may easily lead to excessive consumption of system resources, causing device system instability and affecting user experience. Summary of the invention

[0004] In view of this, the present application provides a screen projection control method, device, electronic device and storage medium to solve the problem in the prior art that when performing multi-channel screen projection, it is easy to consume too many system resources, cause device system instability, and affect user experience.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application discloses a screen projection control method, which is applied to a screen projection receiving end, and the method includes:

[0007] When in the screen projection state, the screen projection display type is determined based on the screen projection window level area information; wherein the screen projection window level area information includes the screen projection window level information, the screen projection window size information and the screen projection window coordinate information; the screen projection display type includes multi-channel simultaneous display and single-channel full-screen maximized display;

[0008] When it is determined that the projection display type is multi-channel simultaneous display and the decoder load is greater than a preset threshold, decoding resources are allocated to each projection window;

[0009] According to the decoding resources allocated to each screen projection window, instruct each screen projection sender to adjust the screen projection video stream.

[0010] Optionally, the above method, when determining that the projection display type is a single-channel full-screen maximized display, controls the projection window in the background non-display state to enter the projection silent state.

[0011] Optionally, in the above method, the step of allocating decoding resources to each screen projection window includes:

[0012] Obtain the idle decoding resources of the decoder and the visible area size of each screen projection window;

[0013] Decoding resources are allocated to each screen projection window based on the idle decoding resources and the visible area size of each screen projection window.

[0014] Optionally, in the above method, the step of instructing each screen projection sending end to adjust the screen projection video stream according to the decoding resources allocated to each screen projection window includes:

[0015] For each screen projection window, the projection screen parameters of the current screen projection window are calculated according to the allocated decoding resources; wherein the projection screen parameters include frame rate, resolution, and bit rate;

[0016] The projection picture parameters are sent to the projection sending end to trigger the projection sending end to adjust the projection video stream based on the projection picture parameters.

[0017] Optionally, in the above method, controlling the projection window in the background non-display state to enter the projection silent state includes:

[0018] Determine the projection window in the background non-display state based on the projection window hierarchical area information;

[0019] For each projection window that is in the background non-display state, the interface of the projection window is set to a hidden state, and the projection video stream is decoded and optimized based on the preset projection protocol.

[0020] Optionally, in the above method, the decoding optimization of the screen projection video stream based on a preset screen projection protocol includes:

[0021] If the screen projection protocol is a private screen projection protocol, a completely discarded and non-decoded screen projection video stream of each screen projection window in the background non-display state is executed, and preset information is generated and sent to the screen projection sending end; wherein the preset information is used to notify the screen projection sending end to stop sending the screen projection video stream;

[0022] If the screen projection protocol is a native screen projection protocol, the strategy of only decoding key frames is executed for the screen projection video stream of each screen projection window in the background non-display state.

[0023] The second aspect of the present application discloses a screen projection control method, which is applied to a screen projection sending end, and the method includes:

[0024] Identify the type of screen content to be projected; wherein the type of screen content includes dynamic pictures and static pictures;

[0025] Generate a screen projection video stream according to the screen projection content type;

[0026] Adjust the screen projection video stream according to the instruction of the screen projection receiving end.

[0027] Optionally, in the above method, generating a screen projection video stream according to the screen projection content type includes:

[0028] Configuring the projection screen parameters of the screen to be projected based on the projection content type; wherein the projection screen parameters include frame rate, resolution, and bit rate;

[0029] The projection video stream is generated based on the projection picture parameters.

[0030] Optionally, in the above method, adjusting the screen projection video stream according to an instruction of the screen projection receiving end includes:

[0031] The projection video stream is adjusted based on the projection picture parameters sent by the projection receiving end to generate an adjusted projection video stream.

[0032] The third aspect of the present application discloses a screen projection control device, which is applied to a screen projection receiving end, comprising:

[0033] A determination unit, used to determine the projection display type based on the projection window hierarchical area information when in projection state; wherein the projection window hierarchical area information includes projection window hierarchical information, projection window size information and projection window coordinate information; the projection display type includes multi-channel simultaneous display and single-channel full-screen maximized display;

[0034] An allocation unit, configured to allocate decoding resources to each screen projection window when it is determined that the screen projection display type is multi-channel simultaneous display and the decoder load is greater than a preset threshold;

[0035] The indication unit is used to instruct each screen projection sending end to adjust the projection video stream according to the decoding resources allocated to each screen projection window.

[0036] Optionally, the above device includes:

[0037] A control unit is used to control the projection window in the background non-display state to enter the projection silent state when it is determined that the projection display type is a single-channel full-screen maximized display.

[0038] Optionally, the above-mentioned device, the distribution unit, includes:

[0039] An acquisition subunit, used to acquire the idle decoding resources of the decoder and the size of the visible area of ​​each screen projection window;

[0040] An allocation subunit is used to allocate decoding resources to each screen projection window based on the idle decoding resources and the visible area size of each screen projection window.

[0041] Optionally, in the above device, the control unit includes:

[0042] A calculation subunit is used to calculate the projection screen parameters of the current projection screen window for each projection screen window according to the allocated decoding resources; wherein the projection screen parameters include frame rate, resolution, and bit rate;

[0043] The sending subunit is used to send the projection picture parameters to the projection sending end to trigger the projection sending end to adjust the projection video stream based on the projection picture parameters.

[0044] Optionally, in the above device, the indicating unit includes:

[0045] A determination subunit, configured to determine a projection window in a background non-display state based on the projection window hierarchical area information;

[0046] The optimization subunit is used to set the interface of each projection window in the background non-display state to a hidden state, and to decode and optimize the projection video stream based on a preset projection protocol.

[0047] Optionally, in the above-mentioned device, the optimization subunit includes:

[0048] The first execution subunit is used for executing a completely discarding and non-decoding strategy for the screen projection video stream of each screen projection window in a background non-display state if the screen projection protocol is a private screen projection protocol, and generating preset information to send to the screen projection sending end; wherein the preset information is used to notify the screen projection sending end to stop sending the screen projection video stream;

[0049] The second execution sub-unit is used to execute the strategy of only decoding key frames for the projection video stream of each projection window in the background non-display state if the projection protocol is a native projection protocol.

[0050] A fourth aspect of the present application discloses a screen projection control device, which is applied to a screen projection sending end, comprising:

[0051] An identification unit, used to identify the type of screen content to be projected; wherein the type of screen content includes dynamic pictures and static pictures;

[0052] A generating unit, configured to generate a screen projection video stream according to the screen projection content type;

[0053] An adjustment unit is used to adjust the projection video stream according to the instruction of the projection receiving end.

[0054] Optionally, in the above device, the generating unit includes:

[0055] A configuration subunit, configured to configure the projection screen parameters of the screen to be projected based on the projection content type; wherein the projection screen parameters include frame rate, resolution, and bit rate;

[0056] A generating subunit is used to generate the projection video stream based on the projection picture parameters.

[0057] Optionally, in the above device, the adjustment unit includes:

[0058] The adjustment subunit is used to adjust the projection video stream based on the projection picture parameters sent by the projection receiving end to generate an adjusted projection video stream.

[0059] A fifth aspect of the present application discloses a screen projection receiving device, comprising:

[0060] one or more processors;

[0061] a storage device having one or more programs stored thereon;

[0062] Display screen;

[0063] Communication interface;

[0064] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect of the present application.

[0065] The present application discloses a screen projection sending device in six aspects, including:

[0066] one or more processors;

[0067] a storage device having one or more programs stored thereon;

[0068] Communication interface;

[0069] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the second aspect of the present application.

[0070] A seventh aspect of the present application discloses a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any one of the first aspect or the second aspect of the present application is implemented.

[0071] It can be seen from the above technical scheme that a screen projection control method provided by the present application is applied to the screen projection receiving end. When in the screen projection state, the screen projection display type is determined based on the hierarchical area information of the screen projection window, and different screen projection adjustment strategies are adopted for different screen projection display types to improve the adjustment effect. When it is determined that the screen projection display type is multi-channel simultaneous display and the decoder load is greater than the preset threshold, decoding resources are allocated to each screen projection window, and according to the decoding resources allocated to each screen projection window, the screen projection sending end of each channel is instructed to adjust the screen projection video stream. In this way, the screen projection windows of multiple simultaneous displays are dynamically adjusted to save system resources and avoid consuming too many system resources without affecting the user experience. It solves the problem in the prior art that when performing multi-channel screen projection, it is easy to consume too many system resources, causing instability of the device system and affecting the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0073] Figure 1 This is a flow chart of a screen projection control method disclosed in an embodiment of the present application;

[0074] Figure 2 A schematic diagram of a multi-channel simultaneous display scenario disclosed in an embodiment of the present application;

[0075] Figure 3 A schematic diagram of a scene of a single-channel full-screen maximized display disclosed in an embodiment of the present application;

[0076] Figure 4 This is a flowchart of an implementation of step S104 disclosed in an embodiment of the present application;

[0077] Figure 5 This is a flow chart of another screen projection control method disclosed in an embodiment of the present application;

[0078] Figure 6 This is a schematic diagram of the screen projection control system architecture disclosed in the embodiment of the present application;

[0079] Figure 7 A schematic diagram of a screen projection control device disclosed in an embodiment of the present application;

[0080] Figure 8 A schematic diagram of another screen projection control device disclosed in an embodiment of the present application;

[0081] Fig. 9A schematic diagram of a screen projection receiving device disclosed in an embodiment of the present application;

[0082] Fig.10 This is a schematic diagram of a screen projection sending device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0084] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0085] Furthermore, in this document, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0086] The present application embodiment provides a screen projection control method, which is applied to a screen projection receiving end, such as Figure 1 As shown, specifically including:

[0087] S101. When in the projection state, determine the projection display type based on the projection window level area information; wherein the projection window level area information includes the projection window level information, the projection window size information and the projection window coordinate information; the projection display type includes multi-channel simultaneous display and single-channel full-screen maximized display.

[0088] It should be noted that when in the screen projection state, the projection window level area information is first obtained, which includes the projection window level information, projection window size information and projection window coordinate information. According to the projection window level information, the window level of each projection window can be known; according to the projection window size information, the display size of each projection window can be known; according to the projection window coordinate information, the position coordinates of each projection window can be known. Therefore, after obtaining the projection window level area information, the projection display type used by the current projection can be determined, among which the projection display types include multi-channel simultaneous display and single-channel full-screen maximized display. Multi-channel simultaneous display such as Figure 2 As shown, each projection has a separate display area and can be displayed simultaneously. The sizes of the areas in the figure are the same. In other embodiments, the sizes of the display areas of different projection windows can also be different. Figure 3 As shown, only one of the projection windows will be maximized and displayed, and the remaining projection windows are in the background non-display state. The images of these projection windows in the background non-display state are not visible to users.

[0089] S102. When it is determined that the projection display type is multi-channel simultaneous display and the decoder load is greater than a preset threshold, decoding resources are allocated to each projection window.

[0090] It should be noted that if the projection display type is multi-channel simultaneous display, it is necessary to decode and render the video stream of each projection. If each projection is fully decoded and rendered, a large amount of system resources will be consumed, and the system resource pressure will be very high. Therefore, when it is determined that the projection display type is multi-channel simultaneous display and the decoder load is greater than the preset threshold, the decoding resources of each projection window are dynamically allocated to reduce the system resource pressure and avoid the projection disconnection and screen freeze due to insufficient system resources. The threshold can be set according to the actual situation, for example, 80%. When the decoder load is not greater than the preset threshold, the system resources are sufficient, and the projection video stream generated by the projection sender is directly decoded and rendered to achieve projection.

[0091] Optionally, in another embodiment of the present application, an implementation of the above S102 may include:

[0092] Get the free decoding resources of the decoder and the visible area size of each projection window.

[0093] Decoding resources are allocated to each projection window based on the idle decoding resources and the visible area size of each projection window.

[0094] It should be noted that the idle decoding resources of the decoder are first calculated based on the total capacity of the decoder and the current load of the decoder. Then, the visible area size of the projection window is obtained based on the projection window size information. The idle decoding resources are allocated according to the visible area size of each projection window, with more allocations for larger visible areas and less allocations for smaller visible areas.

[0095] S103. According to the decoding resources allocated to each screen projection window, instruct each screen projection sending end to adjust the screen projection video stream.

[0096] Among them, for each screen projection window, the projection picture parameters of the current screen projection window are calculated according to the allocated decoding resources, and the projection picture parameters are sent to the screen projection sending end to trigger the screen projection sending end to adjust the projection video stream based on the projection picture parameters; wherein the projection picture parameters include frame rate, resolution and bit rate.

[0097] It should be noted that for each projection window, the new frame rate, resolution, bit rate and other projection screen parameters of the current projection window are calculated according to the decoding resources allocated to the current projection window, and then the calculated projection screen parameters are sent to the projection screen sending end corresponding to the current projection window. After the projection screen sending end receives these projection screen parameters, it configures the frame rate, resolution, bit rate, etc. of the projection screen according to the received projection screen parameters, generates a projection video stream and sends it to the projection screen receiving end.

[0098] Optionally, in another embodiment of the present application, the above method, when determining that the projection display type is a single-channel full-screen maximized display, controls the projection window in the background non-display state to enter the projection silent state.

[0099] It should be noted that if the projection display type is a single-channel full-screen maximized display, one of the projection windows is maximized, and the other projection windows are in the background non-display state, and the user cannot see the other projection images. Therefore, the projection windows in the background non-display state are determined, and these projection windows are silently processed at this time, thereby reducing the consumption of decoding performance, rendering performance, and traffic bandwidth, while not affecting the user experience.

[0100] Optionally, in another embodiment of the present application, a screen projection window in a background non-display state is controlled to enter a screen projection silent state, such as Figure 4 As shown, it may include:

[0101] S401. Determine the projection window in the background non-display state based on the projection window hierarchy area information.

[0102] It should be noted that, based on the projection window hierarchical area information, it is possible to calculate which projection window is blocked, thereby determining the projection window that is in the background non-display state.

[0103] S402. For each screen projection window that is in the background non-display state, the interface of the screen projection window is set to a hidden state, and the screen projection video stream is decoded and optimized based on a preset screen projection protocol.

[0104] It should be noted that for each screen projection window in the background non-display state, the screen projection window interface is set to a hidden state, and the screen is no longer rendered, which reduces the system GPU occupancy and greatly reduces the system rendering performance consumption. At the same time, based on the preset screen projection protocol, the screen projection video stream sent by the screen projection sender is decoded and optimized. Instead of continuously decoding each screen projection, the key frames in the screen projection video stream are decoded or even not decoded at all, which reduces the consumption of decoding performance and rendering performance.

[0105] Optionally, in another embodiment of the present application, an implementation of step S402 may include:

[0106] If the screen projection protocol is a private screen projection protocol, the screen projection video stream of each screen projection window in the background non-display state is completely discarded and not decoded, and preset information is generated and sent to the screen projection sender; wherein the preset information is used to notify the screen projection sender to stop sending the screen projection video stream.

[0107] If the screen projection protocol is a native screen projection protocol, the strategy of only decoding key frames is executed for the screen projection video stream of each screen projection window in the background non-display state.

[0108] It should be noted that if a private screen projection protocol is used, and the private screen projection protocol can be used to achieve complete discarding and non-decoding, then the screen projection video stream of each screen projection window in the background non-display state is completely discarded and non-decoding is implemented, that is, the received screen projection video stream is not decoded. Because the current display state is a single-channel full-screen maximized display, other screen projection windows are in the background and are completely blocked, these projections do not need to be rendered and displayed, and the corresponding ones do not need to be decoded. The corresponding code stream can be directly discarded, reducing the consumption of decoding resources. At the same time, the freezing screen is triggered to be silent, and the preset information is generated and sent to the screen projection sender, notifying the screen projection sender to stop sending the screen projection video stream, saving traffic bandwidth.

[0109] If the native screen projection protocol is used, such as the native screen projection of Windows system / Android system uses the Miracast protocol, and the native screen projection of Mac system uses the Airplay screen projection protocol, this type of native screen projection protocol cannot be completely discarded and not decoded. Therefore, the screen projection video stream of each screen projection window in the background non-display state is executed with the strategy of only decoding the key frames, that is, each video frame of the received screen projection video stream is judged to determine whether it is a key frame. If it is a key frame, it is decoded, and non-key frames are not decoded. Under normal circumstances, there will be a key frame every few seconds, and the performance occupation is basically negligible, which also effectively reduces the decoding resource consumption.

[0110] In a screen projection control method provided in an embodiment of the present application, it is applied to the screen projection receiving end. When in the screen projection state, the screen projection display type is determined based on the hierarchical area information of the screen projection window, and different screen projection adjustment strategies are adopted for different screen projection display types to improve the adjustment effect. When it is determined that the screen projection display type is multi-channel simultaneous display and the decoder load is greater than the preset threshold, decoding resources are allocated to each screen projection window, and according to the decoding resources allocated to each screen projection window, the screen projection sending end of each channel is instructed to adjust the screen projection video stream. In this way, the screen projection windows of multiple simultaneous displays are dynamically adjusted to save system resources and avoid consuming too many system resources without affecting the user experience. This solves the problem in the prior art that when performing multi-channel screen projection, it is easy to consume too many system resources, causing instability of the device system and affecting the user experience.

[0111] In another embodiment of the present application, a screen projection control method is provided, which is applied to a screen projection sending end, such as Figure 5 As shown, specifically including:

[0112] S501. Identify the type of content to be projected; wherein the type of content to be projected includes dynamic images and static images.

[0113] It should be noted that the screen projection content of users is diverse, and can be divided into static screens and dynamic screens according to the angle of screen changes. Static screens are typical office scenes (execl pages, ppt pages, word pages, web pages, windows window pages), etc., and the screen moves only when the user scrolls; dynamic screens are game scenes, video playback scenes, etc. Therefore, the screen content type of the screen to be projected is first identified, so that different frame rates, resolutions, bit rates, etc. are configured for screens of different screen content types.

[0114] S502: Generate a projection video stream according to the projection content type.

[0115] It should be noted that since different types of screen projection content have different display requirements for the screen, the screen projection sender generates different screen projection video streams according to different screen projection content types.

[0116] Optionally, in another embodiment of the present application, an implementation of the above step S502 may include:

[0117] Configure the projection screen parameters of the screen to be projected based on the projection content type; wherein the projection screen parameters include frame rate, resolution and bit rate.

[0118] Generate a projection video stream based on the projection screen parameters.

[0119] It should be noted that static images can be configured with lower frame rates, resolutions and bit rates, such as 60 frames, 1080p resolution, static image bit rates <300kbps, and bit rates <5Mkbps during scrolling, in order to reduce traffic and decoding resource consumption without affecting user experience. Dynamic images can be configured with higher frame rates, resolutions and bit rates to ensure smoothness and clarity, such as 120 frames, 2K resolution, 7.5Mkbps < bit rates <8Mkbps.

[0120] It should also be noted that after configuring the parameters such as the frame rate, resolution and bit rate of the screen to be projected, a projection video stream of the screen to be projected is generated based on the frame rate, resolution and bit rate, and sent to the projection receiver. After the projection receiver receives the projection video stream, it decodes and renders the projection video stream, and displays the projection screen through the display end. At the projection sender, the frame rate, resolution and bit rate of the projection screen can be reasonably controlled according to the type of projection content, thereby reasonably controlling the traffic and image quality. At the projection receiver, the performance occupancy can be reduced, system resources can be saved, and the stability of the projection service can be better guaranteed.

[0121] S503. Adjust the projection video stream according to the instruction of the projection receiving end.

[0122] It should be noted that if a screen adjustment instruction is received by the screen projection, the screen projection parameters of the screen projection video stream are adjusted according to the instruction of the screen projection receiving end to generate an adjusted screen projection video stream.

[0123] Optionally, in another embodiment of the present application, an implementation of the above step S503 may include:

[0124] The projection video stream is adjusted based on the projection picture parameters sent by the projection receiving end to generate an adjusted projection video stream.

[0125] It should be noted that if the projection screen parameters sent by the projection receiving end are received and the screen adjustment is instructed, the projection screen will be configured with a new frame rate, resolution and bit rate according to the received projection screen parameters. After the configuration is completed, the projection video stream is generated and sent to the projection receiving end. After the projection receiving end receives the projection video stream, it can decode and render the projection video stream and display the projection screen through the display end. This can achieve dynamic adjustment of the projection screen and optimize the projection effect.

[0126] In a screen projection control method provided in an embodiment of the present application, applied to the screen projection sending end, the screen projection content type of the screen to be projected is first identified, and then a screen projection video stream is generated according to the screen projection content type. Since the frame rate, resolution and bit rate of the screen projection are reasonably controlled at the screen projection sending end, the performance occupancy can be reduced at the screen projection receiving end, system resources can be saved, and the stability of the screen projection service can be better guaranteed, which can ensure the smoothness and clarity of the screen projection without affecting the user experience. Finally, the screen projection video stream is adjusted according to the instructions of the screen projection receiving end to achieve dynamic adjustment of the screen projection picture and optimize the screen projection effect.

[0127] For ease of understanding, a screen projection control system architecture diagram is provided in another embodiment of the present application, such as Figure 6 As shown, it includes a screen projection sending end, a screen projection receiving end and a display end. Among them, the screen projection sending end includes an acquisition module, an encoding control module, an encoding module, a network transmission module, a screen projection quality control module and a media negotiation module. The acquisition module is used to acquire the picture and configure the picture frame rate. The encoding control module is used to set the encoding mode. The encoding module is used to configure the bit rate and resolution of the projection picture. The network transmission module is used to send data to the projection receiving end. The media negotiation module is used to receive data from the projection receiving end, and the projection quality control module is used to configure the projection picture parameters. The projection receiving end includes a network receiving module, a video cache module, a playback control module, a video decoding module, a video rendering module, a projection window module, a multi-channel control strategy module and a media negotiation module. The network receiving module is used to receive the projection video stream data sent by the projection sending end. The video cache module is used to cache the projection video stream data. The playback control module is used to control the screen playback of the projection window. The video decoding module is used to decode the projection video stream. The video rendering module is used to render the projection video stream and transmit video data to the display end. The projection window module is used to obtain the projection window hierarchical area information. The multi-channel control strategy module is used to adjust the projection screen parameters and the playback status of the projection window. The media negotiation module is used to send data to the projection sender. The display end is used to display the projection screen.

[0128] Another embodiment of the present application further provides a screen projection control device, which is applied to a screen projection receiving end, such as Figure 7 As shown, specifically including:

[0129] Determination unit 701 is used to determine the projection display type based on the projection window hierarchy area information when in projection state; wherein the projection window hierarchy area information includes projection window hierarchy information, projection window size information and projection window coordinate information; the projection display type includes multi-channel simultaneous display and single-channel full-screen maximized display.

[0130] The allocation unit 702 is used to allocate decoding resources to each screen projection window when it is determined that the screen projection display type is multi-channel simultaneous display and the decoder load is greater than a preset threshold.

[0131] The indication unit 703 is used to instruct each screen projection sending end to adjust the screen projection video stream according to the decoding resources allocated to each screen projection window.

[0132] In this embodiment, the specific execution process of the determination unit 701, the allocation unit 702, and the indication unit 703 can be referred to in the corresponding Figure 1 The content of the method embodiment will not be repeated here.

[0133] In a screen projection control device provided in an embodiment of the present application, which is applied to a screen projection receiving end, when in a screen projection state, a determination unit 701 determines the screen projection display type based on the hierarchical area information of the screen projection window, and adopts different screen projection adjustment strategies for different screen projection display types to improve the adjustment effect. When it is determined that the screen projection display type is multi-channel simultaneous display, and the decoder load is greater than the preset threshold, the allocation unit 702 allocates decoding resources to each screen projection window. The indication unit 703 indicates the screen projection sending end of each channel to adjust the screen projection video stream according to the decoding resources allocated to each screen projection window. In this way, the screen projection windows of multiple channels displayed simultaneously are dynamically adjusted to save system resources and avoid consuming too many system resources without affecting the user experience. It solves the problem in the prior art that when performing multi-channel screen projection, it is easy to consume too many system resources, causing instability of the device system and affecting the user experience.

[0134] Optionally, in another embodiment of the present application, an implementation of the allocation unit 702 may include:

[0135] The acquisition subunit is used to obtain the idle decoding resources of the decoder and the visible area size of each projection window.

[0136] The allocation subunit is used to allocate decoding resources to each projection window based on the idle decoding resources and the visible area size of each projection window.

[0137] In this embodiment, the specific execution process of obtaining sub-units and allocating sub-units can be found in the corresponding method embodiment described above, and will not be repeated here.

[0138] Optionally, in another embodiment of the present application, an implementation of the control unit 704 may include:

[0139] A determination subunit is used to determine the projection window in the background non-display state based on the projection window hierarchy area information.

[0140] The optimization subunit is used to set the interface of each projection window in the background non-display state to a hidden state, and to decode and optimize the projection video stream based on a preset projection protocol.

[0141] In this embodiment, the specific execution process of determining the subunit and optimizing the subunit can be referred to in the corresponding Figure 4 The content of the method embodiment will not be repeated here.

[0142] Optionally, in another embodiment of the present application, an implementation of the above optimization subunit may include:

[0143] The first execution sub-unit is used to execute a strategy of completely discarding and not decoding the projection video stream of each projection window in the background non-display state if the projection protocol is a private projection protocol, and generate preset information to send to the projection sending end; wherein the preset information is used to notify the projection sending end to stop sending the projection video stream.

[0144] The second execution sub-unit is used to execute the strategy of only decoding key frames for the projection video stream of each projection window in the background non-display state if the projection protocol is a native projection protocol.

[0145] In this embodiment, the specific execution process of the first execution subunit and the second execution subunit can refer to the corresponding method embodiment content mentioned above, which will not be repeated here.

[0146] Another embodiment of the present application further provides a screen projection control device, which is applied to a screen projection sending end, such as Figure 8 As shown, specifically including:

[0147] The identification unit 801 is used to identify the type of screen content to be projected; wherein the screen content type includes dynamic pictures and static pictures.

[0148] The generation unit 802 is used to generate a screen projection video stream according to the screen projection content type.

[0149] The adjustment unit 803 is used to adjust the projection video stream according to the instruction of the projection receiving end.

[0150] In this embodiment, the specific execution process of the identification unit 801, the generation unit 802, and the adjustment unit 803 can be referred to in the corresponding Figure 5 The content of the method embodiment will not be repeated here.

[0151] In a screen projection control device provided in an embodiment of the present application, it is applied to the screen projection sending end. First, the identification unit 801 identifies the type of screen projection content of the screen to be projected, and then the generation unit 802 generates a screen projection video stream according to the screen projection content type. Since the frame rate, resolution and bit rate of the screen projection picture are reasonably controlled at the screen projection sending end, the performance occupancy can be reduced at the screen projection receiving end, saving system resources, and better guaranteeing the stability of the screen projection service, which can ensure the smoothness and clarity of the screen projection without affecting the user experience. Finally, the adjustment unit 803 adjusts the screen projection video stream according to the instructions of the screen projection receiving end to achieve dynamic adjustment of the screen projection picture and optimize the screen projection effect.

[0152] Optionally, in another embodiment of the present application, the screen projection control device applied to the screen projection sending end may further include:

[0153] The receiving unit is used to generate a projection video stream of the screen to be projected based on the projection picture parameters sent by the projection receiving end if the projection picture parameters sent by the projection receiving end are received.

[0154] In this embodiment, the specific execution process of the receiving unit can refer to the corresponding method embodiment content mentioned above, which will not be repeated here.

[0155] Another embodiment of the application further provides a screen projection receiving device, such as Fig. 9 As shown, specifically including:

[0156] One or more processors 901.

[0157] The storage device 902 stores one or more programs.

[0158] Display screen 903.

[0159] Communication interface 904.

[0160] It should be noted that the screen projection receiving device exchanges data with the screen projection sending device through the communication interface 904, and receives the screen projection video stream into the storage device 902. The received screen projection video stream is processed by one or more processors 901, and the screen projection is displayed through the display screen 903.

[0161] Another embodiment of the application also provides a screen projection sending device, such as Fig.10 As shown, specifically including:

[0162] One or more processors 1001.

[0163] The storage device 1002 stores one or more programs.

[0164] Communication interface 1003.

[0165] It should be noted that the screen projection sending device executes the program on the storage device 1002 through the processor 1001, generates a screen projection video stream or adjusts the screen projection video stream, and interacts with the screen projection receiving device through the communication interface 1003.

[0166] Another embodiment of the present application further provides a computer storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, any one of the methods in the above embodiments is implemented.

[0167] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0168] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0169] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A screen projection control method, characterized in that: Applied to the screen projection receiving end, the method includes: When in the screen projection state, the screen projection display type is determined based on the screen projection window level area information; wherein the screen projection window level area information includes the screen projection window level information, the screen projection window size information and the screen projection window coordinate information; the screen projection display type includes multi-channel simultaneous display and single-channel full-screen maximized display; When it is determined that the projection display type is multi-channel simultaneous display and the decoder load is greater than a preset threshold, decoding resources are allocated to each projection window; According to the decoding resources allocated to each screen projection window, instruct each screen projection sender to adjust the screen projection video stream.

2. The method according to claim 1, characterized in that When it is determined that the screen projection display type is a single-channel full-screen maximized display, the screen projection window in the background non-display state is controlled to enter a screen projection silent state.

3. The method according to claim 1, characterized in that The decoding resource allocation for each screen projection window includes: Obtain the idle decoding resources of the decoder and the visible area size of each screen projection window; Decoding resources are allocated to each screen projection window based on the idle decoding resources and the visible area size of each screen projection window.

4. The method according to claim 1, characterized in that: The method of instructing each screen projection sending end to adjust the screen projection video stream according to the decoding resources allocated to each screen projection window includes: For each screen projection window, the projection screen parameters of the current screen projection window are calculated according to the allocated decoding resources; wherein the projection screen parameters include frame rate, resolution, and bit rate; The projection picture parameters are sent to the projection sending end to trigger the projection sending end to adjust the projection video stream based on the projection picture parameters.

5. The method according to claim 2, characterized in that: The controlling of the screen projection window in the background non-display state to enter the screen projection silent state includes: Determine the projection window in the background non-display state based on the projection window hierarchical area information; For each projection window that is in the background non-display state, the interface of the projection window is set to a hidden state, and the projection video stream is decoded and optimized based on the preset projection protocol.

6. The method according to claim 5, characterized in that The decoding optimization of the screen projection video stream based on the preset screen projection protocol includes: If the screen projection protocol is a private screen projection protocol, a completely discarded and non-decoded screen projection video stream of each screen projection window in the background non-display state is executed, and preset information is generated and sent to the screen projection sending end; wherein the preset information is used to notify the screen projection sending end to stop sending the screen projection video stream; If the screen projection protocol is a native screen projection protocol, the strategy of only decoding key frames is executed for the screen projection video stream of each screen projection window in the background non-display state.

7. A screen projection control method, characterized in that: Applied to the screen projection sending end, the method includes: Identify the type of screen content to be projected; wherein the type of screen content includes dynamic pictures and static pictures; Generate a screen projection video stream according to the screen projection content type; Adjust the screen projection video stream according to the instruction of the screen projection receiving end.

8. The method according to claim 7, characterized in that The generating a screen projection video stream according to the screen projection content type includes: Configuring the projection screen parameters of the screen to be projected based on the projection content type; wherein the projection screen parameters include frame rate, resolution, and bit rate; The projection video stream is generated based on the projection picture parameters.

9. The method according to claim 7, characterized in that: The adjusting the screen projection video stream according to the instruction of the screen projection receiving end includes: The projection video stream is adjusted based on the projection picture parameters sent by the projection receiving end to generate an adjusted projection video stream.

10. A screen projection control device, characterized in that: Applied to screen projection receiving end, including: A determination unit, used to determine the projection display type based on the projection window hierarchical area information when in projection state; wherein the projection window hierarchical area information includes projection window hierarchical information, projection window size information and projection window coordinate information; the projection display type includes multi-channel simultaneous display and single-channel full-screen maximized display; An allocation unit, configured to allocate decoding resources to each screen projection window when it is determined that the screen projection display type is multi-channel simultaneous display and the decoder load is greater than a preset threshold; The indication unit is used to instruct each screen projection sending end to adjust the projection video stream according to the decoding resources allocated to each screen projection window.

11. A screen projection control device, characterized in that: Applied to the screen projection sender, including: An identification unit, used to identify the type of screen content to be projected; wherein the type of screen content includes dynamic pictures and static pictures; A generating unit, configured to generate a screen projection video stream according to the screen projection content type; An adjustment unit is used to adjust the projection video stream according to the instruction of the projection receiving end.

12. A screen projection receiving device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; Display screen; Communication interface; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 6.

13. A screen projection sending device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; Communication interface; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 7 to 9.

14. A computer storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images