Screen projection method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of computers, in particular to a screen projection method and device, equipment and a storage medium, and the scheme comprises the steps that the position of a preset selection box on a display interface of electronic equipment is acquired, and the position of the preset selection box is adjustable; acquiring full-screen data of a display interface recorded by the electronic equipment; screen area data is intercepted from the full-screen data according to the position of a preset selection box on the display interface; performing data coding on the screen area data to obtain screen projection coded data; and sending the projection screen coded data to a display device, so that the display device displays a picture selected by the preset selection box on the display interface according to the projection screen coded data. According to the method and the device, the local data in the full-screen data can be adaptively intercepted according to the preset selection box, and the local data is displayed, so that personalized requirements are met, and better user experience and usability are provided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a screen projection method, device, equipment and storage medium. Background Art

[0002] Currently, screen projection on the Android platform usually requires the use of the API (application programming interface) provided by the Android system to obtain the screen stream. However, the API usually only returns the current complete screen interface data, including the control bar and buttons of the application. However, in some cases, users want to project a specific part of the content, such as courseware, documents or web application content, and do not want to display the control bar and buttons of these applications. This results in general mobile phone projection software can only transmit the content of the entire mobile phone screen when projecting, and cannot selectively capture part of the area. Summary of the invention

[0003] One purpose of the embodiments of the present application is to provide a screen projection method, device, equipment and storage medium to solve the technical problem that the user cannot freely select the projection area in the screen projection scenario.

[0004] In a first aspect, a screen projection method is provided, which is applied to an electronic device, comprising:

[0005] Obtaining a position of a preset selection box on a display interface of the electronic device, wherein the position of the preset selection box is adjustable;

[0006] Acquire full-screen data of the display interface recorded by the electronic device;

[0007] According to the position of the preset selection box on the display interface, extracting screen area data from the full screen data;

[0008] Encoding the screen area data to obtain screen projection coded data;

[0009] The screen projection coding data is sent to the display device so that the display device displays the picture selected by the preset selection box on the display interface according to the screen projection coding data.

[0010] In combination with the first aspect, in a possible implementation method, the screen area data is intercepted from the full-screen data according to the position of the preset selection box on the display interface, including: rendering the full-screen data into a pre-created texture rendering model to obtain full-screen rendering data, the texture rendering model being configured with a rendering component; and intercepting the screen area data from the full-screen data according to the rendering component and the position of the preset selection box on the display interface.

[0011] In combination with the first aspect, in a possible implementation method, the screen area data is intercepted from the full-screen data according to the positions of the rendering component and the preset selection box on the display interface, including: according to the rendering component, converting the position of the preset selection box on the display interface into a interception position corresponding to the texture rendering model; and intercepting the screen area data from the full-screen data according to the interception position.

[0012] In combination with the first aspect, in a possible implementation, the electronic device is configured with an editing component, and before encoding the screen area data, the method also includes: responding to a user's operation on the editing component, processing the screen area data to obtain processed screen area data.

[0013] In combination with the first aspect, in a possible implementation method, in response to the user's operation on the editing component, the screen area data is processed to obtain the processed screen area data, including: obtaining a first layer corresponding to the screen area data; in response to the user's operation on the editing component, determining a second layer corresponding to the editing component; and superimposing the second layer on the first layer to obtain the processed screen area data.

[0014] In combination with the first aspect, in a possible implementation method, obtaining the position of the preset selection box on the display interface of the electronic device includes: responding to a user's moving operation on the preset selection box on the display interface of the electronic device, and determining the position of the preset selection box on the display interface.

[0015] In combination with the first aspect, in a possible implementation method, obtaining the position of the preset selection box on the display interface of the electronic device includes: performing a correlation analysis on the data content of the screen area data to obtain the correlation between each element in the data content; based on the correlation of each element, screening out elements whose correlation exceeds a preset correlation threshold as target elements; and determining the position of the preset selection box on the display interface based on the positions of multiple target elements on the display interface.

[0016] In a second aspect, a screen projection device is provided, which is applied to an electronic device, and the device includes:

[0017] An acquiring unit, configured to acquire a position of a preset selection box on a display interface of the electronic device, wherein the position of the preset selection box is adjustable;

[0018] The acquisition unit is further used to acquire the full screen data of the display interface recorded by the electronic device;

[0019] A capture unit, configured to capture screen area data from the full screen data according to the position of the preset selection box on the display interface;

[0020] An encoding unit, used for encoding the screen area data to obtain screen projection encoding data;

[0021] The display unit is used to send the screen projection coding data to the display device, so that the display device displays the picture selected by the preset selection box on the display interface according to the screen projection coding data.

[0022] In a third aspect, an embodiment of the present invention provides a computer device, including:

[0023] at least one processor; and,

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0026] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method as described in the first aspect.

[0027] In the scheme implemented by the above-mentioned screen projection method, device, equipment and storage medium, for electronic equipment, first obtain the position of the preset selection box on the display interface of the electronic equipment, wherein the position of the preset selection box is adjustable, and secondly, obtain the full-screen data of the display interface recorded by the electronic equipment, and then, according to the position of the preset selection box on the display interface, intercept the screen area data in the full-screen data, encode the screen area data, and obtain the projection coding data, and finally, send the projection coding data to the display device, so that the display device displays the picture selected by the preset selection box on the display interface according to the projection coding data. This method intercepts the local projection area in the full-screen data for projection through the adjustable preset selection box, further flexibly adapts to different application scenarios, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 This is a schematic diagram of a scenario in an embodiment of the present invention;

[0030] Figure 2A It is a flowchart of a screen projection method in one embodiment of the present invention;

[0031] Figure 2B It is a flowchart of a screen projection method in one embodiment of the present invention;

[0032] Figure 3 is a schematic structural diagram of a screen projection device in one embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0035] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0036] The present invention is described in detail below through specific embodiments.

[0037] The technical solution of this application can be applied to various teaching scenarios or meeting scenarios, etc.

[0038] See also Figure 1 , Figure 1A scenario diagram is provided for an embodiment of the present application, in which there are an electronic device 10 and a display device 20 involved in the present solution, wherein the display device 20 may be a computer device with multifunctional imaging such as display, broadcasting, and recording, and the electronic device 10 may be a mobile phone, a tablet, a watch, etc., which is not the only limitation here. The electronic device 10 establishes a connection with the display device 20, and the electronic device 10 can transmit data to the display device 20 for display, which can be achieved in a variety of ways, such as using a wireless network (Wi-Fi), Bluetooth or USB connection. Generally, wireless network is the most common screen projection connection method, and both the electronic device 10 and the display device 20 are connected to the same Wi-Fi network.

[0039] Taking an actual scenario as an example, when the electronic device 10 is a mobile phone and the display device 20 is a tablet computer, the general screen projection process is as follows: the mobile phone and the tablet computer establish a communication connection, select a screen projection application on the mobile phone, such as "screen projection" or "mirror display", and start the screen projection mode; the mobile phone will start to capture the image and content on its screen, and the image and content on the screen will be encoded into a video stream; the mobile phone will encode and compress the captured screen data, usually using a video encoding standard (such as H.264 or H.265) to reduce the amount of data so that it is easier to transmit over the network; the encoded video data is transmitted to the tablet computer through the established connection; on the tablet computer, the received video data will be decoded back to the original image format; the decoded data will be rendered and displayed on the tablet computer's screen, realizing the projection or mirroring of the mobile phone screen content. The above whole process is real-time screen projection, and any changes on the mobile phone screen will be immediately reflected on the tablet computer to maintain synchronization.

[0040] In this solution, after obtaining screen data through MediaProjection in the electronic device, the data is first rendered to a created SurfaceTexture, in which the interface data is clipped according to the coordinates and area selected by the user through OpenGL (Open Graphics Library), and the clipped data is sent to the encoder, which is then encoded and transmitted to the display device through the network. The components mentioned in the above electronic device are explained as follows:

[0041] MediaProjection API: It is an important API provided by Android for capturing and recording the content on the device screen. It is often used to implement functions such as screen recording, screenshots, and screen projection. MediaProjection API allows applications to access screen frame data and process it;

[0042] SurfaceTexture: It is the core component for rendering on Android. It is a combination of Surface and OpenGL ES texture. It is used to provide Surface output to GLES texture. It is used for rendering, processing and displaying images or videos in applications, usually as the output of cameras or video decoders.

[0043] OpenGL (Open Graphics Library): is a cross-platform, open-standard graphics rendering API for high-performance 2D and 3D graphics rendering in computer graphics programming. It provides a set of powerful graphics processing tools that enable developers to create complex graphics and visual effects for game development, computer-aided design (CAD), simulation, scientific visualization, virtual reality, and other graphics applications. In screen projection applications, OpenGL can be used to process and render screen data so that the captured screen content can be cropped, scaled, added with special effects, or other graphics processing operations, and then the processed image data can be transmitted to the receiving end.

[0044] In view of this, the present application proposes a screen projection method to solve the above problems, which is described in detail below.

[0045] See also Figure 2A , Figure 2A A flowchart of a screen projection method provided by an embodiment of the present invention is applied to an electronic device, and is characterized in that the method includes the following steps:

[0046] S10. Obtaining a position of a preset selection box on a display interface of the electronic device, wherein the position of the preset selection box is adjustable.

[0047] The preset selection box may be a view or a control for selecting a specific area on the display interface. The user may customize the size and position of the preset selection box to suit their needs. For example, the preset selection box may be a rectangular box or a box of other shapes, or a movable and resizable control element may be created on the interface.

[0048] The adjustable function can interact with the adjustable preset selection box through input devices (such as a mouse, a touch screen, a keyboard, etc.), voice control, etc. The preset selection box can be further dragged, scaled, rotated, or otherwise adjusted in position and size to meet user needs.

[0049] It can be seen that the adjustable preset selection box enables users to interact with electronic devices and allows users to freely select areas on the screen, thereby increasing the diversity of screen projection methods and improving user experience.

[0050] S20: Obtain full-screen data of the display interface recorded by the electronic device.

[0051] The full-screen data refers to the images, texts, videos and other contents contained in the entire display interface, and the full-screen data is obtained according to the MediaProjection API provided above.

[0052] Among them, before obtaining the full-screen data of the display interface recorded by the electronic device, it is necessary to start the recording device, which can be a camera, screen recording software, video capture card, etc., and there is no sole limitation here. After starting the recording device, it is necessary to configure the recording parameters to specify the recording resolution, frame rate, encoding format and other parameters, and select according to user needs to ensure that the recorded data meets the required quality and performance requirements.

[0053] Among them, the full-screen data can be used by applications in electronic devices, such as screen projection, video editing, video streaming, etc.

[0054] It can be seen that by acquiring the full-screen data of the display interface, a complete data information is provided to the preset selection box, which facilitates the user to control the preset selection box for subsequent selection operations.

[0055] S30. According to the position of the preset selection box on the display interface, extract the screen area data from the full screen data.

[0056] The above-mentioned screen area data includes the boundary coordinates of the screen area and the content displayed in the screen area.

[0057] Among them, firstly, the full screen data needs to be rendered onto the created SurfaceTexture, and the position of the preset selection box on the display interface is recorded. In this step, the coordinates of the four boundary corners of the preset selection box in the display interface are obtained based on the coordinate system of the display interface. Secondly, after intercepting the position of the preset selection box on the display interface, OpenGL is used to perform coordinate conversion on the position of the preset selection box on the display interface to obtain the boundary coordinates of the intercepted screen data area, and obtain the display content written in the corresponding screen area.

[0058] Among them, the coordinates in the OpenGL coordinate system are 0 to 1, 0 is the center point, 1 is the edge of the screen, and the left side of the interface is 0 to the screen resolution size. This method can convert the coordinates of the preset selection box (the upper left point is 0,0) into OpenGL coordinates (the middle of the screen is 0,0) and then pass it to OpenGL.

[0059] For example, to obtain full-screen data, assuming that the full-screen resolution is 1920x1080 pixels, the user selects a rectangular preset selection box on the display interface. For example, the coordinates of the upper left corner of the selection box (x1, y1) are (400, 300), and the coordinates of the lower right corner (x2, y2) are (800, 600). The above coordinates are obtained relative to the coordinate system of the upper left corner of the display interface. Calculate the coordinates of the selection box in the SurfaceTexture texture coordinate system. Assuming that the texture coordinate range is [0.0,1.0], the texture coordinates are (0.25,0.222) in the upper left corner and (0.625,0.444) in the lower right corner. OpenGL intercepts the pixel data of the selection box area on SurfaceTexture based on the texture coordinates. The boundary coordinates and pixel data of the intercepted area are recorded. Assuming that the size of the selection box area is 400x300 pixels, OpenGL is used to perform coordinate conversion on the texture coordinates, and the coordinates of the selection box are converted from the SurfaceTexture texture coordinate system to the OpenGL coordinate system. In the OpenGL coordinate system, the coordinates of the selection box in the upper left corner are (-0.25,-0.111), and the coordinates of the lower right corner are (0.25,0.111). According to these coordinates, the actual position of the preset selection box area is located in the original full-screen data, and the screen area data is obtained. The screen area data is the image content in the preset selection box.

[0060] It can be seen that after the full-screen data is rendered to SurfaceTexture, the display content in the selection box is intercepted according to the preset selection box position selected by the user, and the coordinates are converted according to the OpenGL coordinate system to obtain the corresponding screen content, which is convenient for transmitting the screen area data in subsequent steps.

[0061] S40, encoding the screen area data to obtain screen projection encoding data.

[0062] The data encoding is performed by MediaCodec (codec component). MediaCodec is a multimedia codec API in the Android operating system, which provides hardware-accelerated audio and video encoding and decoding functions. MediaCodec allows applications to encode (compress) and decode (decompress) audio and video data to meet various needs of multimedia processing, including video playback, audio recording, video recording, video editing, streaming media transmission, etc.

[0063] Before encoding the screen area data, the electronic device creates a Surface through MediaCodec, and the Surface is used to pass the image data to the encoder, that is, to render the screen area data to the Surface, so as to input the screen area data into MediaCodec. After MediaCodec receives the screen area data, it encodes the data, generally into the H.264 encoding format.

[0064] Among them, the screen projection encoding data must comply with the standard encoding and decoding format, and in the present invention, it must be set to the H.264 encoding format.

[0065] It can be seen that encoding the screen area data to generate screen projection encoded data provides higher efficiency, accessibility and convenience for multimedia processing and transmission.

[0066] S50. Send the screen projection coded data to the display device, so that the display device displays the picture selected by the preset selection box on the display interface according to the screen projection coded data.

[0067] The display device may be a mobile phone, a computer client, or a device capable of receiving and displaying web pages.

[0068] The encoded projection data needs to be transmitted to the target display device through a network, wireless connection or wired connection, and can be a network address or identifier for sending data to the target device. If the data transmission is carried out through the network, the electronic device and the real device need to establish a communication channel, and a network protocol (such as TCP / IP or UDP) must be used to ensure reliable data transmission.

[0069] Among them, the display device needs to decode the received projection encoded data and use the same encoding and decoding format and codec as the electronic device to restore the original image or video data. The decoded data is rendered on the screen of the display device.

[0070] like Figure 2B As shown, area A is selected in the electronic device in the figure and the screen is projected on the display device synchronously.

[0071] It can be seen that the display device can display the picture selected in the preset selection box according to the projection encoded data, and realize the user's customized selection of the display interface.

[0072] This solution uses an adjustable preset selection box to capture a local projection area in the full-screen data for projection, further flexibly adapting to different application scenarios and improving the user experience.

[0073] In one possible example, the method of capturing screen area data from the full-screen data according to the position of the preset selection box on the display interface includes: rendering the full-screen data into a pre-created texture rendering model to obtain full-screen rendering data, wherein the texture rendering model is configured with a rendering component; and capturing screen area data from the full-screen data according to the rendering component and the position of the preset selection box on the display interface.

[0074] The texture rendering model is a virtual image container, which has properties such as texture, coordinates, rendering parameters, etc., and is used to store and process rendering data. The model can be the above-mentioned SurfaceTexture.

[0075] The rendering component defines how to process image data. The rendering component includes shader programs, filter effects, coordinate conversion, etc. It will guide how to render full-screen data into the model and perform subsequent processing on the data when necessary. This component can be the above-mentioned Opengl.

[0076] The position coordinates of the preset selection box are converted to match the coordinate system of the texture rendering model, which usually involves converting from the display interface coordinate system (usually a coordinate system with the upper left corner of the display interface as the origin) to a coordinate system that matches the texture rendering model (usually a range from 0 to 1).

[0077] Among them, using the configuration and rendering components of the texture rendering model, specific screen area data, including pixel data, images, text or other content, are intercepted from the data of the texture rendering model according to the converted selection box coordinates.

[0078] It can be seen that the projection data is processed through the texture rendering model and its rendering components.

[0079] In one possible example, the method of capturing screen area data from the full-screen data based on the positions of the rendering component and the preset selection box on the display interface includes: converting the position of the preset selection box on the display interface into a capture position corresponding to the texture rendering model according to the rendering component; and capturing screen area data from the full-screen data according to the capture position.

[0080] Among them, the coordinate conversion algorithm defined in the rendering component is used to map the position of the preset selection box from the display interface coordinate system to the texture coordinate system, including scaling, translating and appropriate mapping operations on the coordinates to ensure that the preset selection box is correctly positioned in the texture rendering model.

[0081] The interception position is the position information after the coordinate conversion is completed, that is, the coordinate of the texture rendering model, which represents the position where the data is intercepted on the texture rendering model.

[0082] The corresponding pixel or area is found in the full screen data according to the coordinates of the intercepted position to obtain the screen area data.

[0083] It can be seen that, through the coordinate conversion operation in the rendering component, the position of the preset selection box is converted from the display interface coordinate system to the coordinate system matching the texture rendering model. Then, according to the converted interception position, the screen area data is intercepted from the full screen data for further processing or display.

[0084] In a possible example, the electronic device is configured with an editing component, and before encoding the screen area data, the method further includes: responding to a user's operation on the editing component, processing the screen area data to obtain processed screen area data.

[0085] The editing component is usually a tool or interface element for editing images, texts or other elements on the screen. The operations of the editing component may include but are not limited to dragging, scaling, rotating, filtering and coloring, cropping, etc.

[0086] Among them, the following data processing can be performed according to the editing component: capturing or extracting data from the screen, such as images, text or other content; applying color adjustments, filters and effects to the screen area data to change the appearance of the screen area data, which may include adjusting the brightness, contrast, saturation, or applying filter effects such as blurring and sharpening.

[0087] When the screen area data after data processing is obtained, if there are multiple layers, the editing component needs to merge or overlay them together to generate the final editing effect, that is, to obtain the processed screen area data. This usually involves pixel-level image processing and synthesis operations.

[0088] It can be seen that the editing components and their corresponding data processing operations in this solution provide users with rich tools and functions to customize, modify and enhance the content on the screen.

[0089] In a possible example, in response to the user's operation on the editing component, the screen area data is processed to obtain the processed screen area data, including: obtaining a first layer corresponding to the screen area data; in response to the user's operation on the editing component, determining a second layer corresponding to the editing component; and superimposing the second layer on the first layer to obtain the processed screen area data.

[0090] The operations of the editing component mentioned above may include but are not limited to dragging, scaling, rotating, inserting text, etc. An editing component is usually a tool or interface element for editing images, texts or other elements on the screen.

[0091] The first layer is the original data of the screen area data, that is, the display data of the captured area in the display interface. The display data is only captured and has not been edited.

[0092] The second layer is formed according to the user's operation on the editing component and is used to temporarily store the user's editing content so as to be superimposed with the first layer. For example, if the user draws a line, the second layer may contain the graphic data of the line.

[0093] The above-mentioned step of superimposing the second layer onto the first layer and mixing the pixel data of the first layer and the second layer can be implemented using an image processing algorithm, such as a transparency blending or fusion mode, and further considering the relative position between the layers, merging the content of the second layer onto the first layer according to the user's editing operation, ensuring that the result of the merging correctly reflects the user's editing operation visually, such as smooth connection of lines, correct arrangement of text, etc.

[0094] It can be seen that this method allows the user to interact with the editing component to edit the content on the screen, and the edited results will be superimposed on the original content to generate the final processed screen area data, thereby improving the user experience.

[0095] In a possible example, obtaining the position of the preset selection box on the display interface of the electronic device includes: responding to a user's moving operation on the preset selection box on the display interface of the electronic device, and determining the position of the preset selection box on the display interface.

[0096] Among them, the response to the user's moving operation of the preset selection box on the display interface of the electronic device can be based on the user's touch operation on the display interface; or the user generates a preset selection box through voice control, and then determines the display area of ​​the display interface; or the preset selection box is generated according to the mouse, keyboard, gestures, etc., and then determines the display area of ​​the display interface. No sole limitation is made here.

[0097] In a specific implementation, during a touch operation, the user can dynamically adjust the size or shape of the preset selection box according to the touch, and then determine the display area of ​​the display interface according to the preset selection box adjusted by the user.

[0098] In a specific implementation, when a user generates a preset selection box through voice control, the user's voice command can be collected through a microphone component in the electronic device, such as capturing an image with a width of 1 meter and a height of 1 meter.

[0099] Optionally, before determining the position of the preset selection box on the display interface, the method further includes: verifying the area selected by the preset selection box to ensure that the preset selection box does not exceed the boundary of the display interface or conflict with other elements, and if the position is invalid, moving the preset selection box to a valid area, wherein the valid area is within the boundary of the display interface.

[0100] It can be seen that this method controls the preset selection box to perform local selection on the display interface according to the user's mobile operation, which helps to ensure that the user can accurately select the projection area, reduces the chance of misoperation, and improves the user experience.

[0101] The operation of performing partial display on the display interface, the partial selection is based on the preset selection box.

[0102] In one possible example, obtaining the position of the preset selection box on the display interface of the electronic device includes: performing a correlation analysis on the data content of the screen area data to obtain the correlation between each element in the data content; based on the correlation of each element, screening out elements whose correlation exceeds a preset correlation threshold as target elements; and determining the position of the preset selection box on the display interface based on the positions of multiple target elements on the display interface.

[0103] Optionally, before performing correlation analysis on the data content of the screen area data, the screen area data, such as text, image, button, frame, etc., needs to be identified and extracted.

[0104] The relevance can be evaluated based on one of the following factors between elements: distance: the distance between two elements; color: the color similarity of the elements; shape: the shape similarity of the elements; size: the size similarity of the elements; content: the content similarity of the elements (e.g., text content). One or more relevance factors can be selected to calculate the relevance, and there is no limit here.

[0105] In a specific example, the judgment rules for various factors in calculating the association degree are as follows: distance: the closer the distance between two icons, the higher the association score; color: icons with similar colors have higher association scores; shape: icons with similar shapes have higher association scores; content: if two icons have similar labels or text content, they have higher association scores.

[0106] The preset correlation threshold is used to determine which elements are considered to be correlated, and then determine the elements that reach the preset critical threshold to obtain the target element. The preset correlation threshold can be set manually or set when the electronic device leaves the factory, and is not limited here.

[0107] Optionally, according to the position of the target element on the display interface, the position information of the target element on the display interface is obtained, and the position information may include the coordinates, size, bounding box or other geometric attributes of the element; and the area of ​​the preset selection box is calculated according to the position information of multiple target elements.

[0108] In a specific example, in calculating the area of ​​a preset selection box based on the position information of multiple target elements, it can be done as follows: calculate the average value of the center point coordinates of all target elements to obtain the center position of the concentrated area of ​​the target elements, draw the selection box at the calculated center position, create a larger rectangular selection box around the target elements, or create a selection box that matches the shape of the target elements, and the created selection box must include all target elements.

[0109] In a specific example, in calculating the area of ​​a preset selection box according to the position information of multiple target elements, the following method can be used: Calculate the bounding box of all target elements, that is, the rectangular box surrounding each element. Set the position of the selection box to the area covered by the bounding box, which will ensure that the selection box contains all target elements.

[0110] It can be seen that by automatically identifying relevant elements and positions, the screen projection process can be automated, which reduces user manual intervention and improves the convenience of operation.

[0111] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0112] As another aspect of the embodiment of the present application, the embodiment of the present application provides a screen projection device. The screen projection device can be a software module, and the software module includes a number of instructions stored in a memory. The processor can access the memory and call the instructions for execution to complete the screen projection method described in each of the above embodiments.

[0113] See also Figure 3 , Figure 3 Schematic diagram of a screen projection device provided in an embodiment of the present application. Figure 3As shown, the screen projection device is applied to an electronic device, and is characterized in that the screen projection device comprises:

[0114] An acquiring unit 301 is used to acquire a position of a preset selection box on a display interface of the electronic device, wherein the position of the preset selection box is adjustable;

[0115] The acquisition unit 301 is further used to acquire the full screen data of the display interface recorded by the electronic device;

[0116] A capture unit 302, configured to capture screen area data from the full screen data according to the position of the preset selection box on the display interface;

[0117] The encoding unit 303 is used to encode the screen area data to obtain screen projection encoding data;

[0118] The display unit 304 is used to send the screen projection coding data to the display device, so that the display device displays the picture selected by the preset selection box on the display interface according to the screen projection coding data.

[0119] This solution uses an adjustable preset selection box to capture a local projection area in the full-screen data for projection, further flexibly adapting to different application scenarios and improving the user experience.

[0120] In one embodiment, in capturing screen area data from the full-screen data according to the position of the preset selection box on the display interface, the capture unit 302 is further used to: render the full-screen data into a pre-created texture rendering model to obtain full-screen rendering data, the texture rendering model being configured with a rendering component; and capture screen area data from the full-screen data according to the rendering component and the position of the preset selection box on the display interface.

[0121] In one embodiment, in capturing screen area data from the full-screen data according to the positions of the rendering component and the preset selection box on the display interface, the capture unit 302 is further used to: convert the position of the preset selection box on the display interface into a capture position corresponding to the texture rendering model according to the rendering component; and capture the screen area data from the full-screen data according to the capture position.

[0122] In one embodiment, the electronic device is provided with an editing component, and before encoding the screen area data, the interception unit 302 is further used to: respond to the user's operation on the editing component, process the screen area data, and obtain processed screen area data.

[0123] In one embodiment, in the step of processing the screen area data in response to the user's operation on the editing component to obtain the processed screen area data, the interception unit 302 is further configured to:

[0124] Acquire a first layer corresponding to the screen area data;

[0125] In response to a user's operation on the editing component, determining a second layer corresponding to the editing component;

[0126] The second layer is superimposed on the first layer to obtain processed screen area data.

[0127] In one embodiment, in controlling the main screen to perform a screen off operation, the control unit 302 is further used to: generate a screen off notification; and send the screen off notification to the main screen so that the main screen enters a screen off state according to the screen off notification.

[0128] In one embodiment, in the process of obtaining the position of the preset selection box on the display interface of the electronic device, the acquisition unit 301 is further used to: respond to the user's moving operation on the preset selection box on the display interface of the electronic device, and determine the position of the preset selection box on the display interface.

[0129] In one embodiment, in the process of obtaining the position of the preset selection box on the display interface of the electronic device, the acquisition unit 301 is further used to: perform a correlation analysis on the data content of the screen area data to obtain the correlation between each element in the data content; based on the correlation of each element, screen out elements whose correlation exceeds a preset correlation threshold as target elements; and determine the position of the preset selection box on the display interface based on the positions of multiple target elements on the display interface.

[0130] In some embodiments, the screen projection device can also be constructed by hardware devices. For example, the screen projection device can be constructed by one or more chips, and the chips can work in coordination with each other to complete the screen projection methods described in the above embodiments. For another example, the screen projection device can also be constructed by various logic devices, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0131] It should be noted that the above-mentioned screen projection device can execute the screen projection method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in the embodiment of the screen projection device, please refer to the screen projection method provided in the embodiment of the present application.

[0132] See also Figure 4 , Figure 4 1 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device includes one or more processors and a memory. The memory is connected to the one or more processors, for example, connected to the processor via a bus.

[0133] The processor is configured to support the computer device to perform the corresponding functions in the method in the above method embodiment. The processor can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0134] The memory is used to store program codes, etc. The memory may include volatile memory (VM), such as random access memory (RAM); the memory may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory may also include a combination of the above types of memory.

[0135] The memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the screen projection method in the embodiment of the present application. The processor executes the various functional applications and data processing of the screen projection method and the screen projection device by running the non-volatile software programs, instructions and modules stored in the memory, that is, realizing the functions of the screen projection method and each module or unit of the screen projection device provided in the above method embodiment.

[0136] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function. The data storage area may store data created according to the use of the projection device, etc. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the projection device via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0137] The one or more modules are stored in the memory, and when executed by the one or more processors, the screen projection method in any of the above-mentioned method embodiments is executed, for example, the method steps described in the above-mentioned method embodiments are executed to realize the functions of the modules described in the above-mentioned device embodiments.

[0138] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0139] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0140] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A screen projection method, applied to an electronic device, characterized in that: include: Obtaining a position of a preset selection box on a display interface of the electronic device, wherein the position of the preset selection box is adjustable; Acquire full-screen data of the display interface recorded by the electronic device; According to the position of the preset selection box on the display interface, extracting screen area data from the full screen data; Encoding the screen area data to obtain screen projection coded data; The screen projection coding data is sent to the display device so that the display device displays the picture selected by the preset selection box on the display interface according to the screen projection coding data.

2. The method according to claim 1, characterized in that: The extracting the screen area data from the full screen data according to the position of the preset selection box on the display interface includes: Rendering the full-screen data into a pre-created texture rendering model to obtain full-screen rendering data, wherein the texture rendering model is configured with a rendering component; Screen area data is captured from the full screen data according to the positions of the rendering component and the preset selection box on the display interface.

3. The method according to claim 2, characterized in that The extracting the screen area data from the full screen data according to the positions of the rendering component and the preset selection box on the display interface includes: According to the rendering component, converting the position of the preset selection box on the display interface into a cut-off position corresponding to the texture rendering model; Screen area data is intercepted from the full screen data according to the interception position.

4. The method according to claim 2, characterized in that: The electronic device is provided with an editing component, and before data encoding the screen area data, the method further comprises: In response to the user's operation on the editing component, the screen area data is processed to obtain processed screen area data.

5. The method according to claim 4, characterized in that The step of processing the screen area data in response to the user's operation on the editing component to obtain the processed screen area data comprises: Acquire a first layer corresponding to the screen area data; In response to a user's operation on the editing component, determining a second layer corresponding to the editing component; The second layer is superimposed on the first layer to obtain processed screen area data.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the position of the preset selection box on the display interface of the electronic device includes: In response to a user's moving operation on a preset selection box on a display interface of the electronic device, a position of the preset selection box on the display interface is determined.

7. According to the method according to any one of claims 1 to 5, obtaining the position of the preset selection box on the display interface of the electronic device comprises: Performing correlation analysis on the data content of the screen area data to obtain the correlation between each element in the data content; According to the correlation degree of each of the elements, elements whose correlation degree exceeds a preset correlation threshold are selected as target elements; The position of the preset selection box on the display interface is determined according to the positions of the multiple target elements on the display interface.

8. A screen projection device, characterized in that: Applied to electronic equipment, characterized in that the device comprises: An acquiring unit, configured to acquire a position of a preset selection box on a display interface of the electronic device, wherein the position of the preset selection box is adjustable; The acquisition unit is further used to acquire the full screen data of the display interface recorded by the electronic device; A capture unit, configured to capture screen area data from the full screen data according to the position of the preset selection box on the display interface; An encoding unit, used for encoding the screen area data to obtain screen projection encoding data; The display unit is used to send the screen projection coding data to the display device, so that the display device displays the picture selected by the preset selection box on the display interface according to the screen projection coding data.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images