Screen projection method and device and electronic equipment

Abstract

The invention relates to a screen projection method and device and electronic equipment. The method is applied to electronic equipment, the electronic equipment and projection display equipment are located in the same network, and the method comprises the steps of obtaining screen parameters of the projection display equipment; determining to-be-projected data, and determining target projection data through the first thread and at least according to the screen parameters and the to-be-projected data; and encoding the target projection data through a second thread to obtain target encoded projection data, and sending the target encoded projection data to a projection display device to instruct the projection display device to decode the target encoded projection data and then display the decoded target encoded projection data, the first thread and the second thread being mutually independent threads. By adopting the method, the display speed and the reaction speed of projection can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of data transmission technology, and in particular to a screen projection method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of mobile devices, screen mirroring technology is becoming increasingly widespread. More and more electronic devices are now projecting video or image resources onto display screens for viewing. In traditional technology, when an electronic device connects to a physical screen, it sends its current image to the display screen, which then displays the image. Alternatively, a virtual display screen can be created, and the image displayed on the virtual screen is used to display the image from the virtual display screen.

[0003] However, the first method, which directly sends the image from the current electronic device to the display screen, may cause problems such as resolution mismatch, incomplete image display, and excessive image stretching. The second method, on the other hand, creates an additional virtual display screen, increasing the power consumption of the electronic device. Furthermore, if the video or image is large, it will further deplete the device's performance, slowing down the projection speed and response time, potentially causing stuttering and other issues. Summary of the Invention

[0004] Therefore, it is necessary to provide a screen projection method, device, electronic device, and storage medium to address the aforementioned technical problems.

[0005] Firstly, this disclosure provides a screen projection method. Applied to an electronic device, wherein the electronic device and the projection display device are on the same network, the method includes:

[0006] Obtain the screen parameters of the projection display device;

[0007] Determine the data to be projected, and determine the target projection data through the first thread and at least based on the screen parameters and the data to be projected;

[0008] The target projection data is encoded by a second thread to obtain target encoded projection data, and the target encoded projection data is sent to the projection display device to instruct the projection display device to decode the target encoded projection data and display it. The first thread and the second thread are independent of each other.

[0009] In one embodiment, determining the target projection data via a first thread and at least based on the screen parameters and the data to be projected includes:

[0010] At least based on the screen parameters, and using the first thread to generate constraints that control the image range;

[0011] Based on the constraints, the first thread determines the target projection data that satisfies the constraints from the data to be projected.

[0012] In one embodiment, the electronic device displays virtual control buttons, and the constraint conditions for generating the control image range based at least on the screen parameters and using a first thread include:

[0013] In response to detecting a projection control operation for the virtual control button, determine the control information indicated by the projection control operation;

[0014] Based on the control information and the screen parameters, constraints for controlling the image range are generated using the first thread.

[0015] In one embodiment, the step of generating constraints controlling the image range based at least on the screen parameters and using a first thread includes:

[0016] Acquire inertial sensor data from the electronic device;

[0017] The attitude change of the electronic device in space is calculated based on the inertial sensor data;

[0018] Based on the pose change and the screen parameters, constraints controlling the image range are generated using the first thread.

[0019] In one embodiment, the method further includes:

[0020] Determine the amount of data to be projected and the data transmission speed between the electronic device and the projection display device;

[0021] The display resolution is determined based on the data volume and / or data transmission speed;

[0022] The target projection data is determined through the first thread based on the screen parameters, the display resolution, and the data to be projected.

[0023] In one embodiment, determining the target projection data via a first thread and at least based on the screen parameters and the data to be projected includes:

[0024] Based on the screen parameters, a virtual screen matching the screen parameters is created using a first thread;

[0025] The first thread is used to project the data to be projected onto the virtual screen;

[0026] The target projection data is determined based on the data projected onto the virtual screen.

[0027] In one embodiment, the electronic device displays virtual control buttons, and the step of projecting the data to be projected onto the virtual screen using a first thread includes:

[0028] In response to detecting a projection control operation for the virtual control button, determine the control information indicated by the projection control operation;

[0029] Based on the control information, the data to be projected is projected onto the virtual screen using the first thread.

[0030] In one embodiment, projecting the data to be projected onto the virtual screen using a first thread includes:

[0031] Acquire inertial sensor data from the electronic device;

[0032] The attitude change of the electronic device in space is calculated based on the inertial sensor data;

[0033] Based on the attitude change, the data to be projected is projected onto the virtual screen using the first thread.

[0034] Secondly, this disclosure also provides a screen projection device. Applied to an electronic device, wherein the electronic device and the projection display device are on the same network, the device includes:

[0035] A parameter acquisition module is used to acquire the screen parameters of the projection display device;

[0036] The projection data determination module is used to determine the data to be projected, and to determine the target projection data through a first thread and at least based on the screen parameters and the data to be projected;

[0037] The data transmission module is used to encode the target projection data through a second thread to obtain target encoded projection data, and send the target encoded projection data to the projection display device to instruct the projection display device to decode the target encoded projection data and display it. The first thread and the second thread are independent threads.

[0038] Thirdly, this disclosure also provides an electronic device. The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in any of the above method embodiments.

[0039] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in any of the above method embodiments.

[0040] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0041] In the above embodiments, the first thread focuses on the calculation of the target projection data (such as image transformation and viewpoint adaptation), while the second thread independently handles encoding and transmission, with both executing in parallel. For example, when the encoding process is delayed due to high-resolution data, the first thread can still continue processing the next frame of data, avoiding overall process lag and ensuring the continuity of the projected image (such as no significant delay during real-time viewpoint switching in VR scenes). Utilizing the parallel computing capabilities of a multi-core CPU avoids processor idleness caused by the time consumed by a single thread during "encoding-transmission." For example, the first thread occupies the CPU's image processing core, while the second thread occupies a dedicated encoding module (such as a GPU encoder), achieving efficient allocation of hardware resources. By pre-obtaining the screen parameters of the projection display device (such as resolution, refresh rate, and color gamut), the first thread can process the data to be projected specifically to obtain the target projection data, ensuring display quality. The two threads are independent of each other; if one thread stops working due to an anomaly (such as an encoding module crash), the other thread can continue running, preventing the entire projection system from crashing. For example, if the second thread makes an encoding error, the first thread can still continue generating and caching the target projection data, retransmitting it after the encoding module recovers, reducing the perceived interruption time for the user. Separating thread functions makes the system architecture clearer and facilitates future upgrades. For example, if support for a new encoding format (such as AV1) is required, only the encoding logic of the second thread needs to be modified, without adjusting the data processing flow of the first thread, thus reducing development and maintenance costs. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a screen projection method in one embodiment;

[0044] Figure 2 This is a flowchart illustrating step S104 in one embodiment;

[0045] Figure 3 This is a flowchart illustrating step S202 in one embodiment;

[0046] Figure 4 This is another flowchart illustrating step S202 in one embodiment;

[0047] Figure 5 This is a schematic diagram of the process for determining target projection data using display resolution in one embodiment;

[0048] Figure 6 This is a flowchart illustrating step S104 in one embodiment;

[0049] Figure 7 This is a flowchart illustrating step S604 in one embodiment;

[0050] Figure 8 This is another flowchart illustrating step S604 in one embodiment;

[0051] Figure 9 This is a schematic block diagram of the projection device in one embodiment;

[0052] Figure 10 This is a schematic diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0054] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0055] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0056] In one embodiment, such as Figure 1 As shown, a screen projection method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that the terminal can refer to an electronic device, which may include smartphones, smart cameras, smart portable devices, etc. The electronic device and the projection display device are on the same network. "Same network" can mean that the electronic device and the projection display device are connected to the same Wi-Fi network or the same local area network. The electronic device and the projection display device can be connected to the same network via wired or wireless means. Some embodiments of this disclosure do not limit the manner in which they are on the same network. The method includes the following steps:

[0057] S102, Obtain the screen parameters of the projection display device.

[0058] In some embodiments of this disclosure, the projection display device can be a device that converts electronic image signals into optical images and projects them onto a screen or other surface. Common types include projectors, projection televisions, and holographic projection devices. In this context, it specifically refers to a display terminal used to display images. Screen parameters typically include information such as resolution, size, aspect ratio, refresh rate, and display area.

[0059] Specifically, when an electronic device and a projection display device are on the same network, the electronic device can establish a connection with the projection display device and obtain its screen parameters. For example, it can call the display device management API provided by the operating system to obtain the projection display device's screen parameters.

[0060] S104, determine the data to be projected, and determine the target projection data through the first thread and at least based on the screen parameters and the data to be projected.

[0061] The data to be projected can be image data or video data. More specifically, it can be a panoramic image or panoramic video. The target projection data refers to the data extracted and processed from the data to be projected based on screen parameters and pose constraints, ultimately used for display on the projection display device. Examples include: a cropped image area (a specific viewpoint area extracted from a panoramic image); a reprojected image (a corrected image projected onto a flat screen from spherical data); and a resolution-optimized image (an image adjusted according to the screen resolution).

[0062] Specifically, the data to be projected can be read and determined from the electronic device. Often, the data to be projected may not be compatible with the projection display device. Therefore, a first thread can be used, utilizing screen parameters and the data to be projected, to determine the target projection data. This ensures that the target projection data can be displayed correctly.

[0063] S106, the target projection data is encoded by the second thread to obtain target encoded projection data, and the target encoded projection data is sent to the projection display device to instruct the projection display device to decode the target encoded projection data and display it. The first thread and the second thread are independent threads.

[0064] The second thread can be a dedicated processing thread independent of the first thread (responsible for generating target projection data), primarily handling data encoding and transmission tasks. This multi-threaded architecture parallelizes computational tasks, preventing the encoding process from blocking the image generation flow and ensuring system real-time performance. For example, the first thread can handle image cropping and viewpoint transformation, while the second thread can independently perform encoding and network transmission. Encoding typically involves converting the raw target projection data into a compressed format, reducing data volume by removing redundant information (spatial and temporal redundancy) for easier network transmission and storage. The encoded target projection data can be encoded and compressed target projection data, existing in a specific format (such as H.264 encoded NAL unit streams or JPEG compressed image data). Decoding typically refers to the process by which the projection display device restores the encoded data to the original image, implemented through a hardware decoder (GPU, dedicated decoding chip) or software decoder, and is the final processing step before display.

[0065] Specifically, the second thread can acquire target projection data and encode it to obtain target encoded projection data. Since the electronic device and the projection display device are on the same network, the second thread can be used to send the target encoded projection data to the projection display device. After receiving the target encoded projection data, the projection display device can decode it and display it.

[0066] In some exemplary embodiments, a second thread can be used to configure encoder parameters (such as H.265 encoding, 1920×1080 resolution, 8Mbps bitrate, and 30fps frame rate). A connection is established between the electronic device and the projection display device. The second thread calls the encoder to compress the target projection data into a video stream, generating target encoded projection data. The encoded data is encapsulated into network data packets, and a timestamp and sequence number are added. The data packets are sent to the projection display device over the network. The frame rate is synchronized to ensure stable output. After receiving the network data packets, the projection display device reassembles the complete video frames and displays them.

[0067] In the above projection method, the first thread focuses on calculating the target projection data (such as image transformation and viewpoint adaptation), while the second thread independently handles encoding and transmission, with both executing in parallel. For example, if the encoding process is delayed due to high-resolution data, the first thread can still continue processing the next frame, avoiding overall process lag and ensuring the continuity of the projected image (e.g., no noticeable delay during real-time viewpoint switching in VR scenes). Utilizing the parallel computing capabilities of a multi-core CPU avoids processor idleness caused by the time consumed by a single thread during "encoding-transmission." For example, the first thread occupies the CPU's image processing core, while the second thread occupies a dedicated encoding module (such as a GPU encoder), achieving efficient allocation of hardware resources. By pre-obtaining the screen parameters of the projection display device (such as resolution, refresh rate, and color gamut), the first thread can process the data to be projected specifically, thereby obtaining the target projection data and ensuring display quality. The two threads are independent of each other; if one thread stops working due to an anomaly (such as an encoding module crash), the other thread can continue running, preventing the entire projection system from crashing. For example, if the second thread makes an encoding error, the first thread can still continue generating and caching the target projection data, retransmitting it after the encoding module recovers, reducing the perceived interruption time for the user. Separating thread functions makes the system architecture clearer and facilitates future upgrades. For example, if support for a new encoding format (such as AV1) is required, only the encoding logic of the second thread needs to be modified, without adjusting the data processing flow of the first thread, thus reducing development and maintenance costs.

[0068] In one embodiment, such as Figure 2 As shown, determining the target projection data through a first thread and at least based on the screen parameters and the data to be projected includes:

[0069] S202, at least based on the screen parameters, and using the first thread, generate constraints to control the image range.

[0070] The constraints controlling the image range can be limitations set for the display range of an image or panoramic image, based at least on information such as screen parameters. These limitations include the image's display boundaries in the horizontal and vertical directions (determining the start and end points of the display area), image scaling (controlling the screen size), and viewing angle offset (adjusting the display viewing angle).

[0071] Specifically, a first thread is created and initialized, and assigned the task of image data processing. After starting, the first thread continuously listens for and acquires data such as screen parameters (e.g., device attitude data detected by inertial sensors). When screen parameters are acquired, or new or different screen parameters are acquired, the first thread can use these parameters to process the data. Furthermore, based on the screen parameters (the number of horizontal and vertical pixels, resolution, etc.) and the original resolution of the panoramic image, the first thread calculates the boundary of the image region that can be fully displayed on the current screen. This boundary can be a constraint. For example, if the image is 8K resolution (7680×3840) and the screen resolution is 4K (3840×2160), then the starting coordinates and range for cropping the 4K region from the image need to be calculated. It should be noted that in addition to screen parameters, other data can also be used.

[0072] S204, Based on the constraints, the first thread determines the target projection data that satisfies the constraints from the data to be projected.

[0073] Specifically, using the first thread and constraints, the coordinate range to be intercepted is calculated. Based on the coordinate range, the portion of image data that meets the constraints is extracted and processed from the data to be projected; this portion of image data can be the target projection data.

[0074] In this embodiment, constraints are generated based on screen parameters (such as resolution, size, and refresh rate) to ensure a strict match between the projected data and the physical characteristics of the display device. For example, when the screen resolution is 4K, the constraints limit the upper limit of the output image resolution, preventing the generation of high-resolution data that exceeds the screen's display capabilities and reducing unnecessary computational resource consumption. For irregularly shaped screens (such as curved or spherical screens), the constraints can define special geometric boundaries to ensure that the image completely covers the screen without distortion. The constraints can respond in real time to changes in screen parameters (such as when the user adjusts the screen size or switches display modes), dynamically adjusting the image range.

[0075] In one embodiment, such as Figure 3 As shown, the electronic device displays virtual control buttons, and the constraint conditions for generating the control image range based at least on the screen parameters and using a first thread include:

[0076] S302, in response to detecting a projection control operation for the virtual control button, determine the control information indicated by the projection control operation.

[0077] S304, Based on the control information and the screen parameters, and using the first thread, generate constraints for controlling the image range.

[0078] Virtual control buttons refer to operation buttons presented virtually on the projection display interface. Unlike physical buttons, they are triggered by touch, gestures, voice, etc. For example, if the image involved in this disclosure is a panoramic image, the virtual control buttons on the panoramic image projection interface may include function buttons such as view switching, play / pause, and image zoom. Projection control operations typically refer to user interactions with virtual control buttons, such as clicking virtual buttons with a finger, making specific gestures in the air to trigger button functions, or activating button operations through voice commands. In some embodiments of this disclosure, there are no restrictions on how projection control operations are triggered. Control information is usually the instruction content generated by the projection control operation, clearly instructing the system to perform which function. For example, clicking the "fast forward" virtual button results in the control information "fast forward the video for a specific duration"; operating the view rotation button results in control information including parameters such as rotation direction and angle.

[0079] Specifically, it can detect user actions on virtual control buttons in real time. This detection can be achieved in several ways. If the projection display device supports touch functionality (such as a touchscreen projector), when a user touches the virtual control button area, the device's touch sensor transmits the touch location information to the system, which then identifies the virtual button corresponding to the touch location. Alternatively, a camera or dedicated sensor (such as a depth camera) can capture user gestures, which can then be recognized and analyzed using computer vision algorithms (such as OpenCV and TensorFlow). For example, a user drawing a circle in the air is recognized as a view rotation operation. Furthermore, if the electronic device integrates a voice recognition module (such as the iFlytek voice recognition engine or Google Voice API), when the user speaks a voice command corresponding to the virtual button function (such as "pause play"), the voice recognition module converts the speech into text information, parses the text to determine the corresponding control operation, and thus detects the projection control action.

[0080] Specifically, based on the detected projection control operation, specific control information is determined. The first thread generates constraints based on the control information and screen parameters. For example, the basic range and proportion of the image display can be determined based on parameters such as screen resolution and aspect ratio. For example, if the screen resolution is 1920×1080 and the aspect ratio is 16:9, the maximum boundary of the image display is set based on this. If the control information is view rotation, the first thread calculates the view offset of the image in the horizontal and vertical directions based on the screen's viewing angle range and rotation angle requirements, and updates the boundary coordinates of the image display. For example, if the original display view is [0°, 90°], and the control information requires a 30° rotation to the left, it is updated to [-30°, 60°]. If the control information is image scaling, the first thread calculates an appropriate scaling ratio based on the screen size and the current image size. If the screen size is small, the maximum scaling factor is limited to ensure clear and complete image display; if the screen size is large, the scaling limit can be appropriately increased.

[0081] The calculated parameters such as image display boundary, viewpoint offset, and scaling ratio are integrated to form complete constraints for controlling the image range.

[0082] In this embodiment, virtual control buttons (such as touch, gesture, and voice triggers) replace traditional physical buttons, allowing users to operate directly on the projected screen without additional peripherals (such as remote controls). When generating constraints, screen parameters (resolution, size, etc.) are integrated to ensure a perfect match between the projected content and the display device. When switching to different control information using virtual control buttons, adjustments are made adaptively to avoid image cropping or stretching. For example, when switching from a 1080p screen to a 4K screen, the constraints are dynamically adjusted to guarantee display quality.

[0083] In one embodiment, such as Figure 4 As shown, the step of generating constraints controlling the image range based at least on the screen parameters and using a first thread includes:

[0084] S402, acquire inertial sensor data of the electronic device.

[0085] S404, Calculate the attitude change of the electronic device in space based on the inertial sensor data.

[0086] S406, based on the pose change and the screen parameters, and using the first thread, generate constraints to control the image range.

[0087] Inertial sensor data is typically collected by the inertial measurement unit (IMU) built into the electronic device, and usually includes accelerometer data, gyroscope data, and magnetometer data. Attitude changes are physical quantities that describe changes in the position and orientation of the electronic device in three-dimensional space, and are usually represented by Euler angles (pitch angle, yaw angle, roll angle) or quaternions.

[0088] Specifically, the current inertial sensor data of the electronic device can be acquired. Then, the attitude quaternion is calculated based on the inertial sensor data. Based on the attitude quaternion at each moment, the attitude change of the electronic device in space is calculated. The viewpoint offset is determined based on the attitude change, and then constraints are generated using the viewpoint offset, screen parameters, and the first thread.

[0089] In this embodiment, the device's posture (such as rotation and tilt) is captured in real time by an inertial sensor. The system can calculate the posture change within milliseconds and generate corresponding image constraints. For example, when the user rotates the device, the projected image will adjust the display area synchronously, achieving a natural interaction where "the image follows the user's gaze," avoiding the lag of traditional button operations.

[0090] In one embodiment, such as Figure 5 As shown, the method further includes:

[0091] S502, determine the amount of data to be projected and determine the data transmission speed between the electronic device and the projection display device.

[0092] The amount of data to be projected typically refers to the storage space occupied by the original image or video data or the amount of data transmitted in real time, usually measured in bytes (Byte), megabytes (MB), or bits (Bit). Data transmission speed refers to the amount of data transmitted between the electronic device and the projection display device per unit time, usually measured in bits per second (bps) or megabits per second (Mbps). It should be noted that in some embodiments of this disclosure, the data transmission speed usually refers to the actual transmission speed rather than the theoretical transmission speed.

[0093] Specifically, the amount of data to be projected can be calculated based on the resolution and color depth of the data to be projected. For example, data size = width × height × color depth / 8 (Bytes).

[0094] S504, determine the display resolution based on the data volume and / or data transmission speed.

[0095] Specifically, generally speaking, data transfer speed = resolution × frame rate × bits per pixel × compression ratio. Therefore, it can be determined that data volume and / or data transfer speed are directly proportional to display resolution; that is, the higher the data volume and / or data transfer speed, the higher the display resolution can be. It should be noted that the display resolution here cannot exceed the resolution of the projection display device. For example, if the display device supports a maximum resolution of 4K, and the determined display resolution is 8K, the 8K resolution will be adjusted to 4K to maximize transmission speed without wasting resources. Furthermore, regarding how to determine the display resolution based on data volume and / or data transfer speed, multiple mapping relationships between resolutions and data volume and / or data transfer speed can be preset, and the resolution can be determined through these mapping relationships. Alternatively, predictive resolution adjustment based on machine learning can be used, for example, training a model using historical data to predict the optimal resolution based on data volume and / or data transfer speed.

[0096] S506, through the first thread and based on the screen parameters, the display resolution and the data to be projected, determine the target projection data.

[0097] Specifically, it checks whether the display resolution exceeds the screen's supported range. If the display resolution (e.g., 4K) is higher than the screen's maximum resolution (1080p), it automatically reduces to the maximum supported value. Based on the screen parameters, display resolution, and the area to be displayed in the projected data, the target projection data is determined.

[0098] In some exemplary real-time streams, if the data to be projected is a panoramic image, the various screen parameters of the projection display device can first be analyzed, including: resolution (e.g., 1920×1080), aspect ratio (16:9), pixel density (PPI), etc., to calculate the physical display area size. The actual output resolution (display resolution) is determined (e.g., forced 720p or adaptive 4K), and its screen compatibility is verified. The format (panoramic / planar), original resolution (e.g., 8K), encoding type (H.265), and metadata (e.g., projection method, frame rate) of the data to be projected are identified. A mapping relationship is established between the screen coordinate system (pixel coordinates) and the coordinate system of the data to be projected (e.g., the latitude and longitude of the panoramic image), and the transformation matrix is ​​calculated. Based on the screen aspect ratio and display resolution, the area to be displayed in the data to be projected is calculated. Cross-boundary cases (e.g., the left and right edges of the panoramic image are joined) are handled, ensuring seamless display through mirroring or loop copying. Anti-aliasing algorithms (e.g., Smoothstep) are applied to optimize the cropped edges and avoid jagged edges. Based on the data type to be projected (e.g., equidistant cylindrical or cubic projection), the corresponding transformation algorithm is applied to convert it into a displayable planar image. An interpolation algorithm (e.g., bilinear / bicubic interpolation) is selected and adjusted according to the ratio of the display resolution to the original data resolution. The final target projection data is then obtained.

[0099] In this embodiment, by acquiring the amount of data to be projected (such as file size and bitrate) and the inter-device transmission speed (such as WiFi bandwidth and USB throughput) in real time, the system can automatically adapt to different hardware environments and network conditions. It can automatically reduce the resolution in low-bandwidth scenarios (such as mobile network projection) to reduce transmission pressure and avoid stuttering; and enable high-resolution mode in high-bandwidth environments (such as HDMI direct connection) to ensure lossless image quality. Based on the data volume and transmission speed, it dynamically calculates the optimal display resolution (such as 1080p / 4K adaptive), balancing image quality and transmission efficiency. This avoids resource waste (such as bandwidth idleness when transmitting small amounts of data at 4K resolution) or performance bottlenecks (such as latency caused by forcibly transmitting 8K data under low bandwidth) resulting from a fixed resolution.

[0100] In one embodiment, such as Figure 6 As shown, determining the target projection data through a first thread and at least based on the screen parameters and the data to be projected includes:

[0101] S602, based on the screen parameters, and using the first thread, create a virtual screen that matches the screen parameters.

[0102] S604, the data to be projected is projected onto the virtual screen using the first thread.

[0103] S606, Determine target projection data based on the data projected on the virtual screen.

[0104] A virtual screen typically simulates a physical projection display device's virtual display space at the software level; essentially, it's an image buffer in memory. It has the same parameters as the physical screen (such as resolution, aspect ratio, and color space) and is used for preprocessing data before it's projected onto the physical device. The core function of a virtual screen is to pre-render or transform the data to be projected (such as panoramic images or 3D models) into the virtual space, then extract the target projection data adapted to the physical screen, achieving pre-calculation and optimization of the display effect.

[0105] Specifically, the first thread can obtain physical screen parameters (such as resolution 1920×1080, aspect ratio 16:9, color depth 24-bit) through the EDID protocol or system API, and create a perfectly matching virtual screen in memory. The first thread then acquires the data to be projected and projects it onto the virtual screen according to the screen parameters. Next, the first thread extracts the valid display area from the virtual screen (excluding black borders or portions exceeding the physical screen) and converts it to a format supported by the physical device (such as RGB→YUV420). This yields the target projection device.

[0106] In some exemplary embodiments, such as when the data to be projected is a panoramic image, the latitude and longitude coordinates need to be converted into pixel coordinates of the virtual screen using a coordinate mapping algorithm. Then, the 3D model is transformed into the 2D space of the virtual screen using a perspective projection matrix using the OpenGL pipeline, and vertex coordinates and texture mapping are calculated.

[0107] In this embodiment, the virtual screen acts as a "digital mirror" of the physical screen, synchronizing parameters such as resolution, aspect ratio, and color space in real time. When the projection device is changed (e.g., switching from a 16:9 screen to a 4:3 projector), the first thread can dynamically reconstruct the virtual screen, automatically scaling the data to be projected and cropping black borders to avoid image distortion or incomplete display due to parameter mismatch. By performing projection transformations (such as coordinate mapping and perspective projection) in advance through the virtual screen, rendering tasks that originally needed to be processed in real time on the projection device are shifted to the preprocessing stage. For example, vertex transformations of 3D models can be calculated in advance on the virtual screen, avoiding image stuttering caused by insufficient computing power of the projector and increasing the real-time projection frame rate by 30%-50%.

[0108] In one embodiment, the electronic device displays virtual control buttons, such as... Figure 7 As shown, the step of projecting the data to be projected onto the virtual screen using the first thread includes:

[0109] S702, in response to detecting a projection control operation for the virtual control button, determine the control information indicated by the projection control operation.

[0110] For details on the specific implementation method here, please refer to the above embodiments, which will not be repeated here.

[0111] S704, according to the control information, the data to be projected is projected onto the virtual screen using the first thread.

[0112] Specifically, the first thread continuously listens to the control information queue. When a new instruction arrives, it immediately interrupts the current non-critical task (such as background rendering) and prioritizes user operations. For example, taking view rotation as an example, when the ROTATE_LEFT instruction is detected, the first thread pauses the distant view cache calculation and immediately starts the view rotation process. Based on rotation-related control information (such as {angle:30°}), the projection matrix of the virtual screen is updated. The first thread then re-renders the data to be projected onto the virtual screen based on the updated projection matrix.

[0113] In one embodiment, such as Figure 8 As shown, the step of projecting the data to be projected onto the virtual screen using the first thread includes:

[0114] S802, acquire the inertial sensor data of the electronic device.

[0115] S804, calculate the attitude change of the electronic device in space based on the inertial sensor data.

[0116] For details on the specific implementation method, please refer to the above embodiments, which will not be repeated here.

[0117] S806, based on the attitude change amount, and using the first thread, project the data to be projected onto the virtual screen.

[0118] Specifically, the attitude changes typically include changes in Euler angles or quaternions. These Euler angles or quaternions can be converted into a projection transformation matrix to update the rendering viewpoint of the virtual screen. Generally, only panoramic images adjust the viewpoint; therefore, the following embodiment uses a panoramic image as an example. For instance, yaw corresponds to horizontal rotation, pitch corresponds to vertical rotation, and roll corresponds to screen tilt. The attitude matrix maps latitude and longitude coordinates to pixel coordinates on the virtual screen. For example, increasing the yaw angle by 30° corresponds to a 30° rightward rotation of the panoramic image within the pixel range on the virtual screen.

[0119] In this embodiment, inertial sensors (accelerometers, gyroscopes, magnetometers) are used to collect device motion data in real time (sampling rate can reach 100-1000Hz). Combined with the high-speed processing capability of the first thread, the device's rotation, tilting and other actions are immediately converted into changes in the projection view of the virtual screen. Without touch or buttons, the projected content can be controlled directly through changes in the device's posture.

[0120] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0121] Based on the same inventive concept, this disclosure also provides a screen projection device for implementing the screen projection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more screen projection device embodiments provided below can be found in the limitations of the screen projection method described above, and will not be repeated here.

[0122] In one embodiment, such as Figure 9 As shown, a projection device 900 is provided, applied to an electronic device. The electronic device and the projection display device are on the same network. The device includes: a parameter acquisition module 902, a projection data determination module 904, and a data transmission module 906, wherein:

[0123] Parameter acquisition module 902 is used to acquire the screen parameters of the projection display device;

[0124] Projection data determination module 904 is used to determine the data to be projected, and to determine the target projection data through a first thread and at least based on the screen parameters and the data to be projected;

[0125] The data sending module 906 is used to encode the target projection data through a second thread to obtain target encoded projection data, and send the target encoded projection data to the projection display device to instruct the projection display device to decode the target encoded projection data and display it. The first thread and the second thread are independent threads.

[0126] In one embodiment of the device, the projection data determination module 904 includes:

[0127] A constraint generation module is used to generate constraints that control the range of the image, based at least on the screen parameters and using a first thread.

[0128] The target data determination module is used to determine, based on the constraints, target projection data that satisfies the constraints from the data to be projected through the first thread.

[0129] In one embodiment of the device, the electronic device displays virtual control buttons, and the constraint generation module is further configured to, in response to detecting a projection control operation for the virtual control buttons, determine the control information indicated by the projection control operation; and generate constraints for the control image range based on the control information and the screen parameters, using a first thread.

[0130] In one embodiment of the device, the constraint generation module is further configured to acquire inertial sensor data of the electronic device; calculate the attitude change of the electronic device in space based on the inertial sensor data; and generate constraints for controlling the image range based on the attitude change and the screen parameters using a first thread.

[0131] In one embodiment of the device, the device further includes:

[0132] A transmission speed determination module is used to determine the amount of data to be projected and to determine the data transmission speed between the electronic device and the projection display device.

[0133] The display resolution determination module is used to determine the display resolution based on the data volume and / or data transmission speed;

[0134] The target data determination module is further configured to determine target projection data through a first thread and based on the screen parameters, the display resolution, and the data to be projected.

[0135] In one embodiment of the device, the target data determination module is further configured to create a virtual screen that matches the screen parameters using a first thread; project the data to be projected onto the virtual screen using the first thread; and determine target projection data based on the data projected onto the virtual screen.

[0136] In one embodiment of the device, the electronic device displays virtual control buttons, and the target data determination module includes:

[0137] The projection module is configured to respond to the detection of a projection control operation on the virtual control button, determine the control information indicated by the projection control operation, and project the data to be projected onto the virtual screen using a first thread based on the control information.

[0138] In one embodiment of the device, the projection module is further configured to acquire inertial sensor data of the electronic device; calculate the attitude change of the electronic device in space based on the inertial sensor data; and project the data to be projected onto the virtual screen based on the attitude change and using a first thread.

[0139] Each module in the aforementioned projection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device in hardware form or independent of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0140] In one embodiment, an electronic device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a screen projection method. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the device's casing, or an external keyboard, touchpad, or mouse.

[0141] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above method embodiments.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above method embodiments.

[0144] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this disclosure may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this disclosure may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the appended claims.

Claims

1. A screen projection method, characterized in that, Applied to an electronic device, wherein the electronic device and the projection display device are on the same network, the method includes: Obtain the screen parameters of the projection display device; Determine the data to be projected, and determine the target projection data through the first thread and at least based on the screen parameters and the data to be projected; The target projection data is encoded by a second thread to obtain target encoded projection data, and the target encoded projection data is sent to the projection display device to instruct the projection display device to decode the target encoded projection data and display it. The first thread and the second thread are independent of each other.

2. The method according to claim 1, characterized in that, The process of determining the target projection data via a first thread and at least based on the screen parameters and the data to be projected includes: At least based on the screen parameters, and using the first thread to generate constraints that control the image range; Based on the constraints, the first thread determines the target projection data that satisfies the constraints from the data to be projected.

3. The method according to claim 2, characterized in that, The electronic device displays virtual control buttons, and the constraint conditions for generating the control image range based at least on the screen parameters and using a first thread include: In response to detecting a projection control operation for the virtual control button, determine the control information indicated by the projection control operation; Based on the control information and the screen parameters, constraints for controlling the image range are generated using the first thread.

4. The method according to claim 2, characterized in that, The step of generating constraints controlling the image range based at least on the screen parameters and using a first thread includes: Acquire inertial sensor data from the electronic device; The attitude change of the electronic device in space is calculated based on the inertial sensor data; Based on the pose change and the screen parameters, constraints controlling the image range are generated using the first thread.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Determine the amount of data to be projected and the data transmission speed between the electronic device and the projection display device; The display resolution is determined based on the data volume and / or data transmission speed; The target projection data is determined through the first thread based on the screen parameters, the display resolution, and the data to be projected.

6. The method according to claim 1, characterized in that, The process of determining the target projection data via a first thread and at least based on the screen parameters and the data to be projected includes: Based on the screen parameters, a virtual screen matching the screen parameters is created using a first thread; The first thread is used to project the data to be projected onto the virtual screen; The target projection data is determined based on the data projected onto the virtual screen.

7. The method according to claim 6, characterized in that, The electronic device displays virtual control buttons, and the step of projecting the data to be projected onto the virtual screen using a first thread includes: In response to detecting a projection control operation for the virtual control button, determine the control information indicated by the projection control operation; Based on the control information, the data to be projected is projected onto the virtual screen using the first thread.

8. The method according to claim 6, characterized in that, The step of projecting the data to be projected onto the virtual screen using a first thread includes: Acquire inertial sensor data from the electronic device; The attitude change of the electronic device in space is calculated based on the inertial sensor data; Based on the attitude change, the data to be projected is projected onto the virtual screen using the first thread.

9. A screen projection device, characterized in that, Applied to an electronic device, wherein the electronic device and the projection display device are on the same network, the device includes: A parameter acquisition module is used to acquire the screen parameters of the projection display device; The projection data determination module is used to determine the data to be projected, and to determine the target projection data through a first thread and at least based on the screen parameters and the data to be projected; The data transmission module is used to encode the target projection data through a second thread to obtain target encoded projection data, and send the target encoded projection data to the projection display device to instruct the projection display device to decode the target encoded projection data and display it. The first thread and the second thread are independent threads.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images