Handwriting and screen projection picture superposition system

Abstract

The invention discloses a handwriting and screen projection picture superposition system, which is characterized by comprising a screen projection emitter, a receiving end, a writing device, a camera and a display terminal, the screen projection transmitter communicates with the receiving end through WiFi, and transmits a screen projection picture signal to the receiving end; the writing device communicates with the receiving end through Bluetooth, WiFi or a USB interface, and the sensor module is used for detecting pen starting and pen falling actions; through real-time synchronization of handwriting, screen projection pictures and camera pictures, unified output of a plurality of video sources is ensured, the interaction experience is remarkably improved, a buffer area mechanism is introduced, the problems of packet loss, delay and jitter are effectively solved, the continuity of video streams is ensured, and in addition, through a synchronization mechanism combined with timestamp information, the real-time performance of the system is improved. The real-time synchronous display of the handwriting and the video frame is ensured, the delay or dislocation phenomenon is avoided, meanwhile, an Alpha hybrid processing module is introduced, and the pixel fusion of the handwriting and the screen projection picture is realized through an algorithm to optimize the visual effect.

Description

Technical Field

[0001] The present invention relates to the field of screen projection technology, and in particular to a handwriting and screen projection image superposition system. Background Art

[0002] Screen projection technology is a technology that can wirelessly transmit digital content (such as video, audio, images and applications) from one device to another display device (such as TV, projector, monitor, etc.). It is widely used in education, business, home entertainment and other fields. Screen projection devices mainly include transmitters and receivers. The transmitter is usually a mobile phone, tablet computer, laptop computer, etc., which is responsible for transmitting the content to the receiver. The receiver, such as a smart TV, projector or other display device, receives and displays the projected content. Screen projection technology usually relies on Wi-Fi networks for data transmission. Some devices support other wireless connection methods such as Bluetooth and NFC.

[0003] However, in traditional screen projection systems, it is difficult to achieve real-time synchronization of multiple video sources (such as handwriting, projection screen and camera screen), resulting in information asynchrony. When network conditions are unstable, traditional technologies have weak processing capabilities for packet loss and delay, which can easily lead to video stream interruption. The lack of an effective synchronization mechanism combining timestamp information also causes delays or misalignment between handwriting and projection screen content. These defects significantly limit the application effect of traditional screen projection technology in real-time interaction, user experience and device compatibility. To solve the above problems, we propose a handwriting and projection screen overlay system. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a handwriting and projection image overlay system to solve the problems raised in the background technology.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions:

[0006] A handwriting and projection screen superposition system, comprising: a projection screen transmitter, a receiving end, a writing device, a camera and a display terminal;

[0007] The projection screen signal adopts the H.264 encoding format, and the receiving end includes a network interface module, a buffer, a video decoding module and an image cache module. The network interface module is used to receive the projection screen signal, and the buffer is used to cache the data packets in the projection screen signal to deal with network problems such as packet loss and delay to ensure the continuity of the video stream. The video decoding module is used to decode the projection screen signal into frame images, which are stored in the image cache module. The frame rate of the decoded frame image is 30 or 60 frames per second;

[0008] The writing device communicates with the receiving end via a Bluetooth, WiFi or USB interface. The writing device includes a sensor module and a handwriting capture module. The sensor module is used to detect the start and end of the writing. The handwriting capture module is used to capture the writing trajectory, including the starting point, the end point and the path data. The trajectory data is transmitted to the receiving end in the form of (x, y) coordinates, and combined with the timestamp information to ensure the synchronization of the handwriting and the video frame;

[0009] The receiving end performs superposition processing on the frame image and the writing trajectory data, and outputs a superimposed video stream containing handwriting to a display terminal.

[0010] Preferably, during the transmission of the projection image signal, the buffer mechanism is used to handle packet loss, delay and jitter problems under different network conditions. When the network connection is interrupted or packet loss is serious, the system issues a retransmission request and restores the integrity of the video stream through a packet loss recovery algorithm.

[0011] Preferably, the handwriting capture module of the writing device has pressure detection and speed detection functions, which is used to capture the handwriting pressure and writing speed information during the writing process. The receiving end adjusts the thickness of the handwriting according to the pressure and speed information to improve the user's writing experience and handwriting display effect.

[0012] Preferably, the receiving end includes an Alpha blending processing module for superimposing the pixel color value of the handwriting with the pixel color value of the video frame by an Alpha blending algorithm, and the formula of the Alpha blending algorithm is as follows:

[0013] Pnew(x,y,t)=Pscreen(x,y,t)×(1-α)+Ppen(x,y,t)×α

[0014] Among them, P_new(x,y,t): the color value of the pixel in the new video frame obtained after superposition; P_screen(x,y,t): the color value of the pixel in the original projection screen; P_pen(x,y,t): the color value of the handwriting; α: represents the transparency of the handwriting, usually set between 0 and 1;

[0015] The Alpha blending processing module is used to achieve the fusion of the handwriting trajectory and the current frame image. The user can adjust the Alpha value according to needs to optimize the visual effect.

[0016] Preferably, the receiving end is connected to a camera, which is used to obtain real-time camera images. The receiving end includes an image processing module for adjusting the resolution, frame rate and color space of the camera image, and embedding the camera image in the projection image in a picture-in-picture (PIP) mode. The size, position and transparency of the camera image can be adjusted according to user needs.

[0017] Preferably, after completing the overlay processing, the receiving end generates a composite video stream including the projection screen, writing handwriting and camera screen, and compresses the composite video stream through the H.264 encoder. The receiving end transmits the compressed video stream back to the projection transmitter via WiFi.

[0018] Preferably, the screen projection transmitter is configured to receive and decode the composite video stream transmitted in reverse, and the screen projection transmitter includes a decoding module, which simulates the decoded composite video stream into a UVC (USB Video Class) virtual camera signal. The display terminal recognizes the signal as a camera device to display the superimposed projection screen containing handwriting and camera images in real time.

[0019] Preferably, the receiving end also includes a data synchronization module, which performs synchronization processing according to the timestamp information transmitted by the writing device and the time information of the frame image to ensure real-time synchronous display of the writing handwriting data and the video frame data to avoid delay or misalignment.

[0020] Preferably, the receiving end further comprises a packet loss recovery module, which recovers lost frames through a retransmission mechanism to ensure the integrity and stability of the video stream.

[0021] Preferably, the system dynamically adjusts the bit rate of the video stream according to the network bandwidth through the intelligent bandwidth control module, thereby reducing the transmission pressure of the video stream when the network bandwidth is insufficient.

[0022] Preferably, the receiving end supports a multi-screen display function, can display different video sources on multiple display terminals at the same time, and supports multi-screen synchronous operation.

[0023] Preferably, the display terminal includes a touch screen, and the user can interactively control the display effect, position and size of the handwriting, or adjust the screen layout through the touch screen.

[0024] In summary, the present invention mainly has the following beneficial effects:

[0025] 1. Realize real-time synchronization of multiple video streams

[0026] The present invention realizes real-time synchronization of handwriting, projection screen and camera screen by merging multiple video streams and simulating them as virtual camera signals. This method ensures the unified output of multiple video sources, avoids the delay or dislocation between the video streams, and thus improves the fluency and consistency of multimedia interaction. When users perform interactive operations, they can clearly and in real time see the handwriting, screen content and camera screen synchronous display, which greatly enhances the interactive experience and sense of participation, and is particularly suitable for scenarios with high real-time requirements such as education and remote conferencing.

[0027] 2. Optimize device connection and video streaming stability

[0028] The present invention supports flexible communication between writing devices and receiving ends through Bluetooth, Wi-Fi or USB interfaces by optimizing the connection mode between devices, thereby simplifying the operation process and enhancing the compatibility between devices. In addition, the system adopts the H.264 encoding format and combines the buffer mechanism to effectively deal with network problems such as packet loss, delay and jitter, and ensure the continuity of the video stream. In particular, in the case of network connection interruption or severe packet loss, the system can automatically issue a retransmission request and restore the integrity of the video stream through a packet loss recovery algorithm, ensuring that users can always obtain a smooth and uninterrupted interactive experience.

[0029] 3. Accurate writing synchronization and visual effect optimization

[0030] Combined with the synchronization mechanism of timestamp information, the present invention can ensure the real-time synchronous display of handwriting and video frames, avoiding the misalignment caused by delays or frame losses, thereby improving the user's writing experience. The introduced Alpha blending processing module optimizes the visual effect between handwriting and the projected screen through a pixel-level fusion algorithm. Users can adjust the transparency of handwriting as needed to achieve seamless fusion of handwriting and background images, enhancing visual comfort and naturalness. Whether in education and teaching, creative design or entertainment interaction, users can enjoy precise, clear and expressive writing effects, further improving the interactivity and application flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a system structure block diagram of the present invention.

[0032] Figure 2 It is a data synchronization and image processing flow chart of the present invention.

[0033] Figure 3 It is a block diagram of the intelligent adjustment and optimization module of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0035] The following examples are used to illustrate the present invention, but they cannot be used to limit the scope of protection of the present invention. The conditions in the examples can be further adjusted according to specific conditions. Simple improvements to the method of the present invention under the premise of the concept of the present invention belong to the scope of protection claimed in the present invention.

[0036] Example 1: Basic handwriting and projection screen overlay system

[0037] refer to Figure 1 , a handwriting and projection screen superposition system, including a projection screen transmitter, a receiving end, a writing device and a display terminal;

[0038] The system transmits the projection screen signal through the H.264 encoding format. The receiving end is equipped with a network interface module, a buffer, a video decoding module and an image cache module. The network interface module is used to receive the projection screen signal, and the buffer is used to cache data packets to deal with network problems and ensure the continuity of the video stream.

[0039] The video decoding module decodes the projection screen signal into frame images, which are stored in the image cache module. The frame rate of the decoded frame images is 30 or 60 frames per second.

[0040] The writing device communicates with the receiving end via Bluetooth, WiFi or USB interface. The writing device includes a sensor module and a handwriting capture module. The former is used to detect the start and end of the pen, and the latter is used to capture the writing trajectory. The trajectory data is transmitted to the receiving end in the form of (x, y) coordinates, and combined with the timestamp information to ensure the synchronization of the handwriting and the video frame;

[0041] The receiving end performs superposition processing on the frame image and the writing trajectory data, and outputs a superimposed video stream containing the handwriting to the display terminal;

[0042] In addition, the receiving end handles packet loss, delay and jitter problems under network conditions through a buffer mechanism. When the network connection is interrupted or packet loss is serious, the system will issue a retransmission request and restore the integrity of the video stream through a packet loss recovery algorithm.

[0043] Embodiment 2: Enhanced handwriting capture and video stream processing system

[0044] Based on the first embodiment, this embodiment further enhances the function of the handwriting capture module;

[0045] The handwriting capture module of the writing device adds pressure detection and speed detection functions to capture the handwriting pressure and writing speed information during the writing process. The receiving end adjusts the thickness of the handwriting according to the pressure and speed information to improve the user's writing experience and handwriting display effect;

[0046] At the same time, the receiving end is equipped with an Alpha blending processing module, which uses the Alpha blending algorithm to superimpose the pixel color value of the handwriting with the pixel color value of the video frame. The formula of the Alpha blending algorithm is as follows:

[0047] Pnew(x,y,t)=Pscreen(x,y,t)×(1-α)+Ppen(x,y,t)×α

[0048] Among them, P_new(x,y,t): the color value of the pixel in the new video frame obtained after superposition; P_screen(x,y,t): the color value of the pixel in the original projection screen; P_pen(x,y,t): the color value of the handwriting; α: represents the transparency of the handwriting, usually set between 0 and 1;

[0049] The receiving end is also connected to the camera to obtain real-time camera images. The receiving end includes an image processing module for adjusting the resolution, frame rate and color space of the camera image, and embedding the camera image into the projection screen in a picture-in-picture (PIP) mode. The size, position and transparency of the camera image can be adjusted according to user needs;

[0050] Finally, the receiving end generates a composite video stream containing the projection screen, handwriting and camera screen, compresses the composite video stream through the H.264 encoder, and transmits it back to the projection transmitter.

[0051] The Alpha blending algorithm achieves the fusion of handwriting and video images by dynamically adjusting the transparency (α value). When the α value is close to 0, the handwriting will be more transparent and blend more naturally with the background image; when the α value is close to 1, the handwriting will be more obvious and stand out. This processing module not only improves the naturalness of the visual effect, but also enhances the controllability of users in different usage scenarios, allowing users to adjust the visibility of handwriting according to their needs.

[0052] The effect of the Alpha blending processing module on the display terminal is that the transition between the handwriting and the projection screen becomes smoother, avoiding the sense of separation between the handwriting and the background screen in traditional technology, making the fusion of the two layers more natural and real. For example, in educational applications, the teacher's handwriting and the content of the projected courseware can be seamlessly combined to help students better understand the information without being disturbed by the messy screen segmentation. In order to further improve the user experience, this embodiment also optimizes the synchronization processing of handwriting and video frames. By combining the aforementioned timestamp synchronization mechanism, the receiving end can ensure the synchronous display of handwriting and projection video frames. This synchronization processing technology can eliminate the misalignment problem caused by factors such as network delay and inconsistent frame rate, so that when the user is writing or drawing, the projection screen and the presentation of handwriting are always consistent. When the writing device transmits the handwriting data to the receiving end, the receiving end synchronizes the handwriting data and video frames in real time according to the timestamp information to ensure accurate matching of the handwriting and the projection content. For example, if the timestamp of the video frame is t and the timestamp of the handwriting is t+0.03 seconds, the receiving end will accurately display the handwriting on the corresponding frame image without any delay or misalignment.

[0053] In addition, the receiving end can also dynamically adjust the video stream according to actual needs to ensure the real-time and accuracy of the handwriting. For example, when the network transmission is relatively unstable, the receiving end will adjust the compression method and data transmission rate of the video stream to optimize the system response time, thereby improving the accuracy of the handwriting and screen synchronization.

[0054] Embodiment 3 adds a multi-channel video source synchronization system based on Embodiment 1, which can simultaneously receive input signals from multiple video sources, including:

[0055] Projected screen: The screen image from the projection transmitter, usually used to display videos, pictures, presentations, etc.

[0056] Writing handwriting: Input signals from writing devices, which contain the trajectory information of the user's writing on the display screen, such as the position of the handwriting, pen pressure, writing speed, etc.

[0057] Camera image: A real-time image signal from a camera, usually used to display the lecturer, environment, or on-site situation to enhance interactivity and immersion.

[0058] These signals must be effectively synchronized at the receiving end to ensure that they can be displayed seamlessly on the display terminal and are compatible with different network conditions.

[0059] One of the core innovations of this embodiment is to support the synchronous display of multiple video sources, ensuring that the projection screen, handwriting and camera screen can be displayed simultaneously on the same screen and maintain a consistent timeline. To achieve this goal, the receiving end uses an efficient image processing module to uniformly synchronize and synthesize video signals from different sources. The receiving end synthesizes the images of multiple video sources through the image processing module to create a composite image containing multiple screen elements. The projection screen usually occupies most of the screen, while the handwriting and camera screen can be embedded in the main screen as small windows. Through real-time synthesis, the system can superimpose handwriting with video content and display the camera screen at the same time to achieve seamless display of multi-source video streams.

[0060] In order to ensure the synchronous display of multiple video sources, the system adopts a timestamp synchronization mechanism. A timestamp is assigned to each frame of video between the projection screen, handwriting and camera screen to ensure that all video sources are consistent on the timeline. The system performs real-time frame matching based on these timestamps to eliminate the asynchrony caused by network delay or processing delay, ensuring that the display content of each video source is accurate. Since different video sources may have different frame rates and resolutions, the receiving end also needs to perform unified frame rate synchronization and resolution adjustment on these video sources. For example, the frame rate of the projection screen may be 30fps, while the frame rates of handwriting and camera screens may be different. The system synchronizes the frame rates of all video sources through frame insertion, frame drop or interpolation algorithms to ensure that the output video stream is smooth and unstuck on the display terminal.

[0061] When processing multiple video sources, the efficiency of data transmission and processing is a crucial factor. Especially when the network environment is unstable or the bandwidth is limited, how to ensure the stability and fluency of the video stream is one of the key technologies of this embodiment. To this end, this embodiment adopts the H.264 encoding standard and intelligent compression technology to cope with network fluctuations, packet loss and delays, and ensure the continuity and stability of the video stream. H.264 is a widely used video encoding standard with efficient compression performance and good image quality. In this embodiment, the receiving end performs H.264 encoding on the synthesized multi-channel video stream (including projection screen, handwriting and camera screen). H.264 can compress video data into a smaller file size, thereby reducing the network burden and improving the efficiency of video transmission. In addition, H.264 encoding also has high fault tolerance, and the system can guarantee a certain image quality even in poor network conditions.

[0062] In order to adapt to different network environments, the system of this embodiment has an intelligent bandwidth control function. The system can monitor the network bandwidth in real time and dynamically adjust the encoding parameters of the video stream (such as bit rate, frame rate, resolution, etc.) according to the changes in bandwidth. For example, when the network bandwidth is low, the system will reduce the bit rate of the video stream to reduce the amount of data transmission and avoid packet loss; when the bandwidth is sufficient, the system will restore the higher quality video stream output to ensure that users enjoy the best viewing experience in a high-quality network environment.

[0063] Packet loss and delay are common problems in network transmission. This embodiment addresses these challenges by adopting a packet loss recovery algorithm and a delay compensation mechanism. The receiving end caches video data packets through a buffer. When packet loss occurs, the system can automatically request retransmission of the lost data packets and merge the recovered data with the original data stream to avoid video stream interruption or freeze. In addition, the system also adjusts the timeline of different video sources through a delay compensation mechanism to ensure the synchronization of multiple video sources, and maintains the smoothness of the picture even in the case of high network delay.

[0064] In addition, the system also supports touch screen interaction, and users can use the touch screen to zoom, drag or edit the screen. For example, users can adjust the position, size or transparency of the handwriting on the display terminal, or quickly switch the screen layout through gestures on the touch screen. Through these functions, users can flexibly control multiple video streams to create a more personalized and rich interactive experience.

[0065] Embodiment 4 Based on Embodiment 1, the intelligent data synchronization and dynamic adjustment system added in this embodiment mainly improves the transmission quality and interactive effect of the video stream in the following ways:

[0066] Intelligent data synchronization module: It can dynamically adjust the transmission strategy of video stream according to network conditions, optimize the synchronization effect of multi-source video, reduce delay and dislocation, and ensure seamless synchronization of projection screen, handwriting and camera screen.

[0067] Dynamic adjustment system: real-time adjustment of the handwriting display effect to adapt to the writing habits of different users and provide a personalized interactive experience. This module is designed to meet different environments and user needs, so that the handwriting always maintains the best state in multi-source video synthesis.

[0068] Handwriting capture and display optimization: Combined with the image processing module on the receiving end, fine-tune the display effect of handwriting on the projection screen, including the thickness, color, transparency and position of the handwriting, to ensure the perfect integration of the user's handwriting with other video sources (such as projection screen, camera screen).

[0069] In the multi-source video stream synchronization application, the projection screen, writing handwriting and camera screen are different data sources. How to ensure the real-time synchronization of these video streams, especially in the case of network fluctuations, unstable bandwidth, etc., to always maintain smoothness and consistency, is the key issue in this embodiment. The intelligent data synchronization module solves this problem in the following ways: the intelligent data synchronization module first obtains the current network bandwidth, delay, packet loss rate and other information in real time through the network monitoring function. By monitoring these network parameters, the system can make a quick assessment of the current network environment and provide a basis for subsequent dynamic adjustments. Network delay and insufficient bandwidth will directly affect the synchronization of video streams, so the system must adjust the transmission strategy in real time to avoid time dislocation or frame loss between multi-source video streams. According to the real-time network conditions, the intelligent data synchronization module dynamically adjusts the transmission strategy of the video stream. Specifically, when the network conditions are good, the system can support higher resolution and frame rate to ensure video quality; when the network bandwidth is low or the delay is high, the system automatically reduces the bit rate, frame rate or resolution of the video stream, thereby reducing the amount of data transmission and avoiding the picture asynchrony caused by packet loss and delay. In addition, the system will cache the lost frames and recover them through prediction algorithms and retransmission mechanisms to reduce the impact of network fluctuations. In order to ensure the consistency of different video sources (projected screen, handwriting, camera screen) on the timeline, the intelligent data synchronization module adopts a timestamp synchronization mechanism. A timestamp is attached to the data stream of each video source, indicating the generation time of the data frame. When processing the video stream, the receiving end will match it according to the timestamp to ensure that the display time of each video source is consistent. This synchronization mechanism effectively avoids video dislocation and lag problems caused by network delays or unstable data transmission during real-time interaction.

[0070] When network packet loss is serious, the intelligent data synchronization module can recover packet loss based on the key frames of the video stream. The system automatically requests lost data packets and saves data within a certain time range through the buffer to ensure the continuity of the video. Through this mechanism, the synchronization and stability of the video stream can still be guaranteed even in an unstable network environment, greatly improving the user experience. In addition to the intelligent data synchronization module, the introduction of the dynamic adjustment system enables the handwriting display effect to be adjusted according to the writing habits and real-time interaction needs of different users, further optimizing the interactive experience. The system intelligently adjusts the handwriting display effect in the following ways:

[0071] When the user writes, the pressure and writing speed applied by the user will affect the thickness of the handwriting. This embodiment uses the pressure detection and speed sensing module to monitor the user's writing pressure and speed in real time. According to the real-time feedback, the system can dynamically adjust the thickness of the handwriting. For example, when the user writes quickly, the system can automatically thicken the handwriting so that it is clearer on the display screen; when the user writes slowly, the handwriting is automatically thinned to ensure the exquisiteness of the writing.

[0072] In addition to the thickness of the handwriting, the color and transparency of the handwriting are also key factors in the user interaction experience. The dynamic adjustment system automatically adjusts the color and transparency of the handwriting according to different background content (such as the projection screen or camera screen) to avoid conflicts between the handwriting and the background content. In terms of color adjustment, the system can provide a variety of preset handwriting color options according to different application scenarios or user preferences, and even allow users to customize colors. In terms of transparency adjustment, the system dynamically changes the transparency of the handwriting according to the complexity of the background, making the fusion between the handwriting and the background more natural.

[0073] Intelligent adaptation to writing speed: Dynamic adjustment The system also makes adaptive adjustments based on the user's writing speed. If the user changes the writing speed during the writing process, the system can sense it in real time and automatically adjust the display effect of the handwriting. For example, when writing quickly, the handwriting may become blurred or intermittent. The system can adjust the display algorithm to ensure that the handwriting remains smooth and continuous; when writing slowly, the details of the handwriting are retained to ensure that every stroke is accurately presented.

[0074] At the receiving end, the collaborative work of the handwriting capture module and the image processing module is the basis for the present embodiment to achieve efficient synchronization and smooth display. The handwriting capture module captures the trajectory data (such as position, pressure, speed, etc.) written by the user in real time and transmits these data to the receiving end. The image processing module at the receiving end is responsible for synthesizing these handwriting data with the projection screen or camera screen, and finally outputting them to the display terminal.

[0075] The image processing module not only superimposes the handwriting with the video stream, but also dynamically adjusts the presentation of the handwriting according to different display devices and user needs. For example, for high-resolution display screens, the system can automatically adjust the display details of the handwriting so that the handwriting can remain clear on the large screen; for lower-resolution display devices, the system automatically reduces the level of detail to avoid blurring or loss of handwriting.

[0076] The intelligent data synchronization and dynamic adjustment system of this embodiment has a wide range of application scenarios, and is particularly suitable for distance education applications, where teachers not only need to display multimedia content, but also need to write teaching content in real time. Through the intelligent data synchronization and dynamic adjustment system, the teacher's handwriting can be synchronized with the video screen, and the handwriting display effect can be automatically adjusted according to the teacher's writing habits, providing a smoother and more natural teaching experience. In a video conference, multiple video sources (such as the speaker's camera screen, shared screen content, and whiteboard writing content) need to be displayed at the same time. The intelligent data synchronization module can ensure that all video sources are displayed synchronously, and the dynamic adjustment system adjusts the display effect of the handwriting in real time according to the speaker's handwriting style, ensuring that the visual experience of the participants is clear and coherent.

[0077] The beneficial effects of this embodiment are: real-time synchronization of handwriting, projection screen and camera screen is achieved, thereby ensuring the unified output of multiple video sources, significantly improving the interactive experience, adopting the H.264 encoding format and introducing a buffer mechanism, effectively dealing with packet loss, delay and jitter problems, and ensuring the continuity of the video stream. In addition, by combining the synchronization mechanism of timestamp information, the real-time synchronous display of handwriting and video frames is ensured to avoid delays or misalignment, further improving the user's writing experience. At the same time, an Alpha blending processing module is introduced to achieve pixel fusion of handwriting and projection screen through an algorithm, so that users can adjust the transparency according to their needs and optimize the visual effect.

[0078] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0079] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto, and any changes or substitutions that are not conceived through creative work should be included in the protection scope of the present invention.

Claims

1. A handwriting and projection screen superposition system, characterized in that: include: Screen projection transmitter, receiver, writing device, camera and display terminal; The projection screen transmitter communicates with the receiving end via a wireless network and transmits the projection screen signal to the receiving end; The receiving end includes a network interface module, a buffer, a video decoding module and an image cache module, wherein the network interface module is used to receive a projection screen signal, the buffer is used to cache data packets in the projection screen signal to cope with network problems such as packet loss and delay, and ensure the continuity of the video stream, and the video decoding module decodes the projection screen signal into a frame image and stores it in the image cache module; The writing device communicates with the receiving end via Bluetooth, Wi-Fi or USB interface, and includes a sensor module and a handwriting capture module, wherein the sensor module is used to detect the pen start and pen drop actions, and the handwriting capture module is used to capture the writing trajectory data and transmit the writing trajectory data to the receiving end via timestamp information; The receiving end performs superposition processing on the frame image and the writing trajectory data through an image processing module, and outputs a composite video stream containing the writing traces to a display terminal.

2. The handwriting and projection screen superposition system according to claim 1, characterized in that: The projection screen image signal adopts the H.264 encoding format, and the receiving end decodes the projection screen image signal into a frame image through the H.264 decoding module, and superimposes it with the writing handwriting data.

3. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end also includes an Alpha blending processing module, which superimposes the pixel color values ​​of the handwriting with the pixel color values ​​of the video frame through the Alpha blending algorithm, thereby optimizing the visual fusion effect of the handwriting and the projected screen.

4. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end also includes a data synchronization module, which performs synchronization processing according to the timestamp information transmitted by the writing device and the timestamp information of the video frame to ensure real-time synchronous display of handwriting and video frame.

5. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end dynamically adjusts the transmission strategy of the video stream according to the real-time network status, optimizes the synchronization effect of multi-source videos, and ensures the stability and fluency of the video stream.

6. The handwriting and projection screen superposition system according to claim 1, characterized in that: The handwriting capture module of the writing device includes a pressure detection function and a speed detection function, which is used to detect the pressure and writing speed applied by the user when writing, and adjust the thickness, transparency and display effect of the handwriting according to this information.

7. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end includes an image processing module, which adapts and optimizes the handwriting and video frames according to the resolution, frame rate and display requirements of different display terminals.

8. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end also includes a camera input module for receiving and processing image signals provided by the camera. The camera image is embedded in the projection screen through a picture-in-picture mode. The size, position and transparency of the camera image can be adjusted according to user needs.

9. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end also includes a packet loss recovery module, which recovers lost frames through a retransmission mechanism to ensure the integrity and stability of the video stream.

10. The handwriting and projection screen superposition system according to claim 1, characterized in that: The system dynamically adjusts the bit rate of the video stream according to the network bandwidth through the intelligent bandwidth control module, thereby reducing the transmission pressure of the video stream when the network bandwidth is insufficient.

11. The handwriting and projection screen superposition system according to claim 1, characterized in that: The receiving end supports a multi-screen display function, can display different video sources on multiple display terminals at the same time, and supports multi-screen synchronous operation.

12. The handwriting and projection screen superposition system according to claim 1, characterized in that: The display terminal includes a touch screen, and the user can interactively control the display effect, position and size of the handwriting, or adjust the screen layout through the touch screen.

13. A computer-readable storage medium storing a computer program, which, when executed in a computer, enables the computer to implement the following steps: Receive the projection screen signal and decode it to generate video frames; receiving writing trajectory data from a writing device, and synchronizing the writing trajectory with a video frame according to timestamp information; Overlaying the writing track with the video frame to generate a composite video stream containing the writing trace; Outputting the composite video stream to a display terminal; Adjust the video stream transmission strategy according to the real-time network conditions to optimize the synchronization effect of multiple video sources and ensure the stability and smoothness of the video stream.

14. A handwriting and projection screen superposition device, characterized in that: include: A screen projection transmitter is used to transmit the projection image signal to the receiving end via a wireless network; The receiving end includes: A network interface module, used to receive the projection image signal; Buffer, used to cache data packets in the projection image signal; A video decoding module, used to decode the projection screen signal and generate a video frame; A writing device interface, used for receiving writing trajectory data from a writing device; A data synchronization module, used for synchronizing the writing trajectory data with the video frame according to timestamp information; An image processing module, used for superimposing the synchronized writing track with the video frame to generate a composite video stream; An output module, used for outputting the composite video stream to a display terminal; The dynamic adjustment module is used to adjust the video stream transmission strategy in real time according to network conditions to ensure the stability and smoothness of the video stream.

Images