Stroke data processing method, mobile terminal, display terminal and equipment

Abstract

The invention discloses a stroke data processing method, a mobile terminal, a display terminal and equipment, and the method comprises the steps: responding to a stroke operation of a user, and generating initial stroke data containing a plurality of attribute parameters; according to each attribute parameter, selecting a corresponding compression strategy for compression, and generating a compressed stroke data packet; and sending the compressed stroke data packet to a display terminal, so that the display terminal displays a stroke track corresponding to the initial stroke data based on the compressed stroke data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a pen stroke data processing method, a mobile terminal, a display terminal, and a device. Background Technology

[0002] Currently, in cross-device art co-creation scenarios, multiple users can create art through collaborative painting applications on their mobile devices (such as Procreate's collaboration mode) and synchronize all brushstrokes in real time to the screen of the same display terminal (such as a smart TV).

[0003] However, when the number of collaborating users increases dramatically (e.g., reaching hundreds of users), the mobile terminal needs to transmit massive amounts of pen touch data to the display terminal. Due to the limited processing power and network bandwidth of typical display terminals (such as TVs with only 1GB of memory), the massive data transmission puts enormous pressure on network bandwidth, leading to data transmission delays and ultimately resulting in a poor real-time collaboration experience. Summary of the Invention

[0004] The main purpose of this application is to provide a pen stroke data processing method, a mobile terminal, a display terminal, and a device, which aims to solve the technical problem that when the amount of data transmitted from the mobile terminal to the display terminal is large, it will cause data transmission delay and result in a poor real-time collaboration experience.

[0005] To achieve the above objectives, this application proposes a pen stroke data processing method applied to a mobile terminal, wherein the mobile terminal and a display terminal are communicatively connected, and the method includes: Responding to the user's pen strokes, it generates initial pen stroke data containing multiple attribute parameters; The corresponding compression strategy is selected based on each attribute parameter to compress the data and generate a compressed pen stroke data package. The compressed pen stroke data package is sent to the display terminal so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0006] Furthermore, to achieve the above objectives, this application also proposes a pen stroke data processing method, which is applied to a display terminal and communicates with a mobile terminal. The method includes: The system receives compressed pen touch data sent by the mobile terminal. The compressed pen touch data is obtained by the mobile terminal selecting the corresponding compression strategy according to the attribute parameters in the initial pen touch data. The initial pen touch data is generated by the mobile terminal in response to the user's pen touch operation. The compressed stroke data shows the stroke trajectory corresponding to the initial stroke data.

[0007] In addition, to achieve the above objectives, this application also proposes a mobile terminal, comprising: The user interaction module is used to respond to the user's pen stroke operations and generate corresponding initial pen stroke data containing multiple attribute parameters. The data compression module is used to select the corresponding compression strategy based on each attribute data to compress it and generate a compressed pen stroke data package. The data transmission module is used to send the compressed pen stroke data packet to the display terminal, so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0008] Furthermore, to achieve the above objectives, this application also proposes a display terminal, which includes: The data receiving module is used to receive compressed pen touch data sent by the mobile terminal. The compressed pen touch data is obtained by the mobile terminal selecting the corresponding compression strategy according to the attribute parameters in the initial pen touch data. The initial pen touch data is generated by the mobile terminal in response to the user's pen touch operation. The data rendering module is used to display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0009] In addition, to achieve the above objectives, this application also proposes a pen touch data processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pen touch data processing method applied to a mobile terminal or display terminal as described above. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the first embodiment of the pen touch data processing method applied to a mobile terminal according to this application. Figure 2 This is a flowchart illustrating the second embodiment of the pen touch data processing method applied to a mobile terminal according to this application. Figure 3 This is a schematic diagram of the two-end interaction of the pen stroke data processing method in this application; Figure 4 This is a flowchart illustrating an embodiment of the pen touch data processing method applied to a display terminal according to this application; Figure 5This is a schematic diagram of the module structure of the mobile terminal of this application; Figure 6 This is a schematic diagram of the module structure of the display terminal in this application; Figure 7 This is a schematic diagram of the pen touch data processing device of this application.

[0013] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0014] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0015] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0016] The main solution of this application embodiment is: responding to the user's pen stroke operation, generating initial pen stroke data containing multiple attribute parameters; selecting the corresponding compression strategy according to each attribute parameter to compress the data, generating a compressed pen stroke data packet; and sending the compressed pen stroke data packet to the display terminal so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0017] Because existing technologies can cause data transmission delays when mobile terminals transmit large amounts of data to display terminals, resulting in a poor real-time collaboration experience.

[0018] This application provides a solution that compresses the initial pen stroke data by selecting the corresponding compression strategy according to different attribute parameters, thereby reducing the amount of pen stroke data transmitted to the display terminal, thus reducing the pressure on network bandwidth and transmission latency when transmitting data to the display terminal, and effectively improving the real-time collaboration experience.

[0019] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or pen-based data processing device capable of performing the above functions. The following description uses a pen-based data processing device as an example to illustrate this embodiment and the subsequent embodiments.

[0020] Based on this, embodiments of this application provide a pen touch data processing method applied to a mobile terminal, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the pen touch data processing method applied to a mobile terminal according to this application.

[0021] This embodiment applies to a mobile terminal, specifically, the pen touch data processing device is a mobile terminal, and the mobile terminal is communicatively connected to the display terminal. For ease of understanding, a mobile terminal is used instead of a pen touch data processing device in the description. The pen touch data processing method includes steps S10~S30: Step S10: In response to the user's pen stroke operation, generate initial pen stroke data containing multiple attribute parameters.

[0022] It should be noted that the initial stroke data can be uncompressed or unencoded raw data, used to fully describe the characteristics of a single stroke operation. Attribute parameters can be the various data components that make up the initial stroke data. Each attribute parameter describes a specific dimension of the stroke's characteristics; the attribute parameters are independent of each other but work together to fully define the visual presentation of the stroke.

[0023] In the specific implementation, the user touches the touchscreen of the mobile terminal with their finger or stylus and begins to move it, triggering a pen touch operation on the mobile terminal. The mobile terminal's operating system responds to the user's pen touch operation, capturing touch events at a set frequency (e.g., 120 times per second). Each sample generates a trajectory point, and then outputs a sequence of trajectory points arranged in chronological order. Simultaneously, the operating system also associates the pen touch with the current rendering state parameters. When the pen touch operation ends (the user's finger or stylus leaves the screen), the operating system encapsulates the trajectory point sequence and rendering state parameters into a structured data object. This structured data object is the initial pen touch data. Among them, the trajectory point sequence and different rendering state parameters constitute attribute parameters.

[0024] It should be understood that the rendering state parameters associated with a brush stroke can be a set of preset or user-defined attribute parameters that remain constant throughout the duration of a single brush stroke operation and globally affect the final visual presentation of that stroke. Examples include brush stroke color and brush type.

[0025] Step S20: Select the corresponding compression strategy according to each attribute parameter to compress and generate a compressed pen stroke data package.

[0026] In practical implementation, the optimal compression strategy can be pre-configured for different attribute parameters of the pen stroke data through testing. This ensures that the required compression rate can be achieved when compressing different attribute parameters while ensuring the accuracy of the core data. The mobile terminal's operating system can select the corresponding compression strategy based on the different attribute parameters in the initial pen stroke data and use the selected compression strategy to compress each attribute parameter, thereby achieving differentiated compression of different attribute parameters of the initial pen stroke data.

[0027] In one feasible implementation, the attribute parameters include dynamic trajectory parameters, brush style parameters, and color appearance parameters, and step S20 includes steps S201 to S206: Step S201: Calculate the coordinate difference between the next data point and the previous data point in the dynamic trajectory parameters to obtain the dynamic trajectory compressed data.

[0028] It should be noted that dynamic trajectory parameters describe the spatial and temporal trajectory of the pen stroke, including a series of sampling points (x, y, p, t) arranged in chronological order. Here, x is the horizontal coordinate of the point on the screen, y is the vertical coordinate, p is the pressure applied during the stroke, and t is the timestamp of that point. These four dimensions of data are generated by the mobile terminal in response to the user's drawing actions using a finger or pen on the touch area.

[0029] In practical implementation, for dynamic trajectory parameters, the mobile terminal's operating system can store the absolute coordinates (x0, y0, p0, t0) of the first data point, and for the subsequent i-th point (xi, yi, pi, ti), store the difference between it and the previous point. , , , ), , , , The storage space required for this difference is smaller than that required for storing absolute coordinate values. Furthermore, since the differences can be positive or negative, the sign of each difference can be stored using a sign bit, and ZigZag encoding maps the signed differences to unsigned integers, further reducing the space required for storing the differences. Finally, the unsigned differences, after the above processing, and the first data point are recorded together as dynamic trajectory compressed data.

[0030] Step S202: Obtain the preset brush dictionary.

[0031] The preset brush dictionary includes dictionary indices corresponding to different brush style parameters.

[0032] It should be noted that brush style parameters can be parameters describing the rendering style of a brush stroke, including at least one of brush type, size, texture, and opacity. These brush style parameters can be generated by the mobile terminal in response to a single brush selection operation performed by the user.

[0033] In the specific implementation, brush style parameters include a variety of complex parameters such as type (pencil, brush), thickness, texture, and transparency. Each brush style can be configured with a corresponding number as a dictionary index, that is, the dictionary index is used to replace the lengthy brush style parameters, and each dictionary index is added to the same brush dictionary to obtain a preset brush dictionary.

[0034] Step S203: Obtain the target dictionary index corresponding to the brush style parameter from the preset brush dictionary, and use the target dictionary index as the compressed data of the brush style.

[0035] In the specific implementation, the mobile terminal can look up the corresponding dictionary index in the preset brush dictionary according to the current brush style parameters, and use the queried target dictionary index as the brush style compression data, thereby compressing the brush parameters that may occupy hundreds of bytes into an integer of only a few bytes (such as 4B), achieving the ultimate compression.

[0036] It should be understood that the same preset brush dictionary can be pre-synchronized in both the mobile terminal and the display terminal, so that after the display terminal receives the compressed brush style data transmitted by the mobile terminal, it can restore the brush style parameters based on the same preset brush dictionary.

[0037] Step S204: Convert the color appearance parameters using the YUV color space to obtain compressed color appearance data with a preset color sampling ratio.

[0038] It should be noted that the color appearance parameter can be an RGB value describing the color information of the brush stroke. This color appearance parameter can be generated by the mobile terminal in response to the user's color selection operation with a single brush stroke.

[0039] Understandably, the preset sampling ratio can be a pre-set sampling rate, such as YUV420 (4:2:0 sampling).

[0040] In practical implementation, the mobile terminal can perform color space conversion on the color appearance parameters in the YUV color space, separating the color appearance parameters into Y (luminance), U, and V (chrominance). Y (luminance) represents the brightness and darkness details of the image, while U and V (chrominance) represent the color information of the image. Then, the color appearance parameters in the YUV color space are sampled for chrominance at a preset sampling ratio, realizing the format conversion of the color appearance parameters and obtaining color appearance compressed data with the preset color sampling ratio. By converting the RGB color appearance parameters, which are not sensitive to the human eye, to the YUV color space and sampling them, the amount of color data is effectively reduced.

[0041] Step S205: Perform layer hash mapping on the layer identifier of the current painting layer to obtain compressed layer data.

[0042] It should be noted that the layer identifier can be a parameter that identifies the current painting layer.

[0043] In the specific implementation, the mobile terminal can perform a hash function calculation on the layer identifier of the current painting layer, map the layer identifier to a number of fixed length, and then limit the result of the hash value to the range of 16-bit binary numbers through bit operations, so as to obtain a 16-bit layer hash value, that is, layer compressed data, so that no matter how many times the ID of the current painting layer is used, it will only occupy 2 bytes of storage space in the end.

[0044] Step S206: Integrate the dynamic trajectory compression data, brush style compression data, color appearance compression data, and layer compression data to obtain the compressed brush stroke data package.

[0045] In practical implementation, the mobile terminal can integrate dynamic trajectory compression data, brush style compression data, color appearance compression data, and layer compression data, and encapsulate them into a structured stroke data object, namely the compressed stroke data package.

[0046] Step S30: The compressed pen stroke data packet is sent to the display terminal so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0047] In a practical implementation, the mobile terminal can send the compressed pen stroke data packet to the display terminal, which then decompresses and renders the compressed pen stroke data before displaying the pen stroke trajectory corresponding to the initial pen stroke data on the display terminal's screen.

[0048] This embodiment generates initial stroke data containing multiple attribute parameters in response to user stroke operations; it then compresses the data using a corresponding compression strategy based on each attribute parameter, generating a compressed stroke data packet; finally, it sends the compressed stroke data packet to the display terminal, enabling the display terminal to display the stroke trajectory corresponding to the initial stroke data based on the compressed stroke data. This embodiment reduces the amount of stroke data transmitted to the display terminal by selecting appropriate compression strategies based on different attribute parameters, thereby reducing the pressure on network bandwidth and transmission latency when transmitting data to the display terminal, effectively improving the real-time collaboration experience.

[0049] Based on the first embodiment of this application, a second embodiment of this application is proposed. In the second embodiment, content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the pen touch data processing method applied to a mobile terminal according to this application.

[0050] In this embodiment, step S30 includes steps S301 to S305: Step S301: Extract the dynamic trajectory compressed data and header data from the compressed pen stroke data package.

[0051] The header data includes compressed brush style data, compressed color appearance data, and compressed layer data.

[0052] In the specific implementation, the brush style compression data, color appearance compression data, and layer compression data are the core rendering attributes of the brush stroke. After extracting the dynamic trajectory compression data, brush style compression data, color appearance compression data, and layer compression data from the compressed brush stroke data package, the mobile terminal can use the brush style compression data, color appearance compression data, and layer compression data as header data.

[0053] Step S302: Repeat the sending of header data to the display terminal a preset number of times.

[0054] In the actual implementation, the header data is the most critical data. If it is lost, the entire stroke cannot be rendered. Therefore, the mobile terminal can send the header data to the display terminal repeatedly a preset number of times (such as 3 times) to increase the probability of the header data being successfully sent to the display terminal and ensure that the display terminal can correctly parse the stroke style.

[0055] Step S303: Cut the dynamic trajectory compressed data into a preset number of data blocks.

[0056] In a practical implementation, the mobile terminal can divide all coordinate point data (differences) of the dynamic trajectory compression data into a preset number of data blocks (such as 50). For example, a data stream containing 1000 Delta points can be divided into blocks of 50 points each. Data block 1 contains points 1 to 50, data block 2 contains points 51 to 100, and so on.

[0057] Step S304: Add a corresponding sequence number to each data block according to the time order of the data blocks in the original stroke.

[0058] In the specific implementation, a series of coordinate points (x, y, p, t) obtained through touch sampling are accompanied by timestamps for each coordinate point. The time sequence can be determined based on these timestamps. The mobile terminal can assign a unique, incremental sequence number to each data block according to the time sequence. This sequence number clearly identifies the positional order of the data block within the initial pen stroke data or dynamic trajectory parameters.

[0059] Step S305: Select data blocks sequentially according to the order of their serial numbers and send them to the display terminal so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the received main data and each data block.

[0060] In practice, the mobile terminal can select data blocks sequentially according to their serial numbers and send them to the display terminal in a continuous, pipeline manner. After successfully receiving the main data and each data block, the display terminal can decompress and render the main data and each data block, and then display the pen stroke trajectory corresponding to the initial pen stroke data on the screen of the display terminal.

[0061] In one feasible implementation, the step of sequentially selecting data blocks according to the order of their serial numbers and sending them to the display terminal includes steps S3051~S3053: Step S3051: Select the target data block to be sent according to the sequence number.

[0062] In the actual implementation, for each data block, the mobile terminal selects the data block as the target data block in order of the sequence number and sends it.

[0063] Step S3052: Send the target data block to the display terminal so that the display terminal can perform integrity verification on the target data block.

[0064] Step S3053: Upon receiving a confirmation signal from the display terminal indicating that the target data block has passed verification, the next data block is selected as the target data block according to the sequence number. The process of sending the target data block to the display terminal to enable the display terminal to perform integrity verification on the target data block continues until confirmation signals for each data block are received.

[0065] Reference Figure 3 , Figure 3 This is a schematic diagram illustrating the two-terminal interaction of the pen stroke data processing method of this application. After the mobile terminal sends header data to the display terminal, it performs block transmission of the dynamic trajectory compressed data. Specifically, it selects any target data block from the dynamic trajectory compressed data packet and sends it to the display terminal, executing a loop operation. Upon receiving any target data block, the display terminal checks its integrity. If the target data block is intact, the verification is considered successful, and an acknowledgment signal (ACK) is sent back to the mobile terminal. This acknowledgment signal contains the sequence number of the target data block, which identifies its confirmation. When the mobile terminal receives the acknowledgment signal corresponding to the target data block, it determines that the target data block was successfully transmitted. Based on the sequence number of the successfully transmitted target data block, it selects the next data block corresponding to the next sequence number, using this next data block as the new target data block. Then, it returns to the step of sending the target data block to the display terminal for integrity verification. This process is repeated until acknowledgment signals for each data block are received, confirming the transmission is complete and thus completing the transmission of the compressed pen stroke data packet.

[0066] This application also provides a pen touch data processing method applied to a display terminal, referring to... Figure 4 , Figure 4 This is a flowchart illustrating an embodiment of the pen touch data processing method applied to a display terminal according to this application.

[0067] This embodiment applies to a display terminal, specifically, the pen touch data processing device is the display terminal, which is communicatively connected to the mobile terminal. For ease of understanding, the display terminal is used in place of the pen touch data processing device in the description. The pen touch data processing method includes steps S40~S50: Step S40: Receive compressed pen stroke data sent by the mobile terminal.

[0068] The compressed pen stroke data is obtained by the mobile terminal selecting the corresponding compression strategy based on the attribute parameters in the initial pen stroke data. The initial pen stroke data is generated by the mobile terminal in response to the user's pen stroke operation.

[0069] Step S50: Display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0070] In the implementation, the user touches the mobile terminal's touchscreen with their finger or stylus and begins to move it, triggering a stroke operation on the mobile terminal. The mobile terminal's operating system responds to the user's stroke operation, capturing touch events at a set frequency (e.g., 120 times per second). Each sample generates a trajectory point, which is then output as a sequence of trajectory points arranged in chronological order. Simultaneously, the operating system associates the stroke with current rendering state parameters. When the stroke operation ends (the user's finger or stylus leaves the screen), the operating system encapsulates the trajectory point sequence and rendering state parameters into a structured data object, which is the initial stroke data. The mobile terminal's operating system can select a corresponding compression strategy based on different attribute parameters in the initial stroke data and use the selected compression strategy to compress each attribute parameter, achieving differentiated compression of different attribute parameters in the initial stroke data. The mobile terminal can send the compressed stroke data packet to the display terminal, where the display terminal decompresses and renders the compressed stroke data, displaying the stroke trajectory corresponding to the initial stroke data on the display terminal's screen.

[0071] Further, step S50 includes steps S501 to S505: Step S501: Extract the header data and each data block from the compressed pen stroke data.

[0072] Step S502: Decompress the header data to obtain the decompressed header data.

[0073] In the specific implementation, the header data includes compressed brush style data, compressed color appearance data, and compressed layer data. The display terminal can restore the compressed brush style data (i.e., all data in the target dictionary) to brush style parameters based on a preset brush dictionary; convert the YUV format compressed color appearance data to RGB format to obtain color appearance parameters; and convert the compressed layer data, i.e., the 16-bit layer hash value, into a layer identifier, using the layer identifier to determine the layer to be drawn on. After decompression, the header data includes brush style parameters, color appearance parameters, and the layer identifier of the drawing layer.

[0074] Step S503: Reassemble each data block according to its sequence number to obtain the target dynamic trajectory compressed data.

[0075] In practice, the display terminal can start from the absolute coordinates of the first data point according to the sequence number of each data block, and gradually accumulate the difference between each data block to reassemble each data block, restore the coordinate sequence of the dynamic trajectory parameters, and obtain the target dynamic trajectory compressed data.

[0076] Furthermore, packet loss may occur during transmission. To compensate for the impact of packet loss, in one feasible implementation, step S503 includes: Step S5031: Establish an expected sequence number list based on the sequence number of each data block, and add the sequence number of each received data block to the actual received list; Step S5032: Determine whether there are any lost serial numbers by comparing the expected serial number list with the actual received list.

[0077] In its implementation, the display terminal can establish a continuous list of expected sequence numbers based on the sequence numbers of the first and last received data blocks. For example, if the sequence number of the first data block is 1 and the sequence number of the last data block is 10, then the resulting list of expected sequence numbers would be 1, 2, 3, ..., 10. The display terminal can then add the sequence numbers of each received data block to the actual received list and compare the actual received list with the list of expected sequence numbers. By detecting whether there are sequence numbers present in the expected sequence model but not in the actual received list, the terminal can determine whether the sequence numbers of each data block in the initial dynamic trajectory compressed data have been lost.

[0078] Step S5033: When a lost sequence number exists, determine the preceding and following target data blocks adjacent to the lost sequence number.

[0079] In the specific implementation, when a sequence number exists in the expected sequence model but not in the actual received list, it is determined that the sequence number is lost. This sequence number in the expected sequence model is identified as the lost sequence number in the actual received list, and the preceding and following target data blocks adjacent to this lost sequence number are determined. For example, if the expected sequence model is [1,2,3,4,5] and the actual received list is [1,2,4,5], by comparison, sequence number 3 can be determined to be a lost sequence number. The preceding target data block is the data block corresponding to sequence number 2, and the following target data block is the data block corresponding to sequence number 4.

[0080] Step S5034: Perform curve interpolation between the coordinates of the previous target data block and the coordinates of the next target data block to generate interpolation coordinate points corresponding to the lost sequence number.

[0081] Step S5035: Add the interpolated coordinate points to the positions corresponding to the lost sequence numbers in the initial dynamic trajectory compression data to obtain the target dynamic trajectory compression data.

[0082] In its implementation, the display terminal can generate interpolation coordinate points corresponding to the lost sequence numbers using a Bézier curve algorithm based on the coordinates of the previous and next target data blocks. These interpolation coordinate points are then added to the positions corresponding to the lost sequence numbers in the initial dynamic trajectory compressed data to obtain the target dynamic trajectory compressed data.

[0083] For example, a feasible implementation of generating tweening coordinate points may be as follows: calculate the midpoint of the coordinates of the previous target data block and the coordinates of the next target data block to obtain an intermediate control point; take the coordinates of the previous target data block as the starting point and the coordinates of the next target data block as the ending point; calculate the intermediate transition point based on the mathematical formula of Bézier curves using the starting point, control point and ending point; and use the intermediate transition point as the tweening coordinate point corresponding to the lost sequence number.

[0084] Step S504: Decompress the target dynamic trajectory compressed data to obtain the decompressed dynamic trajectory parameters.

[0085] In the specific implementation, after the display terminal completes the above tweening operation, it can gradually accumulate the difference between each data block according to the sequence number of each data block, starting from the absolute coordinates of the first data point, to reassemble each data block, restore the coordinate sequence of the dynamic trajectory parameters, and obtain the target dynamic trajectory compressed data. This can compensate for the impact of packet loss during data transmission and ensure smooth drawing for multiple people.

[0086] Step S505: The GPU renders the decompressed header data and the decompressed dynamic trajectory parameters to display the stroke trajectory corresponding to the initial stroke data.

[0087] In the specific implementation, the display terminal can select the corresponding vertex shader and fragment shader according to the brush style parameters, set the brush texture, then set the color value according to the color appearance parameters, generate the brush stroke geometry according to the dynamic trajectory parameters, calculate the texture coordinates for each vertex, and ensure that the brush texture is correctly stretched along the brush stroke direction. Afterwards, the data obtained through the above processing is uploaded to the Central Processing Unit (GPU), which executes the drawing and displays the brush stroke trajectory corresponding to the initial brush stroke data.

[0088] This embodiment receives compressed stroke data sent by a mobile terminal and displays the stroke trajectory corresponding to the initial stroke data based on the compressed stroke data. Since the compressed stroke data is obtained by the mobile terminal compressing the initial stroke data using a corresponding compression strategy selected based on the attribute parameters in the initial stroke data, the amount of data received by the display terminal is reduced. This reduces the pressure on network bandwidth and transmission latency when transmitting data to the display terminal, effectively improving the real-time collaboration experience.

[0089] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the pen stroke data processing method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0090] This application also provides a mobile terminal, please refer to... Figure 5 , Figure 5 This is a schematic diagram of the module structure of the mobile terminal of this application.

[0091] Mobile terminals include: User interaction module 10 is used to respond to the user's pen stroke operation and generate corresponding initial pen stroke data containing multiple attribute parameters.

[0092] The data compression module 20 is used to select the corresponding compression strategy according to each attribute data to compress it and generate a compressed pen stroke data package.

[0093] The data sending module 30 is used to send the compressed pen stroke data packet to the display terminal so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0094] The mobile terminal provided in this application, employing the pen touch data processing method for mobile terminals described in the above embodiments, can solve the technical problem in the prior art where a large amount of data transmitted from a mobile terminal to a display terminal leads to data transmission delays and a poor real-time collaboration experience. Compared with the prior art, the beneficial effects of the mobile terminal provided in this application are the same as those of the pen touch data processing method for mobile terminals provided in the above embodiments, and other technical features of the mobile terminal are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0095] This application also provides a display terminal, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the module structure of the display terminal in this application.

[0096] The display terminal includes: The data receiving module 40 is used to receive compressed pen stroke data sent by the mobile terminal.

[0097] The compressed pen stroke data is obtained by the mobile terminal selecting the corresponding compression strategy based on the attribute parameters in the initial pen stroke data. The initial pen stroke data is generated by the mobile terminal in response to the user's pen stroke operation.

[0098] The data rendering module 50 is used to display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

[0099] The display terminal provided in this application, employing the pen touch data processing method applied to the display terminal in the above embodiments, can solve the technical problem in the prior art where a large amount of data transmitted from a mobile terminal to a display terminal leads to data transmission delay, resulting in a poor real-time collaboration experience. Compared with the prior art, the beneficial effects of the display terminal provided in this application are the same as those of the pen touch data processing method applied to the display terminal provided in the above embodiments, and other technical features in the display terminal are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0100] This application provides a pen touch data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the pen touch data processing method in the above embodiments.

[0101] When the pen touch data processing device is a mobile terminal, the pen touch data processing method applied to the mobile terminal in the above embodiments is executed when the instructions stored in its memory are executed by the processor. When the pen touch data processing device is a display terminal, the pen touch data processing method applied to the display terminal in the above embodiments is executed when the instructions stored in its memory are executed by the processor.

[0102] The following is for reference. Figure 7 , Figure 7This is a schematic diagram of the pen touch data processing device of this application. The pen touch data processing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The pen touch data processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0103] like Figure 7 As shown, the pen touch data processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the pen touch data processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the pen touch data processing device to communicate wirelessly or wiredly with other devices to exchange data. Although pen touch data processing devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0104] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0105] The pen touch data processing device provided in this application, employing the pen touch data processing method in the above embodiments, can solve the technical problem in the prior art where a large amount of data transmitted from a mobile terminal to a display terminal leads to data transmission delay, resulting in a poor real-time collaboration experience. Compared with the prior art, the beneficial effects of the pen touch data processing device provided in this application are the same as those of the pen touch data processing method provided in the above embodiments, and other technical features in this pen touch data processing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0106] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0108] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A pen stroke data processing method, characterized in that, The method is applied to a mobile terminal, which is communicatively connected to a display terminal, and the method includes: Responding to the user's pen strokes, it generates initial pen stroke data containing multiple attribute parameters; Based on the attribute parameters described, the corresponding compression strategy is selected for compression to generate a compressed pen touch data package. The compressed pen stroke data packet is sent to the display terminal so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

2. The pen stroke data processing method as described in claim 1, characterized in that, The attribute parameters include dynamic trajectory parameters, brush style parameters, and color appearance parameters. The step of selecting a corresponding compression strategy based on each attribute parameter to compress and generate a compressed brush stroke data package includes: Calculate the coordinate difference between the next data point and the previous data point in the dynamic trajectory parameters to obtain the dynamic trajectory compressed data; Obtain a preset brush dictionary, which includes dictionary indices corresponding to different brush style parameters; Obtain the target dictionary index corresponding to the brush style parameter from the preset brush dictionary, and use the target dictionary index as the compressed brush style data. The color appearance parameters are converted using the YUV color space to obtain compressed color appearance data with a preset color sampling ratio. Perform layer hash mapping on the layer identifier of the current painting layer to obtain compressed layer data; The dynamic trajectory compression data, the brush style compression data, the color appearance compression data, and the layer compression data are integrated to obtain a compressed brush stroke data package.

3. The pen stroke data processing method as described in claim 2, characterized in that, The step of sending the compressed pen stroke data packet to the display terminal so that the display terminal displays the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data includes: Extract the dynamic trajectory compressed data and header data from the compressed brush stroke data package. The header data includes the brush style compressed data, the color appearance compressed data, and the layer compressed data. The header data is sent to the display terminal a preset number of times; The dynamic trajectory compressed data is divided into a preset number of data blocks; Add a corresponding sequence number to each data block according to the time order of the data blocks in the original stroke; Data blocks are selected sequentially according to the order of the serial numbers and sent to the display terminal, so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the received main data and each of the data blocks.

4. The pen stroke data processing method as described in claim 3, characterized in that, The step of selecting data blocks sequentially according to the sequence number and sending them to the display terminal includes: Select the target data block to be sent according to the order of the sequence numbers; The target data block is sent to the display terminal so that the display terminal performs an integrity check on the target data block. Upon receiving a confirmation signal from the display terminal indicating that the verification of the target data block has passed, the next data block is selected as the target data block according to the order of the sequence number, and the process of sending the target data block to the display terminal to enable the display terminal to perform integrity verification on the target data block continues until confirmation signals for each data block are received.

5. A pen stroke data processing method, characterized in that, The method is applied to a display terminal, which is communicatively connected to a mobile terminal, and the method includes: The mobile terminal receives compressed pen touch data, which is obtained by the mobile terminal compressing the initial pen touch data by selecting a corresponding compression strategy based on the attribute parameters in the initial pen touch data. The initial pen touch data is generated by the mobile terminal in response to the user's pen touch operation. The compressed stroke data displays the stroke trajectory corresponding to the initial stroke data.

6. The pen stroke data processing method as described in claim 5, characterized in that, The step of displaying the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data includes: Extract the header data and each data block from the compressed pen stroke data; The header data is decompressed to obtain the decompressed header data; The data blocks are reassembled according to their sequence numbers to obtain compressed data of the target dynamic trajectory. The compressed data of the target dynamic trajectory is decompressed to obtain the decompressed dynamic trajectory parameters; The GPU renders the decompressed header data and the decompressed dynamic trajectory parameters to display the stroke trajectory corresponding to the initial stroke data.

7. The pen stroke data processing method as described in claim 6, characterized in that, The step of reassembling the data blocks according to their sequence numbers to obtain compressed target dynamic trajectory data includes: A list of expected sequence numbers is established based on the sequence number of each data block, and the sequence number of each received data block is added to the actual received list. The presence of any lost serial numbers is determined by comparing the expected serial number list with the actual received list. When the lost sequence number exists, determine the preceding and following target data blocks adjacent to the lost sequence number; The curve difference is calculated between the coordinates of the previous target data block and the coordinates of the next target data block to generate the interpolation coordinate points corresponding to the lost sequence number; The interpolated coordinate points are added to the positions corresponding to the lost sequence numbers in the initial dynamic trajectory compressed data to obtain the target dynamic trajectory compressed data.

8. A mobile terminal, the mobile terminal comprising: The user interaction module is used to respond to the user's pen stroke operations and generate corresponding initial pen stroke data containing multiple attribute parameters. The data compression module is used to select the corresponding compression strategy according to each attribute data to compress it and generate a compressed pen stroke data package; The data transmission module is used to send the compressed pen stroke data packet to the display terminal, so that the display terminal can display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

9. A display terminal, the display terminal comprising: The data receiving module is used to receive compressed pen touch data sent by the mobile terminal. The compressed pen touch data is obtained by the mobile terminal selecting a corresponding compression strategy according to each attribute parameter in the initial pen touch data. The initial pen touch data is generated by the mobile terminal in response to the user's pen touch operation. The data rendering module is used to display the pen stroke trajectory corresponding to the initial pen stroke data based on the compressed pen stroke data.

10. A pen stroke data processing device, characterized in that, The pen stroke data processing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pen stroke data processing method as claimed in any one of claims 1 to 5 or 6 to 8.

Images