Writing feedback method, system and equipment of electronic whiteboard and storage medium

Abstract

The invention provides a writing feedback method, system and device for an electronic whiteboard and a storage medium, and the method comprises the steps: collecting original handwriting coordinate data generated by a touch screen when a user writes on the touch screen of the electronic whiteboard, and obtaining corresponding stroke parameters; dynamically selecting a currently applicable target sampling processing strategy based on the stroke parameter; based on a target sampling processing strategy, sampling the original handwriting coordinate data to obtain target handwriting coordinate data; calling the target kernel event queue to transmit the target handwriting coordinate data to a target event processing thread; and calling the target event processing thread to render the target handwriting coordinate data so as to display the handwriting written by the user on the touch screen. According to the application, real-time writing response of the electronic whiteboard can be realized, writing feedback delay of the electronic whiteboard can be reduced, and writing experience of a user can be improved.

Description

Technical Field

[0001] The present application relates to the field of electronic whiteboard writing technology, and in particular to a writing feedback method, system, device and storage medium for an electronic whiteboard. Background Art

[0002] When it comes to handwriting input on electronic whiteboards, the core metric that users are most sensitive to and most impacts the user experience is the time difference between the pen movement and the appearance of the handwriting on the screen, known as the writing feedback delay. A large writing feedback delay can result in unsmooth handwriting display or even noticeable lag, causing users to experience a sense of "stuck hand," "drifting," or "stuttering," severely disrupting the smoothness and naturalness of writing and degrading the user's writing experience.

[0003] Therefore, it is necessary to provide a solution that can reduce the writing feedback delay of the electronic whiteboard to improve the user's writing experience. Summary of the Invention

[0004] The present application provides a method, system, device, and storage medium for providing writing feedback on an electronic whiteboard to address the problems existing in related technologies. The technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present application provides a writing feedback method for an electronic whiteboard, comprising:

[0006] When a user writes on the touch screen of the electronic whiteboard, the original handwriting coordinate data generated by the touch screen is collected and the corresponding stroke parameters are obtained;

[0007] Based on the brush stroke parameters, dynamically select a currently applicable target sampling processing strategy;

[0008] Based on the target sampling and processing strategy, sampling and processing are performed on the original handwriting coordinate data to obtain target handwriting coordinate data;

[0009] Calling a target kernel event queue to pass the target handwriting coordinate data to a target event processing thread, wherein the target kernel event queue is a high-priority kernel event queue, and the target event processing thread is a high-priority thread dedicated to processing handwriting input;

[0010] The target event processing thread is called to render the target handwriting coordinate data to display the handwriting written by the user on the touch screen.

[0011] In one embodiment, the stroke parameters include: writing speed and writing acceleration; based on the stroke parameters, dynamically selecting the currently applicable target sampling processing strategy includes:

[0012] The writing speed in the stroke parameters is compared with a preset speed threshold, and the writing acceleration is compared with a preset acceleration threshold, and then a currently applicable target sampling processing strategy is dynamically selected according to the comparison results.

[0013] In one embodiment, dynamically selecting a currently applicable target sampling processing strategy based on the comparison result includes:

[0014] When the comparison result shows that the writing speed in the stroke parameters is greater than or equal to the preset speed threshold, and the writing acceleration is less than the preset acceleration threshold, selecting a first sampling processing strategy as the target sampling processing strategy, wherein the first sampling processing strategy is to use a linear interpolation method to perform sampling processing on the collected handwriting coordinate data;

[0015] When the comparison result shows that the writing speed in the stroke parameters is less than the preset speed threshold, and the writing acceleration is less than the preset acceleration threshold, selecting a second sampling processing strategy as the target sampling processing strategy, wherein the second sampling processing strategy is to use a cubic spline interpolation method to sample and process the collected handwriting coordinate data;

[0016] When the comparison result includes that the writing acceleration in the stroke parameter is greater than or equal to the preset acceleration threshold, the third sampling processing strategy is selected as the target sampling processing strategy, and the third sampling processing strategy is to disable the linear interpolation method and the cubic spline difference method to perform sampling processing on the collected handwriting coordinate data.

[0017] In one embodiment, based on the target sampling and processing strategy, sampling and processing the original handwriting coordinate data to obtain target handwriting coordinate data includes:

[0018] When the target sampling processing strategy is the first sampling processing strategy, two adjacent handwriting coordinates P in the original handwriting coordinate data are i-1 and handwriting coordinates P i Insert the handwriting coordinates in proportion to the time to obtain the target handwriting coordinate data, where i is an integer greater than or equal to 2;

[0019] When the target sampling processing strategy is the second sampling processing strategy, the three adjacent handwriting coordinates P in the original handwriting coordinate data are j-2 , handwriting coordinates P j-1 and handwriting coordinates P j Fit into a smooth curve, and then at the handwriting coordinate P j-1 and handwriting coordinates P j Uniformly interpolating the handwriting coordinates located on the smooth curve between the two to obtain the target handwriting coordinate data, wherein j is an integer greater than or equal to 3;

[0020] When the target sampling and processing strategy is the third sampling and processing strategy, the original handwriting coordinate data is used as the target handwriting coordinate data.

[0021] In one embodiment, calling the target kernel event queue to pass the target handwriting coordinate data to the target event processing thread includes:

[0022] Packing the target handwriting coordinate data into a writing input event package;

[0023] Putting the writing input event packet into the target kernel event queue;

[0024] Calling the target inter-process communication channel will extract the writing input event package from the target kernel event queue and pass it to the target event processing thread. The target inter-process communication channel is used to extract the event package from the target kernel event queue and pass it to the target event processing thread.

[0025] In one embodiment, calling the target event processing thread to render the target handwriting coordinate data to display the handwriting written by the user on the touch screen includes:

[0026] Calling the target event processing thread to use target computing resources to render the target handwriting data to obtain rendered handwriting data, where the target computing resources are part of the system computing resources reserved by the electronic whiteboard for writing applications;

[0027] The handwriting of the user is displayed on the touch screen directly according to the handwriting data by adopting a rendering output priority submission method.

[0028] In one embodiment, directly displaying the handwriting of the user on the touch screen according to the handwriting data in a rendering output priority submission manner includes:

[0029] The handwriting data is directly pushed to the touch screen using the immediate mode of the graphics API, and the touch screen displays the handwriting of the user according to the handwriting data.

[0030] In a second aspect, an embodiment of the present application further provides a writing feedback system for an electronic whiteboard, comprising:

[0031] The acquisition unit is used to acquire original handwriting coordinate data generated by the touch screen of the electronic whiteboard when the user writes on the touch screen, and obtain corresponding stroke parameters;

[0032] A selection unit, configured to dynamically select a currently applicable target sampling processing strategy based on the brush stroke parameters;

[0033] a processing unit, configured to perform sampling processing on the original handwriting coordinate data based on the target sampling processing strategy to obtain target handwriting coordinate data;

[0034] a transfer unit, configured to call a target kernel event queue to transfer the target handwriting coordinate data to a target event processing thread, wherein the target kernel event queue is a kernel event queue of high priority, and the target event processing thread is a high-priority thread dedicated to processing handwriting input;

[0035] The display unit is used to call the target event processing thread to render the target handwriting coordinate data so as to display the handwriting written by the user on the touch screen.

[0036] In one embodiment, the stroke parameters include: writing speed and writing acceleration; when the selection unit is used to dynamically select the currently applicable target sampling processing strategy based on the stroke parameters, it is specifically used to:

[0037] The writing speed in the stroke parameters is compared with a preset speed threshold, and the writing acceleration is compared with a preset acceleration threshold, and then a currently applicable target sampling processing strategy is dynamically selected according to the comparison results.

[0038] In one embodiment, when the selection unit is used to dynamically select a currently applicable target sampling processing strategy based on the comparison result, it is specifically used to:

[0039] When the comparison result shows that the writing speed in the stroke parameters is greater than or equal to the preset speed threshold, and the writing acceleration is less than the preset acceleration threshold, selecting a first sampling processing strategy as the target sampling processing strategy, wherein the first sampling processing strategy is to use a linear interpolation method to perform sampling processing on the collected handwriting coordinate data;

[0040] When the comparison result shows that the writing speed in the stroke parameters is less than the preset speed threshold, and the writing acceleration is less than the preset acceleration threshold, selecting a second sampling processing strategy as the target sampling processing strategy, wherein the second sampling processing strategy is to use a cubic spline interpolation method to sample and process the collected handwriting coordinate data;

[0041] When the comparison result includes that the writing acceleration in the stroke parameter is greater than or equal to the preset acceleration threshold, the third sampling processing strategy is selected as the target sampling processing strategy, and the third sampling processing strategy is to disable the linear interpolation method and the cubic spline difference method to perform sampling processing on the collected handwriting coordinate data.

[0042] In one embodiment, when the processing unit is used to perform sampling processing on the original handwriting coordinate data based on the target sampling processing strategy to obtain the target handwriting coordinate data, it is specifically used to:

[0043] When the target sampling processing strategy is the first sampling processing strategy, two adjacent handwriting coordinates P in the original handwriting coordinate data are i-1 and handwriting coordinates P i Insert the handwriting coordinates in proportion to the time to obtain the target handwriting coordinate data, where i is an integer greater than or equal to 2;

[0044] When the target sampling processing strategy is the second sampling processing strategy, the three adjacent handwriting coordinates P in the original handwriting coordinate data are j-2 , handwriting coordinates P j-1 and handwriting coordinates P j Fit into a smooth curve, and then at the handwriting coordinate P j-1 and handwriting coordinates P j Uniformly interpolating the handwriting coordinates located on the smooth curve between the two to obtain the target handwriting coordinate data, wherein j is an integer greater than or equal to 3;

[0045] When the target sampling and processing strategy is the third sampling and processing strategy, the original handwriting coordinate data is used as the target handwriting coordinate data.

[0046] In one embodiment, when the transfer unit is used to call the target kernel event queue to transfer the target handwriting coordinate data to the target event processing thread, it is specifically used to:

[0047] Packing the target handwriting coordinate data into a writing input event package;

[0048] Putting the writing input event packet into the target kernel event queue;

[0049] Calling the target inter-process communication channel will extract the writing input event package from the target kernel event queue and pass it to the target event processing thread. The target inter-process communication channel is used to extract the event package from the target kernel event queue and pass it to the target event processing thread.

[0050] In one embodiment, when the display unit is used to call the target event processing thread to render the target handwriting coordinate data to display the handwriting written by the user on the touch screen, it is specifically used to:

[0051] Calling the target event processing thread to use target computing resources to render the target handwriting data to obtain rendered handwriting data, where the target computing resources are part of the system computing resources reserved by the electronic whiteboard for writing applications;

[0052] The handwriting of the user is displayed on the touch screen directly according to the handwriting data by adopting a rendering output priority submission method.

[0053] In one embodiment, when the display unit is used to directly display the handwriting of the user on the touch screen according to the handwriting data in a rendering output priority submission manner, it is specifically used to:

[0054] The handwriting data is directly pushed to the touch screen using the immediate mode of the graphics API, and the touch screen displays the handwriting of the user according to the handwriting data.

[0055] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned embodiments, wherein the memory and the processor communicate with each other through an internal connection path.

[0056] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method in any one of the above-mentioned embodiments is implemented.

[0057] The advantages or beneficial effects of the above technical solution include at least:

[0058] This application can achieve real-time writing response of the electronic whiteboard and reduce the writing feedback delay of the electronic whiteboard by increasing the sampling frequency of the touch screen of the electronic whiteboard, optimizing the event response stack of the electronic whiteboard to writing input, and reducing the system response time of the electronic whiteboard to handwriting rendering when the user writes on the touch screen of the electronic whiteboard, so as to ensure that the user will not feel "out of hand tracking", "drifting" or "stuttering", thereby ensuring the smoothness and naturalness of writing on the electronic whiteboard and improving the user's writing experience.

[0059] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0061] Figure 1 This is a flowchart illustrating a method for providing feedback on writing on an electronic whiteboard provided in an embodiment of the present application;

[0062] Figure 2 A structural block diagram of a writing feedback system for an electronic whiteboard provided in an embodiment of the present application;

[0063] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0065] Figure 1 FIG. 1 is a flow chart showing a method for providing writing feedback on an electronic whiteboard according to an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0066] S110 . When a user writes on the touch screen of the electronic whiteboard, original handwriting coordinate data generated by the touch screen is collected, and corresponding stroke parameters are obtained.

[0067] In one embodiment, when it is detected that a user has opened a writing application (e.g., a handwriting application, a drawing application, etc.) on the electronic whiteboard, a monitoring touch event (e.g., a touchmove event) may be created. When the user writes on the touch screen of the electronic whiteboard, the monitoring touch event may be called to collect the original handwriting coordinate data generated by the touch screen.

[0068] As an example, the touch event monitoring function may be called to collect the original handwriting coordinate data generated by the touch screen based on the sampling frequency of the touch screen.

[0069] In one embodiment, improvements can be made at the hardware level to increase the sampling frequency of the touch screen of the electronic whiteboard. For example, a touch sensor or electromagnetic induction sensor with a high sampling frequency can be used on the touch screen.

[0070] Current electronic whiteboards generally use touch screens with a sampling frequency of 100Hz or 120Hz. Therefore, in the above embodiment, a touch sensor or electromagnetic induction sensor with a sampling frequency of 240Hz or 480Hz can be used on the touch screen of the electronic whiteboard, thereby increasing the sampling frequency of the touch screen of the electronic whiteboard.

[0071] In one embodiment, the stroke parameters may include but are not limited to: writing speed and writing acceleration.

[0072] As an example, the touch event monitor can collect the timestamp corresponding to the raw handwriting coordinate data in addition to the raw handwriting coordinate data. In this case, the writing speed and writing acceleration can be quickly calculated based on the raw handwriting coordinate data and its corresponding timestamp to obtain the stroke parameters. It can be understood that writing speed is the distance the pen or finger moves per unit time, and writing acceleration is the change in writing speed per unit time.

[0073] S120 : Dynamically select a currently applicable target sampling processing strategy based on the stroke parameters.

[0074] In one embodiment, the writing speed in the stroke parameters may be compared with a preset speed threshold, and the writing acceleration may be compared with a preset acceleration threshold, and then a currently applicable target sampling processing strategy may be dynamically selected based on the comparison results.

[0075] For example, when the comparison result shows that the writing speed in the stroke parameters is greater than or equal to the preset speed threshold and the writing acceleration is less than the preset acceleration threshold, the first sampling processing strategy is selected as the target sampling processing strategy. The first sampling processing strategy is to use a linear interpolation method to sample and process the collected handwriting coordinate data.

[0076] When the comparison result shows that the writing speed in the stroke parameters is less than the preset speed threshold and the writing acceleration is less than the preset acceleration threshold, the second sampling processing strategy is selected as the target sampling processing strategy. The second sampling processing strategy is to use a cubic spline interpolation method to sample and process the collected handwriting coordinate data.

[0077] When the comparison result includes a writing acceleration in the stroke parameters being greater than or equal to the preset acceleration threshold, the third sampling and processing strategy is selected as the target sampling and processing strategy. The third sampling and processing strategy disables the use of linear interpolation and cubic spline interpolation methods for sampling the collected handwriting coordinate data. In other words, the third sampling and processing strategy does not sample the collected handwriting coordinate data.

[0078] It can be understood that, when the writing acceleration in the stroke parameters is less than the preset acceleration threshold, the currently applicable target sampling processing strategy is dynamically selected according to the writing speed in the stroke parameters.

[0079] In specific implementation, the preset speed threshold and the preset acceleration threshold can be set according to actual needs. For example, the preset speed threshold can be set to 0.3m / s, and the preset acceleration threshold can be set to 5m / s. 2 , the embodiments of this application do not limit this.

[0080] In the embodiment of the present application, by executing step S120, the sampling and processing strategy can be dynamically switched according to the stroke parameters, thereby supporting matching the optimal sampling and processing strategy for the real-time writing situation.

[0081] S130 : Based on the target sampling and processing strategy, sample and process the original handwriting coordinate data to obtain target handwriting coordinate data.

[0082] In one embodiment, when the target sampling processing strategy is the first sampling processing strategy, two adjacent handwriting coordinates P in the original handwriting coordinate data are i-1 and handwriting coordinates P i The handwriting coordinates are inserted in proportion to the time to obtain the target handwriting coordinate data, where i is an integer greater than or equal to 2.

[0083] As an example, two adjacent handwriting coordinates P in the original handwriting coordinate data can be obtained according to actual needs. i-1 and handwriting coordinates P i Insert a handwriting coordinate in proportion to the time. i-1 is (x i-1 ,y i-1 ), the corresponding timestamp t i-1 , handwriting coordinates P i is (x i ,y i ), the corresponding timestamp t i , then the inserted handwriting coordinates are (x, y), and the corresponding timestamp is t. Where x, y, and t can be determined by the following formula (1).

[0084] In the embodiment of the present application, by adopting the first sampling processing strategy to sample and process the original handwriting coordinate data, it is possible to obtain target handwriting coordinate data with a 2x upscaling frequency while reducing the amount of real-time calculations. For example, taking the sampling frequency of the touch screen of an electronic whiteboard as 100Hz, the above-described sampling processing can achieve a 200Hz acquisition output, that is, the sampling frequency corresponding to the original handwriting coordinate data is 100Hz, and the sampling frequency corresponding to the target handwriting coordinate data is 200Hz.

[0085] In one embodiment, when the target sampling processing strategy is the second sampling processing strategy, the three adjacent handwriting coordinates P in the original handwriting coordinate data are j-2 , handwriting coordinates P j-1 and handwriting coordinates P j Fit into a smooth curve, and then at the handwriting coordinate P j-1 and handwriting coordinates P j The handwriting coordinates located on the smooth curve are uniformly inserted between them to obtain the target handwriting coordinate data, where j is an integer greater than or equal to 3.

[0086] As an example, the handwriting coordinate P can be set according to actual needs. j-1 and handwriting coordinates P j The two handwriting coordinates located on the smooth curve are uniformly inserted between the timestamps of j-1 The corresponding timestamp t j-1 , handwriting coordinates P j The corresponding timestamp t j The timestamps corresponding to the two inserted handwriting coordinates are t1 and t2 respectively. Where t1 and t2 can be determined by the following formula (2).

[0087]

[0088] In the embodiment of the present application, by adopting the second sampling processing strategy to sample and process the original handwriting coordinate data, the generated handwriting path can be made smoother and more natural while obtaining the target handwriting coordinate data with a 3x upsampling frequency. For example, taking the sampling frequency of the touch screen of an electronic whiteboard as 100Hz, the sampling processing of the above example can achieve a 300Hz acquisition output, that is, the sampling frequency corresponding to the original handwriting coordinate data is 100Hz, and the sampling frequency corresponding to the target handwriting coordinate data is 300Hz.

[0089] In one embodiment, when the target sampling processing strategy is the third sampling processing strategy, the original handwriting coordinate data is used as the target handwriting coordinate data, so as to avoid the "broken line feeling" or ghosting at the sharp turning points of the handwriting.

[0090] In this embodiment of the present application, by executing step S130, the sampling frequency of the electronic whiteboard's touch screen can be increased at the data processing level, without having to improve the electronic whiteboard's touch screen hardware. This lowers the cost of increasing the sampling frequency. Furthermore, the output target handwriting coordinate data can seamlessly replace the original handwriting coordinate data, eliminating the need for application-layer adaptation of the electronic whiteboard and ensuring smooth compatibility.

[0091] Of course, in specific implementation, the sampling frequency can be increased simultaneously at the hardware level and the data processing level, so that the sampling frequency of the touch screen of the electronic whiteboard can be greatly increased.

[0092] In combination with the above steps S120 and S130, dynamic switching of sampling processing strategies can be achieved to balance writing feedback delay and accuracy, adapt to the real-time situation of user writing, help achieve real-time writing response, and reduce writing feedback delay.

[0093] S140 , calling the target kernel event queue to pass the target handwriting coordinate data to the target event processing thread.

[0094] In one embodiment, the target kernel event queue is a high-priority kernel event queue. The target event processing thread is a high-priority thread dedicated to processing writing input.

[0095] As an example, for an electronic whiteboard, the interrupt request (IRQ) generated by the kernel driver (Kernel Driver) of the touch input can be set to have a higher priority, so that a kernel event queue (such as URIS_Kqueue) with a high priority can be created, that is, the target kernel event queue.

[0096] As an example, when a writing application is opened, it can be called to create a high-priority thread dedicated to handling writing input events, namely the target event processing thread. This ensures that when the target event processing thread is called, it can preempt most other non-critical threads on the CPU (such as other background threads of the electronic whiteboard's operating system). For example, when the target event processing thread is running, low-priority threads such as system message push and cloud synchronization are temporarily suspended.

[0097] In one embodiment, the implementation process of step S140 may include the following steps:

[0098] S141. Pack the target handwriting coordinate data into a writing input event package.

[0099] As an example, the touch control chip (IC) of the electronic whiteboard may be called to package the target handwriting coordinate data into a writing input event package.

[0100] S142: Put the writing input event packet into the target kernel event queue.

[0101] As an example, after receiving the writing input event packet, the kernel driver may immediately put it into the target kernel event queue.

[0102] In the embodiment of the present application, by executing step S142, the transmission priority of the target handwriting coordinate data can be increased. For example, ordinary touch sliding events may be queued in the default queue, while writing input events are directly entered into the target kernel event queue through the kernel driver, so that they can be transmitted with priority.

[0103] S143. Call the target inter-process communication channel to extract the writing input event packet from the target kernel event queue and pass it to the target event processing thread.

[0104] As an example, you can register the writing application as a URIS client in advance, and then establish an efficient target inter-process communication channel between the writing application and the target kernel event queue, such as an eBPF map or shared memory mapping. In this way, the target inter-process communication channel can be used to extract event packets from the target kernel event queue (such as URIS_Kqueue) and pass them to the target event processing thread of the writing application. In this way, you can avoid using the standard and inefficient system call path. For example, you can directly poll or receive signal notifications through the shared memory mapping to pull new event packet data from the shared memory area, which is 1-2 orders of magnitude faster than system calls.

[0105] In the embodiment of the present application, by executing step S143, the distribution and delivery of writing input events can be optimized and the delivery delay of target handwriting coordinate data can be reduced.

[0106] In the embodiment of the present application, by executing step S140, the event response stack of the writing feedback of the electronic whiteboard can be optimized, which helps to achieve real-time writing response, thereby reducing the writing feedback delay of the electronic whiteboard.

[0107] S150: Calling a target event processing thread to render the target handwriting coordinate data to display the handwriting written by the user on the touch screen.

[0108] In one embodiment, the implementation process of step S150 may include the following steps:

[0109] S151. Call the target event processing thread to use the target computing resources to render the target handwriting data to obtain rendered handwriting data.

[0110] As an example, the target computing power resources are the portion of system computing power resources reserved by the electronic whiteboard for the writing application. For example, when the writing application is detected to be running, the electronic whiteboard's application manager can be called to dynamically reserve some system computing power resources, such as CPU resources and GPU resources (e.g., CPU core time slices and GPU computing power), for the writing application.

[0111] As an example, after receiving the writing input event package, the target event processing thread parses and extracts the target handwriting coordinate data, and then uses the target calculation resources to render the target handwriting data to obtain the rendered writing handwriting data.

[0112] In the embodiment of the present application, by executing step S151, it can be ensured that the target handwriting coordinate data can be rendered and processed in a timely manner, and the system response time of the electronic whiteboard can be reduced.

[0113] S152: Display the handwriting of the user on the touch screen directly based on the handwriting data using a rendering output priority submission method.

[0114] As an example, the immediate mode of the graphics API may be used to push the handwriting data directly to the touch screen, and the touch screen may display the handwriting of the user based on the handwriting data.

[0115] Exemplarily, the graphics API may be, but is not limited to, one of OpenGL ES, Vulkan, Metal, and DirectX.

[0116] In the embodiment of the present application, by executing step S152, the display of handwriting does not need to wait for the touch screen to complete the refresh cycle, which can reduce the display delay of handwriting and reduce the sense of delay in writing feedback.

[0117] In the embodiment of the present application, by executing step S150, the system response time of the electronic whiteboard writing feedback can be reduced, and the sense of delay in writing feedback can be reduced.

[0118] From the above description, it can be seen that the writing feedback method of the electronic whiteboard provided in the embodiment of the present application can achieve real-time writing response of the electronic whiteboard and reduce the writing feedback delay of the electronic whiteboard by increasing the sampling frequency of the touch screen of the electronic whiteboard, optimizing the event response stack of the electronic whiteboard to writing input, and reducing the system response time of the electronic whiteboard to handwriting rendering when the user writes on the touch screen of the electronic whiteboard, so as to ensure the smoothness and naturalness of the writing on the electronic whiteboard and enhance the user's writing experience.

[0119] Figure 2 FIG. 1 shows a structural block diagram of a writing feedback system of an electronic whiteboard according to an embodiment of the present application. Figure 2 As shown, the system may include:

[0120] The collection unit 210 is used to collect the original handwriting coordinate data generated by the touch screen when the user writes on the touch screen of the electronic whiteboard, and obtain the corresponding stroke parameters;

[0121] A selection unit 220 is used to dynamically select a currently applicable target sampling processing strategy based on the stroke parameters;

[0122] The processing unit 230 is used to perform sampling processing on the original handwriting coordinate data based on the target sampling processing strategy to obtain target handwriting coordinate data;

[0123] The transfer unit 240 is used to call the target kernel event queue to transfer the target handwriting coordinate data to the target event processing thread. The target kernel event queue is a high-priority kernel event queue, and the target event processing thread is a high-priority thread dedicated to processing handwriting input.

[0124] The display unit 250 is used to call the target event processing thread to render the target handwriting coordinate data so as to display the handwriting written by the user on the touch screen.

[0125] In one embodiment, the stroke parameters include writing speed and writing acceleration. When the selection unit 220 is used to dynamically select the currently applicable target sampling processing strategy based on the stroke parameters, it is specifically used to:

[0126] The writing speed in the stroke parameters is compared with the preset speed threshold, and the writing acceleration is compared with the preset acceleration threshold, and then the currently applicable target sampling processing strategy is dynamically selected according to the comparison results.

[0127] In one embodiment, when the selection unit 220 is used to dynamically select the currently applicable target sampling processing strategy based on the comparison result, it is specifically used to:

[0128] When the comparison result shows that the writing speed in the stroke parameters is greater than or equal to the preset speed threshold and the writing acceleration is less than the preset acceleration threshold, the first sampling processing strategy is selected as the target sampling processing strategy, and the first sampling processing strategy is to use a linear interpolation method to sample and process the collected handwriting coordinate data;

[0129] When the comparison result shows that the writing speed in the stroke parameters is less than the preset speed threshold and the writing acceleration is less than the preset acceleration threshold, the second sampling processing strategy is selected as the target sampling processing strategy, and the second sampling processing strategy is to use the cubic spline interpolation method to sample and process the collected handwriting coordinate data;

[0130] When the comparison result includes that the writing acceleration in the stroke parameters is greater than or equal to the preset acceleration threshold, the third sampling processing strategy is selected as the target sampling processing strategy, and the third sampling processing strategy is to disable the linear interpolation method and the cubic spline difference method to sample and process the collected handwriting coordinate data.

[0131] In one embodiment, when the processing unit 230 is used to perform sampling processing on the original handwriting coordinate data based on the target sampling processing strategy to obtain the target handwriting coordinate data, it is specifically used to:

[0132] When the target sampling processing strategy is the first sampling processing strategy, two adjacent handwriting coordinates P in the original handwriting coordinate data are i-1 and handwriting coordinates P i The handwriting coordinates are inserted in proportion to the time to obtain the target handwriting coordinate data, where i is an integer greater than or equal to 2;

[0133] When the target sampling processing strategy is the second sampling processing strategy, the three adjacent handwriting coordinates P in the original handwriting coordinate data are used. j-2 , handwriting coordinates P j-1 and handwriting coordinates P j Fit into a smooth curve, and then at the handwriting coordinate P j-1 and handwriting coordinates P j The handwriting coordinates located on the smooth curve are uniformly interpolated between them to obtain the target handwriting coordinate data, where j is an integer greater than or equal to 3;

[0134] When the target sampling processing strategy is the third sampling processing strategy, the original handwriting coordinate data is used as the target handwriting coordinate data.

[0135] In one embodiment, when the transfer unit 240 is used to call the target kernel event queue to transfer the target handwriting coordinate data to the target event processing thread, it is specifically used to:

[0136] Packing the target handwriting coordinate data into a writing input event packet;

[0137] Put the writing input event packet into the target kernel event queue;

[0138] Calling the target inter-process communication channel will extract the writing input event package from the target kernel event queue and pass it to the target event processing thread. The target inter-process communication is generally used to extract the event package from the target kernel event queue and pass it to the target event processing thread.

[0139] In one embodiment, when the display unit 250 is used to call the target event processing thread to render the target handwriting coordinate data so as to display the handwriting written by the user on the touch screen, it is specifically used to:

[0140] Call the target event processing thread to use the target computing power resources to render the target handwriting data to obtain the rendered handwriting data. The target computing power resources are part of the system computing power resources reserved by the electronic whiteboard for the writing application.

[0141] The user's handwriting is displayed on the touch screen directly based on the handwriting data using a rendering output priority submission method.

[0142] In one embodiment, when the display unit 250 is used to directly display the handwriting of the user on the touch screen according to the handwriting data in a rendering output priority submission manner, it is specifically used to:

[0143] The immediate mode of the graphics API is used to push the handwriting data directly to the touch screen, and the touch screen displays the user's handwriting based on the handwriting data.

[0144] The functions of each unit in the writing feedback system of the electronic whiteboard in the embodiment of the present application can be referred to the corresponding description in the above method, and will not be repeated here.

[0145] Figure 3 FIG. 1 shows a structural block diagram of an electronic device according to an embodiment of the present application. Figure 3 As shown, the electronic device includes: a memory 310 and a processor 320. The memory 310 stores instructions, which are loaded and executed by the processor 320 to implement the electronic whiteboard writing feedback method in the above embodiment. The number of memory 310 and processor 320 can be one or more. The electronic device can be an electronic whiteboard or other device that is communicatively connected to the electronic whiteboard, which is not limited in this embodiment of the application.

[0146] Optionally, the electronic device further includes:

[0147] The communication interface 330 is used to communicate with external devices and perform data exchange transmission.

[0148] If the memory 310, processor 320, and communication interface 330 are implemented independently, the memory 310, processor 320, and communication interface 330 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0149] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0150] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method provided in the embodiment of the present application is implemented.

[0151] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

[0152] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory. The input interface, the output interface, the processor and the memory are connected through an internal connection path. The processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0153] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0154] Furthermore, optionally, the above-mentioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory may include random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0155] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0156] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0157] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0158] Any process or method description in a flow chart or otherwise described herein can be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations in which the functions may be performed in a different order than shown or discussed, including in a substantially simultaneous manner or in a reverse order depending on the functions involved.

[0159] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device).

[0160] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0161] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the aforementioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0162] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A writing feedback method for an electronic whiteboard, characterized in that: include: When a user writes on the touch screen of the electronic whiteboard, the original handwriting coordinate data generated by the touch screen is collected and the corresponding stroke parameters are obtained; Based on the brush stroke parameters, dynamically select a currently applicable target sampling processing strategy; Based on the target sampling and processing strategy, sampling and processing are performed on the original handwriting coordinate data to obtain target handwriting coordinate data; Calling a target kernel event queue to pass the target handwriting coordinate data to a target event processing thread, wherein the target kernel event queue is a high-priority kernel event queue, and the target event processing thread is a high-priority thread dedicated to processing handwriting input; The target event processing thread is called to render the target handwriting coordinate data to display the handwriting written by the user on the touch screen.

2. The method according to claim 1, characterized in that The stroke parameters include: writing speed and writing acceleration; based on the stroke parameters, dynamically selecting the currently applicable target sampling processing strategy includes: The writing speed in the stroke parameters is compared with a preset speed threshold, and the writing acceleration is compared with a preset acceleration threshold, and then a currently applicable target sampling processing strategy is dynamically selected according to the comparison results.

3. The method according to claim 2, characterized in that Dynamically select the currently applicable target sampling processing strategy based on the comparison results, including: When the comparison result shows that the writing speed in the stroke parameters is greater than or equal to the preset speed threshold, and the writing acceleration is less than the preset acceleration threshold, selecting a first sampling processing strategy as the target sampling processing strategy, wherein the first sampling processing strategy is to use a linear interpolation method to perform sampling processing on the collected handwriting coordinate data; When the comparison result shows that the writing speed in the stroke parameters is less than the preset speed threshold, and the writing acceleration is less than the preset acceleration threshold, selecting a second sampling processing strategy as the target sampling processing strategy, wherein the second sampling processing strategy is to use a cubic spline interpolation method to sample and process the collected handwriting coordinate data; When the comparison result includes that the writing acceleration in the stroke parameter is greater than or equal to the preset acceleration threshold, the third sampling processing strategy is selected as the target sampling processing strategy, and the third sampling processing strategy is to disable the linear interpolation method and the cubic spline difference method to perform sampling processing on the collected handwriting coordinate data.

4. The method according to claim 3, characterized in that Based on the target sampling and processing strategy, the original handwriting coordinate data is sampled and processed to obtain target handwriting coordinate data, including: When the target sampling processing strategy is the first sampling processing strategy, two adjacent handwriting coordinates P in the original handwriting coordinate data are i-1 and handwriting coordinates P i Insert the handwriting coordinates in proportion to the time to obtain the target handwriting coordinate data, where i is an integer greater than or equal to 2; When the target sampling processing strategy is the second sampling processing strategy, the three adjacent handwriting coordinates P in the original handwriting coordinate data are used. j-2 , handwriting coordinates P j-1 and handwriting coordinates P j Fit into a smooth curve, and then at the handwriting coordinate P j-1 and handwriting coordinates P j Uniformly interpolating the handwriting coordinates located on the smooth curve between the two to obtain the target handwriting coordinate data, wherein j is an integer greater than or equal to 3; When the target sampling and processing strategy is the third sampling and processing strategy, the original handwriting coordinate data is used as the target handwriting coordinate data.

5. The method according to claim 1, wherein Calling the target kernel event queue to pass the target handwriting coordinate data to the target event processing thread includes: Packing the target handwriting coordinate data into a writing input event package; Putting the writing input event packet into the target kernel event queue; Calling the target inter-process communication channel will extract the writing input event package from the target kernel event queue and pass it to the target event processing thread. The target inter-process communication channel is used to extract the event package from the target kernel event queue and pass it to the target event processing thread.

6. The method according to any one of claims 1 to 5, characterized in that Calling the target event processing thread to render the target handwriting coordinate data to display the handwriting written by the user on the touch screen includes: Calling the target event processing thread to use target computing resources to render the target handwriting data to obtain rendered handwriting data, where the target computing resources are part of the system computing resources reserved by the electronic whiteboard for writing applications; The handwriting of the user is displayed on the touch screen directly according to the handwriting data by adopting a rendering output priority submission method.

7. The method according to claim 6, characterized in that The method of directly displaying the handwriting of the user on the touch screen according to the handwriting data by using the rendering output priority submission method includes: The handwriting data is directly pushed to the touch screen using the immediate mode of the graphics API, and the touch screen displays the handwriting of the user according to the handwriting data.

8. A writing feedback system for an electronic whiteboard, characterized in that: include: The acquisition unit is used to acquire original handwriting coordinate data generated by the touch screen of the electronic whiteboard when the user writes on the touch screen, and obtain corresponding stroke parameters; A selection unit, configured to dynamically select a currently applicable target sampling processing strategy based on the brush stroke parameters; a processing unit, configured to perform sampling processing on the original handwriting coordinate data based on the target sampling processing strategy to obtain target handwriting coordinate data; a transfer unit, configured to call a target kernel event queue to transfer the target handwriting coordinate data to a target event processing thread, wherein the target kernel event queue is a kernel event queue of high priority, and the target event processing thread is a high-priority thread dedicated to processing handwriting input; The display unit is used to call the target event processing thread to render the target handwriting coordinate data so as to display the handwriting written by the user on the touch screen.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the method according to any one of claims 1 to 7 is implemented.

Images