Table editing method, apparatus and device for electronic whiteboard, and storage medium

Abstract

The invention provides a table editing method and device for an electronic whiteboard, equipment and a storage medium, and the table editing method for the electronic whiteboard comprises the steps: collecting operation data of a user on a table and a table screenshot under the condition that the electronic whiteboard displays the table, carrying out the feature extraction of the operation data, obtaining a multi-dimensional operation feature vector, and carrying out the feature extraction of the operation feature vector; the operation feature vector is processed through a machine learning algorithm, the operation type is determined, and the accuracy of determining the operation type is improved through multi-dimensional data analysis of the touch pressure value, the contact area, the direction angle, the speed change rate, the touch distance and the like; according to the method, the table screenshot is subjected to image recognition, the position information of the cell segmentation line is determined, the operation range is determined according to the operation data and the position information, the determination accuracy of the operation range is improved, the cells in the operation range are edited according to the operation type, and the table editing experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of electronic whiteboards, and in particular to a table editing method, apparatus, device and storage medium for an electronic whiteboard. Background Art

[0002] In electronic whiteboard applications, table tools are one of the most frequently used tools in daily teaching, office work, and meetings. Table tools typically provide editing functions, such as merging and splitting cells. Currently, traditional table editing methods on electronic whiteboards often rely on clicking function area buttons or menu operations. Users need to select cells and then operate buttons or menus to merge or split cells, which is inconvenient and inefficient. Although there are currently some gesture operation solutions, their recognition and accuracy of selected cell ranges need to be improved, resulting in a poor user experience. Summary of the Invention

[0003] The present application provides a table editing method, apparatus, device, and storage medium for an electronic whiteboard to solve at least one problem existing in the related art. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for editing a table on an electronic whiteboard, comprising:

[0005] When a table is displayed on the electronic whiteboard, collect the user's operation data on the table and the table screenshot;

[0006] Extracting features from the operation data to obtain a multi-dimensional operation feature vector, and processing the operation feature vector using a machine learning algorithm to determine the operation type. The multi-dimensional dimensions include touch pressure value, contact area, direction angle, speed change rate, and touch distance. The operation type includes merging cells or splitting cells.

[0007] Perform image recognition on the table screenshot to determine the position information of the cell dividing line, and determine the operation range based on the operation data and position information;

[0008] Edit the cells within the operation range according to the operation type.

[0009] In one embodiment, extracting features from the operation data to obtain a multi-dimensional operation feature vector includes:

[0010] Determine the coordinates, touch pressure, touch time, and ellipse parameters of each contact point in the operation data;

[0011] Determine the touch distance and direction angle based on the coordinates of the initial contact point and the coordinates of the final contact point;

[0012] Based on the coordinates of each contact point and the touch time, the speed of the contact segments formed by each contact point is calculated. The speed difference between two adjacent contact segments is calculated based on the speed of each contact segment. The speed change rate is obtained based on the ratio of the sum of all speed differences to the number of speed differences.

[0013] According to the touch pressure of each contact point, the average value is calculated to obtain the touch pressure value;

[0014] The contact area size corresponding to each contact point is determined according to the ellipse parameters, and the average value is calculated based on the contact area size to obtain the final contact area. The multi-dimensional operation feature vector is determined based on the touch distance, direction angle, speed change rate, touch pressure value and contact area.

[0015] In one embodiment, determining the contact area corresponding to each contact point according to the ellipse parameters includes:

[0016] Determine the product of the major axis a, minor axis b and π in the ellipse parameters of each contact point to obtain the contact area size corresponding to each contact point; the ellipse parameters can be obtained by reading the MotionEvent API of the electronic whiteboard screen or the SDK of the stylus.

[0017] In one embodiment, processing the operation feature vector using a machine learning algorithm to determine the operation type includes:

[0018] When the touch pressure value is greater than the touch threshold, the operation type is determined to be merging cells;

[0019] When the touch pressure value is less than or equal to the touch threshold, the operation feature vector is processed by the support vector machine (SVM) to obtain the output result;

[0020] If the output result is the first value, the operation type is determined to be merge cells; if the output result is the second value, the operation type is determined to be split cells.

[0021] In one embodiment, the position information includes the horizontal coordinate of the vertical dividing line and the vertical coordinate of the horizontal dividing line of each cell; and determining the operation range according to the operation data and the position information includes:

[0022] Determine the transformation coordinates corresponding to each contact point based on the point coordinates of each contact point and the transformation matrix;

[0023] Determine a minimum enclosing rectangle surrounding each contact point based on the transformed coordinates corresponding to each contact point, where the minimum enclosing rectangle includes a top side, a bottom side, a left side, and a right side;

[0024] According to the minimum enclosing rectangle, the horizontal coordinate of the vertical dividing line of each cell, and the vertical coordinate of the horizontal dividing line, respectively determine a first horizontal dividing line closest to the top and with a larger vertical coordinate than the vertical coordinate of the top, a second horizontal dividing line closest to the bottom and with a smaller vertical coordinate than the vertical coordinate of the bottom, a first vertical dividing line closest to the left and with a smaller horizontal coordinate than the horizontal coordinate of the left, and a second vertical dividing line closest to the right and with a larger horizontal coordinate than the horizontal coordinate of the right;

[0025] An operating range is determined according to an area enclosed by the first horizontal dividing line, the second horizontal dividing line, the first vertical dividing line, and the second vertical dividing line.

[0026] In one embodiment, the transformation matrix is ​​determined by the following steps:

[0027] Get the corner point coordinate vectors of the four corner points of the table in the table screenshot, and set the target values ​​of the corner point coordinate vectors of the four corner points to (0,0)(1,0)(1,1)(0,1);

[0028] Taking the target values ​​of the corner point coordinate vectors of the four corner points as the target, the original transformation matrix is ​​optimized according to the product of the corner point coordinate vectors of the four corner points and the original transformation matrix to obtain the transformation matrix.

[0029] In one embodiment, editing cells within the operation range according to the operation type includes:

[0030] When the operation type is Merge Cells, the cells in the operation range will be merged. If the original formats of the cells in the operation range are the same, the merged cells will retain the original formats. If the cells in the operation range have different original formats, the original format with the largest number will be selected as the target format of the merged cells.

[0031] When the operation type is split cells, the cells in the operation range are split based on the default splitting rule, or a split option is displayed for the user to select a target split option, and the cells in the operation range are split based on the target split option.

[0032] In a second aspect, an embodiment of the present application provides a table editing device for an electronic whiteboard, comprising:

[0033] The collection module is used to collect the user's operation data on the form and form screenshots when the electronic whiteboard displays the form;

[0034] a processing module for extracting features from the operation data to obtain a multi-dimensional operation feature vector, and processing the operation feature vector using a machine learning algorithm to determine the operation type, where the multi-dimensional dimensions include touch pressure value, contact area, direction angle, speed change rate, and touch distance, and the operation type includes merging cells or splitting cells;

[0035] The determination module is used to perform image recognition on the table screenshot, determine the position information of the cell dividing line, and determine the operation range based on the operation data and the position information;

[0036] The editing module is used to edit cells within the operation range according to the operation type.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method in any one of the above-mentioned embodiments.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, it implements the method in any one of the above-mentioned embodiments.

[0039] The beneficial effects of the above technical solution include at least:

[0040] By collecting the user's operation data and table screenshots on the electronic whiteboard when displaying a table, feature extraction is performed on the operation data to obtain a multi-dimensional operation feature vector, and the operation feature vector is processed by a machine learning algorithm to determine the operation type. The accuracy of determining the operation type is improved through multi-dimensional data analysis such as touch pressure value, contact area, direction angle, speed change rate and touch distance; image recognition is performed on the table screenshot to determine the position information of the cell dividing line, and the operation range is determined based on the operation data and position information, thereby improving the accuracy of determining the operation range. According to the operation type, the cells within the operation range are edited to improve the user's table editing experience.

[0041] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0043] Figure 1 This is a flowchart of the steps of a table editing method for an electronic whiteboard according to an embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of an operation range determination process according to an embodiment of the present application;

[0045] Figure 3 This is a structural block diagram of a table editing device for an electronic whiteboard according to an embodiment of the present application;

[0046] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0048] Reference Figure 1 , showing a flow chart of a table editing method for an electronic whiteboard according to an embodiment of the present application, the table editing method for an electronic whiteboard may include at least steps S100-S400:

[0049] S100 : When a table is displayed on the electronic whiteboard, collect user operation data on the table and a screenshot of the table.

[0050] S200 , extracting features from the operation data to obtain a multi-dimensional operation feature vector, and processing the operation feature vector through a machine learning algorithm to determine the operation type.

[0051] Optionally, the multiple dimensions include touch pressure value, contact area, direction angle, speed change rate and touch distance, and the operation type includes merging cells or splitting cells.

[0052] S300: Perform image recognition on the table screenshot to determine the position information of the cell dividing line, and determine the operation range based on the operation data and the position information.

[0053] S400. Edit cells within the operation range according to the operation type.

[0054] The table editing method of the electronic whiteboard in the embodiment of the present application can be executed by the processor of the electronic whiteboard.

[0055] The technical solution of the embodiment of the present application is to collect the user's operation data on the table and the table screenshot when the table is displayed on an electronic whiteboard, perform feature extraction on the operation data to obtain a multi-dimensional operation feature vector, and process the operation feature vector through a machine learning algorithm to determine the operation type, and improve the accuracy of determining the operation type through multi-dimensional data analysis such as touch pressure value, contact area, direction angle, speed change rate and touch distance; perform image recognition on the table screenshot to determine the position information of the cell dividing line, and determine the operation range based on the operation data and position information, thereby improving the accuracy of determining the operation range, and edit the cells within the operation range according to the operation type to improve the user's table editing experience.

[0056] In one embodiment, when a table is displayed on an electronic whiteboard, a user can edit the table by performing gestures or operating a stylus on the electronic whiteboard screen. The electronic whiteboard screen or stylus can then capture the user's table operation data, and the processor can capture a screenshot of the table currently displayed on the screen. The operation data includes, but is not limited to, each contact point, its coordinates, touch pressure, touch time, and ellipse parameters.

[0057] In one embodiment, step S200 performs feature extraction on the operation data to obtain a multi-dimensional operation feature vector, including steps S210-S250:

[0058] S210: Determine the coordinates, touch pressure, touch time, and ellipse parameters of each contact point in the operation data.

[0059] It should be noted that the ellipse parameters include the major axis a and the minor axis b (unit: pixel). The ellipse parameters can be obtained by reading the MotionEvent API of the electronic whiteboard screen (for example, Android MotionEvent.getTouchMajor() or getTouchMinor()), or through the stylus SDK. For example, if the stylus is Wacom, query WacomProAPI.getEllipseAxes() to obtain it.

[0060] S220 : Determine the touch distance and direction angle according to the coordinates of the initial contact point and the coordinates of the final contact point.

[0061] For example, the coordinates of the initial contact point are (x_start, y_start), and the coordinates of the final contact point are (x_end, y_end). The Euclidean distance is calculated based on these two coordinates to obtain the touch distance L.

[0062] For example, the direction angle θ is determined based on the formula:

[0063] θ=arctan[(y_end-y_start) / (x_end-x_start)]

[0064] S230. Calculate the speed of contact segments formed by each contact point based on the coordinates of each contact point and the touch time. Calculate the speed difference between two adjacent contact segments based on the speed of each contact segment. Calculate the speed change rate based on the ratio of the sum of all speed differences to the number of speed differences.

[0065] For example, the distance between each pair of contact points is calculated based on their coordinates, yielding the length of the contact segment formed by each pair of contact points. The speed of each contact segment is then determined based on this length and the time difference between each pair of contact points. The speed difference between each pair of adjacent contact segments is then calculated based on the speed of each contact segment. Finally, the speed change rate Δv is calculated based on the ratio of the sum of all speed differences to the number of speed differences. For example, if there are 10 speed differences, the sum of all 10 speed differences is divided by 10 to obtain the speed change rate Δv.

[0066] S240 : Calculate an average value according to the touch pressure at each contact point to obtain a touch pressure value.

[0067] For example, the touch pressure at each contact point is summed, and the sum is divided by the number of contact points to obtain an average value, which is the touch pressure value P.

[0068] S250. Determine the contact area size corresponding to each contact point based on the ellipse parameters, calculate the average value based on the contact area size to obtain the final contact area, and determine the multi-dimensional operation feature vector based on the touch distance, direction angle, speed change rate, touch pressure value and contact area.

[0069] In one embodiment, the contact area corresponding to each contact point is determined based on the ellipse parameters. Specifically, the product of the major axis a and the minor axis b in the ellipse parameters of each contact point and π is determined to obtain the contact area corresponding to each contact point S = a × b × π. Then, all the contact areas S are summed and divided by the number of contact points to calculate the average value to obtain the final contact area A. Therefore, the final multi-dimensional operational feature vector F = [θΔv LPA] T , where T is the transpose.

[0070] In one embodiment, the operation feature vector is processed by a machine learning algorithm in step S200 to determine the operation type, including steps S260-S280:

[0071] S260: When the touch pressure value is greater than the touch threshold, determine that the operation type is merging cells.

[0072] In one embodiment, a touch threshold may be set. When the touch pressure value is greater than the touch threshold, the operation type is determined to be cell merging, and the cells are forcibly merged, so that the user can press the screen hard to merge the cells.

[0073] S270 : When the touch pressure value is less than or equal to the touch threshold, the operation feature vector is processed by a support vector machine (SVM) to obtain an output result.

[0074] Optionally, when the touch pressure value is less than or equal to the touch threshold, the operation feature vector is processed by a support vector machine (SVM) to obtain the output result ActionType:

[0075] ActionType=sign(w T F+c)

[0076] Wherein, c is a bias term, and w is a weight vector; wherein, the weight vector w is obtained through training, for example, collecting a large amount of historical data in advance, and determining the weight vector w based on the touch pressure value, contact area, direction angle, speed change rate, and touch distance in the historical data.

[0077] S280: If the output result is the first value, determine the operation type as merging cells; if the output result is the second value, determine the operation type as splitting cells.

[0078] Optionally, if the output result ActionType is a first value 1, the operation type is determined to be merging cells; if the output result ActionType is a second value -1, the operation type is determined to be splitting cells.

[0079] In one embodiment, in step S300, the table screenshot can be image recognized by an image recognition neural network to determine the position information of the cell dividing lines. The position information can include the horizontal coordinate of the vertical dividing line and the vertical coordinate of the horizontal dividing line of each cell.

[0080] In one embodiment, in step S300, determining the operating range based on the operating data and the location information includes steps S310-S340:

[0081] S310 : Determine the conversion coordinates corresponding to each contact point according to the point coordinates of each contact point and the transformation matrix.

[0082] In one embodiment, the transformation matrix T z Determine this by following these steps:

[0083] First, get the corner point coordinate vectors (two-dimensional vectors) of the four corner points of the table in the table screenshot, and set the target values ​​of the corner point coordinate vectors of the four corner points to (0,0)(1,0)(1,1)(0,1). This is equivalent to using the lower left corner point as the origin of the coordinate axis (0,0), and setting the coordinates of the remaining corner points with the total length 1 and total width 1 of the table.

[0084] Next, using the target values ​​of the corner point coordinate vectors of the four corner points as the target, the original transformation matrix is ​​optimized based on the product of the corner point coordinate vectors of the four corner points and the original transformation matrix to obtain the transformation matrix. For example, taking the corner point in the lower left corner as an example, the product of the corner point coordinate vector in the lower left corner and the original transformation matrix is ​​determined to obtain the converted coordinates. The Euclidean distance between this coordinate and (0,0) is then calculated. The same process is repeated for the other corner points. The sum of all Euclidean distances is used as the error value. The original transformation matrix is ​​continuously optimized until the error value is within the allowable error range. The original transformation matrix at this time serves as the transformation matrix.

[0085] Therefore, after the transformation matrix is ​​determined, the conversion coordinates corresponding to each contact point can be determined according to the product of the point coordinates of each contact point and the transformation matrix.

[0086] S320 . Determine a minimum enclosing rectangle surrounding each contact point based on the transformed coordinates corresponding to each contact point. The minimum enclosing rectangle includes a top side, a bottom side, a left side, and a right side.

[0087] Optionally, the minimum bounding rectangle MBR surrounding each contact point is determined based on the transformed coordinates corresponding to each contact point. Figure 2 As shown, the minimum enclosing rectangle includes the top, bottom, left and right sides.

[0088] S330. According to the minimum enclosing rectangle, the horizontal coordinate of the vertical dividing line of each cell, and the vertical coordinate of the horizontal dividing line, determine respectively the first horizontal dividing line that is closest to the top and whose vertical coordinate is greater than the vertical coordinate of the top, the second horizontal dividing line that is closest to the bottom and whose vertical coordinate is smaller than the vertical coordinate of the bottom, the first vertical dividing line that is closest to the left and whose horizontal coordinate is smaller than the horizontal coordinate of the left, and the second vertical dividing line that is closest to the right and whose horizontal coordinate is greater than the horizontal coordinate of the right.

[0089] For example, Figure 2As shown, with upward as the positive direction of the Y axis and rightward as the positive direction of the X axis, according to the minimum enclosing rectangle, the horizontal coordinate of the vertical dividing line of each cell and the vertical coordinate of the horizontal dividing line, the first horizontal dividing line (i.e. 21) closest to the top and with a vertical coordinate greater than the vertical coordinate of the top, the second horizontal dividing line (i.e. 22) closest to the bottom and with a vertical coordinate less than the vertical coordinate of the bottom, the first vertical dividing line (i.e. 23) closest to the left and with a horizontal coordinate less than the horizontal coordinate of the left, and the second vertical dividing line (i.e. 24) closest to the right and with a horizontal coordinate greater than the horizontal coordinate of the right are determined respectively.

[0090] S340: Determine an operation range according to an area enclosed by the first horizontal dividing line, the second horizontal dividing line, the first vertical dividing line, and the second vertical dividing line.

[0091] Specifically, if Figure 2 As shown, at this time, the final operation range, ie, the cells surrounded by the four dividing lines, can be determined based on the first horizontal dividing line 21, the second horizontal dividing line 22, the first vertical dividing line 23 and the second vertical dividing line 24.

[0092] In one embodiment, step S400 includes steps S410-S420:

[0093] S410. When the operation type is merge cells, merge the cells in the operation range. If the original formats of the cells in the operation range are the same, the merged cells retain the original formats. If the cells in the operation range have different original formats, the original format with the largest number is selected as the target format of the merged cells.

[0094] Optionally, when the operation type is merge cells, the cells in the operation range are merged. When the original formats of the cells in the operation range are the same, the merged cells retain the original formats. On the contrary, if the cells in the operation range have different original formats, for example, one or more cells may have different formats from other cells, then the target format of the merged cells is the original format with the largest number in the operation range, that is, the most common original format, thereby realizing intelligent format retention or determination, and can realize intelligent alignment to improve neatness.

[0095] S420: When the operation type is to split cells, the cells within the operation range are split based on the default splitting rule, or a splitting option is displayed for the user to select a target splitting option, and the cells within the operation range are split based on the target splitting option.

[0096] Optionally, when the operation type is to split cells, the cells within the operation range are split based on the default splitting rules. For example, the user can set the default splitting rules, such as splitting into two or three, or based on the cells adjacent to the operation range, so that the split cells are aligned with the cells adjacent to the operation range. Alternatively, the splitting options are displayed, such as how many parts to split into, the format after splitting, etc. After the user selects the target splitting option, the cells within the operation range are split based on the target splitting option, which is more user-friendly and improves the flexibility of table editing.

[0097] Reference Figure 3 , shows a structural block diagram of a table editing device for an electronic whiteboard according to an embodiment of the present application, the device may include:

[0098] The collection module is used to collect the user's operation data on the form and form screenshots when the electronic whiteboard displays the form;

[0099] a processing module for extracting features from the operation data to obtain a multi-dimensional operation feature vector, and processing the operation feature vector using a machine learning algorithm to determine the operation type, where the multi-dimensional dimensions include touch pressure value, contact area, direction angle, speed change rate, and touch distance, and the operation type includes merging cells or splitting cells;

[0100] The determination module is used to perform image recognition on the table screenshot, determine the position information of the cell dividing line, and determine the operation range based on the operation data and the position information;

[0101] The editing module is used to edit cells within the operation range according to the operation type.

[0102] The functions of each module in the device of the embodiment of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0103] Reference Figure 4 , shows a block diagram of an electronic device according to an embodiment of the present application. The electronic device includes a memory 310 and a processor 320. The memory 310 stores instructions executable on the processor 320. The processor 320 loads and executes the instructions to implement the table editing method for the electronic whiteboard according to the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0104] In one embodiment, the electronic device further includes a communication interface 330 for communicating with external devices and performing data exchange transmission. If the memory 310, the processor 320, and the communication interface 330 are implemented independently, the memory 310, the processor 320, and the communication interface 330 can be interconnected through a bus and complete communication with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0105] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0106] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the table editing method of the electronic whiteboard provided in the above embodiment is implemented.

[0107] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

[0108] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory. The input interface, the output interface, the processor and the memory are connected through an internal connection path. The processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0109] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the advanced reduced instruction set machine (ARM) architecture.

[0110] Furthermore, optionally, the above-mentioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory may include random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0111] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0112] In the description of this specification, the reference terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0113] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, features specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0114] Any process or method description in a flow chart or otherwise described herein can be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations in which the functions may be performed in a different order than shown or discussed, including in a substantially simultaneous manner or in a reverse order depending on the functions involved.

[0115] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device).

[0116] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0117] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the aforementioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0118] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A table editing method for an electronic whiteboard, characterized in that: include: When a table is displayed on the electronic whiteboard, collect the user's operation data on the table and the table screenshot; Extracting features from the operation data to obtain a multi-dimensional operation feature vector, and processing the operation feature vector using a machine learning algorithm to determine the operation type. The multi-dimensional dimensions include touch pressure value, contact area, direction angle, speed change rate, and touch distance. The operation type includes merging cells or splitting cells. Perform image recognition on the table screenshot to determine the position information of the cell dividing line, and determine the operation range based on the operation data and position information; Edit the cells within the operation range according to the operation type.

2. The table editing method of the electronic whiteboard according to claim 1, characterized in that: The feature extraction of the operation data to obtain a multi-dimensional operation feature vector includes: Determine the coordinates, touch pressure, touch time, and ellipse parameters of each contact point in the operation data; Determine the touch distance and direction angle based on the coordinates of the initial contact point and the coordinates of the final contact point; Based on the coordinates of each contact point and the touch time, the speed of the contact segments formed by each contact point is calculated. The speed difference between two adjacent contact segments is calculated based on the speed of each contact segment. The speed change rate is obtained based on the ratio of the sum of all speed differences to the number of speed differences. According to the touch pressure of each contact point, the average value is calculated to obtain the touch pressure value; The contact area size corresponding to each contact point is determined according to the ellipse parameters, and the average value is calculated based on the contact area size to obtain the final contact area. The multi-dimensional operation feature vector is determined based on the touch distance, direction angle, speed change rate, touch pressure value and contact area.

3. The table editing method of the electronic whiteboard according to claim 2, characterized in that: Determining the contact area corresponding to each contact point according to the ellipse parameters includes: Determine the product of the major axis a, minor axis b and π in the ellipse parameters of each contact point to obtain the contact area size corresponding to each contact point; the ellipse parameters can be obtained by reading the MotionEvent API of the electronic whiteboard screen or the SDK of the stylus.

4. The table editing method of the electronic whiteboard according to claim 2, characterized in that: Processing the operation feature vector using a machine learning algorithm to determine the operation type includes: When the touch pressure value is greater than the touch threshold, the operation type is determined to be merging cells; When the touch pressure value is less than or equal to the touch threshold, the operation feature vector is processed by the support vector machine (SVM) to obtain the output result; If the output result is the first value, the operation type is determined to be merge cells; if the output result is the second value, the operation type is determined to be split cells.

5. The table editing method of the electronic whiteboard according to any one of claims 2 to 4, characterized in that: The position information includes the horizontal coordinate of the vertical dividing line and the vertical coordinate of the horizontal dividing line of each cell; Determining the operation range according to the operation data and the location information includes: Determine the transformation coordinates corresponding to each contact point based on the point coordinates of each contact point and the transformation matrix; Determine a minimum enclosing rectangle surrounding each contact point based on the transformed coordinates corresponding to each contact point, where the minimum enclosing rectangle includes a top side, a bottom side, a left side, and a right side; According to the minimum enclosing rectangle, the horizontal coordinate of the vertical dividing line of each cell, and the vertical coordinate of the horizontal dividing line, respectively determine a first horizontal dividing line closest to the top and with a larger vertical coordinate than the vertical coordinate of the top, a second horizontal dividing line closest to the bottom and with a smaller vertical coordinate than the vertical coordinate of the bottom, a first vertical dividing line closest to the left and with a smaller horizontal coordinate than the horizontal coordinate of the left, and a second vertical dividing line closest to the right and with a larger horizontal coordinate than the horizontal coordinate of the right; An operating range is determined according to an area enclosed by the first horizontal dividing line, the second horizontal dividing line, the first vertical dividing line, and the second vertical dividing line.

6. The table editing method of the electronic whiteboard according to claim 5, characterized in that: The transformation matrix is ​​determined by the following steps: Get the corner point coordinate vectors of the four corner points of the table in the table screenshot, and set the target values ​​of the corner point coordinate vectors of the four corner points to (0,0)(1,0)(1,1)(0,1); Taking the target values ​​of the corner point coordinate vectors of the four corner points as the target, the original transformation matrix is ​​optimized according to the product of the corner point coordinate vectors of the four corner points and the original transformation matrix to obtain the transformation matrix.

7. The table editing method of the electronic whiteboard according to claim 1, characterized in that: The editing of cells within the operation range according to the operation type includes: When the operation type is Merge Cells, the cells in the operation range will be merged. If the original formats of the cells in the operation range are the same, the merged cells will retain the original formats. If the cells in the operation range have different original formats, the original format with the largest number will be selected as the target format of the merged cells. When the operation type is split cells, the cells in the operation range are split based on the default splitting rule, or a split option is displayed for the user to select a target split option, and the cells in the operation range are split based on the target split option.

8. A table editing device for an electronic whiteboard, characterized in that: include: The collection module is used to collect the user's operation data on the form and form screenshots when the electronic whiteboard displays the form; a processing module for extracting features from the operation data to obtain a multi-dimensional operation feature vector, and processing the operation feature vector using a machine learning algorithm to determine the operation type, where the multi-dimensional dimensions include touch pressure value, contact area, direction angle, speed change rate, and touch distance, and the operation type includes merging cells or splitting cells; The determination module is used to perform image recognition on the table screenshot, determine the position information of the cell dividing line, and determine the operation range based on the operation data and the position information; The editing module is used to edit cells within the operation range according to the operation type.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images