Method and device for learning calligraphy

Abstract

The embodiment of the present invention provides a method and device for learning calligraphy. According to the method of the embodiment, firstly, the recognition image of the calligraphy copybook is collected, and the first characteristic information of the calligraphy copybook is obtained according to the recognition image; then, the calligraphy practice track of the user practicing the calligraphy copybook is collected, and the second characteristic information of the calligraphy copybook is obtained according to the calligraphy practice track, and finally, the score of the user practicing the calligraphy copybook is determined according to the similarity between the first characteristic information and the second characteristic information. The scheme provided by the embodiment of the present invention enables the user to obtain timely feedback.

Description

Technical Field

[0001] One or more embodiments of the present invention relate to the field of computer technology, and in particular, to a method and device for learning calligraphy. Background Art

[0002] Copybooks are one of the commonly used tools for practicing calligraphy. The traditional way of practicing calligraphy is to cover the copybook with transparent paper and trace according to the transparent traces of the characters, or just look at the characters on the copybook and write them out on another piece of paper completely based on your feeling.

[0003] The existing calligraphy practice process is relatively boring, does not provide a good calligraphy practice atmosphere for users, and users cannot get timely feedback during the calligraphy practice process, which leads to low calligraphy practice efficiency. Therefore, in view of the above shortcomings, a calligraphy practice learning solution that provides timely feedback is needed. Summary of the invention

[0004] One or more embodiments of the present invention describe a method and device for learning calligraphy, which can enable users to obtain timely feedback.

[0005] According to a first aspect, a method for learning calligraphy is provided, comprising:

[0006] Collect recognition images of copybooks;

[0007] Obtaining first characteristic information of the copybook according to the recognition image;

[0008] Collecting the calligraphy practice track of the user based on the calligraphy copybook;

[0009] According to the calligraphy practice track, second characteristic information of the calligraphy copybook is obtained;

[0010] The score of the user for the copybook practice is determined according to the similarity between the first feature information and the second feature information.

[0011] In one embodiment, obtaining the second characteristic information of the copybook according to the calligraphy practice trajectory includes:

[0012] Inputting the calligraphy practice track into a trained neural network to obtain a target calligraphy practice track output by the trained neural network; wherein the trained neural network is a neural network that propagates from front to back, including an input node and an output node, and the target loss function in the training stage includes a regularization term;

[0013] Obtaining a residual between the calligraphy practice trajectory and the target calligraphy practice trajectory;

[0014] determining abnormal trajectory points in the residual;

[0015] The second characteristic information of the copybook is obtained according to the target calligraphy practice trajectory after removing the abnormal trajectory points.

[0016] In one embodiment, the trained neural network determines at least one of the following based on the characteristics of the calligraphy practice trajectory during the training phase:

[0017] A regularization coefficient of the regularization term;

[0018] Normalize and initialize the weights of the neurons of the neural network;

[0019] The following data for the hidden layers: number of layers and number of neurons in each layer.

[0020] In one embodiment, the target loss function of the trained neural network in the training phase further includes: an original loss function; the original loss function includes: a mean square error function or a mean absolute error function;

[0021] The trained neural network is determined through the following training steps: based on the parameters of the currently trained neural network, the original loss function and the regularization term in the target loss function of this training are updated, the parameters of the currently trained neural network are updated using the gradient descent algorithm and the back propagation algorithm, and the training steps are jumped to be executed until the number of training times reaches the iteration number threshold or the return value of the target loss function is less than the target threshold, thereby obtaining a trained neural network.

[0022] In one embodiment, the step of using a gradient descent algorithm and a back propagation algorithm to determine the parameters of the currently trained neural network includes:

[0023] Based on a preset learning rate, determining a learning rate for adjusting the weights of neurons of a currently trained neural network; wherein the preset learning rate is determined based on a target loss function of this training and a characteristic of the calligraphy practice track;

[0024] Based on the determined learning rate, the parameters of the neural network currently being trained are determined using a gradient descent algorithm and a back propagation algorithm.

[0025] In one embodiment, determining the learning rate for adjusting the weights of neurons of the currently trained neural network based on a preset learning rate includes:

[0026] Based on a preset learning rate and a learning rate adjustment rule, a learning rate for adjusting the weights of neurons of a currently trained neural network is determined.

[0027] In one embodiment, the regularization term is an L1 norm or an L2 norm.

[0028] According to a second aspect, a calligraphy learning device is provided, comprising:

[0029] The first acquisition module is used to acquire the recognition image of the copybook;

[0030] A first feature information obtaining module, used for obtaining first feature information of the copybook according to the recognition image;

[0031] The second collection module is used to collect the calligraphy practice track of the user based on the calligraphy copybook;

[0032] A second feature information obtaining module, used for obtaining the second feature information of the copybook according to the calligraphy practice track;

[0033] The score determination module is used to determine the score of the user for the copybook practice according to the similarity between the first feature information and the second feature information.

[0034] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute any one of the methods according to the first aspect.

[0035] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in any one of the first aspects is implemented.

[0036] According to the method and device provided in the embodiment of the specification, the recognition image of the copybook is first collected, and the first characteristic information of the copybook is obtained according to the recognition image; then the writing trajectory of the user practicing the copybook is collected, and the second characteristic information of the copybook is obtained according to the writing trajectory, and finally the score of the user practicing the copybook is determined according to the similarity between the first characteristic information and the second characteristic information. The above process can score the user's copybook practice after the user has used the copybook for a period of time, and timely feedback the user's practice results, so that the user can get timely feedback, and then the efficiency of writing can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flow chart of a method for learning calligraphy provided by an embodiment of the present invention;

[0039] Figure 2is a schematic diagram of a neural network provided by an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of a device where a calligraphy learning device is provided in one embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of a calligraphy learning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Before describing the solution provided by the present invention, the inventive concept of the solution is first explained.

[0043] Figure 1 The flowchart of the method for learning calligraphy provided by one embodiment of the present invention. The execution subject of the method can be a device with processing capabilities: a server or a terminal device. Figure 1 As shown, the calligraphy learning method may specifically include the following steps:

[0044] Step 101: Collect the recognition image of the copybook.

[0045] In this step, the copybook may include copybook content, and the copybook content may be made into a groove shape, and the user may practice writing along the groove. The handwriting formed by practicing writing may automatically disappear within a certain period of time, so that the copybook can be used repeatedly.

[0046] The copybook content may include Chinese characters, English, numbers, pictures, etc., for example, the stroke order of Chinese characters, poems, phrases, English letters, English words, etc.

[0047] The copybook may also include an identification code corresponding to the copybook content, the terminal device may obtain an identification image, and may obtain the copybook content through the identification image, and the terminal device may also generate audio information matching the identification image based on the received identification image, etc., to achieve interaction between the user and the terminal device. It is understandable that the terminal device may be a voice copybook machine, which includes a speaker for sounding.

[0048] The terminal device may also have wireless communication function, display function, touch screen function, visual perception function, image processing function, intelligent AI function, AR function, etc. For example, it may be a mobile phone, IPAD, intelligent robot, wireless projector, electronic whiteboard, etc. The terminal device may also realize statistics of the user's learning situation by exchanging information with the user, or may realize interaction with the user by receiving the user's voice information.

[0049] In this step, a recognition device may be provided on the terminal device, such as an infrared sensor, a photoelectric recognition module, a camera, etc. The terminal device may obtain a recognition image of the copybook through the recognition device.

[0050] Step 102: Obtain first characteristic information of the copybook based on the recognition image.

[0051] In this step, the copybook content may include Chinese characters, English, numbers, pictures, etc., for example, the stroke order of Chinese characters, poems, words, English letters, English words, etc., so the recognized image may include the contour information (i.e., the first feature information) of Chinese characters, English, numbers, pictures, etc. It can be considered that the first feature information is the information recognized by the recognition device, so the first feature information can be considered as standard feature information to be compared with the second feature information corresponding to the calligraphy practice track collected from the user later.

[0052] Step 103: Collect the calligraphy practice track of the user based on the copybook.

[0053] In this step, the calligraphy practice track can be collected in the following ways:

[0054] Obtaining a pressure signal fed back by a pressure sensor arranged in the terminal device;

[0055] Determine the calligraphy practice trajectory of the user based on the copybook according to the pressure signal;

[0056] or,

[0057] Obtaining the optical signal fed back by the infrared tube arranged in the terminal device;

[0058] Determine the calligraphy practice trajectory of the user based on the copybook according to the light signal;

[0059] or,

[0060] Obtaining a distance signal fed back by a distance sensor arranged in the terminal device;

[0061] The writing trajectory of the user practicing the copybook is determined based on the distance signal.

[0062] In this step, a pressure sensor can be set in the terminal device. The pressure sensor can be set at the bottom of the terminal device. When the child writes in the terminal device with his fingers or a pen, the pressure signal can be detected by the pressure sensor. Finally, the user's calligraphy practice trajectory for the copybook can be determined based on the pressure signal.

[0063] In this step, a light transmitting and receiving device can also be set in the terminal device. The light transmitting device can be set on one side of the terminal device, and the light receiving device is relatively set on the other side of the terminal device opposite to the light transmitting device. The light transmitting device can also be set at the bottom of the terminal device, and the light receiving device is relatively set at a position above the terminal device opposite to the light transmitting device. The position above the terminal device can be any position that does not affect the user's writing in the terminal device and the optical signal can be normally sent and received. When a child writes in the terminal device with a finger or a pen, it will block the light, thereby generating an optical signal. Finally, the user's calligraphy practice trajectory for the copybook can be determined based on the optical signal.

[0064] In this step, a distance sensor can also be set in the terminal device. The distance sensor can be set on one side of the terminal device or on the bottom of the terminal device. When a child writes in the terminal device with his fingers or a pen, the time it takes for the measurement light pulse sent by the distance sensor to be reflected back will be different, thereby generating a distance signal. Finally, the user's calligraphy practice trajectory for the copybook can be determined based on the distance signal.

[0065] Step 104: Obtain the second characteristic information of the copybook according to the calligraphy practice trajectory.

[0066] The specific steps include:

[0067] Step 1041, input the calligraphy practice trajectory into the trained neural network to obtain the target calligraphy practice trajectory output by the trained neural network.

[0068] In this step, combine Figure 2 The process of obtaining the target calligraphy practice trajectory is described. Figure 2 In the example, the trained neural network is a neural network that propagates from front to back, including an input node and an output node (i.e., the input layer of the trained neural network has an input node, and the output layer has an output node), and also includes two hidden layers, each of which includes a number of hidden nodes. In other words, Figure 2 The neural network in is a multilayer perceptron (MLP), which adopts a fully connected structure (that is, any neuron in the previous layer is connected to all neurons in the next layer). In some embodiments, the hidden layer included in the neural network provided in the embodiment of the present invention is not limited to two layers, for example, it can also be one layer, three layers or more layers. It should be noted that when the hidden layer is two or more layers, the hidden layer can better solve the nonlinear problem.

[0069] In one embodiment, the target loss function of the trained neural network in the training phase includes a regularization term, so that the second feature information of the abnormal trajectory points can be obtained, which improves the accuracy of the second feature information. In some embodiments, the regularization term can be an L1 norm or an L2 norm. Among them, the L1 norm refers to the sum of the absolute values ​​of each element in the weight vector. The L1 norm can be used for feature selection, that is, the coefficient of the feature becomes 0; the L2 norm refers to the sum of the squares of each element in the weight vector and then takes the square root. The L2 norm can prevent overfitting and improve the generalization ability of the neural network. Since the trained neural network is a neural network that propagates from front to back and includes an input node and an output node, the target loss function in the training phase includes a regularization term, so the second feature information of the abnormal trajectory points can be obtained, which improves the accuracy of the second feature information.

[0070] In one embodiment, the trained neural network determines at least one of the following based on the characteristics of the calligraphy practice trajectory during the training phase:

[0071] The regularization coefficient of the regularization term;

[0072] Normalize and initialize the weights of the neurons of the neural network;

[0073] The following data for the hidden layers: number of layers and number of neurons in each layer.

[0074] In this embodiment, the regularization coefficient is a value less than 1, and the weight of the target loss function can be constrained by the regularization term. For the L1 norm, to a certain extent, the L1 norm can also prevent overfitting, that is, when the regularization coefficient of the L1 norm is very small, the optimal solution obtained will be very small, which can achieve an effect similar to the L2 norm regularization. For the L2 norm, the larger the regularization coefficient, the more obvious the regularization effect, but it may also mean that the neural network is more difficult to converge, that is, it is easy to cause underfitting.

[0075] In addition, initializing the weights of the neurons of the neural network can avoid the gradient vanishing or gradient exploding as the number of neural network layers increases; and the weights of the neurons should be initialized to avoid zero, which will cause the symmetry problem of the neural network. In order to facilitate the subsequent processing of the weights of the neurons (that is, when the gradient descent algorithm determines the appropriate weights, it is conducive to faster and more accurate convergence), the initialized weights can be normalized, that is, normalized to the range of [0, 1] or [-1, 1].

[0076] In addition, the number of hidden layers and the number of neurons in each hidden layer are correlated with the calligraphy practice trajectory. For example, the more data points of the calligraphy practice trajectory, the more it is necessary to increase the number of hidden layers and the number of neurons in each hidden layer, so as to more accurately represent the calligraphy practice trajectory.

[0077] In one embodiment, the characteristics of the calligraphy practice track include any one of the following: length, mean, variance, and autocorrelation.

[0078] In this embodiment, the length of the calligraphy track refers to the span from the starting time point to the end time point of the calligraphy track. The longer the length, the more data that can characterize the data point, and vice versa. The less data characterizing the data point. In other words, the length of the calligraphy track is related to the number of layers of the hidden layer and the number of neurons in each hidden layer. The mean and variance of the calligraphy track are related to the weight of the neurons in each hidden layer and the regular coefficient in the target loss function. According to the mean and variance of the calligraphy track, the weight of the neurons in each hidden layer within the appropriate value range and the regular coefficient in the target loss function can be determined to reduce the complexity of the neural network, obtain a smaller generalization error, and reduce the degree of overfitting. In addition, autocorrelation is a comparison of the calligraphy track with itself at different times. Autocorrelation can be quantitatively characterized by the autocorrelation coefficient (i.e., Pearson correlation coefficient), and its purpose is to detect the time trend and stability of the calligraphy track. The autocorrelation of the calligraphy practice trajectory is also related to the weights of the neurons in each hidden layer and the regularization coefficient in the target loss function. Similarly, based on the autocorrelation of the calligraphy practice trajectory, the weights of the neurons in each hidden layer and the regularization coefficient in the target loss function can also be determined within an appropriate value range to reduce the complexity of the neural network, obtain a smaller generalization error, and reduce the degree of overfitting.

[0079] In one embodiment, in order to prevent overfitting and improve the generalization performance of the neural network, the target loss function can be set to a combination of the original loss function and the regularization term, that is, the target loss function of the trained neural network in the training stage also includes: the original loss function. The target loss function is used to measure the difference between the predicted value and the input value of each training. The original loss function includes: the mean square error (MSE) function or the mean absolute error (MAE) function, wherein: the mean square error function can evaluate the degree of change of the calligraphy practice trajectory data, the smaller the value of MSE, the better the accuracy of the neural network in describing the calligraphy practice trajectory data; the mean absolute error function can better reflect the actual situation of the prediction value error.

[0080] In addition, the trained neural network is determined through the following training steps: based on the parameters (i.e., weights) of the currently trained neural network, the original loss function and regularization term in the target loss function of this training are updated, the parameters of the currently trained neural network are updated using the gradient descent algorithm and the back propagation algorithm, and the training step is jumped to be executed until the number of training times reaches the iteration number threshold or the return value of the target loss function is less than the target threshold, and the trained neural network is obtained. The weight of the target loss function of the trained neural network obtained has been adjusted to a suitable value. At this time, the target loss function is the smallest, which means that the neural network has better accuracy in describing the calligraphy practice trajectory data.

[0081] In one embodiment, the parameters of the currently trained neural network are determined using a gradient descent algorithm and a back propagation algorithm, including:

[0082] Based on a preset learning rate, determine a learning rate for adjusting the weights of neurons of the currently trained neural network; wherein the preset learning rate is determined based on the target loss function of this training and the characteristics of the calligraphy practice trajectory;

[0083] Based on the determined learning rate, the parameters of the neural network currently being trained are determined using a gradient descent algorithm and a back propagation algorithm.

[0084] In this embodiment, based on the preset learning rate, the learning rate can be fixed to adjust the weight of the neuron, where the learning rate * the gradient of the target loss function = the weight decrement value. However, the method of fixing the learning rate has the following problems: if the learning rate is too small, the convergence of the target loss function will be too slow; if the learning rate is too large, the target loss function will oscillate, and may even fail to converge. When a fixed learning rate is selected to adjust the weight of the neuron, when the target loss function reaches a convergence state, it will swing in a larger area near the optimal value; and when the learning rate is reduced as the number of iterations increases, the target loss function will swing in a smaller area near the optimal value when converging. Therefore, for a fixed learning rate to adjust the weight of the neuron, choosing a suitable learning rate is crucial for the training of the neural network. It should be noted that when the gradient of the target loss function approaches 0, an initialization method or a special value can be used instead.

[0085] In some embodiments, different fixed learning rates can be tried, such as 3, 1, 0.5, 0.1, 0.05, 0.01, 0.005, 0.005, 0.0001, 0.00001, etc., to observe the changing relationship between the number of iterations and the target loss function, and find the learning rate corresponding to the fastest decrease in the target loss function.

[0086] In one embodiment, determining a learning rate for adjusting the weights of neurons of a currently trained neural network based on a preset learning rate includes:

[0087] Based on a preset learning rate and a learning rate adjustment rule, a learning rate for adjusting the weights of neurons of a currently trained neural network is determined.

[0088] In this embodiment, during the optimization of the target loss function, the learning rate should be gradually reduced. The closer to the minimum value of the target loss function, the smaller the learning rate should be. In some embodiments, the learning rate adjustment rules may include: 1) initially setting a larger learning rate, and then as the number of iterations increases, for example, after every n rounds of training, the learning rate is halved; 2) the learning rate decreases exponentially according to the number of training rounds, for example, a n =0.95 n *a n -1, where a n Represents the learning rate at the nth training round, where n represents the number of training rounds.

[0089] Step 1042: Obtain the residual between the calligraphy practice trajectory and the target calligraphy practice trajectory.

[0090] Step 1043: Determine abnormal trajectory points in the residual.

[0091] In this step, the abnormal trajectory point can be determined specifically in the following manner: based on the comparison result between the residual and the detection threshold, the abnormal trajectory point is determined; or

[0092] Based on the abnormal points in the residual that do not conform to the preset distribution, the abnormal trajectory points are determined.

[0093] In this embodiment, by using a detection threshold or a preset distribution (such as T distribution, normal distribution) to determine whether there are abnormal points in the residual, this detection method is simple, so it is helpful to quickly detect abnormal trajectory points. In some embodiments, the N-σ method can be used to determine whether there are abnormal points in the residual that do not conform to the preset distribution.

[0094] Step 1044, obtaining the second characteristic information of the copybook according to the target calligraphy practice trajectory after eliminating abnormal trajectory points.

[0095] In this way, second feature information with higher accuracy can be obtained.

[0096] Step 105: Determine the user's score for the copybook practice based on the similarity between the first feature information and the second feature information.

[0097] In this step, for example, a clustering algorithm may be used to obtain the similarity between the first feature information and the second feature information. If the similarity is higher, the user's score for the copybook practice will be higher, and vice versa.

[0098] In this embodiment, the recognition image of the copybook is first collected, and the first characteristic information of the copybook is obtained according to the recognition image; then the writing trajectory of the user practicing the copybook is collected, and the second characteristic information of the copybook is obtained according to the writing trajectory, and finally the score of the user practicing the copybook is determined according to the similarity between the first characteristic information and the second characteristic information. The above process can score the user's copybook practice after the user has used the copybook for a period of time, and timely feedback the user's practice results, so that the user can get timely feedback, and then the efficiency of writing can be improved.

[0099] like Figure 3 and Figure 4 As shown, the embodiment of the present invention provides a device where a calligraphy learning device is located and a calligraphy learning device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, such as Figure 3 As shown, a hardware structure diagram of the device where the calligraphy learning device provided by the embodiment of the present invention is located, except Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown in the figure, the device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 4 As shown, as a device in a logical sense, the CPU of the device in which it is located reads the corresponding computer program instructions in the non-volatile memory into the internal memory and runs them.

[0100] like Figure 4 As shown, the calligraphy learning device provided in this embodiment includes:

[0101] The first acquisition module 401 is used to acquire the recognition image of the copybook;

[0102] A first feature information obtaining module 402 is used to obtain first feature information of the copybook according to the recognition image;

[0103] The second collection module 403 is used to collect the calligraphy practice track of the user based on the calligraphy copybook;

[0104] The second characteristic information obtaining module 404 is used to obtain the second characteristic information of the copybook according to the calligraphy practice track;

[0105] The score determination module 405 is used to determine the score of the user for the copybook practice according to the similarity between the first feature information and the second feature information.

[0106] In an embodiment of the present invention, the first acquisition module 401 can be used to execute step 101 in the above method embodiment, the first feature information acquisition module 402 can be used to execute step 102 in the above method embodiment, the second acquisition module 403 can be used to execute step 103 in the above method embodiment, the second feature information acquisition module 404 can be used to execute step 104 in the above method embodiment, and the score determination module 405 can be used to execute step 105 in the above method embodiment.

[0107] In one embodiment of the present invention, the second feature information obtaining module 404 is used to perform the following operations:

[0108] Inputting the calligraphy practice trajectory into the trained neural network to obtain the target calligraphy practice trajectory output by the trained neural network; wherein the trained neural network is a neural network that propagates from front to back, including an input node and an output node, and the target loss function in the training stage includes a regularization term;

[0109] Obtain the residual between the practice writing trajectory and the target practice writing trajectory;

[0110] Identify outlier trajectory points in the residuals;

[0111] According to the target calligraphy practice trajectory after eliminating abnormal trajectory points, the second feature information of the copybook is obtained.

[0112] In one embodiment of the present invention, the trained neural network determines at least one of the following based on the characteristics of the calligraphy practice trajectory during the training phase:

[0113] The regularization coefficient of the regularization term;

[0114] Normalize and initialize the weights of the neurons of the neural network;

[0115] The following data for the hidden layers: number of layers and number of neurons in each layer.

[0116] In one embodiment of the present invention, the target loss function of the trained neural network in the training phase also includes: an original loss function; the original loss function includes: a mean square error function or a mean absolute error function;

[0117] The trained neural network is determined through the following training steps: based on the parameters of the currently trained neural network, the original loss function and the regularization term in the target loss function of this training are updated, the parameters of the currently trained neural network are updated using the gradient descent algorithm and the back propagation algorithm, and the training step is jumped to be executed until the number of training times reaches the iteration number threshold or the return value of the target loss function is less than the target threshold, thereby obtaining the trained neural network.

[0118] In one embodiment of the present invention, the parameters of the currently trained neural network are determined using a gradient descent algorithm and a back propagation algorithm, including:

[0119] Based on a preset learning rate, determine a learning rate for adjusting the weights of neurons of the currently trained neural network; wherein the preset learning rate is determined based on the target loss function of this training and the characteristics of the calligraphy practice trajectory;

[0120] Based on the determined learning rate, the parameters of the neural network currently being trained are determined using a gradient descent algorithm and a back propagation algorithm.

[0121] In one embodiment of the present invention, determining a learning rate for adjusting the weights of neurons of a currently trained neural network based on a preset learning rate includes:

[0122] Based on a preset learning rate and a learning rate adjustment rule, a learning rate for adjusting the weights of neurons of a currently trained neural network is determined.

[0123] In one embodiment of the present invention, the regularization term is an L1 norm or an L2 norm.

[0124] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the calligraphy learning device. In other embodiments of the present invention, the calligraphy learning device may include more or fewer components than those shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0125] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0126] The embodiment of the present invention also provides a computer-readable medium storing instructions for causing a computer to execute the method for learning calligraphy as described herein. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code for implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads out and executes the program code stored in the storage medium.

[0127] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0128] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0129] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0130] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0131] The above describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for learning calligraphy, characterized in that: include: Collect recognition images of copybooks; According to the recognition image, first feature information of the copybook is obtained, wherein the first feature information is standard feature information; Collecting the calligraphy practice track of the user based on the calligraphy copybook; According to the calligraphy practice track, second characteristic information of the calligraphy copybook is obtained; Determining a score of the user for the copybook practice according to the similarity between the first feature information and the second feature information; The step of obtaining the second characteristic information of the copybook according to the calligraphy practice track includes: Inputting the calligraphy practice track into a trained neural network to obtain a target calligraphy practice track output by the trained neural network; wherein the trained neural network is a neural network that propagates from front to back, including an input node and an output node, and the target loss function in the training stage includes a regularization term; Obtaining a residual between the calligraphy practice trajectory and the target calligraphy practice trajectory; determining abnormal trajectory points in the residual; According to the target calligraphy practice trajectory after removing the abnormal trajectory points, second feature information of the calligraphy copybook is obtained; The trained neural network determines at least one of the following based on the characteristics of the calligraphy practice trajectory during the training phase: A regularization coefficient of the regularization term; Normalize and initialize the weights of the neurons of the neural network; The following data for the hidden layers: the number of layers and the number of neurons in each layer; The characteristics of the calligraphy practice track include any of the following: length, mean, variance, and autocorrelation; The target loss function of the trained neural network in the training phase also includes: an original loss function; the original loss function includes: a mean square error function or a mean absolute error function; The trained neural network is determined by the following training steps: based on the parameters of the currently trained neural network, the original loss function and the regularization term in the target loss function of this training are updated, the parameters of the currently trained neural network are updated using a gradient descent algorithm and a back propagation algorithm, and the training step is executed until the number of training times reaches an iteration number threshold or the return value of the target loss function is less than a target threshold, thereby obtaining a trained neural network; The method of using a gradient descent algorithm and a back propagation algorithm to determine the parameters of the currently trained neural network includes: Based on a preset learning rate, determining a learning rate for adjusting the weights of neurons of a currently trained neural network; wherein the preset learning rate is determined based on a target loss function of this training and a characteristic of the calligraphy practice track; Based on the determined learning rate, a gradient descent algorithm and a back propagation algorithm are used to determine the parameters of the neural network currently being trained; The step of determining the learning rate for adjusting the weights of neurons of the currently trained neural network based on the preset learning rate includes: Based on a preset learning rate and a learning rate adjustment rule, determining a learning rate for adjusting the weights of neurons of a currently trained neural network; the regularization term is an L1 norm or an L2 norm; The calligraphy practice track is collected in the following manner: Obtaining a pressure signal fed back by a pressure sensor arranged in the terminal device; Determine the calligraphy practice trajectory of the user based on the copybook according to the pressure signal; or, Obtaining the optical signal fed back by the infrared tube arranged in the terminal device; Determine the calligraphy practice trajectory of the user based on the copybook according to the light signal; or, Obtaining a distance signal fed back by a distance sensor arranged in the terminal device; The writing trajectory of the user practicing the copybook is determined based on the distance signal.

2. A calligraphy learning device, characterized in that: include: The first acquisition module is used to acquire the recognition image of the copybook; A first feature information obtaining module, used for obtaining first feature information of the copybook according to the recognition image, wherein the first feature information is standard feature information; The second collection module is used to collect the calligraphy practice track of the user based on the calligraphy copybook; A second feature information obtaining module, used for obtaining the second feature information of the copybook according to the calligraphy practice track; A score determination module, used to determine the score of the user for the copybook practice according to the similarity between the first feature information and the second feature information; The second feature information obtaining module is used to perform the following operations: Inputting the calligraphy practice trajectory into the trained neural network to obtain the target calligraphy practice trajectory output by the trained neural network; wherein the trained neural network is a neural network that propagates from front to back, including an input node and an output node, and the target loss function in the training stage includes a regularization term; Obtain the residual between the practice writing trajectory and the target practice writing trajectory; Identify outlier trajectory points in the residuals; According to the target calligraphy practice trajectory after removing abnormal trajectory points, the second feature information of the calligraphy copybook is obtained; The trained neural network determines at least one of the following based on the characteristics of the calligraphy practice trajectory during the training phase: Regularization coefficient of the regularization term; Normalize and initialize the weights of the neurons of the neural network; The following data for the hidden layers: the number of layers and the number of neurons in each layer; The target loss function of the trained neural network in the training phase also includes: the original loss function; the original loss function includes: the mean square error function or the mean absolute error function; The trained neural network is determined by the following training steps: based on the parameters of the currently trained neural network, the original loss function and the regularization term in the target loss function of this training are updated, the parameters of the currently trained neural network are updated using the gradient descent algorithm and the back propagation algorithm, and the training step is jumped to be executed until the number of training times reaches the iteration number threshold or the return value of the target loss function is less than the target threshold, thereby obtaining the trained neural network; The gradient descent algorithm and back propagation algorithm are used to determine the parameters of the currently trained neural network, including: Based on a preset learning rate, determine a learning rate for adjusting the weights of neurons of the currently trained neural network; wherein the preset learning rate is determined based on the target loss function of this training and the characteristics of the calligraphy practice trajectory; Based on the determined learning rate, a gradient descent algorithm and a back propagation algorithm are used to determine the parameters of the neural network currently being trained; Based on the preset learning rate, determining the learning rate for adjusting the weights of the neurons of the currently trained neural network includes: Based on a preset learning rate and a learning rate adjustment rule, determining a learning rate for adjusting the weights of neurons of a currently trained neural network; The regularization term is L1 norm or L2 norm; The calligraphy practice track is collected in the following manner: Obtaining a pressure signal fed back by a pressure sensor arranged in the terminal device; Determine the calligraphy practice trajectory of the user based on the copybook according to the pressure signal; or, Obtaining the optical signal fed back by the infrared tube arranged in the terminal device; Determine the calligraphy practice trajectory of the user based on the copybook according to the light signal; or, Obtaining a distance signal fed back by a distance sensor arranged in the terminal device; The writing trajectory of the user practicing the copybook is determined based on the distance signal.

3. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method of claim 1.

4. A computing device, characterized in that The invention comprises a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method described in claim 1 is implemented.

Images