Method for recognizing components of a writing device and the content of the writing.

Abstract

Embodiments of the present invention provide components for a writing device and a method for recognizing written content. [Solution] The component includes a writing device and a writing medium, the writing device includes a pressure sensing unit, an optical recognition unit and a processor, the writing medium has code points printed on it, the pressure sensing unit generates a pressure sensing signal when it detects that the writing device has come into contact with the writing medium, the optical recognition unit collects a code point image of the code points on the writing medium when the pressure sensing unit generates a pressure sensing signal, analyzes the code point image to obtain a code point value and transmits the code point value to the processor, the processor recognizes the code point type from the code point value, and if it recognizes that the code point type is a coordinate code, it analyzes the code point value to determine the writing trajectory and performs content recognition on the writing trajectory to obtain the written content.

Description

Technical Field

[0001] This application relates to the field of handwriting devices, and more particularly to components of a writing device and a method for recognizing written content.

Background Art

[0002] With the rapid development of educational informatization, smart education support devices have gradually become important tools for improving learning efficiency and quality.

[0003] A handwriting device is a type of input device for smart education support devices, enabling a user to interact with an electronic device (such as a computer, tablet, mobile phone, etc.) by writing or drawing on a writing medium.

[0004] Currently, handwriting devices only have a handwriting function and a single function. Therefore, how to improve the functionality of handwriting devices has become an urgent technical problem to be solved.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The main objective of the embodiments of this application is to provide components of a writing device and a method for recognizing written content, which can immediately recognize and broadcast the written content when the user writes using the handwriting device, immediately provide feedback on learning results, enhance the interactivity of learning, and also have a speech playback function, thereby enhancing the functionality and practicality of the writing device.

Means for Solving the Problems

[0006] To achieve the above objective, a first aspect of this application provides components of a writing device, which include a writing device and a writing medium. The writing device includes a pressure sensing unit, an optical recognition unit, and a processor. Code points are printed on the writing medium. The pressure sensing unit is used to generate a pressure sensing signal when it detects that the writing device has come into contact with the writing medium. The optical recognition unit is used to collect code point images of the code points on the writing medium when the pressure sensing unit generates a pressure sensing signal, analyze the code point images to obtain code point values, and transmit the code point values ​​to the processor. The processor recognizes the code point type from the code point value, and if it recognizes the code point type as a coordinate code, it performs coordinate code analysis on the code point value, determines the writing trajectory based on the analysis results, and performs content recognition on the writing trajectory to obtain the written content.

[0007] In some embodiments, the writing device further includes an audio playback unit. The audio playback unit is used to broadcast the written content after the processor has recognized it.

[0008] In some embodiments, the processor further, It is used to recognize the meaning of written content and to obtain the meaning of the written text. If the written content indicates that it is question data, the processor further: Used to generate response data corresponding to question data, The audio playback unit is further used to broadcast the corresponding audio response after the processor has generated the response data.

[0009] In some embodiments, the audio playback unit further includes: When a question broadcast command is received, it is used to broadcast the question audio corresponding to the question broadcast command. The processor further, It is used to compare written content with the corresponding answers in the question audio, obtain the answer matching result, and determine the matching result audio based on the answer matching result. The audio playback unit further includes: After the processor determines the matching result audio, it is used to broadcast the matching result audio.

[0010] In some embodiments, the components of the writing device further include a display module for displaying the written content after the processor has recognized the written content.

[0011] In some embodiments, the processor, upon recognizing the code point type of the code point value, is used to determine the content to be spoken based on the code point value if it is determined that the code point type is a speaking code. The audio playback unit is further used to broadcast the spoken content after the processor has determined what to speak.

[0012] In some embodiments, the processor further, Analyze the writing angle based on the image sequence of code point images to obtain writing angle data. Based on writing angle data and writing content, the system will determine the writing posture in real time, and If the writing posture is found to be incorrect as a result of the posture detection, it is used to generate a voice message correcting the writing posture. The audio playback unit is further used to broadcast the writing posture correction audio after the processor has generated it.

[0013] In some embodiments, the pressure sensing unit includes a writing member and a sensing member. The writing member includes a first end that contacts the writing medium during writing and a second end opposite to the first end, and the second end of the writing member is provided with a conductive silicone rubber pad and a first magnetic element. The sensing member has a sensing end positioned opposite the second end of the writing member, and the sensing end is provided with a second magnetic element and an unconnected first wire and second wire, and the first magnetic element and the second magnetic element are arranged to repel each other magnetically. If the first end of the writing element is not in contact with the writing medium, the conductive silicone rubber pad and the sensing end are separated by the repulsive force between the first and second magnetic elements, the first and second wires are not electrically connected, and no pressure sensing signal is generated. When the first end of the writing member contacts the writing medium, and the contact pressure between the writing member and the writing medium is greater than the repulsive force between the first and second magnetic elements, the conductive silicone rubber pad contacts the sensing end, the first and second wires become electrically connected, a pressure sensing signal is generated, or... The writing element includes a first end that contacts the writing medium during writing and a second end opposite to the first end, and a slider is provided at the second end of the writing element. The sensing member includes a pressure sensor positioned opposite the second end of the writing member. If the first end of the writing element is not in contact with the writing medium, the slider slides downward due to gravity, the slider and pressure sensor separate, and the pressure sensor does not generate a pressure sensing signal. When the first end of the writing element contacts the writing medium, and the contact pressure between the writing element and the writing medium is greater than the gravity of the writing element, the slider moves upward and contacts the pressure sensor, which generates a pressure sensing signal.

[0014] In some embodiments, the writing device further includes an audio acquisition unit, and the processor is used to transmit a recording prompt voice to an audio playback unit and a recording command to an audio acquisition unit if, by recognizing the code point type of the code point value, it is determined that the code point type is a recording code. The audio playback unit is further used to broadcast the recording prompt audio after receiving it. The audio acquisition unit is used to receive a recording command and to start recording after the audio playback unit has played a recording prompt sound.

[0015] A second aspect of the present application provides a method for recognizing written content, and the method for recognizing written content is: It is used in the processor of a writing device, which is a component of the writing device in the first aspect of the present application. Continuously obtaining the code point values returned from the optical recognition unit to obtain a code point value sequence, where the code point values are obtained by analyzing the code point images obtained by the optical recognition unit capturing the code points on the writing medium. Recognizing the code point type for the code point values. When the code point type indicated by the code point values is a coordinate code, performing coordinate recognition on the code point value sequence to obtain a coordinate sequence, and drawing a writing trajectory based on the coordinate sequence. Including the step of calling a preset trajectory recognition model to recognize the writing trajectory and obtain the writing content.

Advantages of the Invention

[0016] The components of the writing device proposed in the present application include a writing device and a writing medium. The writing device includes a pressure sensing unit, an optical recognition unit, and a processor. Code points are printed on the writing medium. The pressure sensing unit is used to generate a pressure sensing signal when it detects that the writing device touches the writing medium. The optical recognition unit is used to collect the code point images of the code points on the writing medium, analyze the code point images to obtain code point values, and transmit the code point values to the processor when the pressure sensing unit generates a pressure sensing signal. The processor is used to recognize the code point type for the code point values, perform coordinate code analysis on the code point values when it recognizes that the code point type is a coordinate code, determine the writing trajectory based on the analysis result, and perform content recognition on the writing trajectory to obtain the writing content.

[0017] According to the above method, the pressure sensing unit of the writing device immediately senses the contact with the writing medium, commands the optical recognition unit to collect the code point image of the writing medium, analyzes the code point value from the code point image, and when the processor recognizes that the type of the code point value is a coordinate code, it analyzes the code point value to determine the writing trajectory, and further recognizes the writing content for the writing trajectory. Thereby, the writing device can digitize the handwritten content immediately without the need for an additional recognition device, and can improve the recognition efficiency and portability.

Brief Description of the Drawings

[0018] [Figure 1] It is a structural schematic diagram of the components of the writing device according to an embodiment of the present application. [Figure 2] It is a structural schematic diagram of the components of the writing device according to an embodiment of the present application. [Figure 3] It is another structural schematic diagram of the components of the writing device according to an embodiment of the present application. [Figure 4] It is another structural schematic diagram of the components of the writing device according to an embodiment of the present application. [Figure 5] It is a structural schematic diagram of the pressure sensing unit according to an embodiment of the present application. [Figure 6] It is another structural schematic diagram of the components of the writing device according to an embodiment of the present application. [Figure 7] It is a flow schematic diagram of the method for recognizing the writing content according to an embodiment of the present application. [Figure 8] It is a process flow schematic diagram of the components of the writing device according to an embodiment of the present application.

Modes for Carrying Out the Invention

[0019] To make the purpose, technical solution and advantages of the present application clearer, the present application will be described in more detail below with reference to the drawings and embodiments. The specific embodiments described in this specification are for the purpose of explaining the present application and not for limiting the present application.

[0020] The division of functional modules is shown in the schematic diagram of the device, and the logical order is shown in the flowchart; however, in some cases, this may differ from the division of modules in the device, or the steps shown or described may be performed in a different order from that shown in the flowchart. Terms such as "First," "Second," etc., in the specification and claims, and in the drawings above, are terms used to distinguish similar objects and are not necessarily used to describe a specific order or priority.

[0021] The term "exemplary" as used here specifically means "used as an example, implementation, or explanation." None of the examples described here as "exemplary" are necessarily interpreted as being superior or taking precedence over other examples.

[0022] In the embodiments of this application, the terms “module” or “unit” mean a computer program or part of a computer program having a predetermined function, which operates in conjunction with other relevant parts to achieve a predetermined purpose, and which can be implemented in whole or in part using software, hardware (e.g., processing circuits or memory), or a combination thereof. Similarly, one processor (or more processors or memory) may be used to implement one or more modules or units. Furthermore, each module or unit may be part of an overall module or unit that includes the function of that module or unit.

[0023] Furthermore, in order to better illustrate the present invention, many specific details are provided in the following specific embodiments. Those skilled in the art should understand that the present invention can be similarly implemented even without these specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail in order to emphasize the spirit of the present invention.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art. The terms used herein are for illustrative purposes only and are not intended to limit the application.

[0025] First, we will analyze some of the nouns related to this application.

[0026] Optical identification (OID) is an infrared optical identification technology. The biggest difference between it and other optical identification technologies is that its tiny code points not only have the properties of confidentiality and low visual interference, but can also be hidden beneath the color pattern of printed materials, allowing the use of ordinary printing inks, thereby reducing costs.

[0027] A writing device is a type of input / output device for smart educational support devices, allowing users to interact with and make decisions with electronic devices (e.g., computers, tablets, mobile phones) by writing or drawing on a writing medium (including, but not limited to, paper) using conventional speaking code points. While conventional speaking pens can achieve instant reading of book content, they lack coordinate recognition, immediate feedback, and the ability to judge the characteristics of handwriting results. On the other hand, handwriting recognition technology focuses on the digitization and conversion of handwriting input into other electronic devices.

[0028] Based on this, an embodiment of the present application provides a component of a writing device, the component of which includes a writing device and a writing medium, the writing device including a pressure sensing unit, an optical recognition unit and a processor, the writing medium having code points printed on it, the pressure sensing unit is used to generate a pressure sensing signal when it detects that the writing device has come into contact with the writing medium, the optical recognition unit is used to collect a code point image of the code points on the writing medium when the pressure sensing unit generates a pressure sensing signal, analyze the code point image to obtain a code point value and transmit the code point value to the processor, the processor is used to recognize the code point type from the code point value, and if it recognizes that the code point type is a coordinate code, it performs coordinate code analysis on the code point value and determines the writing trajectory based on the analysis result and performs content recognition on the writing trajectory to obtain the written content.

[0029] According to the above method, the pressure sensing unit of the writing device instantly detects contact with the writing medium, commands the optical recognition unit to collect a code point image of the writing medium, analyzes the code point value from the code point image, and if the processor recognizes that the type of code point value is a coordinate code, it analyzes the code point value to determine the writing trajectory, and further recognizes the written content based on the writing trajectory. As a result, the writing device can instantly digitize handwritten content without requiring additional recognition devices, improving recognition efficiency and portability.

[0030] The components of the writing device and the method for recognizing the written content according to the embodiments of this application will be specifically described in the following embodiments. First, the components of the writing device in the embodiments of this application will be described.

[0031] In embodiments of the present invention, the processing of relevant data may be performed based on artificial intelligence technology. Here, artificial intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and augment human intelligence, sense the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0032] The fundamental technologies of artificial intelligence generally include technologies such as sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technologies, operating / interaction systems, and mechatronics. Artificial intelligence software technologies encompass several major aspects, primarily computer vision technologies, robotics technologies, biometric authentication technologies, audio processing technologies, natural language processing technologies, and machine learning / deep learning.

[0033] This application can be used in the environment or configuration of numerous general-purpose or dedicated computer systems. Examples include personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, a program module includes routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be implemented in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located on local and remote computer storage media, including storage devices.

[0034] Furthermore, in each embodiment of this application, user information is not used, and the user's handwriting is recognized and broadcast by a recognition model built into the handwriting device, thus eliminating the need for server-side judgment and ensuring the security of the user's personal information. When it is necessary to perform related processing via the server based on data related to the user's identity and characteristics, such as user information, user behavior data, user history data, and user location information, the user's permission or consent will be obtained, and the collection, use, and processing of this data will be carried out in compliance with relevant laws and standards. In addition, in the embodiments of this application, when it is necessary to obtain the user's confidential personal information, the user's individual permission or consent will be obtained through a pop-up window or by jumping to a confirmation page, and after explicitly obtaining the user's individual permission or consent, the user-related data necessary for the embodiments of this application to function properly will be obtained.

[0035] Referring to Figure 1, which shows a schematic diagram of the structure of component 1 of a writing device according to an embodiment of the present invention, component 1 of the writing device includes a writing device 11 and a writing medium 12. When the writing device 11 comes into contact with the writing medium 12, it collects code point images of the code points of the writing medium 12. If the code point values ​​of the code point images are recognized as coordinate codes, the code point values ​​are analyzed to determine the writing trajectory, and content recognition is performed on the writing trajectory to obtain the written content.

[0036] The components of the writing device according to the embodiment of this application will be described below, and the method is as follows.

[0037] Referring to Figure 2, which shows a schematic diagram of the structure of component 1 of a writing device according to an embodiment of the present invention, component 1 of the writing device includes a writing device 11 and a writing medium 12, the writing device 11 includes a pressure sensing unit 111, an optical recognition unit 112 and a processor 113, and the writing medium 12 has code points printed on it. The pressure sensing unit 111 is used to generate a pressure sensing signal when it detects that the writing device has come into contact with the writing medium 12. The optical recognition unit 112 is used to collect a code point image of the code point on the writing medium when the pressure sensing unit 111 generates a pressure sensing signal, analyze the code point image to obtain a code point value, and transmit the code point value to the processor. The processor 113 recognizes the code point type from the code point value, and if it recognizes that the code point type is a coordinate code, it performs coordinate code analysis on the code point value, determines the writing trajectory based on the analysis result, and performs content recognition on the writing trajectory to obtain the written content.

[0038] The writing device 11 may be a tool or device for writing, and the user can write letters or draw pictures using the writing device 11.

[0039] The writing medium 12 may be any of several types of media, such as books with OID code points, LCD tablets, or whiteboards. The user can write on the writing medium 12 using the writing device 11 and generate characters or figures with code points.

[0040] The pressure sensing unit 111 responds to pressure changes by outputting an electrical signal proportional to the pressure change, and can determine the current usage state of the writing device 11 based on the change in the electrical signal. That is, the writing device 11 can detect the current usage state of the writing medium 12, including pressure state and release state, using a pressure detector. Here, the pressure state is the state in which the user is writing on the writing medium 12 using the writing device 11, and may be, for example, the state in which pressure is generated on the writing member when the writing member of the writing device 11 comes into contact with the writing medium 12, and the pressure sensing unit 111 can indicate this pressure state by generating a pressure sensing signal. The release state may be the state in which the pressure received by the writing member of the writing device 11 is released, i.e., the writing member is separated from the writing medium 12, or the writing member is in contact with the writing medium 12 but is not subjected to pressure.

[0041] The optical recognition unit 112 may be a camera integrating optical recognition technology that can recognize and process image data to recognize specific patterns or objects. The writing device 11 may acquire an image by scanning the writing medium 12 with the optical recognition unit 112. The optical recognition unit 112 may collect images by a pressure-sensing signal, i.e., a pressure-sensing signal indicating that the user is writing. The optical recognition unit 112 can immediately collect an image on the writing medium 12, i.e., it can capture changes in light refraction on the surface of the writing medium 12, which are usually caused by small code points (e.g., OID codes) printed on the paper. The optical recognition unit 112 may generate a code point image based on this optical information, sense these small code points on the code point image, and convert these code points into electrical signals using image recognition technology. The optical recognition unit 112 may decode these electrical signals to recognize the code point values ​​of the code points in the code point image and transmit them to the processor 113.

[0042] The processor 113 may be a component that processes various data from the writing process. The processor 113 may recognize the code point type for the code point value from the optical recognition unit 112, and may perform different operations depending on the code point type. If the code point type of this code point value is a coordinate code, the processor 113 may analyze the numerical coordinates representing the position of the writing trajectory on the writing medium 12 for that coordinate code. That is, the processor 113 may determine the X and Y coordinate values ​​of each coordinate code based on the coordinate code. Once the coordinate values ​​are obtained, the processor 113 can connect these points to form a writing trajectory. The processor 113 may also integrate a machine learning model or a deep learning model that can learn from a large number of handwriting samples, and can recognize the handwritten content corresponding to the writing trajectory. For the writing trajectory determined by the processor 113, the writing trajectory can be input into the learning model to perform content recognition and obtain the writing content on the writing medium 12 by the user.

[0043] This recognition model can be selected according to specific application scenarios and needs, and may include recurrent neural networks (RNNs), long short-term memory (LSTMs), convolutional neural networks (CNNs), combinations of RNNs (e.g., CRNNs), and Transformers. This recognition model is obtained by training an initial model using sample features and their corresponding labels as sample data. This correspondence between sample features and labels may be manually tagged or downloaded from the internet. The sample data is input into the initial model, and the model parameters are adjusted using a backpropagation algorithm so that the model can accurately map handwriting coordinate data to handwriting content. Furthermore, the recognition model can be retrained based on the current user's handwriting and user feedback to improve the personalization of the recognition model.

[0044] In the components of the writing device according to the embodiment of the present invention, the pressure sensing unit of the writing device immediately senses contact with the writing medium, commands the optical recognition unit to collect a code point image of the writing medium, analyzes the code point value from the code point image, and if the processor recognizes that the type of code point value is a coordinate code, it analyzes the code point value to determine the writing trajectory, and further recognizes the written content with respect to the writing trajectory. As a result, the writing device can instantly digitize handwritten content without requiring an additional recognition device, thereby improving recognition efficiency and portability.

[0045] In one feasible embodiment, referring to another schematic diagram of the writing device component 1 shown in Figure 3, the writing device 11 further includes an audio playback unit 114 for broadcasting the written content after the processor has recognized the written content.

[0046] The audio playback unit 114 may convert the written content into an audio signal after it has been determined. For example, when using TTS technology, preprocessing is performed on the input written content, including text normalization, punctuation processing, and conversion of numbers and dates, to meet the requirements for audio synthesis. Next, text analysis can be performed using a language model to determine the optimal audio output. Finally, the audio playback unit 114 broadcasts the audio signal via a speaker or other audio output device. The written content can be played back instantly offline, thereby improving the interaction effect.

[0047] In one feasible embodiment, the processor 113 further: It is used to recognize the meaning of written content and to obtain the meaning of the written text. If the written content indicates that it is question data, the processor 113 is further used to generate answer data corresponding to the question data. The audio playback unit 114 is further used to broadcast the response audio corresponding to the response data after the processor has generated the response data.

[0048] Semantic recognition can refer to understanding and interpreting the deeper meanings and intentions contained within natural language text. It can be related to natural language processing (NLP) techniques, including grammatical analysis, semantic analysis, and contextual understanding; that is, it requires not only focusing on the meaning of textual vocabulary but also understanding the meaning these words express within sentences and chapters. This means that semantic recognition requires semantic analysis, disambiguation, and corresponding semantic reorganization at the text, vocabulary, syntactic, lexical, and chapter levels.

[0049] The meaning of writing refers to the deeper meaning and intent contained within the written content, going beyond the superficial form of the characters and involving the understanding and interpretation of the written content. For example, it involves understanding the structure of a sentence, combining contextual explanations, sentiment analysis, and other factors to understand and interpret the written content.

[0050] Question data refers to a question expressed in written form. Based on the meaning of the writing, if the processor 113 determines that the written content is question data, it may determine the type of question, such as a fact inquiry, a request for suggestion, or an operation command, and select an appropriate answer generation strategy.

[0051] The response data may be a standard answer corresponding to the question data. The processor 113 may search for relevant information corresponding to the question data by accessing an internal knowledge base, external databases, or internet resources, and determine a standard answer from that relevant information, or the processing unit may use its built-in mathematical engine or calculator function to logically analyze the extracted calculation formula and perform the calculation. For example, if the written content is "1+1", the meaning of the written content may be recognized as a mathematical operation of adding 1 and 1, in which case the processor 113 may generate the answer "2" corresponding to that mathematical operation. The audio playback unit 114 performs text normalization, punctuation processing, and conversion of numbers and dates etc. on the response data in response to the request for audio synthesis, performs text analysis via a language model, determines the optimal response audio, and broadcasts that response audio. Students can not only see their scores but also receive feedback by listening to them, thereby increasing the interactivity and the enjoyment of learning. The broadcast content may include statements such as, "The calculation result was correct," or "There is a small error here, please review the calculation process again," but the specific broadcast content will be determined based on the actual evaluation results and design needs. In this way, the system not only provides immediate feedback but also improves the learning experience, makes the learning process more dynamic, and enhances its interactivity.

[0052] In one feasible embodiment, the audio playback unit 114 further includes: When a question broadcast command is received, it is used to broadcast the question audio corresponding to the question broadcast command. The processor 113 is further used to compare the written content with the answer corresponding to the question audio, obtain the answer matching result, and determine the matching result audio based on the answer matching result. The audio playback unit 114 further includes: After the processor 113 determines the matching result audio, it is used to broadcast the matching result audio.

[0053] The question broadcast command may be a command to instruct the writing device 11 to broadcast a question, may be determined by the writing device 11 scanning a code point on the writing medium 12, may be generated by the user via the control of the writing device 11, or may be transmitted by the user to the writing device 11 via another device. Therefore, this question broadcast command may be commanded by the user to the writing device 11 before writing, that is, the user's current writing content may be a user response written based on the broadcasted question.

[0054] The question audio may be an audio signal that can be broadcast directly by a speaker. The question audio may be stored directly in the writing device 11, or the writing device 11 may acquire it from another device, or the writing device may store the question content and, after receiving a question broadcast command, convert the question content into question audio.

[0055] The answer corresponding to the question audio may be a standard answer for that question audio. This standard answer may be retrieved by the writing device 11; that is, the writing device 11 may store the correspondence between the question audio and the standard answer in advance, or it may directly generate a standard answer for the question content corresponding to the question audio.

[0056] The answer matching result may be a conclusion reached by the processor 113 after comparing the user's written answer with the correct answer, and this result may be a simple match or mismatch, or a detailed similarity score. The processor 113 may perform text preprocessing on the user's answer and the standard answer as indicated by the written content, and compare the similarity of the two texts using a specific algorithm. This may include a simple string matching algorithm, an edit distance algorithm (e.g., Levenshtein distance), or a more complex natural language processing technique such as semantic similarity analysis. The processor 113 interprets the matching result based on the result of the matching algorithm. If the result is a simple match / mismatch, the processor directly concludes whether the answer is correct or not. In the case of a similarity score, the processor may determine whether the answer is close enough to the correct answer based on a pre-set threshold.

[0057] The matching result audio may be an audio signal broadcast to the user by the audio playback unit 114 after the processor 113 has completed matching the written content with the correct answer and converted the matching result into audio information. The processor 113 may also convert the matching result into easily understandable text information. For example, if the matching result indicates that the user's answer is a perfect match to the correct answer, the matching result audio may say "The answer is perfectly correct." If the answer is partially correct, the matching result audio may say "The answer is partially correct, but there is room for improvement." If the answer is incorrect, the matching result audio may say "The answer is incorrect. Please refer to the correct answer below."

[0058] In one feasible embodiment, referring to another schematic diagram of the components of the writing device shown in Figure 4, the component 1 of the writing device further includes a display module 13 for displaying the written content after the processor 113 has recognized the written content.

[0059] The display module 13 may be a component that displays the written content on a display screen. The display module 13 may format the written content, including adjusting the font size, type, color, and layout, so that the written content can be clearly read on the screen. The display module 13 may be provided in the writing device 11, that is, the processor 113 may recognize the written content and transmit it to the display module 13 via internal wiring, and the display module 13 may display the written content on the screen. Alternatively, the display module 13 may be a device other than the writing device 11, and the processor 113 of the writing device 11 may recognize the written content and transmit it to the display module 13 via wireless communication, and the display module 13 may display this written content. An example of the display module 13 being a device other than the writing device 11 is shown in Figure 4.

[0060] In one feasible embodiment, the processor 113 further recognizes the code point type for a code point value, and if it determines that the code point type is a speaking code, it uses the code point value to determine the speaking content. The audio playback unit 114 is further used to broadcast the spoken content after the processor 113 has determined the spoken content.

[0061] A speaking code may be a code point arranged on the writing medium 12 according to certain rules, and each speaking code represents specific information or a command. When the processor 113 recognizes the code point type of a code point value, it analyzes whether or not there is a speaking code in that code point value. After recognizing the speaking code, it may search for the speaking content corresponding to the information indicated by the speaking code in a database inside the writing device 11 or a database connected to an external server. The speaking content in the database may include multimedia information such as text, audio, and video. After the processor 113 obtains the speaking content corresponding to the speaking code from the database, it sends the generated audio data to the audio playback unit 114. The audio playback unit 114 performs necessary processing on this audio data, such as decoding and format conversion, converts it to a format suitable for playback, and plays the format-converted data on an audio output device such as a speaker, allowing the user to hear the speaking content.

[0062] In one feasible embodiment, the processor 113 further: Analyze the writing angle based on the image sequence of code point images to obtain writing angle data. Based on writing angle data and writing content, the system will determine the writing posture in real time, and If the writing posture is found to be incorrect as a result of the posture detection, it is used to generate a voice message correcting the writing posture. The audio playback unit 114 is further used to broadcast the writing posture correction audio after the processor has generated it.

[0063] An image sequence can be a sequence consisting of a series of still images, which are arranged in chronological order and can represent process images of a specific time series or dynamic process. In embodiments of the present invention, the optical recognition unit 112 may continuously capture an image sequence of writing actions.

[0064] The writing angle data may refer to information about the angle between the writing element of the writing device 11 and the writing medium 12 during the process of a user writing. The image sequence may include features related to the writing angle, such as the position of the writing element, the direction of the stroke, and the angle between the stroke and the writing medium, and the writing angle may be calculated according to the change in the position of the writing element. For example, when the writing device 11 is perpendicular to the writing medium 12, the tip of the writing element, such as the pen tip, is positioned in the middle of the stroke, with the deepest part of the stroke being the same distance from the edge. When the writing device 11 is not perpendicular to the writing medium 12, the position of the pen tip is biased towards the stroke side, and the distance between the deepest part of the stroke and the edge is not equal. By measuring and summarizing this ratio, the writing angle of the stroke can be determined. The change in the pen tip angle in the image sequence can be analyzed, and the dynamic change in the writing angle during writing can be determined.

[0065] The result of determining writing posture may refer to a conclusion regarding whether the writing posture is correct or not, obtained by analyzing various data during the writing process. The correctness of the writing posture may also be determined by comparing the writing angle data with a preset standard angle. If the angle exceeds the normal range, it is considered that there is a problem with the writing method. The standard angle may be an optimal one based on ergonomics, medical research, or writing habits, and can reduce pressure on the hand and wrist. The processor 113 may also determine writing posture, such as writing consistency or stroke order, in accordance with the writing content. Regarding writing consistency, if the writing content is judged to be consistent and fluent, it indicates that the writing posture is relatively natural and comfortable. If there are frequent stops, corrections, or sudden changes in the handwriting in the writing content, it indicates a problem with the writing posture. Regarding stroke order, the direction and order of strokes in the writing content can reflect the trajectory of the pen tip when the user writes, and if the stroke order is disordered or the direction is incorrect, it can be determined that an inappropriate writing posture is the cause.

[0066] The writing posture correction voice may be an audio signal that guides and corrects the user's writing posture through voice, and the processor 113 may determine that the user's writing posture is incorrect if it determines that the writing angle in the writing angle data exceeds the normal range, or that the writing content is inconsistent, or that the writing order is incorrect, and may generate text to correct the user's writing posture. The correction text may include the following:

[0067] For example, you might point out a specific problem, such as, "The angle between the pen tip and the paper is too large, which could be causing hand fatigue."

[0068] For example, they might instruct students on the correct posture, such as, "Try holding the pen tip at a 45-degree angle to the surface of the paper." For example, they might suggest, "Try loosening your grip on the pen so that it rests naturally between your index finger and thumb," suggesting practice and adjustments.

[0069] For example, it offers encouraging words such as, "Maintaining correct posture improves writing efficiency and prevents hand pain."

[0070] The processor 113 converts this corrective text into speech for correcting writing posture using text-to-speech technology and transmits it to the audio playback unit 114. The audio playback unit 114 receives the speech for correcting writing posture and may control an audio output device such as a speaker to play the speech for correcting writing posture. The user can then hear instructions on how to correct their writing posture.

[0071] In addition to analyzing the writing angle using image sequences, the writing angle can also be directly analyzed based on code points. For example, the information that can be analyzed from the coordinate code of a code point image may include angle information in addition to coordinates. If the mounting position of the optical recognition unit 112 on the writing device 11 is fixed, and the image captured by the optical recognition unit 112 differs depending on the rotation of the writing device 11, then the arrangement of the captured code points changes depending on the shooting angle, and the coordinates of the handwriting and the shooting angle can be determined based on the captured arrangement. For example, when photographed from above, the code points may exhibit a specific arrangement pattern, and when photographed from below, the arrangement pattern may differ. By analyzing these arrangement patterns, the shooting angle, i.e., the direction of the code points relative to the camera (also called the angle of the writing member of the writing device 11 relative to the writing medium 12), can be estimated, and therefore, it can be determined whether the user's writing posture is incorrect from the difference between this shooting angle and the standard angle.

[0072] In one feasible embodiment, referring to the schematic structural diagram of the pressure sensing unit shown in Figure 5, the pressure sensing unit 111 includes a writing member 1111 and a sensing member 1112.

[0073] The writing member 1111 includes a first end that contacts the writing medium during writing and a second end opposite to the first end, and the second end of the writing member is provided with a conductive silicone rubber pad and a first magnetic element. The sensing member 1112 has a sensing end positioned opposite the second end of the writing member 1111, and the sensing end is provided with a second magnetic element and an unconnected first wire and second wire, and the first magnetic element and the second magnetic element are arranged to repel each other magnetically. When the first end of the writing member 1111 is not in contact with the writing medium, the conductive silicone rubber pad and the sensing end are separated by the repulsive force between the first and second magnetic elements, the first and second wires are not electrically connected, and no pressure sensing signal is generated. When the first end of the writing member 1111 contacts the writing medium, and the contact pressure between the writing member and the writing medium is greater than the repulsive force between the first magnetic element and the second magnetic element, the conductive silicone rubber pad contacts the sensing end, the first wire and the second wire become electrically connected, a pressure sensing signal is generated, or, The writing element includes a first end that contacts the writing medium during writing and a second end opposite to the first end, and a slider is provided at the second end of the writing element. The sensing member includes a pressure sensor positioned opposite the second end of the writing member. If the first end of the writing element is not in contact with the writing medium, the slider slides downward due to gravity, the slider and pressure sensor separate, and the pressure sensor does not generate a pressure sensing signal. When the first end of the writing element contacts the writing medium, and the contact pressure between the writing element and the writing medium is greater than the gravity of the writing element, the slider moves upward and contacts the pressure sensor, which generates a pressure sensing signal.

[0074] The pressure sensing unit 111 consists of a writing member 1111 and a sensing member 1112. The writing member 1111 is designed to have both ends such that the first end contacts the writing medium and the second end faces the sensing member 1112. A conductive silicone rubber pad and a first magnetic element are provided at the second end of the writing member 1111. The conductive silicone rubber pad may be a silicone rubber material that performs conductive function by doping it with conductive particles (such as metal powder or carbon black), and the conductive particles that form conductive paths are uniformly dispersed within the conductive silicone rubber pad. When the conductive silicone rubber pad is not compressed, the contact between these conductive particles is broken, and a closed circuit is not formed. When pressure is applied to the conductive silicone rubber pad, the conductive silicone rubber pad deforms, the conductive particles are pushed out more tightly, and a continuous conductive path is formed. The conductive silicone rubber pad deforms when the first end of the writing member 1111 comes into contact with the writing medium and a certain amount of pressure is applied, forming a conductive path that comes into contact with the sensing end of the sensing member 1112 to establish a circuit connection.

[0075] The sensing end of the sensing member 1112 is equipped with a second magnetic element and two unconnected wires, namely a first wire and a second wire. The first and second magnetic elements may be designed to repel each other magnetically. This repulsive force ensures that, when there is no pressure from writing, the first magnetic element keeps the conductive silicone rubber pad separated from the sensing end. At this time, contact between the conductive particles inside the conductive silicone rubber pad is broken, and the first and second wires are not electrically connected. In the non-contact state, the repulsive force between the first and second magnetic elements holds the conductive silicone rubber pad away from the sensing end, and since the first and second wires are not in communication, no current flows, and the sensing member 1112 is not triggered. When the user is writing, the first end of the writing member 1111 receives pressure from the writing medium, and is pressed inward by that pressure, that is, when the applied pressure exceeds the repulsive force between the first and second magnetic elements, the conductive silicone rubber pad overcomes the magnetic repulsion force and contacts the sensing end and deforms, the conductive silicone rubber pad communicates with the first and second wires, an electric current flows, and at this time the sensing member 1112 is triggered and generates a pressure sensing signal.

[0076] The conductive silicone rubber pad operates within the pressure-sensing unit due to its conductivity and elasticity, converting pressure changes during writing or drawing into electrical signals to achieve pressure sensing. This design allows the pressure-sensing unit to accurately detect and respond to writing pressure, instantly sensing the user's writing.

[0077] In addition to controlling contact between the conductive silicone rubber pad and the sensing end using a magnetic element, the writing state of the writing member 1111 can be detected by a pressure sensor. When the first end of the writing member 1111 is not in contact with the writing medium, gravity causes the writing member 1111 to slide downward, and the slider provided at the second end separates from the pressure sensor in the sensing member 1112. In this state, the pressure sensor does not detect pressure, and therefore no pressure sensing signal is generated.

[0078] However, if the first end of the writing member 1111 is in contact with the writing medium and the applied pressure is so great that it exceeds the gravity of the writing member 1111 itself, the second end of the writing member 1111 will rise due to the action of the pressure, and the slider will come into contact with the pressure sensor. At this time, the pressure sensor will be affected by the pressure of the slider and will generate a pressure sensing signal.

[0079] In one feasible embodiment, referring to another schematic diagram of the writing device component 1 shown in Figure 6, the writing device 11 further includes an audio acquisition unit 115, and a processor 113 is used to recognize the code point type for a code point value and, if it is determined that the code point type is a recording code, to send a recording prompt voice to an audio playback unit 114 and to send a recording command to the audio acquisition unit 115.

[0080] The audio playback unit 114 is further used to broadcast the recording prompt audio after receiving it. The audio acquisition unit 115 is used to receive a recording command and to start recording after the audio playback unit has played a recording prompt sound.

[0081] In this embodiment, the writing medium 12 may be further provided with a recording code. The recording code may be a code point that marks the need for recording, and if the code point image contains a recording code, i.e., if the processor 113 recognizes from the code point value that the code point type is a recording code, it may determine that the user needs to record.

[0082] The processor 113 can send a recording prompt voice message to the audio playback unit 114 to notify the user device that recording is about to begin, ensuring that the user knows when they can start speaking. Upon receiving this recording prompt voice message, the audio playback unit 114 immediately plays it, prompting the user to begin recording.

[0083] The processor 113 may send a recording command to the audio acquisition unit 115 at the same time that the audio playback unit 114 broadcasts a recording prompt voice. This command notifies the audio acquisition unit 115 to prepare to start recording. When the audio playback unit 114 finishes broadcasting the recording prompt voice, the audio acquisition unit 115 may turn on the recording function and start capturing audio data. This ensures the smoothness and accuracy of the recording process, avoids delays at the start of recording, and ensures the completeness and clarity of the recorded content.

[0084] In one example, the optical recognition unit may be an infrared camera, the processor may be a main control processor, and the audio playback unit may be a speaker. The OID infrared optical camera can capture the user's writing and speaking actions on a book, LCD tablet, or whiteboard. Specifically, it can collect code point images of the writing medium based on the user's writing actions and analyze the code point images to obtain code point values.

[0085] The main control processor is a built-in high-performance processor that executes local handwriting recognition algorithms and speaking control programs, enabling rapid data processing and response. If the code point type of a code point value is a speaking code, it retrieves the speaking content corresponding to the speaking code value from the local database. If the code point type of a code point value is a coordinate code, it determines the writing trajectory from the coordinate code value, calls a built-in pre-configured trajectory recognition model to analyze the writing trajectory, and obtains the user's writing content.

[0086] The speaker is used to broadcast spoken content and recognized written content, and also supports recording and shadowing functions. After acquiring the recognition results, it broadcasts the corresponding results, hint sounds from the system itself, and recordings.

[0087] Handwriting devices can instantly provide feedback on what the user is writing by recognizing and broadcasting the content as they write, thereby enhancing the interactivity of learning and improving the functionality and practicality of the handwriting device.

[0088] The above describes the components of a writing device; below, we will describe how the processor in those components recognizes the written content.

[0089] Referring to Figure 7, which shows a schematic flowchart of the method for recognizing written content in an embodiment of the present invention, this recognition method includes the following steps 701 to 703.

[0090] Step 701: Continuously acquire the code point values ​​returned from the optical recognition unit to obtain a code point value sequence, and obtain the code point values ​​by analyzing the code point images obtained when the optical recognition unit photographs the code points in the writing medium.

[0091] A code point value may be a number or mark represented by a data unit consisting of a grid of multiple code points. For example, 6 × 6 = 36 points constitute the smallest data unit, and these grids can represent numerous array forms for representing numbers.

[0092] The code point value sequence may also be a sequence of code point values ​​in a series of code point images taken sequentially by the optical recognition unit, that is, it may include code points at different time points and obtain code point values ​​for different states or positions.

[0093] The code point image may be an image of the collected OID code point. That is, an image collected by an optical recognition unit from a writing medium having an OID code point can be called a code point image. The optical recognition unit may obtain a code point image by capturing a picture of the code point in the writing medium when it receives a pressure detection signal transmitted from a pressure sensing unit.

[0094] The writing medium may be one of several types, such as books or LCD tablets with OID code points.

[0095] The optical recognition unit may be a camera that integrates optical recognition technology and is responsible for collecting images.

[0096] The pressure sensing unit may be a unit within the writing device that detects pressure changes when the writing element of the writing device comes into contact with the writing medium.

[0097] The pressure detection signal may be a signal indicating that the user is writing using the writing device, and the writing device may generate a pressure detection signal indicating that it is currently writing when the pressure applied to the writing element of the writing device reaches a preset value.

[0098] Step 702: Recognize the code point type for the code point value. If the code point type indicated by the code point value is a coordinate code, perform coordinate recognition on the code point value sequence to obtain a coordinate sequence, and draw the writing trajectory based on the coordinate sequence.

[0099] A code point type may be a functional type of a specific code consisting of multiple OID code points, and the arrangement of code points differs depending on the functional type.

[0100] A coordinate code may be a data unit used to determine coordinate information within a grid of multiple code points, where these grids can represent numbers or a number of array forms for representing coordinate information. These code points typically have unique visual features that allow them to be distinguished from the background, such as the XY axis coordinates on a writing medium. Based on the corresponding coordinate information at these code points, the processing unit may identify the specific coordinates of the writing motion and determine the handwriting coordinates. This coordinate information may be predefined during the OID coding process, enabling the system to accurately map the handwriting to a physical location on the writing medium.

[0101] A coordinate sequence may be a set of coordinates, i.e., a set of coordinates corresponding to each code point value in a code point value sequence. The processor can recognize the grid of each code point value in the code point value sequence, analyze the coordinates corresponding to this grid based on an algorithm, and construct a coordinate sequence by sequentially arranging these coordinates based on the order in which the code point values ​​were collected in the code point value sequence.

[0102] The writing trajectory may simulate a combination of strokes drawn based on a coordinate sequence, the stroke order in the user's writing process, or the movement path of the writing element on the writing medium.

[0103] Step 703: Call up a pre-configured trajectory recognition model to recognize the writing trajectory and obtain the written content.

[0104] The pre-configured trajectory recognition model may be a recognition model built into the writing device to recognize the user's handwriting content, and can be selected according to the specific application scenario and needs. This may include recurrent neural networks (RNNs), long short-term memory (LSTMs), convolutional neural networks (CNNs) combined with RNNs (e.g., CRNNs), and Transformers. This recognition model is obtained by training an initial model using sample features and their corresponding labels as sample data. This correspondence between sample features and labels may be manually tagged or downloaded from the internet. The sample data is input into the initial model, and the model parameters are adjusted using a backpropagation algorithm so that the model can accurately map the handwriting trajectory to the handwriting content. Furthermore, the recognition model can be retrained based on the current user's handwriting and user feedback to improve the personalization of the recognition model.

[0105] In one possible form, the code point type of a code point value may be a speaking code. Different combinations of codes may recognize different content, and these codes are typically very small, hidden on the page, invisible to the naked eye, and recognizable by OID. The writing medium in the embodiments of this application may be provided with speaking codes arranged according to certain rules, where each speaking code represents specific information or instructions. After acquiring a code point value, the processor first recognizes whether the code point value contains a speaking code. If a speaking code is recognized, it may search a local database or a database connected to an external server for the speaking content corresponding to the information indicated by the speaking code. The speaking content in the local database may include multimedia information such as text, audio, and video. After acquiring the speaking content corresponding to the speaking code from the local database, the processor performs necessary processing such as decoding and format conversion to convert it to a format suitable for playback and broadcasts it by an audio playback unit.

[0106] In one possible configuration, the processor may further store the written content in a local database and generate the user's learning outcomes in the local database based on the written content.

[0107] Learning outcomes may be determined by the Q&A results shown in the written content, including questions and answers. The processing unit may determine the user's writing accuracy, which is the user's learning outcome, based on the relationship between the answers and questions. Accordingly, the processing unit can distinguish between the question content and the answer content in the written content.

[0108] The learning outcome may be the user's learning progress. Specifically, the written content corresponds to the writing time and is stored in a local database. The format of the written content may be what the user actually wrote, or it may be text, images, or other data formats, or a combination of code point images and text. The writing time is the timestamp that recorded the user's writing. The processing unit may analyze the user's learning progress based on the user's written content and writing time. For example, a writing template may be stored in a local database, and the user can copy the content of the writing template. The processing unit can then recognize the written content to determine the user's writing progress in that writing template, and may also create a line graph of the user's writing in combination with the writing time to reflect differences in writing volume over time. The analysis method may involve training a model using a machine learning algorithm to infer the user's learning progress based on the user's written content. Alternatively, the learning progress can be evaluated using simple statistical methods such as the user's daily writing volume and weekly learning time.

[0109] For example, the processing flow of the components of a writing device may be illustrated by the schematic diagram of the broadcast flow of a handwriting device shown in Figure 8. In step 801, the optical recognition unit acquires a code point image, and when the user writes or speaks on the writing medium, the optical recognition unit scans the writing medium in real time to form a code point image. In step 802, the optical recognition unit recognizes the code point value from the code point image and transmits this code point value to the processor. The optical recognition unit can preprocess the code point image to obtain a preprocessed image. That is, by performing preprocessing operations such as noise reduction and contrast enhancement on the captured code point image, and then recognizing the code point value on the preprocessed image, the recognition accuracy can be improved. In step 803, the processor recognizes whether the code point type of the code point value is a coordinate code. If so, it performs step 805; otherwise, it performs step 804. That is, the code point value that the optical recognition unit sends to the processor may be a coordinate code value or a speaking code value. In step 804, the speaking content is recognized from the local database based on the speaking code. The local database has a pre-stored correspondence between the speaking code and the speaking content. This local database may be updated periodically, or it may connect to a database on an external server. In step 805, the writing trajectory is determined based on the coordinate code. Coordinate values ​​are determined one by one based on the coordinate code, and the writing trajectory is constructed based on the coordinate values. In step 806, a pre-configured trajectory recognition model is called up to recognize the writing trajectory and obtain the written content. The writing trajectory is input into the built-in recognition model for analysis, and the written content (e.g., letters, numbers, etc.) is recognized through comparison and classification. In step 807, the audio playback unit provides audio feedback. When the audio processing unit recognizes written content, it converts the written content into audio and outputs it. When it recognizes spoken content, it converts it into a format suitable for playback and broadcasts it.

[0110] Students can write directly on books using handwriting devices, which broadcast their writing instantly to support memorization and pronunciation practice. Teachers use LCD tablets and whiteboards for lessons, and the handwriting devices are broadcast as educational support tools.

[0111] In the embodiment of the present invention, when the processor receives a pressure detection signal transmitted from the pressure sensing unit, it acquires a code point image obtained by the optical recognition unit capturing the code points of the writing medium, performs code point type recognition on the code point image, and if the code point type indicated by the code point image is a coordinate code, it continuously acquires the code point image returned from the optical recognition unit to obtain a code point image sequence, performs coordinate recognition on the code point image sequence to obtain a coordinate sequence, draws a writing trajectory based on the coordinate sequence, calls a preset trajectory recognition model to recognize the writing trajectory and obtain the writing content. As a result, the writing device can instantly recognize the writing trajectory based on the coordinate code in the code point image, analyze the writing trajectory using the built-in recognition model to determine the writing content, instantly recognize the writing content while the user is writing, and instantly provide feedback on the writing content, thereby improving the functionality and practicality of the handwriting device.

[0112] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not limit the technical solutions provided by the embodiments of this application. As technology evolves and new application scenarios emerge, those skilled in the art will see that the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0113] The technical solutions shown in the figures are not limited to the embodiments of this application and may include more or fewer steps than those shown, or may include specific combinations of steps or different steps.

[0114] The embodiments of the apparatus described above are merely schematic, and the units described as separate components may or may not be physically separated. That is, they may be located in one place or distributed across multiple network units. Depending on the actual needs, some or all of these modules may be selected to achieve the objectives of the embodiments of this embodiment.

[0115] All or part of the steps in the methods disclosed above, the functional modules / units in the system, device, etc., may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0116] The terms “first,” “second,” “third,” “fourth,” etc. (if any) in the specification and drawings of this application are terms used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The data used in this manner is interchangeable where appropriate so that the embodiments of this application described herein may be carried out in an order other than that illustrated or described herein. Furthermore, the terms “includes” and “having” and any variations thereof are intended to cover non-exclusive inclusions, for example, of a process, method, system, product, or device including a series of steps or units, which are not limited to the explicitly described steps or units and may include other steps or units not explicitly described or specific to these processes, methods, products, or devices.

[0117] In this application, "at least one" means one or more, and "multiple" means two or more. "and / or" is used to describe the relationship between related objects and indicates that there are three types of relationships. For example, "A and / or B" may indicate three cases: when only A exists, when only B exists, and when A and B exist simultaneously, where A and B may be singular or plural. The symbol " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or a similar expression means any combination of these terms, including any combination of single or multiple terms. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a, b and c", where a, b, and c may be singular or plural.

[0118] In some embodiments provided herein, the disclosed apparatus and methods may be implemented in other ways. For example, the embodiments of the apparatus described above are schematic, and for example, the division of the units described above is merely a division of logical functions, and in actual implementation, there may be other methods of division, such as combining multiple units or components, integrating them into another system, or ignoring or not performing certain features. Also, the interconnections, direct connections or communication connections shown or considered may be indirect connections or communication connections via several interfaces, devices or units, and may be in electrical, mechanical or other forms.

[0119] The units described above as isolated components may or may not be physically separated, and the components displayed as units may or may not be physical components; that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected as needed in order to achieve the objectives of the embodiment described herein.

[0120] Furthermore, each functional unit in each embodiment of the present application may be integrated into a single processing unit, each unit may exist individually in physical form, or two or more units may be integrated into a single unit. The integrated unit may be implemented in hardware form or in the form of a software functional unit.

[0121] The integrated unit may be implemented as a software functional unit and, if sold or used as an independent product, may be stored on a single computer-readable storage medium. Based on this understanding, the technical solutions of the present application may be essentially or contribute to the prior art, or all or part of the technical solutions may be embodied in the form of a computer software product, which is stored on a single storage medium and includes a number of instructions for a single computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application. The storage mediums include various media capable of storing programs, such as USB memory, removable hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] Preferred embodiments of the embodiments of the present application have been described above with reference to the drawings, but this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent substitutions, and improvements made by a person skilled in the art that do not depart from the scope and substance of the embodiments of the present application should also be within the scope of the rights of the embodiments of the present application.

Claims

1. A component of a writing device, The writing device includes a writing medium, the writing device includes a pressure sensing unit, an optical recognition unit, and a processor, and the writing medium has code points printed on it. The pressure sensing unit is used to generate a pressure sensing signal when it detects that the writing device has come into contact with the writing medium. The optical recognition unit is used to collect a code point image of the code point on the writing medium when the pressure sensing unit generates the pressure sensing signal, analyze the code point image to obtain a code point value, and transmit the code point value to the processor. A component of a writing device, characterized in that the processor is used to recognize the code point type for the code point value, and if the code point type is recognized as a coordinate code, to perform coordinate code analysis on the code point value, determine the writing trajectory based on the analysis result, and perform content recognition on the writing trajectory to obtain the written content.

2. The writing device further includes an audio playback unit, The component of the writing device according to claim 1, characterized in that the audio playback unit is used to broadcast the written content after the processor has recognized the written content.

3. The aforementioned processor further, This is used to perform semantic recognition on the aforementioned written content and to obtain the meaning of the written text. If the meaning of the written content indicates that the written content is question data, the processor further: Used to generate response data corresponding to the aforementioned question data, The component of the writing device according to claim 2, wherein the audio playback unit is further used to broadcast response audio corresponding to the response data after the processor has generated the response data.

4. The aforementioned audio playback unit further, Upon receiving a question broadcast command, it is used to broadcast the question audio corresponding to the said question broadcast command. The aforementioned processor further, This system is used to compare the written content with the answer corresponding to the question audio, obtain an answer matching result, and determine the matching result audio based on the answer matching result. The aforementioned audio playback unit further, The component of the writing device according to claim 2, characterized in that the processor is used to broadcast the matching result audio after determining the matching result audio.

5. The component of the writing device according to claim 1, further comprising a display module for displaying the written content after the processor has recognized the written content.

6. The processor, upon recognizing the code point type of the code point value, is used to determine the speaking content based on the code point value if it is determined that the code point type is a speaking code. The component of the writing device according to claim 2, wherein the audio playback unit is further used to broadcast the speaking content after the processor has determined the speaking content.

7. The aforementioned processor further, Based on the image sequence of the aforementioned code point images, the writing angle is analyzed to obtain writing angle data. Based on the aforementioned writing angle data and the aforementioned writing content, the writing posture is determined in real time, and If the result of the aforementioned writing posture determination indicates an incorrect posture, it is used to generate a writing posture correction voice. The component of the writing device according to claim 2, wherein the audio playback unit is further used to broadcast the writing posture correction sound after the processor has generated the writing posture correction sound.

8. The pressure sensing unit includes a writing member and a sensing member, The writing member includes a first end that contacts the writing medium during writing and a second end opposite to the first end, and the second end of the writing member is provided with a conductive silicone rubber pad and a first magnetic element. The sensing member has a sensing end positioned opposite the second end of the writing member, and the sensing end is provided with a second magnetic element and an unconnected first wire and second wire, and the first magnetic element and the second magnetic element are arranged to repel each other magnetically. If the first end of the writing member is not in contact with the writing medium, the conductive silicone rubber pad and the sensing end are separated by the repulsive force between the first magnetic element and the second magnetic element, the first wire and the second wire are not electrically connected, and the pressure sensing signal is not generated. When the first end of the writing member contacts the writing medium, and the contact pressure between the writing member and the writing medium is greater than the repulsive force between the first magnetic element and the second magnetic element, the conductive silicone rubber pad contacts the sensing end, the first wire and the second wire become electrically connected, and the pressure sensing signal is generated, or The writing member includes a first end that contacts the writing medium during writing and a second end opposite to the first end, and a slider is provided at the second end of the writing member. The sensing member includes a pressure sensor positioned opposite the second end of the writing member. If the first end of the writing member is not in contact with the writing medium, the slider slides downward due to gravity, the slider and the pressure sensor separate, and the pressure sensor does not generate the pressure sensing signal. The component of the writing device according to claim 1, characterized in that when the first end of the writing member contacts the writing medium, and the contact pressure between the writing member and the writing medium is greater than the gravity of the writing member, the slider moves upward and contacts the pressure sensor, and the pressure sensor generates the pressure sensing signal.

9. The writing device further includes an audio acquisition unit, and the processor, upon recognizing the code point type of the code point value, determines that the code point type is a recording code, is further used to transmit a recording prompt sound to the audio playback unit and to transmit a recording command to the audio acquisition unit. The audio playback unit is further used to broadcast the recording prompt audio after receiving it. The component of the writing device according to claim 2, wherein the audio acquisition unit is used to receive the recording command and to start recording after the audio playback unit has played the recording prompt sound.

10. A method of recognizing written content, Used in a processor of a writing device which is a writing device in a component of a writing device according to any one of claims 1 to 9, A step of continuously acquiring code point values ​​returned from the optical recognition unit to obtain a code point value sequence, wherein the code point values ​​are obtained by analyzing a code point image obtained when the optical recognition unit photographs the code points on the writing medium. The steps include: recognizing the code point type for the code point value; if the code point type indicated by the code point value is a coordinate code, performing coordinate recognition on the code point value sequence to obtain a coordinate sequence; and drawing a writing trajectory based on the coordinate sequence. A method for recognizing written content, comprising the steps of: calling up a pre-configured trajectory recognition model to recognize the writing trajectory and obtain the written content.

Images