Writing device assembly and written content identification method

Abstract

The embodiment of the invention provides a writing equipment component and a writing content identification method. The assembly comprises a writing device and a writing medium, the writing device comprises a pressure sensing unit, an optical recognition unit and a processor, and code points are printed on the writing medium; the pressure sensing unit generates a pressure sensing signal when sensing that the writing device is in contact with the writing medium; when the pressure sensing unit generates the pressure sensing signal, the optical identification unit collects a code point image of a code point in the writing medium, analyzes the code point image to obtain a code point value, and sends the code point value to the processor; and the processor performs code point type identification on the code point value, analyzes the code point value to determine a writing track when identifying that the code point type is a coordinate code, and performs content identification on the writing track to obtain writing content. When a user writes, the writing content can be recognized in real time, the writing content can be fed back in real time, the learning interactivity is improved, meanwhile, the point-to-read playing function is achieved, and the functionality and practicability of the writing equipment are improved.

Description

Technical Field

[0001] The present application relates to the field of handwriting devices, and in particular to a writing device component and a writing content recognition method. Background Art

[0002] With the rapid development of educational informatization, intelligent educational auxiliary equipment has gradually become an important tool to improve learning efficiency and quality.

[0003] A handwriting device is an input device among intelligent educational auxiliary devices. It allows users to interact with electronic devices (such as computers, tablets, mobile phones, etc.) by handwriting or drawing on writing media.

[0004] At present, handwriting devices only have handwriting functions and are single in function. Therefore, how to improve the functionality of handwriting devices has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose a writing device component and a writing content recognition method, in which the handwriting device instantly recognizes the writing content and broadcasts it when the user is writing, which can provide instant feedback on learning outcomes and increase learning interactivity. It also has a point-reading and playback function, thereby improving the functionality and practicality of the writing device.

[0006] To achieve the above-mentioned object, the first aspect of the present application proposes a writing device assembly, including a writing device and a writing medium, the writing device including a pressure sensing unit, an optical recognition unit and a processor, and a code point is printed on the writing medium; The pressure sensing unit is used to generate a pressure sensing signal when sensing that the writing device contacts the writing medium; The optical recognition unit is used to collect a code point image of a code point in the writing medium when the pressure sensing unit generates a pressure sensing signal, analyze the code point image, obtain a code point value, and send the code point value to the processor; The processor is used to identify the code point type of the code point value. When the code point type is identified as a coordinate code, the code point value is parsed as a coordinate code and a writing trajectory is determined based on the parsing result. The processor is also used to identify the content of the writing trajectory to obtain the writing content.

[0007] In some embodiments, the writing device further includes an audio playback unit. The audio playback unit is used to broadcast the written content after the processor recognizes the written content.

[0008] In some embodiments, the processor is further configured to: Perform semantic recognition on the written content to obtain the written semantics; When the writing semantics indicates that the written content is question data, the processor is further configured to generate answer data corresponding to the question data; The audio playback unit is also used to broadcast the answer audio corresponding to the answer data after the processor generates the answer data.

[0009] In some embodiments, the audio playback unit is further configured to: When receiving a problem broadcast instruction, broadcast the problem audio corresponding to the problem broadcast instruction; The processor is also used to: compare the written content with the answer corresponding to the question audio to obtain an answer comparison result, and determine the comparison result audio according to the answer comparison result; The audio playback unit is also used to play the comparison result audio after the processor determines the comparison result audio.

[0010] In some embodiments, the writing device assembly further includes a display module, the display module being configured to: After the processor recognizes the written content, the written content is displayed.

[0011] In some embodiments, when the processor identifies the code point type of the code point value and determines that the code point type is a point-read code, the processor is further configured to determine the point-read content according to the code point value; The audio playback unit is also used to broadcast the point-to-read content after the processor determines the point-to-read content.

[0012] In some embodiments, the processor is further configured to: Perform writing angle analysis based on the image sequence of the code point image to obtain writing angle data; Real-time writing posture identification based on writing angle data and writing content; When the writing posture identification result is that the posture is wrong, a writing posture correction voice is generated; The audio playback unit is also used to broadcast the writing posture correction voice after the processor generates the writing posture correction voice.

[0013] In some embodiments, the pressure sensing unit includes a writing component and a sensing component; The writing component includes a first end in contact with the writing medium during writing and a second end opposite to the first end; the second end of the writing component is provided with a conductive silicone pad and a first magnetic device; The induction component includes an induction end arranged opposite to the second end of the writing component, the induction end is provided with a second magnetic device and a first wire and a second wire that are not connected, and the first magnetic device and the second magnetic device are arranged to magnetically repel each other; When the first end of the writing component is not in contact with the writing medium, the conductive silicone pad and the sensing end are separated under the repulsive force of the first magnetic device and the second magnetic device, the first wire and the second wire are not conducting, and no pressure sensing signal is generated; When the first end of the writing component contacts the writing medium and the contact pressure between the writing component and the writing medium is greater than the repulsive force between the first magnetic device and the second magnetic device, the conductive silicone pad contacts the sensing end, so that the first wire and the second wire are connected, generating a pressure sensing signal; or, The writing member includes a first end in contact with the writing medium during writing and a second end opposite to the first end; a sliding block is provided at the second end of the writing member; The sensing component includes a pressure sensor disposed opposite to the second end of the writing component; When the first end of the writing component is not in contact with the writing medium, the sliding block of the writing component slides down under the influence of gravity, the sliding block is separated from the pressure sensor, and the pressure sensor does not generate a pressure sensing signal; When the first end of the writing component contacts the writing medium and the contact pressure between the writing component and the writing medium is greater than the gravity of the writing component, the sliding block moves upward to contact the pressure sensor, and the pressure sensor generates a pressure sensing signal.

[0014] In some embodiments, the writing device further includes an audio collection unit, and when the processor identifies the code point type of the code point value and determines that the code point type is a recording code, the processor is further configured to send a recording prompt voice to the audio playback unit, and initiate a recording instruction to the audio collection unit; The audio playback unit is also used to play the recorded prompt voice after receiving the recorded prompt voice; The audio acquisition unit is used to start recording after receiving a recording instruction and the audio playback unit plays the recording prompt voice.

[0015] A second aspect of the present application provides a method for identifying written content, which is used for a processor in a writing device, wherein the writing device is the writing device in the writing device assembly in the first aspect of the present application, and the method comprises: Continuously acquiring code point values ​​returned by the optical recognition unit to obtain a code point value sequence, wherein the code point value is obtained by parsing a code point image obtained by the optical recognition unit when photographing the code point in the writing medium; Perform code point type recognition on the code point value. When the code point type indicated by the code point value is a coordinate code, perform coordinate recognition on the code point value sequence to obtain a coordinate sequence, and draw a writing trajectory according to the coordinate sequence. The preset trajectory recognition model is called to recognize the writing trajectory and obtain the writing content.

[0016] A writing device assembly proposed in the present application includes a writing device and a writing medium, the writing device includes a pressure sensing unit, an optical recognition unit and a processor, and code points are printed on the writing medium; the pressure sensing unit is used to generate a pressure sensing signal when sensing that the writing device contacts the writing medium; the optical recognition unit is used to collect a code point image of the code point in the writing medium when the pressure sensing unit generates a pressure sensing signal, and to parse the code point image to obtain the code point value, and send the code point value to the processor; the processor is used to identify the code point type of the code point value, and when the code point type is identified as a coordinate code, the code point value is parsed as a coordinate code, and a writing trajectory is determined according to the parsing result, and the writing trajectory is content identified to obtain the writing content.

[0017] Through the above method, the pressure sensing unit in the writing device instantly senses the contact with the writing medium, instructs the optical recognition unit to collect the code point image on the writing medium, and parses the code point value from the code point image. When the processor recognizes that the category of the code point value is a coordinate code, the code point value is parsed to determine the writing trajectory, and then the writing content is recognized from the writing trajectory. The writing device can instantly digitize the handwritten content without the need for additional equipment recognition, which can improve recognition efficiency and portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of the architecture of a writing device component provided in an embodiment of the present application; Figure 2 is a structural schematic diagram of a writing device assembly provided in an embodiment of the present application; Figure 3 is another structural schematic diagram of a writing device assembly provided in an embodiment of the present application; Figure 4 is another structural schematic diagram of a writing device assembly provided in an embodiment of the present application; Figure 5 is a structural schematic diagram of a pressure sensing unit provided in an embodiment of the present application; Figure 6 is another structural schematic diagram of a writing device assembly provided in an embodiment of the present application; Figure 7 is a flowchart of a method for identifying written content provided in an embodiment of the present application; Figure 8 It is a schematic diagram of the processing flow of the writing device component provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0021] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0022] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0023] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0025] First, some nouns involved in this application are analyzed: Optical identification (OID) is an infrared optical recognition technology. The biggest difference between it and other optical recognition technologies is that its tiny code points not only have the characteristics of confidentiality and low visual interference, but can also be hidden under the color patterns of printed materials. It is a low-cost solution that can be achieved by using only ordinary printing ink.

[0026] Writing devices are an input and output device in intelligent educational auxiliary equipment. Using the previous point reading code points, users interact and judge with electronic devices (such as computers, tablets, mobile phones, etc.) by handwriting or drawing on writing media (including but not limited to paper). Although traditional point reading pens can realize instant reading of book contents, they lack coordinate recognition and instant feedback and judgment of handwriting results. Handwriting recognition technology focuses on inputting into other electronic devices for digital conversion of handwriting.

[0027] Based on this, an embodiment of the present application provides a writing equipment component, including a writing device and a writing medium, the writing device including a pressure sensing unit, an optical recognition unit and a processor, and code points are printed on the writing medium; the pressure sensing unit is used to generate a pressure sensing signal when it senses that the writing device contacts the writing medium; the optical recognition unit is used to collect a code point image of the code point in the writing medium when the pressure sensing unit generates a pressure sensing signal, and to parse the code point image to obtain the code point value, and to send the code point value to the processor; the processor is used to identify the code point type of the code point value, and when the code point type is identified as a coordinate code, the code point value is parsed as a coordinate code and a writing trajectory is determined according to the parsing result, and the writing trajectory is content identified to obtain the writing content.

[0028] Through the above method, the pressure sensing unit in the writing device instantly senses the contact with the writing medium, instructs the optical recognition unit to collect the code point image on the writing medium, and parses the code point value from the code point image. When the processor recognizes that the category of the code point value is a coordinate code, the code point value is parsed to determine the writing trajectory, and then the writing content is recognized from the writing trajectory. The writing device can instantly digitize the handwritten content without the need for additional equipment recognition, which can improve recognition efficiency and portability.

[0029] The writing device component and writing content recognition method provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the writing device component in the embodiments of the present application is described.

[0030] The embodiments of the present application can process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0031] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0032] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0033] It should be noted that in each specific implementation of the present application, personal information is not involved. The user's written content is recognized and broadcast by the recognition model built into the handwriting device, and there is no need to judge through the server, so the user's personal information security is guaranteed. When it comes to the need to process data related to the user's identity or characteristics through the server based on user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0034] See also Figure 1 ,like Figure 1 What is shown is a schematic diagram of the architecture of the writing device component 1 of an embodiment of the present application, including a writing device 11 and a writing medium 12. The writing device 11 can collect a code point image of a code point in the writing medium 12 when in contact with the writing medium 12. When the code point value in the code point image is identified as a coordinate code, the code point value is parsed to determine a writing trajectory, and the writing content is identified from the writing trajectory.

[0035] The writing device assembly provided in the embodiment of the present application is described below, and the method is as follows: See also Figure 2 ,like Figure 2 The figure is a schematic diagram of the structure of a writing device assembly 1 provided in an embodiment of the present application, comprising: a writing device 11 and a writing medium 12, the writing device 11 comprising a pressure sensing unit 111, an optical recognition unit 112 and a processor 113, and the writing medium 12 is printed with code points; The pressure sensing unit 111 is used to generate a pressure sensing signal when sensing that the writing device contacts the writing medium 12; The optical recognition unit 112 is used to collect the code point image of the code point in the writing medium when the pressure sensing unit 111 generates the pressure sensing signal, analyze the code point image, obtain the code point value, and send the code point value to the processor; The processor 113 is used to identify the code point type of the code point value. When the code point type is identified as a coordinate code, the code point value is parsed into a coordinate code, and a writing trajectory is determined according to the parsing result, and the writing trajectory is identified to obtain the writing content.

[0036] The writing device 11 may be a tool or device for writing, and a user may write text or draw pictures through the writing device 11 .

[0037] The writing medium 12 may be a book with OID code points, a liquid crystal handwriting board, a whiteboard, etc. The user can write on the writing medium 12 through the writing device 11 to generate text or graphics with code points.

[0038] The pressure sensing unit 111 can respond to pressure changes and output an electrical signal proportional to the pressure change. The current use state of the writing device 11 can be determined based on the change of the electrical signal. That is, the writing device 11 can detect the current use state on the writing medium 12 through the pressure detector. The use state includes a pressure state and a release state, wherein the pressure state is the state when the user uses the writing device 11 to write on the writing medium 12. For example, it can be a state in which pressure is generated on the writing component of the writing device 11 when it is detected to be in contact with the writing medium 12. The pressure sensing unit 111 can generate a pressure sensing signal to indicate the pressure state. The release state can be a state in which the pressure on the writing component of the writing device 11 is released, that is, the writing component leaves the writing medium 12, or the writing component is in contact with the writing medium 12 but is not subjected to pressure.

[0039] The optical recognition unit 112 may be a camera integrated with optical recognition technology, which can recognize and process image data to identify specific patterns or objects. The writing device 11 may scan the writing medium 12 through the optical recognition unit 112 to obtain an image. The image acquisition by the optical recognition unit 112 may be triggered by a pressure sensing signal, that is, the pressure sensing signal indicates that the user is writing, and the optical recognition unit 112 may instantly acquire an image on the writing medium 12, that is, capture the light refraction changes on the surface of the writing medium 12, which are usually caused by tiny code points (such as OID codes) printed on paper. The optical recognition unit 112 may generate a code point image based on the optical information, sense the tiny code points on the code point image, and convert the code points into electrical signals through image recognition technology. The optical recognition unit 112 may decode the electrical signals to identify the code point values ​​of the code points in the code point image, and may send the code point values ​​to the processor 113.

[0040] The processor 113 can be a component that processes various data from the writing process. The processor 113 can identify the code point type for the code point value from the optical recognition unit 112, and can perform different operations for different code point types. If the code point type of the code point value is a coordinate code, the processor 113 can parse the coordinate code to obtain digital coordinates, which represent the position of the writing track on the writing medium 12, that is, the processor 113 can determine the X and Y coordinate values ​​of each coordinate code based on the coordinate code. With the coordinate values, the processor 113 can connect these points to form a writing track. The processor 113 can also be integrated with a machine learning model or a deep learning model. These recognition models can learn from a large number of handwriting samples and can identify the handwritten content corresponding to the writing track. For the writing track determined by the processor 113, the writing track can be input into the learning model for content recognition to obtain the user's writing content on the writing medium 12.

[0041] The recognition model can be selected according to specific application scenarios and requirements, which may include recurrent neural networks (RNN), long short-term memory networks (LSTM), convolutional neural networks (CNN) combined with RNN (such as CRNN), Transformer, etc. The recognition model can be obtained by training the initial model with sample features and labels corresponding to the sample features as sample data. The correspondence between the sample features and the labels can be manually annotated or downloaded from the Internet. The sample data is input into the initial model, and the parameters of the model are adjusted through the back propagation algorithm so that the model can accurately map the handwriting coordinate data to the written content. Furthermore, the recognition model can be retrained based on the handwriting of the current user and user feedback to improve the personalization of the recognition model.

[0042] In the writing device assembly provided in the embodiment of the present application, the pressure sensing unit in the writing device instantly senses the contact with the writing medium, instructs the optical recognition unit to collect the code point image on the writing medium, and parses the code point value from the code point image. When the processor recognizes that the category of the code point value is a coordinate code, the code point value is parsed to determine the writing trajectory, and then the writing content is recognized from the writing trajectory. The writing device can instantly digitize the handwritten content without the need for additional equipment recognition, thereby improving recognition efficiency and portability.

[0043] For a possible implementation, see Figure 3 Another structural diagram of the writing device assembly 1 shown in FIG. 1 , the writing device 11 further includes an audio playback unit 114 , and the audio playback unit 114 is used to broadcast the writing content after the processor recognizes the writing content.

[0044] The audio playback unit 114 can convert the written content into an audio signal after the written content is determined. For example, the application of TTS technology can pre-process the input written content, including text normalization, punctuation processing, conversion of numbers and dates, etc., to meet the requirements of speech synthesis. Then, a language model can be used to analyze the text and determine the most suitable voice output. Finally, the audio playback unit 114 will broadcast the audio signal through a speaker or other audio output device. The written content can be played offline instantly to improve the interactive effect.

[0045] In a feasible implementation manner, the processor 113 is further configured to: Perform semantic recognition on the written content to obtain the written semantics; When the writing semantics indicates that the written content is question data, the processor 113 is further configured to generate answer data corresponding to the question data; The audio playback unit 114 is further configured to broadcast the answer audio corresponding to the answer data after the processor generates the answer data.

[0046] Semantic recognition can refer to understanding and interpreting the deep meaning and intention contained in natural language text. It can involve natural language processing (NLP) technology, including grammatical analysis, semantic analysis and contextual understanding, that is, not only focusing on the meaning of text words, but also understanding the meaning of these words in sentences and chapters. This means that semantic recognition requires semantic analysis and ambiguity elimination at the text, vocabulary, syntax, morphology, and chapter levels, as well as corresponding meaning reorganization.

[0047] Writing semantics can be the deep meaning and intention contained in the writing content. It goes beyond the surface form of the text and involves the understanding and interpretation of the writing content. For example, understanding the sentence structure, combining the previous and next context interpretation, sentiment analysis and context to understand and interpret the writing content.

[0048] Question data may refer to questions expressed in written form. When determining that the written content is question data based on the writing semantics, the processor 113 may determine the type of question, such as fact query, suggestion request, operation instruction, etc., in order to select an appropriate answer generation strategy.

[0049] The answer data may be the standard answer corresponding to the question data. The processor 113 may retrieve the relevant information corresponding to the question data in the internal knowledge base or by accessing external databases and Internet resources, and determine the standard answer through the relevant information. It may also be obtained by the processing unit using the built-in mathematical engine or calculator function to perform logical analysis and calculation on the extracted calculation expression. Exemplarily, if the written content is "1+1", the written semantic recognition may be a mathematical calculation of adding 1 to 1, and the processor 113 may generate the answer "2" corresponding to the mathematical calculation. The audio playback unit 114 may perform text normalization, punctuation processing, conversion of numbers and dates, etc. on the answer data to meet the requirements of speech synthesis, and then perform text analysis through the language model to determine the most appropriate answer audio, and broadcast the answer audio. Students can not only see their scores, but also get feedback by listening, which increases interactivity and learning interest. The content of the broadcast may include "your calculation result is correct" or "there is a small error here, please check your calculation process again", etc. The specific content of the broadcast will be determined according to the actual evaluation results and design requirements. In this way, the system not only provides immediate feedback but also enhances the learning experience, making the learning process more dynamic and interactive.

[0050] In a feasible implementation manner, the audio playback unit 114 is further configured to: When receiving a problem broadcast instruction, broadcast the problem audio corresponding to the problem broadcast instruction; The processor 113 is further configured to: compare the written content with the answer corresponding to the question audio to obtain an answer comparison result, and determine a comparison result audio according to the answer comparison result; The audio playback unit 114 is further configured to broadcast the comparison result audio after the processor 113 determines the comparison result audio.

[0051] The question broadcasting instruction may be an instruction to instruct the writing device 11 to broadcast the question, and the question broadcasting instruction may be determined by the writing device 11 scanning the code point on the writing medium 12, or may be generated by the user through the control on the writing device 11, or may be sent by the user to the writing device 11 through other devices. Accordingly, the question broadcasting instruction may be an instruction from the user to the writing device 11 before writing, that is, the current writing content of the user may be the user's answer written based on the broadcasted question.

[0052] The question audio can be an audio signal that can be directly broadcast by a speaker. The question audio can be directly stored in the writing device 11, or obtained by the writing device 11 from other devices. The writing device can also store the question content and then convert the question content into question audio after receiving the question broadcast instruction.

[0053] The answer corresponding to the question audio may be a standard answer to the question audio, and the standard answer may be retrieved by the writing device 11, that is, the writing device 11 may pre-save the correspondence between the question audio and the standard answer, or it may be a standard answer directly generated by the writing device 11 for the question content corresponding to the question audio.

[0054] The answer comparison result can be the conclusion drawn by the processor 113 after comparing the user's written answer with the correct answer. This result can be a simple match or mismatch, or a detailed similarity score. The processor 113 can perform text preprocessing on the user's answer indicated by the written content and the standard answer, and use a specific algorithm to compare the similarity of the two texts. This can include a simple string matching algorithm, an edit distance algorithm (such as Levenshtein distance), or a more complex natural language processing technology, such as semantic similarity analysis. The processor 113 interprets the comparison result based on the result of the comparison algorithm. If it is a simple match / mismatch result, the processor will directly conclude whether the answer is correct. If it is a similarity score, the processor may determine whether the answer is close enough to the correct answer based on a preset threshold.

[0055] The comparison result audio may be an audio signal that is broadcasted to the user by the audio playback unit 114 after the processor 113 completes the comparison between the written content and the correct answer and converts the comparison result into voice information. The processor 113 may convert the comparison result into easy-to-understand text information. For example, if the comparison result shows that the user's answer is completely consistent with the correct answer, the comparison result audio may be "Your answer is completely correct"; if the answer is partially correct, the comparison result audio may be "Your answer is partially correct, but there are some areas that need improvement"; if the answer is incorrect, the comparison result audio may be "Your answer is incorrect, please refer to the correct answer below".

[0056] For a possible implementation, see Figure 4 Another structural schematic diagram of the writing device assembly shown, the writing device assembly 1 further includes a display module 13, and the display module 13 is used for: After the processor 113 recognizes the written content, the written content is displayed.

[0057] The display module 13 can be a component that displays the written content on the display screen. The display module 13 can format the written content, including adjusting the font size, type, color and layout, etc., to ensure that the content is clear and readable on the screen. Among them, the display module 13 can be configured on the writing device 11, that is, after the processor 113 recognizes the written content, it can transmit the written content to the display module 13 through internal wiring, and the display module 13 then displays the written content on the screen. The display module 13 can also be a device other than the writing device 11. After the processor 113 in the writing device 11 recognizes the written content, it can send the written content to the display module 13 through wireless communication, and the display module 13 then displays the written content. Figure 4 Take the example that the display module 13 is a device other than the writing device 11 .

[0058] In a feasible implementation manner, when the processor 113 performs code point type identification on the code point value and determines that the code point type is a point reading code, the processor 113 is further configured to determine the point reading content according to the code point value; The audio playback unit 114 is further configured to broadcast the content after the processor 113 determines the content.

[0059] The point reading code may be a code point arranged according to a certain rule on the writing medium 12, and each point reading code represents a specific information or instruction. When the processor 113 identifies the code point type of the code point value, it may parse whether the code point value is a point reading code. After identifying the point reading code, the processor 113 may search for the point reading content corresponding to the information indicated by the point reading code in the database inside the writing device 11 or in the database connected to the external server. The point reading content in the database may include multimedia information such as text, audio, and video. After obtaining the point reading content corresponding to the point reading code from the database, the processor 113 sends the generated voice data to the audio playback unit 114. The audio playback unit 114 may perform necessary processing, such as decoding, format conversion, etc., to convert it into a format suitable for playback, and then play the converted data through a speaker or other audio output device so that the user can hear the point reading content.

[0060] In a feasible implementation manner, the processor 113 is further configured to: Perform writing angle analysis based on the image sequence of the code point image to obtain writing angle data; Real-time writing posture identification based on writing angle data and writing content; When the writing posture identification result is that the posture is wrong, a writing posture correction voice is generated; The audio playback unit 114 is further configured to broadcast the writing posture correction voice after the processor generates the writing posture correction voice.

[0061] The image sequence may be a sequence of still images arranged in time order, which may represent a certain time sequence or a process image of a dynamic process. In the embodiment of the present application, the image sequence may be an image sequence of writing actions captured continuously by the optical recognition unit 112 .

[0062] The writing angle data may refer to the angle information between the writing component of the writing device 11 and the writing medium 12 during the writing process of the user. The image sequence may include features related to the writing angle, such as the position of the writing component, the direction of the stroke, and the angle between the stroke and the writing medium, and the writing angle may be calculated based on the position change of the writing component. For example, when the writing device 11 is perpendicular to the writing medium 12, the writing head of the writing component, such as the pen tip, is located in the middle of the stroke, and the deepest point of the pen mark is at the same distance from the edge. When the writing device 11 is not perpendicular to the writing medium 12, the position of the pen tip is biased toward one side of the stroke, and the deepest point of the pen mark is at an unequal distance from the edge. By measuring and counting this ratio, the writing angle of the stroke can be characterized. The pen tip angle changes in the image sequence can be analyzed to determine the dynamic changes of the writing angle during the writing process.

[0063] The result of writing posture discrimination may refer to the conclusion on whether the writing posture is correct or not obtained by analyzing various data in the writing process. The writing angle in the writing angle data may be compared with the preset standard angle to determine whether the writing posture is correct. If the angle exceeds the normal range, it may be considered that there is a problem with the writing posture. The standard angle may be based on the best practices of ergonomics, medical research and writing habits, and may be used to reduce hand and wrist pressure. The processor 113 may also discriminate the writing posture in combination with the writing content, such as writing coherence, stroke order, etc. In writing coherence, if it is determined that the writing content is coherent and fluent, it may indicate that the writing posture is relatively natural and comfortable. If there are frequent pauses, corrections or sudden changes in handwriting in the writing content, it may be determined that there is a problem with the writing posture. In the stroke order, the direction and order of the strokes in the writing content may reflect the trajectory of the pen tip movement when the user is writing. If the stroke order is chaotic or the direction is wrong, it may be determined that it is caused by improper writing posture.

[0064] The writing posture correction voice may be an audio signal for guiding and correcting the user's writing posture through voice. When the processor 113 determines that the writing angle in the writing angle data exceeds the normal range, or the writing content is incoherent or the stroke order is incorrect, it may be determined that the user's posture is wrong, and a text for correcting the user's writing posture may be generated. The correction text may include the following content: Point out specific issues: For example, "Your pen tip is at too sharp an angle to the paper, which may cause hand fatigue." Provide guidance on proper posture: For example, “Try to keep the tip of your pen at a 45-degree angle to the paper.” Suggested exercises or adjustments: For example, "Try relaxing your pen hand so that the pen rests naturally between your index finger and thumb." Encouraging language: For example, "Maintaining a correct posture can improve writing efficiency and prevent hand pain.

[0065] The processor 113 can convert the correction text into writing posture correction speech using text-to-speech technology and send it to the audio playback unit 114. The audio playback unit 114 receives the writing posture correction speech and controls the speaker or other audio output device to play the writing posture correction speech. The user can then hear guidance on how to correct the writing posture.

[0066] In addition to analyzing the writing angle through image sequences, the writing angle can also be analyzed directly based on the code points. For example, the coordinate code of the code point image can include not only coordinates but also angle information. The assembly position of the optical recognition unit 112 on the writing device 11 is fixed. As the writing device 11 rotates, the image captured by the optical recognition unit 112 is also different. Accordingly, the arrangement of the code points captured at different shooting angles will also change. The coordinates of the handwriting and the shooting angle can be determined based on the captured arrangement. For example, if the image is taken from a bird's-eye view, the code points may present a specific arrangement pattern, while when the image is taken from an upward angle, the arrangement pattern will be different. By analyzing these arrangement patterns, the shooting angle can be inferred, that is, the direction of the code point relative to the camera, which can also be called the angle of the writing component of the writing device 11 relative to the writing medium 12. Accordingly, the difference between the shooting angle and the standard angle can be used to determine whether the user's writing posture is wrong.

[0067] For a possible implementation, see Figure 5 A schematic structural diagram of a pressure sensing unit is shown, wherein the pressure sensing unit 111 includes a writing component 1111 and a sensing component 1112; The writing component 1111 includes a first end that contacts the writing medium during writing and a second end opposite to the first end; the second end of the writing component is provided with a conductive silicone pad and a first magnetic device; The induction component 1112 includes an induction end arranged opposite to the second end of the writing component 1111, the induction end is provided with a second magnetic device and a first wire and a second wire that are not connected, and the first magnetic device and the second magnetic device are arranged to magnetically repel each other; When the first end of the writing component 1111 is not in contact with the writing medium, the conductive silicone pad and the sensing end are separated by the repulsive force of the first magnetic device and the second magnetic device, the first wire and the second wire are not conducting, and no pressure sensing signal is generated; When the first end of the writing component 1111 contacts the writing medium and the contact pressure between the writing component and the writing medium is greater than the repulsive force between the first magnetic device and the second magnetic device, the conductive silicone pad contacts the sensing end, so that the first wire and the second wire are connected, generating a pressure sensing signal; or, The writing member includes a first end in contact with the writing medium during writing and a second end opposite to the first end; a sliding block is provided at the second end of the writing member; The sensing component includes a pressure sensor disposed opposite to the second end of the writing component; When the first end of the writing component is not in contact with the writing medium, the sliding block of the writing component slides down under the influence of gravity, the sliding block is separated from the pressure sensor, and the pressure sensor does not generate a pressure sensing signal; When the first end of the writing component contacts the writing medium and the contact pressure between the writing component and the writing medium is greater than the gravity of the writing component, the sliding block moves upward to contact the pressure sensor, and the pressure sensor generates a pressure sensing signal.

[0068] The pressure sensing unit 111 is composed of a writing component 1111 and a sensing component 1112. The writing component 1111 is designed with two ends, the first end is used to contact the writing medium, and the second end is opposite to the sensing component 1112. A conductive silicone pad and a first magnetic device are installed at the second end of the writing component 1111. The conductive silicone pad can be a silicone material that realizes the conductive function by doping conductive particles (such as metal powder or carbon black). The conductive silicone pad contains evenly dispersed conductive particles inside, and these particles form a conductive path in the silicone pad. When the conductive silicone pad is not under pressure, the contact between these conductive particles is disconnected, so no closed circuit is formed. When pressure is applied, the conductive silicone pad is deformed, and the conductive particles are squeezed more tightly, thereby forming a continuous conductive path. When the first end of the writing component 1111 contacts the writing medium and a certain pressure is applied, the conductive silicone pad can be deformed to form a conductive path, and contact the sensing end of the sensing component 1112, thereby realizing the connection of the circuit.

[0069] The sensing end of the sensing component 1112 is equipped with a second magnetic device and two unconnected wires, namely, a first wire and a second wire. The first magnetic device and the second magnetic device can be designed to repel each other magnetically. This repulsive force ensures that in the absence of writing pressure, the first magnetic device drives the conductive silicone pad to remain separated from the sensing end. At this time, the contact between the conductive particles inside the conductive silicone pad is disconnected, so that the first wire and the second wire are not conductive. In the non-contact state, due to the repulsive force between the first magnetic device and the second magnetic device, the conductive silicone pad remains at a position away from the sensing end, and there is no connection between the first wire and the second wire, so no current will pass, and the sensing component 1112 is not triggered. When the user is writing, the first end of the writing component 1111 is subjected to the pressure of the writing medium. When the first end is pressed against the inside of the writing component 1111 by the pressure, that is, the applied pressure exceeds the repulsive force between the first magnetic device and the second magnetic device, the conductive silicone pad overcomes the magnetic repulsion and contacts the sensing end and deforms. At this time, the conductive silicone pad is connected to the first wire and the second wire, and the current can pass. At this time, the sensing component 1112 is triggered to generate a pressure sensing signal.

[0070] The conductive silicone pad works in the pressure sensing unit through its conductivity and elasticity, converting the pressure changes during writing or drawing into electrical signals, thereby realizing the pressure sensing function. This design enables the pressure sensing unit to accurately detect and respond to writing pressure, instantly sensing the user's writing.

[0071] In addition to using magnetic devices to control the contact between the conductive silicone pad and the sensing end, the writing state of the writing component 1111 can also be detected by a pressure sensor. When the first end of the writing component 1111 is not in contact with the writing medium, the writing component 1111 slides downward due to gravity, causing the sliding block set at the second end to separate from the pressure sensor in the sensing component 1112. In this state, the pressure sensor does not detect pressure, so no pressure sensing signal is generated.

[0072] However, when the first end of the writing component 1111 contacts the writing medium and the pressure applied is large enough to exceed the weight of the writing component 1111, the second end of the writing component 1111 rises due to the pressure, causing the sliding block to contact the pressure sensor. At this time, the pressure sensor is affected by the pressure of the sliding block, thereby generating a pressure sensing signal.

[0073] For a possible implementation, see Figure 6 Another structural schematic diagram of the writing device assembly 1 shown, the writing device 11 also includes an audio acquisition unit 115, when the processor 113 performs code point type identification on the code point value and determines that the code point type is a recording code, the processor 113 is also used to send a recording prompt voice to the audio playback unit 114, and initiate a recording instruction to the audio acquisition unit 115; The audio playback unit 114 is also used to play the recorded prompt voice after receiving the recorded prompt voice; The audio collection unit 115 is used to start recording after receiving a recording instruction and after the audio playback unit plays the recording prompt voice.

[0074] In this embodiment, a recording code may also be provided on the writing medium 12. The recording code may be a code point that indicates that a recording operation is required. When the code point image includes a recording code, that is, when the processor 113 identifies the code point type as a recording code from the code point value, it can be determined that the user needs to record.

[0075] The processor 113 may send a recording prompt voice to the audio playback unit 114. The purpose of this voice is to inform the user that the device is about to start recording, so that the user knows when to start speaking. After receiving this recording prompt voice, the audio playback unit 114 will play it immediately so that the user can hear the prompt of the upcoming recording.

[0076] While the audio playback unit 114 is broadcasting the recording prompt voice, the processor 113 can also send a recording instruction to the audio acquisition unit 115. This instruction tells the audio acquisition unit 115 to prepare to start recording. Once the audio playback unit 114 completes the broadcast of the recording prompt voice, the audio acquisition unit 115 can start the recording function and start capturing audio data. This can ensure the smoothness and accuracy of the recording process, avoid any delay at the start of recording, and ensure the integrity and clarity of the recorded content.

[0077] In one example, the optical recognition unit may be an infrared camera, the processor may be a main control processor, and the audio playback unit may be a speaker. The OID infrared optical camera can capture the user's writing and reading actions on books, LCD handwriting tablets or whiteboards, that is, it obtains the code point image of the writing medium based on the user's writing operation, and analyzes the code point image to obtain the code point value.

[0078] The main control processor is a built-in high-performance processor that runs the local handwriting recognition algorithm and point reading control program to achieve fast data processing and response. When the code point type of the code point value is a point reading code, the point reading content corresponding to the point reading code value can be obtained from the local database. When the code point type of the code point value is a coordinate code, the writing trajectory can be determined from the value of the coordinate code, and the built-in preset trajectory recognition model can be called to parse the writing trajectory to obtain the user's writing content.

[0079] The speaker is used to announce the point reading content and the recognized written content, and also supports recording and follow-up reading. After obtaining the recognition result, the corresponding result and some system prompt sounds, recordings, etc. are announced.

[0080] The handwriting device can provide instant feedback on the written content by recognizing the written content and announcing it when the user is writing, thereby increasing the interactivity of learning and improving the functionality and practicality of the handwriting device.

[0081] The writing device component is described above, and the writing content recognition method executed by the processor in the writing device component is described below.

[0082] See also Figure 7 ,like Figure 7 The figure is a flow chart of the writing content recognition method in the embodiment of the present application, including: Step 701: continuously obtain code point values ​​returned by the optical recognition unit to obtain a code point value sequence, wherein the code point value is obtained by analyzing a code point image captured by the optical recognition unit on the code point in the writing medium.

[0083] The code point value may be a numerical value or identifier indicated by a data unit composed of a dot matrix consisting of multiple code points. For example, every 6×6=36 points constitute a minimum data unit. These dot matrices can represent a large number of arrangements for representing numbers.

[0084] The code point value sequence may be a sequence of code point values ​​in a series of code point images continuously photographed by the optical recognition unit, that is, it includes code points at different time points, and code point values ​​at different states or positions may be obtained.

[0085] The code point image can be an image of the OID code point, that is, the image collected by the optical recognition unit from the writing medium with the OID code point can be called the code point image. When the optical recognition unit receives the pressure sensing signal sent by the pressure sensing unit, the code point image can be obtained by photographing the code point in the writing medium.

[0086] The writing medium can be a book with OID code points, a liquid crystal writing tablet, and other media.

[0087] The optical recognition unit may be a camera integrated with optical recognition technology, which is responsible for collecting images.

[0088] The pressure sensing unit may be a unit in the writing device that is used to detect the pressure change when the writing component of the writing device contacts the writing medium.

[0089] The pressure sensing signal may be a signal indicating that the user is writing with the writing device. When the pressure applied to the writing component of the writing device reaches a preset value, a pressure sensing signal may be generated to indicate that the writing device is currently writing.

[0090] Step 702: perform code point type identification on the code point value. When the code point type indicated by the code point value is a coordinate code, perform coordinate identification on the code point value sequence to obtain a coordinate sequence, and draw a writing trajectory according to the coordinate sequence.

[0091] The code point type can be a specific coded function type composed of multiple OID code points, and the arrangement of code points of different function types is different.

[0092] The coordinate code can be a data unit that can be used to determine the coordinate information in a dot matrix composed of multiple code points. These dot matrices can represent a large number of arrangements to represent digital or coordinate information. For example, the XY axis coordinates on the writing medium, these code points usually have unique visual features, so that they can be distinguished from the background. The processing unit can determine the specific coordinates of the writing operation based on the corresponding coordinate information in these code points to determine the handwriting coordinates. These coordinate information can be pre-defined during the OID encoding process, which allows the system to accurately map the handwriting to the physical location on the writing medium.

[0093] The coordinate sequence may be a set of a series of coordinates, that is, a set of coordinates corresponding to each code point value in the code point value sequence. The processor may identify a dot matrix in each code point value in the code point value sequence, analyze the coordinates corresponding to the dot matrix based on an algorithm, and arrange the coordinates in sequence based on the collection order of the code point values ​​in the code point value sequence to form the coordinate sequence.

[0094] The writing trajectory can be a combination of strokes drawn based on a coordinate sequence simulation, which can simulate the stroke sequence of the user during the writing process, and can also be understood as a simulated movement path of the writing component on the writing medium.

[0095] Step 703: Call a preset trajectory recognition model to recognize the writing trajectory to obtain the writing content.

[0096] The preset trajectory recognition model can be a recognition model built into the writing device for recognizing the user's writing content. The recognition model can be selected according to specific application scenarios and requirements, and can include a recurrent neural network (RNN), a long short-term memory network (LSTM), a combination of a convolutional neural network (CNN) and RNN (such as CRNN), a Transformer, etc. The recognition model can be obtained by training the initial model using sample features and labels corresponding to the sample features as sample data. The correspondence between the sample features and the labels can be manually annotated or downloaded from the Internet. The sample data is input into the initial model, and the parameters of the model are adjusted through the back propagation algorithm so that the model can accurately map the writing trajectory to the writing content. Furthermore, the recognition model can be retrained based on the current user's handwriting and user feedback to improve the personalization of the recognition model.

[0097] In one possible way, the code point type in the code point value can also be a point-reading code, which identifies different contents through different combinations of codes. These codes are usually very small, hidden in the page, and difficult to detect with the naked eye. They can be identified by OID. The writing medium in the embodiment of the present application can be equipped with point-reading codes arranged according to a certain pattern, and each point-reading code represents a specific information or instruction. After obtaining the code point value, the processor can first identify whether the code point value is a point-reading code. After identifying the point-reading code, the processor searches for the point-reading content corresponding to the information indicated by the point-reading code in the local database or in a database connected to an external server. The point-reading content in the local database may include multimedia information such as text, audio, and video. After obtaining the point-reading content corresponding to the point-reading code from the local database, necessary processing will be performed, such as decoding, format conversion, etc., to convert it into a format suitable for playback, and then the audio playback unit will broadcast it.

[0098] In one possible manner, the processor may also save the written content in a local database; and generate the user's learning achievements in the local database according to the written content.

[0099] The learning outcome can be determined by the question and answer results indicated by the written content, that is, the written content includes questions and answers, and the processing unit can determine the writing accuracy of the user based on the relevance of the answers to the questions, and the writing accuracy is the learning outcome of the user. Accordingly, the processing unit can distinguish the question content and the answer content in the written content.

[0100] The learning outcome can also be the user's learning progress, that is, the writing content corresponds to the writing time. In the local database, the form of the writing content can be the actual content written by the user, which can be text, image or other format data, or code point image plus text. The writing time is a timestamp for recording the user's writing. The processing unit can analyze the user's learning progress based on the user's writing content and writing time. Exemplarily, a writing template can be saved in the local database, and the user can copy the content of the writing template. The processing unit can determine the user's writing progress in the writing template by identifying the writing content, and can also draw a line chart of the user's writing in combination with the writing time to reflect the difference in the amount of writing in each period of time. The analysis method can be to use a machine learning algorithm to train the model and infer the user's learning progress based on the user's writing content. Simple statistical methods can also be used, such as calculating the user's daily writing volume, weekly learning time, etc., to evaluate their learning progress.

[0101] In one example, the processing flow of the writing device component can refer to Figure 8The handwriting device broadcasting process diagram shown in step 801, the optical recognition unit obtains the code point image. When the user writes or reads on the writing medium, the optical recognition unit scans the writing medium in real time to form a code point image. Step 802, the optical recognition unit identifies the code point value from the code point image and sends the code point value to the processor. The optical recognition unit can pre-process the code point image to obtain a pre-processed image, that is, perform pre-processing operations such as denoising and contrast enhancement on the captured code point image, and then identify the code point value of the pre-processed image to improve the recognition accuracy. Step 803, the processor identifies whether the code point type is a coordinate code for the code point value. If yes, step 805 is executed, otherwise step 804 is executed, that is, the code point value sent by the optical recognition unit to the processor can be the value of the coordinate code or the value of the point reading code. Step 804, identify the point reading content from the local database according to the point reading code. The local database pre-stores the corresponding relationship between the point reading code and the point reading content. The local database can be updated regularly or connected to the database of an external server. Step 805, determine the writing trajectory according to the coordinate code. Based on the coordinate code, coordinate values ​​are determined one by one, and the writing trajectory is formed based on the coordinate values. Step 806, call the preset trajectory recognition model to recognize the writing trajectory and obtain the writing content. The writing trajectory is input into the built-in recognition model for analysis, and the writing content (such as letters, numbers, etc.) is recognized through comparison and classification. Step 807, audio playback unit voice feedback, when the writing content is recognized, the audio processing unit converts the writing content into voice output, and when the point reading content is recognized, it is converted into a format suitable for playback and then broadcast.

[0102] Students can use handwriting devices to write directly on books, and handwriting devices can broadcast in real time to assist memory and pronunciation practice. Teachers use LCD handwriting boards or whiteboards to teach, and handwriting devices are used as auxiliary teaching tools to broadcast.

[0103] In the embodiment of the present application, when the processor receives the pressure sensing signal sent by the pressure sensing unit, the processor obtains the code point image obtained by the optical recognition unit shooting the code point in the writing medium; performs code point type recognition on the code point image, and when the code point type indicated by the code point image is a coordinate code, the processor continuously obtains the code point image returned by the optical recognition unit to obtain a code point image sequence; performs coordinate recognition on the code point image sequence to obtain a coordinate sequence, and draws a writing trajectory according to the coordinate sequence; and calls a preset trajectory recognition model to recognize the writing trajectory to obtain the writing content. In the above manner, the writing device can instantly recognize the writing trajectory according to the coordinate code in the code point image, and parse the writing trajectory through the built-in recognition model to determine the writing content. The writing content can be instantly recognized when the user is writing, and the writing content can be instantly fed back, thereby improving the functionality and practicality of the handwriting device.

[0104] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0105] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0106] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0108] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0109] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0111] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0114] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A writing device assembly, characterized in that: It includes a writing device and a writing medium, wherein the writing device includes a pressure sensing unit, an optical recognition unit and a processor, and the writing medium is printed with code points; The pressure sensing unit is used to generate a pressure sensing signal when sensing that the writing device contacts the writing medium; The optical recognition unit is used to collect a code point image of the code point in the writing medium when the pressure sensing unit generates the pressure sensing signal, analyze the code point image, obtain a code point value, and send the code point value to the processor; The processor is used to identify the code point type of the code point value, and when the code point type is identified as a coordinate code, parse the code point value into a coordinate code, determine the writing trajectory based on the parsing result, and identify the content of the writing trajectory to obtain the writing content.

2. The writing instrument assembly according to claim 1, characterized in that The writing device also includes an audio playback unit, The audio playback unit is used to broadcast the written content after the processor recognizes the written content.

3. The writing instrument assembly according to claim 2, characterized in that The processor is further configured to: Performing semantic recognition on the written content to obtain the written semantics; When the writing semantics indicates that the writing content is question data, the processor is further configured to generate answer data corresponding to the question data; The audio playback unit is further used to broadcast the answer audio corresponding to the answer data after the processor generates the answer data.

4. The writing instrument assembly according to claim 2, characterized in that The audio playback unit is also used for: Upon receiving a problem broadcast instruction, broadcasting the problem audio corresponding to the problem broadcast instruction; The processor is further configured to: compare the written content with the answer corresponding to the question audio to obtain an answer comparison result, and determine a comparison result audio according to the answer comparison result; The audio playback unit is further configured to broadcast the comparison result audio after the processor determines the comparison result audio.

5. The writing instrument assembly according to claim 1, wherein: The writing device assembly further comprises a display module, wherein the display module is used for: After the processor recognizes the written content, the written content is displayed.

6. The writing device assembly according to any one of claims 2 to 4, when the processor identifies the code point type of the code point value and determines that the code point type is a point reading code, the processor is further used to determine the point reading content according to the code point value; The audio playback unit is further configured to broadcast the point-to-read content after the processor determines the point-to-read content.

7. A writing instrument assembly according to any one of claims 2 to 4, characterized in that The processor is further configured to: Perform writing angle analysis based on the image sequence of the code point image to obtain writing angle data; Performing real-time writing posture identification based on the writing angle data and the writing content; When the writing posture identification result is that the posture is wrong, a writing posture correction voice is generated; The audio playback unit is further used to broadcast the writing posture correction voice after the processor generates the writing posture correction voice.

8. The writing instrument assembly according to claim 1, wherein: The pressure sensing unit includes a writing component and a sensing component; The writing component comprises a first end in contact with the writing medium during writing and a second end opposite to the first end; the second end of the writing component is provided with a conductive silicone pad and a first magnetic device; The induction component comprises an induction end arranged opposite to the second end of the writing component, the induction end is provided with a second magnetic device and a first wire and a second wire which are not connected, and the first magnetic device and the second magnetic device are arranged to magnetically repel each other; When the first end of the writing component is not in contact with the writing medium, the conductive silicone pad and the sensing end are separated under the repulsive force of the first magnetic device and the second magnetic device, the first wire and the second wire are not conducting, and the pressure sensing signal is not generated; When the first end of the writing component contacts the writing medium and the contact pressure between the writing component and the writing medium is greater than the repulsive force between the first magnetic device and the second magnetic device, the conductive silicone pad contacts the sensing end, so that the first wire and the second wire are connected to generate the pressure sensing signal; or, The writing member comprises a first end in contact with the writing medium during writing and a second end opposite to the first end; a sliding block is provided at the second end of the writing member; The sensing component includes a pressure sensor disposed opposite to the second end of the writing component; When the first end of the writing component is not in contact with the writing medium, the writing component slides downward with the sliding block under the influence of gravity, the sliding block is separated from the pressure sensor, and the pressure sensor does not generate the pressure sensing signal; When the first end of the writing component contacts the writing medium and the contact pressure between the writing component and the writing medium is greater than the gravity of the writing component, the sliding block moves upward and contacts the pressure sensor, and the pressure sensor generates the pressure sensing signal.

9. The writing instrument assembly according to claim 2, wherein: The writing device further includes an audio acquisition unit. When the processor identifies the code point type of the code point value and determines that the code point type is a recording code, the processor is further configured to send a recording prompt voice to the audio playback unit and initiate a recording instruction to the audio acquisition unit. The audio playback unit is also used to play the recorded prompt voice after receiving the recorded prompt voice; The audio acquisition unit is used to start recording after receiving the recording instruction and the audio playback unit plays the recording prompt voice.

10. A method for identifying written content, characterized in that: A processor for a writing device, wherein the writing device is a writing device in a writing device assembly according to any one of claims 1 to 9, the method comprising: Continuously acquiring the code point values ​​returned by the optical recognition unit to obtain a code point value sequence, wherein the code point values ​​are obtained by parsing a code point image obtained by the optical recognition unit when photographing the code point in the writing medium; Performing code point type identification on the code point value, and when the code point type indicated by the code point value is a coordinate code, performing coordinate identification on the code point value sequence to obtain a coordinate sequence, and drawing a writing trajectory according to the coordinate sequence; A preset trajectory recognition model is called to recognize the writing trajectory to obtain the writing content.

Images