Smart pen

Abstract

A smart pen according to the present invention comprises: an ink accommodation unit in which inks of different colors are accommodated; a scan unit that extracts position coordinates through a first point cloud pattern previously inserted into a printed matter when a user writes handwritten content on the printed matter; a pen point unit that prints according to the user's operation through an ink discharged from the ink accommodation unit; and a control unit that receives the position coordinates from the scan unit and controls a specific ink among the inks of the ink accommodation unit to be discharged according to the position coordinates while being operated by the user.

Description

smart pen

[0001] The present invention relates to a smart pen, and more particularly, to a smart pen that can eject ink according to a mode set by a user using position coordinates through a point cloud pattern pre-inserted in a printed matter.

[0002] As digital devices become more prevalent in modern society, many people use computers, tablets, smartphones, and other devices to record and store information. However, many still prefer analog handwriting.

[0003] The act of writing by hand on paper is known to be natural and intuitive, and to help enhance creativity and critical thinking. Smart pens were developed to preserve the benefits of analog handwriting while also offering the benefits of digital technology.

[0004] Existing digital input devices have limited ability to accurately digitize handwriting data. Smart pens recognize point cloud patterns pre-embedded in printed materials, enabling them to accurately track the user's handwriting position, enabling more accurate digitization of handwriting data. This allows users to synchronize their writing on paper with their digital devices in real time.

[0005] Smart pens can be utilized in a variety of fields, including education, business, and creative endeavors. For example, using smart pens in education allows students to share and save their handwriting in real time, thereby enhancing learning efficiency.

[0006] In business environments, notes taken during meetings or brainstorming sessions can be easily digitized and shared with team members. Furthermore, artists and designers can use smart pens to digitally record and edit their creative processes.

[0007] Advances in sensor technology, wireless communication technology, and data processing technology played a significant role in the emergence of smart pens. In particular, advancements in technology capable of recognizing and interpreting point cloud patterns embedded in printed materials are a key element of smart pens. These technologies enable smart pens to provide users with a more precise and intuitive writing experience.

[0008] The purpose of the present invention is to provide a smart pen that extracts position coordinates through a first point group pattern pre-inserted in a printed matter, recognizes handwriting according to the position coordinates, and then prints a second point group pattern according to the handwriting so that the second point group pattern can be used for authentication of the printed matter.

[0009] In addition, the present invention aims to provide a smart pen that, when writing on a printed matter, ejects ink corresponding to the handwriting along with ink corresponding to the point cloud pattern around the handwriting, thereby inserting a point cloud pattern around the handwriting on the printed matter.

[0010] In addition, the present invention aims to provide a smart pen that can extract position coordinates through a first point group pattern pre-inserted in a printed matter and print by ejecting ink through an inkjet nozzle as the position coordinates change.

[0011] In addition, the present invention aims to provide a smart pen that can receive an electronic document corresponding to the code value from a printed information provision server according to a mode selected by a user, convert the contents of the electronic document into voice data and output it as voice, or provide an AI service for the collected voice data.

[0012] In addition, the present invention aims to provide a smart pen that can extract a code value using a point cloud pattern inserted in a printed matter, and then receive an electronic document corresponding to the code value and provide it as a voice.

[0013] In addition, the present invention aims to provide a smart pen capable of voice recognition that can generate and provide a summary of sound collection content.

[0014] In addition, the present invention aims to provide a smart pen capable of voice recognition that can translate and provide sound collection content in real time.

[0015] A smart pen for achieving this purpose includes an ink receptacle that holds ink of different colors, a scanning unit that extracts position coordinates through a first point group pattern pre-inserted in a printed matter when a user writes on the printed matter, a pen tip unit that prints according to a user's operation using ink discharged from the ink receptacle, and a control unit that receives position coordinates from the scanning unit during a process operated by the user and controls specific ink from the ink receptacle to be discharged according to the position coordinates.

[0016] In one embodiment, the scanning unit scans the printed matter to generate a photographed image, analyzes the photographed image to extract a reference pattern in which boundary dots are formed between a plurality of vertical dotted lines of different lengths, and analyzes the pattern of the plurality of vertical dotted lines forming the reference pattern to correct the recognition direction to the positive direction to extract the position coordinates indicated by the first point group pattern.

[0017] In one embodiment, the scanning unit can scan the printed matter to generate a photographed image, and then analyze the scanned image to detect a boundary dot located in the middle, calculate a distance between other boundary dots centered on the boundary dot located in the middle, and detect other boundary dots with close distances as area setting dots.

[0018] In one embodiment, the scanning unit can detect a vertical dotted line in an upward or downward direction based on another boundary dot that is close to the distance, and detect a boundary dot close to the end of the vertical dotted line in the upward or downward direction as an area setting dot.

[0019] In one embodiment, the scanning unit can determine the upper and lower portions of a specific region using a pattern of the lengths of at least three vertical dotted lines constituting the region setting dot, correct the region based on the determination result, and then extract the position coordinates of each first point group pattern of the region.

[0020] In one embodiment, the control unit can control a specific ink among the inks in the ink receiving unit to be ejected so that a second point group pattern is output according to the position coordinates.

[0021] In one embodiment, the second point group pattern may be scanned by a user terminal upon authentication of the printed matter, and then a code value extracted therefrom may be compared with a pre-stored code value to verify the authenticity of the printed matter.

[0022] In one embodiment, the first point cloud pattern may include a reference pattern composed of a plurality of reference dots indicating position coordinates and a reference pattern in which vertical dotted lines of different lengths indicating directions are formed at specific intervals in the horizontal direction, and a boundary dot is formed between the vertical dotted lines of different lengths.

[0023] In one embodiment, the smart pen may further include a voice collection unit for collecting external voice data and a voice output unit for outputting voice data.

[0024] In one embodiment, the control unit may receive an electronic document corresponding to a combination of position coordinates received from the scanning unit from a printed information providing server, convert the contents of the electronic document into voice data and output it through the voice output unit, or provide an AI service for voice data collected through the voice collection unit.

[0025] In one embodiment, the control unit transmits the code value extracted by the scanning unit to the printed information providing server when the mode selected by the user is a document content voice output mode, and when receiving an electronic document corresponding to the code value from the printed information providing server through the communication unit, the control unit can control the text data of the electronic document to be converted into voice data and provided through the voice output unit.

[0026] In one embodiment, if the mode selected by the user is a translation mode, the control unit may collect voice data through the voice collection unit, convert the voice data into text data, and then translate the text data into a target language and provide it through the voice output unit.

[0027] In one embodiment, if the mode selected by the user is a summary mode, the control unit can collect voice data through the voice collection unit and then generate summary data of the collected content and provide the collected data to the user terminal.

[0028] In one embodiment, the control unit receives position coordinates from the scanning unit during the process of being operated by the user after the printing mode is set by the user, and can control specific ink among the inks in the ink receiving unit to be ejected as the position coordinates change.

[0029] In one embodiment, the print mode may include at least one of a user-defined color printing mode, a texture printing mode, and a handwriting mode.

[0030] In one embodiment, the printed matter is created by being printed through a printing program after being written as an electronic document, and the combination of the electronic document and location coordinates can be stored in a printed matter information providing server.

[0031] According to the present invention as described above, there is an advantage in that position coordinates are extracted through a first point group pattern pre-inserted in a printed matter, handwriting content is recognized according to the position coordinates, and then a second point group pattern is printed according to the handwriting content, which can be used for authentication of the printed matter.

[0032] In addition, according to the present invention, there is an advantage in that position coordinates are extracted through a first point group pattern pre-inserted in a printed matter, handwriting content is recognized according to the position coordinates, and then a second point group pattern is printed according to the handwriting content, which can be used for authentication of the printed matter.

[0033] In addition, according to the present invention, when writing on a printed matter, there is an advantage in that a point cloud pattern can be inserted around the written content of the printed matter by ejecting ink corresponding to the point cloud pattern around the written content together with ink corresponding to the written content.

[0034] In addition, according to the present invention, there is an advantage in that the position coordinates can be extracted through a first point group pattern pre-inserted in the printed matter, and printing can be performed by ejecting ink through an inkjet nozzle as the position coordinates change.

[0035] In addition, according to the present invention, there is an advantage in that, after receiving an electronic document corresponding to the code value from a printed information provision server according to a mode selected by a user, the contents of the electronic document can be converted into voice data and output as voice, or an AI service can be provided for the collected voice data.

[0036] In addition, according to the present invention, there is an advantage in that a code value can be extracted using a point cloud pattern inserted in a printed matter, and then an electronic document corresponding to the code value can be received and provided as a voice.

[0037] In addition, according to the present invention, there is an advantage in that voice recognition is possible to generate and provide a summary of sound collection contents.

[0038] In addition, according to the present invention, there is an advantage in that voice recognition is possible to provide real-time translation of sound collection contents.

[0039] FIG. 1 is a drawing for explaining a smart pen according to one embodiment of the present invention.

[0040] FIG. 2 is a block diagram illustrating the internal structure of a smart pen according to one embodiment of the present invention.

[0041] FIG. 3 is a drawing for explaining an authentication information insertion system using a smart pen according to one embodiment of the present invention.

[0042] FIG. 4 is an exemplary diagram illustrating a point cloud pattern automatic insertion system using a smart pen according to one embodiment of the present invention.

[0043] FIGS. 5 to 7 are drawings for explaining a point group pattern formed on paper according to one embodiment of the present invention.

[0044] The above-described objects, features, and advantages will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can easily practice the technical idea of ​​the present invention. In describing the present invention, if it is determined that a detailed description of known technologies related to the present invention may unnecessarily obscure the gist of the present invention, a detailed description thereof will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

[0045]

[0046] The term “printed material” used in this specification may include all printed materials on which a user can take notes with a smart pen, such as business cards, diaries, galleries, and contracts.

[0047] The term “written content” used in this specification may vary depending on the type of printed matter and may include signatures, etc.

[0048] Among the terms used in this specification, the “first point cloud pattern” is a pattern that is pre-inserted into the background screen of a printed matter to enable the recognition of the positional coordinates of each point constituting the point cloud. The “first point cloud pattern” includes a reference pattern composed of a plurality of reference dots indicating positional coordinates, and a reference pattern in which vertical dotted lines of different lengths indicating directions are formed at specific intervals in the horizontal direction, and boundary dots are formed between the vertical dotted lines of different lengths.

[0049] Among the terms used in this specification, “second point cloud pattern” means a pattern that indicates printed authentication information according to location coordinates when a user writes something.

[0050]

[0051] FIG. 1 is a drawing for explaining a smart pen according to one embodiment of the present invention.

[0052] Referring to FIG. 1, a smart pen (100) is a pen that extracts position coordinates through a first point group pattern pre-inserted in a printed matter (400) when writing on a printed matter, recognizes authentication information based on the position coordinates, and simultaneously prints a second point group pattern for authentication.

[0053] That is, when writing on a printed matter (400) in a state where a first point group pattern is pre-inserted on the background screen of the printed matter (400), the smart pen (100) extracts position coordinates through the first point group pattern and ejects ink according to the position coordinates to insert a second point group pattern around the written matter.

[0054] As described above, since the first point group pattern recognizable through the smart pen (100) is inserted into the printed matter (400), the position coordinates indicated by the reference dot of the reference pattern constituting the first point group pattern can be recognized when the smart pen (100) moves. Here, recognition (or recognition) can be understood as an action of becoming aware of the shade information of the first point group pattern.

[0055] The above smart pen (100) can recognize the position coordinates of each reference dot that constitutes the reference pattern (101) inserted into the printed matter (400) when writing in a specific position on the printed matter (400) while writing in the correct direction on the printed matter (400).

[0056] However, when the smart pen (100) writes at a specific location on the printed matter (400) described above, if the printed matter (400) is written in a reverse or sideways direction rather than a forward direction, and the printed matter (400) is photographed in a reverse or sideways direction rather than a forward direction, there is a problem in that the shooting direction cannot be recognized, and thus the position coordinates of each reference dot constituting the first point group pattern cannot be recognized.

[0057] In order to solve this problem, the present invention forms a reference pattern (201) indicating a direction in a state where a reference pattern (101) of a printed matter (400) is formed, so that no matter from which direction the reference pattern (101) is photographed, the photographing direction can be recognized through the reference pattern (201), and the position coordinates indicated by the reference dot constituting the first point group pattern can be extracted and provided.

[0058] The smart pen (100) scans the printed matter (400) to create a photographed image, then analyzes the photographed image to detect a boundary dot located in the middle. Thereafter, the smart pen (100) extracts a reference pattern (201) and can detect other boundary dots centered on the boundary dot located in the middle of the reference pattern (201).

[0059] In one embodiment, the smart pen (100) can calculate the distance between other boundary dots centered on a boundary dot located in the middle of the reference pattern (201), and detect other boundary dots with close distances as area setting dots.

[0060] After that, the smart pen (100) can detect a vertical dotted line in an upward or downward direction based on another boundary dot that is close to the distance, and can detect a boundary dot close to the end of the vertical dotted line in the upward or downward direction as an area setting dot.

[0061] As described above, the smart pen (100) can determine the upper and lower portions of a specific area by using a pattern of the lengths of at least three vertical dotted lines that constitute the area setting dot.

[0062] That is, the smart pen (100) can determine that the first point group pattern has been scanned in the forward direction if the pattern of the length of the vertical dotted line is a pattern that gradually becomes longer, and can determine that the first point group pattern has been scanned in the reverse direction if the pattern of the length of the vertical dotted line is a pattern that gradually becomes shorter.

[0063] For example, if the pattern of the lengths of the vertical dotted lines is A, B, C, it can be determined that the first point group pattern was scanned in the forward direction, and if the pattern of the lengths of the vertical dotted lines is C, B, A, it can be determined that the first point group pattern was scanned in the reverse direction.

[0064] At this time, the smart pen (100) can rotate and correct to the correct direction if it is determined that a specific area is in the reverse direction by using a pattern of the length of at least three vertical dotted lines that constitute the area setting dot.

[0065] The smart pen (100) can determine the upper and lower sides of a specific area, correct them, and then extract the position coordinates indicated by the reference dot that constitutes the first point group pattern of the specific area. As described above, the smart pen (100) can then insert a second point group pattern for authentication based on the extracted position coordinates.

[0066] Additionally, one side of the smart pen (100) can perform functions such as inkjet printing when an inkjet nozzle is combined.

[0067] More specifically, when the smart pen (100) is operated after the printing mode is set by the user, it can extract position coordinates through a point cloud pattern pre-inserted in the printed matter and print by ejecting ink through an inkjet nozzle as the position coordinates change. Such printing modes may include a user-defined color printing mode, a texture printing mode, a handwriting mode, etc.

[0068] This smart pen (100) receives a print mode input through a user interface, and when the print mode is set, it checks whether there is movement of the user through a change in the position coordinates, and then prints by ejecting ink through an inkjet nozzle.

[0069] At this time, the smart pen (100) scans the background of the printed matter (400) and then recognizes the reference pixel value for the background of the printed matter as RGB using light information, and then converts the RGB corresponding to the value increased by a specific pixel value from the RGB (Red, Green, Blue) corresponding to the reference pixel value of the background or the RGB corresponding to the value decreased by a specific pixel value from the RGB corresponding to the reference pixel value of the background into a CMYK (Cyan, Magenta, Yello, Back) value based on [Mathematical Formula 1] and [Mathematical Formula 2], and then combines ink according to the CMYK value and then ejects it.

[0070] For example, the smart pen (100) scans the background of the printed matter (400) and then uses the light information to convert RGB (256, 256, 256) corresponding to a value increased by a specific pixel value when RGB corresponding to the reference pixel value for the background of the printed matter is (255, 255, 255) to a CMYK (Cyan, Magenta, Yello, Back) value, and then combines ink according to the CMYK value and then ejects it.

[0071]

[0072] [Mathematical Formula 1]

[0073] R'= R / 255

[0074] G'= G / 255

[0075] B'= B / 255

[0076]

[0077] R: The value corresponding to Red,

[0078] G: The value corresponding to Green,

[0079] B: The value corresponding to Blue,

[0080] R': The value corresponding to Red is changed to a range of 0 to 1.

[0081] G': The value corresponding to Green is changed to a range of 0 to 1.

[0082] B': The value corresponding to Blue is changed to a range of 0 to 1.

[0083]

[0084] [Mathematical formula 1] Since the range of values ​​of R, G, and B is 0 to 255, the range can be changed to 0 to 1 by dividing by 255, and thus can be calculated as R', G', and B'.

[0085]

[0086] [Equation 2]

[0087] K = 1-max(R', G' B')

[0088] C = (1-R'-K') / (1-K)

[0089] M = (1-G'-K) / (1-K)

[0090] Y = (1-B'-K) / (1-K)

[0091]

[0092] [Mathematical formula 1] Since the range of values ​​of R, G, and B is 0 to 255, the range can be changed to 0 to 1 by dividing by 255, and thus can be calculated as R', G', and B'.

[0093]

[0094] In addition, the smart pen (100) can provide voice output of the document contents of the printed matter (400). The printed matter (400) described above stores a predetermined combination of position coordinates and the electronic document in the printed matter information providing server (300) when printing the electronic document through a printing program after creating an electronic document. Accordingly, the smart pen (100) can take a picture of the printed matter (400), extract position coordinates using the first point cloud pattern of the printed matter (400) using the taken image, and then receive an electronic document corresponding to the combination of position coordinates from the printed matter information providing server (300) to output voice output.

[0095] Accordingly, the smart pen (100) can output voice when it provides the location coordinates to the printed information providing server (300) and then receives an electronic document corresponding to the location coordinate combination from the printed information providing server (300).

[0096] Additionally, the smart pen (100) can collect voice data and then translate and provide it in real time.

[0097] To this end, the smart pen (100) can collect voice data, convert the voice data into text data, and then translate the text data into a target language and provide it.

[0098] Additionally, the smart pen (100) collects voice data and then generates and provides summary data of the collected content.

[0099] To this end, the smart pen (100) collects voice data, converts the voice data into text data, tokenizes the text data to divide it into individual words or phrases, and then identifies keywords based on the frequency of each word.

[0100] First, the smart pen (100) calculates the number of times each word appears (TF).

[0101] Then, the smart pen (100) calculates the inverse document frequency (IDF) by taking the reciprocal of the number of sentences or paragraphs in which each word appears. At this time, the inverse document frequency (IDF) is a value obtained by taking the reciprocal of the frequency, and the inverse document frequency (IDF) of a commonly appearing word is low, and the inverse document frequency (IDF) of a rarely appearing word is high. That is, the smart pen (100) can calculate the inverse document frequency (IDF) using the following [Mathematical Formula 3].

[0102]

[0103] [Equation 3]

[0104]

[0105]

[0106] N: Number of entire sentences or paragraphs

[0107] DF(w): The number of sentences or paragraphs in which a specific word w appears

[0108]

[0109] The reason for adding 1 to the denominator in the above [Mathematical Formula 1] is to prevent the result of the natural logarithm function from becoming infinite because the denominator becomes 0 if a certain word does not appear in the entire document.

[0110] Therefore, the inverse document frequency (IDF) value indicates how rare a specific word is in the entire document set, and the higher the value, the higher the importance of the word.

[0111] The smart pen (100) calculates the importance score of a word using word frequency (TF) and inverse document frequency (IDF(W)) as in [Mathematical Formula 4].

[0112]

[0113] [Equation 4]

[0114]

[0115]

[0116] TF-IDF(w): importance score of a word,

[0117] TF(w): frequency of a specific word w,

[0118] IDF(w): rarity of a particular word w,

[0119]

[0120] The TF-IDF above indicates the importance of a word by considering its frequency of occurrence within a document while also considering its rarity within the document set. A higher TF-IDF reflects the importance of a word within a document, while a higher IDF indicates that the word is a uniquely important word within the document.

[0121] Afterwards, the smart pen (100) can identify important content that can be used for summarizing among text data using keywords and provide summary data.

[0122]

[0123] FIG. 2 is a block diagram illustrating the internal structure of a smart pen according to one embodiment of the present invention.

[0124] Referring to FIG. 2, the smart pen (100) includes a pen tip portion (110), a scanning portion (120), an ink receiving portion (130), a voice collection portion (140), a voice output portion (150), and a control portion (160).

[0125] The nib (110) generates handwriting according to the user's operation using ink discharged from the ink receiving portion (130). As described above, an ink module is discharged from the nib portion of the nib (110) to perform the functions of a general pen.

[0126] The scanning unit (120) can scan the printed matter (400) and extract the position coordinates by scanning the first point group pattern inserted in advance.

[0127] In one embodiment, the scanning unit (120) can extract position coordinates through the first point group pattern and provide them to the control unit (160) when writing on the printed matter (400) in a state where the first point group pattern is pre-inserted on the background screen of the printed matter (400).

[0128] As described above, since the first point group pattern recognizable through the smart pen (100) is inserted into the printed matter (400), the position coordinates indicated by the reference dot of the reference pattern constituting the first point group pattern can be recognized when the smart pen (100) moves. Here, recognition (or recognition) can be understood as an action of becoming aware of the shade information of the first point group pattern.

[0129] The above scanning unit (120) can recognize the position coordinates of each reference dot that constitutes the reference pattern (101) inserted into the printed matter (400) when writing on the printed matter (400) in the correct direction during the process of writing on a specific location of the printed matter (400).

[0130] However, when the smart pen (100) writes at a specific location on the printed matter (400) described above, if the printed matter (400) is written in a reverse or sideways direction rather than a forward direction, and the printed matter (400) is photographed in a reverse or sideways direction rather than a forward direction, there is a problem in that the shooting direction cannot be recognized, and thus the position coordinates of each reference dot constituting the first point group pattern cannot be recognized.

[0131] In order to solve this problem, the present invention forms a reference pattern (201) indicating a direction in a state where a reference pattern (101) of a printed matter (400) is formed, so that no matter from which direction the reference pattern (101) is photographed, the photographing direction can be recognized through the reference pattern (201), and the position coordinates indicated by the reference dot constituting the first point group pattern can be extracted and provided.

[0132] The scanning unit (120) scans the printed matter (400) to create a photographed image, and then analyzes the photographed image to detect a boundary dot located in the middle. Thereafter, the smart pen (100) can extract a reference pattern (201) and then detect other boundary dots centered on the boundary dot located in the middle of the reference pattern (201).

[0133] In one embodiment, the scanning unit (120) can calculate the distance between other boundary dots centered on a boundary dot located in the middle of the reference pattern (201), and detect other boundary dots with close distances as area setting dots.

[0134] After that, the scanning unit (120) can detect a vertical dotted line in the upper or lower direction based on another boundary dot that is close in distance, and can detect a boundary dot close to the end of the vertical dotted line in the upper or lower direction as an area setting dot.

[0135] As described above, the scanning unit (120) can determine the upper and lower parts of a specific area by using a pattern of the lengths of at least three vertical dotted lines that constitute the area setting dot.

[0136] That is, the scanning unit (120) can determine that the first point group pattern has been scanned in the forward direction if the pattern of the length of the vertical dotted line is a pattern that gradually becomes longer, and can determine that the first point group pattern has been scanned in the reverse direction if the pattern of the length of the vertical dotted line is a pattern that gradually becomes shorter.

[0137] For example, if the pattern of the lengths of the vertical dotted lines is A, B, C, it can be determined that the first point group pattern was scanned in the forward direction, and if the pattern of the lengths of the vertical dotted lines is C, B, A, it can be determined that the first point group pattern was scanned in the reverse direction.

[0138] At this time, the scanning unit (120) can correct to the correct direction by rotating if it is determined that a specific area is in the reverse direction by using the pattern of the length of at least three vertical dotted lines that constitute the area setting dot. The scanning unit (120) can correct by determining the upper and lower sides of the specific area, and then extract the position coordinates indicated by the reference dot that constitutes the first point group pattern of the specific area and provide them to the control unit (160).

[0139] The ink receiving portion (130) contains inks of different colors.

[0140] In one embodiment, each ink receptacle (130) stores color ink and transparent ink corresponding to CMY (Cyan, Magenta, Yellow). At this time, the color ink and the projection ink can be used to insert handwritten content and a second point cloud pattern around the handwritten content according to the user's operation, can be used to generate the handwritten content, and can be used to express a printing mode set by the user.

[0141] In another embodiment, each of the ink receiving units (130) stores color ink and transparent ink corresponding to CMYK (Cyan, Magenta, Yellow, Back). At this time, the color ink and the projection ink can be used to insert handwritten content and a second point cloud pattern around the handwritten content according to the user's operation, can be used to create the handwritten content, and can be used to express a printing mode set by the user.

[0142] Ink of different colors stored in each ink receiving unit (130) is discharged under the control of the control unit (160), mixed, and provided to the outside through the pen tip unit (110), thereby printing on the printed matter (400) according to the written content, the second point group pattern, and the printing mode set by the user.

[0143] The control unit (160) can control a specific ink among the inks in the ink receiving unit (130) to be ejected.

[0144] In one embodiment, the control unit (160) can control a specific ink among the inks in the ink receiving unit (130) to be ejected to insert a second point group pattern for authentication according to the position coordinates received from the scanning unit (120).

[0145] In another embodiment, the control unit (160) may receive position coordinates from the scan unit (120) during a process operated by the user after the print mode is set by the user, and control a specific ink among the inks in the ink receiving unit (130) to be ejected as the position coordinates change.

[0146] In another embodiment, the control unit (160) may receive position coordinates from the scan unit (120) during a process in which a color is selected by the user and is manipulated by the user, and may control a specific ink among the inks in the ink receiving unit (130) to be ejected as the position coordinates change.

[0147] The control unit (160) provides the location coordinates to the printed information providing server (300), and then, when receiving an electronic document corresponding to the location coordinate combination from the printed information providing server (300), the control unit (160) can output the electronic document as a voice through the voice output unit (150).

[0148] Additionally, the control unit (160) can collect voice data and then translate and provide it in real time.

[0149] To this end, the control unit (160) can collect voice data through the voice collection unit (140), convert the voice data into text data, translate the text data into a target language, and provide it through the voice output unit (150).

[0150] In addition, if the mode selected by the user is the voice data summary mode, the control unit (160) collects voice data through the voice collection unit (140) and then generates and provides summary data of the collected content.

[0151] To this end, the control unit (160) collects voice data, converts the voice data into text data, tokenizes the text data to divide it into individual words or phrases, and then identifies keywords based on the frequency of each word.

[0152] First, the control unit (160) calculates the number of times each word appears (TF).

[0153] Then, the control unit (160) calculates the inverse document frequency (IDF) by taking the reciprocal of the number of sentences or paragraphs in which each word appears. At this time, the inverse document frequency (IDF) is a value obtained by taking the reciprocal of the frequency, and the inverse document frequency (IDF) of a word that appears frequently is low, and the inverse document frequency (IDF) of a word that appears rarely is high. That is, the smart pen (100) can calculate the inverse document frequency (IDF) using the following [Mathematical Formula 1].

[0154] After that, the control unit (160) can calculate the importance score of a word using the word frequency (TF) and inverse document frequency (IDF(W)) as in [Mathematical Formula 2] using the AI ​​pen (100) capable of voice recognition.

[0155] The TF-IDF above indicates the importance of a word by considering its frequency of occurrence within a document while also considering its rarity within the document set. A higher TF-IDF reflects the importance of a word within a document, while a higher IDF indicates that the word is a uniquely important word within the document.

[0156]

[0157] FIG. 3 is a drawing for explaining an authentication information insertion system using a smart pen according to one embodiment of the present invention.

[0158] Referring to FIG. 3, the automatic point cloud pattern insertion system using a smart pen includes a smart pen (100), a printed matter information provision server (300), and a user terminal (500).

[0159] When writing on a printed matter (400), the smart pen (100) can insert a second point group pattern around the written content of the printed matter (400) by ejecting ink corresponding to the written content and ink corresponding to the second point group pattern around the written content.

[0160] This smart pen (100) can extract position coordinates through a first point group pattern pre-inserted in the printed matter when writing on the printed matter, recognize authentication information according to the position coordinates, and simultaneously print a second point group pattern for authentication.

[0161] The body of the smart pen (100) is a pipe-shaped body with a cylindrical or polygonal cross-section, similar to that of a conventional ballpoint pen, and has the same size as a ballpoint pen. Inside the smart pen (100), there are multiple ink modules, and the ink modules may include color ink corresponding to CMYK (Cyan, Magenta, Yellow, Back) and transparent ink.

[0162] An ink module is ejected from the nib portion of the smart pen (100) to perform the functions of a general pen.

[0163] In one embodiment, the smart pen (100) may eject transparent ink to create a second pre-generated dot cloud pattern around the handwriting along with the handwriting when the user writes on the printed matter (400).

[0164] At this time, the second point group pattern can be created in an invisible form with a transparent printing layer having a code value, so that no physical color difference exists, providing a form that cannot be replicated by optical equipment such as a copier. In addition, the transparent printing layer can be formed by mixing general ink and resin.

[0165] Of course, it is desirable that the second point group pattern have a code value corresponding to the identification information. Since the purpose is to extract the code value corresponding to the identification information from the printed matter (400) using the diffuse reflection of light, the transparent printed layer may take various shapes, but it is desirable that it have a convex embossed shape.

[0166] Additionally, any shape of the second point cloud pattern can be formed as long as it is visually undetectable. For example, it can be a square shape, or the transparent printing layer can be formed intaglio. In this case, the intaglio can indicate 0 and 1. That is, it can be predefined that the positive indicates 0 and the negative indicates 1, or it can be predefined that the positive indicates 1 and the negative indicates 0.

[0167] That is, the transparent printed layer is a state in which a code value corresponding to identification information recognizable through a user terminal (500) is printed, for example, using transparent ink, and thus connects information of the printed matter corresponding to the identification information. The above identification information can be various types of information utilizing the characteristics of light.

[0168] For example, various information using the characteristics of light can be the reflection speed of light, the pixel value of a photographed image, that is, an image containing diffuse reflection, or the luminance value among the pixel values.

[0169] In this way, embodiments of the present invention can use grayscale values ​​in the absence of diffuse reflection and grayscale values ​​in the presence of diffuse reflection to extract code values ​​corresponding to identification information according to embodiments of the present invention. In addition, various factors due to diffuse reflection can be used to provide information, and thus embodiments of the present invention will not be specifically limited to any one of them.

[0170] In another embodiment, the smart pen (100) may eject colored ink to create a second pre-generated dot cloud pattern around the handwriting along with the handwriting when the user writes on the printed matter (400).

[0171] To this end, when a user writes on a printed matter (400), the smart pen (100) can recognize the reference pixel value of the background of the printed matter (400) and then combine and discharge ink to produce a color corresponding to a value increased by a specific pixel value based on the reference pixel value of the background or decreased by a specific pixel value based on the reference pixel value.

[0172] More specifically, the smart pen (100) scans the background of the printed matter (400) and then recognizes the reference pixel value for the background of the printed matter as RGB using light information, and then converts the RGB corresponding to the value increased by a specific pixel value from the RGB (Red, Green, Blue) corresponding to the reference pixel value of the background or the RGB corresponding to the value decreased by a specific pixel value from the RGB corresponding to the reference pixel value of the background into a CMYK (Cyan, Magenta, Yello, Back) value, and then combines ink according to the CMYK value and then ejects it.

[0173] As described above, the smart pen (100) can insert a second point cloud pattern around the handwriting by ejecting color ink to create a second point cloud pattern pre-generated around the handwriting together with the handwriting when the user writes on the printed matter (400).

[0174] The user terminal (500) is a terminal that recognizes the handwriting of a printed matter (400), extracts an identification code, and then authenticates the printed matter (400) according to the identification code.

[0175] The user terminal (500) can execute an application (hereinafter, "app") for scanning printed matter (400). Accordingly, when the user executes the app to scan printed matter (400) and points the camera at the surface of the printed matter (400), the flash automatically operates and a photo is taken. It is preferable to capture a still image through a single photo, rather than a video recording.

[0176] In other words, since the printed matter (400) is printed with a code value recognizable through the app, for example, using transparent ink, the code value reflected and displayed through the flash built into the user terminal (500) is recognized and then identification information is connected. Here, the code value can be various information utilizing the characteristics of light.

[0177] For example, it can be the reflection speed of light, the pixel value of a captured image, that is, an image containing diffuse reflection, or a luminance value among the pixel values. In this way, the user terminal (500) extracts a code value from the diffuse reflection of the second point cloud pattern and transmits the extracted code value to the printed matter information providing server (300) to extract and provide the code value inserted in the printed matter (400).

[0178] Extraction of code values ​​may be performed through an app on a user terminal (500), but may also be performed on a printed information provision server (300), and therefore, the embodiment of the present invention will not be particularly limited to either form.

[0179] A photographed image captured by a user terminal (500) can be provided to a printed matter information providing server (300), and a code value can be extracted from a printed matter (400) and provided using a photographed image captured directly from the app. Alternatively, the app can extract only the code value from the photographed image and transmit the extracted code value to the printed matter information providing server (300). At this time, the code value can be in binary bit form, and the code value when there is diffuse reflection is clearly different from the code value when there is no diffuse reflection.

[0180] After that, the user terminal (500) provides the code value to the print information providing server (300), and then receives and displays the print owner request data linked to the identification information among the code values ​​from the print information providing server (300), and then stores the identification information and the photographed image in correspondence.

[0181]

[0182] FIG. 4 is an exemplary diagram illustrating a system for automatically inserting point cloud patterns using a smart pen according to one embodiment of the present invention. The embodiment of FIG. 4 relates to an embodiment in which a point cloud pattern can be automatically inserted when writing a signature using a smart pen in the case of a printed document such as a contract.

[0183] Referring to FIG. 4, the second point group pattern automatic insertion system using a smart pen includes a smart pen (100), a printed matter information provision server (300), and a user terminal (500).

[0184] The smart pen (100) has the advantage of being able to insert a second point group pattern around the signature of the contract (410) by ejecting ink corresponding to the second point group pattern together with ink corresponding to the signature when writing a signature on the contract (410).

[0185] The body of the smart pen (100) is a pipe-shaped body with a cylindrical or polygonal cross-section, similar to that of a conventional ballpoint pen, and has the same size as a ballpoint pen. Inside the smart pen (100), there are multiple ink modules, and the ink modules may include color ink corresponding to CMYK (Cyan, Magenta, Yellow, Back) and transparent ink.

[0186] An ink module is ejected from the nib portion of the smart pen (100) to perform the functions of a general pen.

[0187] In one embodiment, the smart pen (100) may eject transparent ink to create a second pre-generated dot cloud pattern around the signature along with the signature when the user writes a signature on the contract (410).

[0188] At this time, the second point group pattern can be created in an invisible form with a transparent printing layer having a code value, so that no physical color difference exists, providing a form that cannot be replicated by optical equipment such as a copier. In addition, the transparent printing layer can be formed by mixing general ink and resin.

[0189] Of course, it is desirable that the second point group pattern have a code value corresponding to the identification information. Since the purpose is to extract the code value corresponding to the identification information from the contract (410) using the diffuse reflection of light, the transparent printing layer can take various shapes, but it is desirable to have a convex embossed shape.

[0190] Additionally, any shape of the second point cloud pattern can be formed as long as it is visually undetectable. For example, it can be a square shape, or the transparent printing layer can be formed intaglio. In this case, the intaglio can indicate 0 and 1. That is, it can be predefined that the positive indicates 0 and the negative indicates 1, or it can be predefined that the positive indicates 1 and the negative indicates 0.

[0191] That is, the transparent printing layer is a state in which a code value corresponding to identification information recognizable through a user terminal (500) is printed, for example, using transparent ink, and thus connects the information of the contract (410) corresponding to the identification information. The above identification information can be various information utilizing the characteristics of light.

[0192] For example, various information using the characteristics of light can be the reflection speed of light, the pixel value of a photographed image, that is, an image containing diffuse reflection, or the luminance value among the pixel values.

[0193] In this way, embodiments of the present invention can use grayscale values ​​in the absence of diffuse reflection and grayscale values ​​in the presence of diffuse reflection to extract code values ​​corresponding to identification information according to embodiments of the present invention. In addition, various factors due to diffuse reflection can be used to provide information, and thus embodiments of the present invention will not be specifically limited to any one of them.

[0194] In another embodiment, the smart pen (100) may eject colored ink to create a second pre-generated dot cloud pattern around the signature along with the signature when the user writes a signature on the contract (410).

[0195] To this end, when a user writes a signature on a contract (410), the smart pen (100) can recognize the reference pixel value of the background of the contract (410) and then combine and discharge ink to create a color corresponding to a value increased by a specific pixel value based on the reference pixel value of the background or decreased by a specific pixel value based on the reference pixel value.

[0196] More specifically, the smart pen (100) scans the background of the contract (410) and then recognizes the reference pixel value for the background of the contract (410) as RGB using light information, and then converts the RGB corresponding to the value increased by a specific pixel value from the RGB (Red, Green, Blue) corresponding to the reference pixel value of the background or the RGB corresponding to the value decreased by a specific pixel value from the RGB corresponding to the reference pixel value of the background into a CMYK (Cyan, Magenta, Yello, Back) value, and then combines ink according to the CMYK value and then ejects it.

[0197] For example, the smart pen (100) scans the background of the contract (410) and then uses light information to convert RGB (256, 256, 256) corresponding to a value increased by a specific pixel value when RGB corresponding to the reference pixel value for the background of the contract (410) is (255, 255, 255) into a CMYK (Cyan, Magenta, Yello, Back) value, and then combines ink according to the CMYK value and then ejects it.

[0198] As described above, the smart pen (100) can insert a second point cloud pattern around the signature by ejecting color ink to create a second point cloud pattern pre-generated around the signature together with the signature when the user writes a signature on the contract (410).

[0199] The user terminal (500) is a terminal that recognizes the signature of the contract (410), extracts an identification code, and then authenticates the contract (410) according to the identification code.

[0200] The user terminal (500) can run an application (hereinafter, "app") for scanning a contract (410). Accordingly, when the user runs the app to scan the contract (410) and points the camera at the surface of the contract (410), the flash automatically activates and a photo is taken. It is preferable to capture a still image through a single photo, rather than a video recording.

[0201] In other words, the contract (410) is printed with a code value recognizable through the app, for example, using transparent ink, and then recognizes the code value reflected and displayed through the flash built into the user terminal (500) and connects the identification information. Here, the code value can be various information utilizing the characteristics of light.

[0202] For example, it can be the reflection speed of light, the pixel value of a captured image, i.e., an image containing diffuse reflection, or a luminance value among the pixel values. In this way, the user terminal (500) extracts a code value from the diffuse reflection of the second point cloud pattern and transmits the extracted code value to the printed information provision server (300) to extract and provide the code value inserted in the contract (410).

[0203] Extraction of code values ​​may be performed through an app on a user terminal (500), but may also be performed on a printed information provision server (300), and therefore, the embodiment of the present invention will not be particularly limited to either form.

[0204] A photographed image captured by a user terminal (500) can be provided to a printed information provision server (300), and a code value can be extracted from a contract (410) and provided using a photographed image captured directly from the app.

[0205] Alternatively, the app can extract only the code value from the captured image and transmit the extracted code value to the print information provision server (300). At this time, the code value can be in binary bit form, and the code value when there is diffuse reflection is clearly different from the code value when there is no diffuse reflection.

[0206] After that, the user terminal (500) provides the code value to the print information providing server (300), and then receives and displays the print owner request data linked to the identification information among the code values ​​from the print information providing server (300), and then stores the identification information and the photographed image in correspondence.

[0207]

[0208] FIGS. 5 to 7 are drawings for explaining a point group pattern formed on paper according to one embodiment of the present invention.

[0209] Referring to FIGS. 5 to 7, the first point group pattern formed on the paper may include a reference pattern (101) composed of a plurality of reference dots indicating position coordinates and a reference pattern (201) in which vertical dotted lines of different lengths indicating directions are formed at a specific interval in the horizontal direction and a boundary dot is formed between the vertical dotted lines of different lengths.

[0210] Each of the plurality of reference dots constituting the reference pattern (101) can indicate a specific position coordinate on a predetermined global coordinate. For example, if the global coordinate is from (0, 0) to (100, 100), the plurality of reference dots can be determined to have any one position coordinate between (0, 0) and (100, 100) depending on the position.

[0211] When the smart pen decodes the above-mentioned reference pattern (101) and scans the reference pattern (101) in the correct direction, the position coordinates of each reference dot constituting the reference pattern (101) can be recognized.

[0212] However, when the smart pen decodes the first point group pattern described above and scans the reference pattern (101), if the reference pattern (101) is photographed in a reverse or sideways direction rather than a forward direction, there is a problem in that the shooting direction cannot be recognized and thus the position coordinates of each reference dot constituting the first point group pattern cannot be recognized.

[0213] In order to solve this problem, the present invention forms a reference pattern (201) indicating a direction while a reference pattern (101) is formed, so that no matter from which direction the reference pattern (101) is photographed, the shooting direction can be recognized through the reference pattern (201), and the position coordinates indicated by the reference dot constituting the first point group pattern can be extracted and provided.

[0214] A reference pattern (201) for this purpose is formed by vertical dotted lines of different lengths indicating directions being spaced apart in the horizontal direction by a specific interval, and boundary dots are formed between the vertical dotted lines of different lengths.

[0215] In one embodiment, the vertical dotted lines of the reference pattern (201) may be formed by a first vertical dotted line (210) of a first length, a second vertical dotted line (220) of a second length that is a specific length longer than the first length, and a third vertical dotted line (230) of a third length that is a specific length longer than the second length, which are spaced apart from each other by a specific interval in the horizontal direction.

[0216] Additionally, each of the first vertical dotted line (210), the second vertical dotted line (220), and the third vertical dotted line (230) may be formed spaced apart from each other by a specific interval, and a boundary dot may be formed between the first vertical dotted line (210), the second vertical dotted line (220), and the third vertical dotted line.

[0217] For example, if the pattern of the lengths of the vertical dotted lines is A, B, C, it can be determined that the first point group pattern was scanned in the forward direction, and if the pattern of the lengths of the vertical dotted lines is C, B, A, it can be determined that the first point group pattern was scanned in the reverse direction.

[0218] As described above, the reason why the lengths of the first vertical dotted line (210), the second vertical dotted line (220), and the third vertical dotted line (230) are implemented differently is that a vertical dotted line of a shorter length can indicate one of the upward and downward directions of the entire coordinates, and a vertical dotted line of a longer length can indicate the remaining direction of the upward and downward directions.

[0219] Accordingly, when the smart pen scans the reference pattern (101) to generate a first point cloud pattern image, a boundary dot located in the middle of the point cloud pattern image is detected, another boundary dot is detected based on the boundary dot located in the middle, and then the other boundary dots are connected to enable detection of a specific area.

[0220] That is, the smart pen scans the first point group pattern as shown in Fig. 9 and then detects a boundary dot (400) located in the middle of the reference pattern (201).

[0221] The smart pen detects other boundary dots (410, 420, 430) centered around the boundary dot (400) located in the middle.

[0222] In one embodiment, the smart pen can calculate the distance between other boundary dots (410, 420, 430) centered on a boundary dot (400) located in the middle, and detect other boundary dots (410) with close distances as area setting dots.

[0223] Thereafter, the smart pen can detect a vertical dotted line in the upper or lower direction based on another boundary dot (410) that is close to the distance, and detect a boundary dot (420, 430) close to the end of the vertical dotted line in the upper or lower direction as an area setting dot.

[0224] The smart pen connects different boundary dots to create a specific area (500) as shown in Fig. 7 and determines the upper and lower parts of the specific area (500).

[0225] That is, the smart pen detects a vertical dotted line in an upward or downward direction based on another boundary dot that is close to the center, and compares the length between the vertical dotted lines in the upward and downward directions. If the vertical dotted line in the upward direction is longer than the vertical dotted line in the downward direction, the downward direction can be determined as the upward direction of the point cloud pattern.

[0226] The smart pen determines the upper and lower sides of a specific area (500) and corrects them, and then decodes the second point group pattern of the specific area (500).

[0227]

[0228] While described with reference to limited embodiments and drawings, the present invention is not limited to the above-described embodiments, and various modifications and variations are possible based on this disclosure by those skilled in the art. Accordingly, the scope of the present invention should be understood solely by the scope of the claims set forth below, and all equivalent or equivalent modifications thereof are deemed to fall within the scope of the present invention.

Claims

1. Ink container containing inks of different colors; A scanning unit that extracts location coordinates through a first point cloud pattern pre-inserted in a printed matter when a user writes on the printed matter; A pen tip portion that prints according to the user's operation using ink discharged from the ink receiving portion; and It is characterized by including a control unit that receives position coordinates from a scanning unit in a process operated by a user and controls a specific ink among inks in an ink receiving unit to be ejected according to the position coordinates. Smart pen.

2. In paragraph 1, The above scanning section The method comprises: scanning the above printed matter to generate a photographed image, analyzing the photographed image to extract a reference pattern in which a boundary dot is formed between a plurality of vertical dotted lines of different lengths, analyzing the pattern of the plurality of vertical dotted lines forming the reference pattern to correct the recognition direction to the positive direction, and extracting the position coordinates indicated by the first point group pattern. Smart pen.

3. In paragraph 2, The above scanning section The above printed matter is scanned to generate a photographed image, and then analyzed to detect a boundary dot located in the middle, calculate the distance between other boundary dots centered on the boundary dot located in the middle, and detect other boundary dots with close distances as area setting dots. Smart pen.

4. In paragraph 3, The above scanning section It is characterized in that a vertical dotted line in the upper or lower direction is detected based on another boundary dot having a close distance, and a boundary dot close to the end of the vertical dotted line in the upper or lower direction is detected as an area setting dot. Smart pen.

5. In paragraph 3, The above scanning section A method characterized in that the upper and lower portions of a specific area are determined using a pattern of the lengths of at least three vertical dotted lines constituting the above-mentioned area setting dot, and the area is corrected according to the determination result, and then the position coordinates of each first point group pattern of the area are extracted. Smart pen.

6. In paragraph 1, The above control unit It is characterized in that it controls to eject a specific ink among the ink of the ink receiving unit so that a second point group pattern is output according to the above position coordinates. Smart pen.

7. In paragraph 7, The above second point group pattern is When authenticating the above printed matter, the code value is extracted after being scanned by the user terminal and compared with the pre-stored code value to be used to verify the authenticity of the printed matter. Smart pen.

8. In paragraph 1, The above first point group pattern is A reference pattern comprising a plurality of reference dots indicating position coordinates and a reference pattern in which vertical dotted lines of different lengths indicating directions are formed at specific intervals in the horizontal direction, and a boundary dot is formed between the vertical dotted lines of different lengths. Smart pen.

9. In paragraph 1, A voice collection unit that collects external voice data; and It is characterized by further including a voice output unit that outputs voice data. Smart pen.

10. In paragraph 9, The above control unit It is characterized in that an electronic document corresponding to a combination of position coordinates received from the above scanning unit is received from a printed information providing server, the contents of the electronic document are converted into the above voice data and output through the above voice output unit or an AI service is provided for the voice data collected through the above voice collection unit. Smart pen.

11. In paragraph 10, The above control unit If the mode selected by the user is a document content voice output mode, the code value extracted by the scanning unit is transmitted to the print information providing server, and if an electronic document corresponding to the code value is received from the print information providing server through the communication unit, the text data of the electronic document is converted into voice data and controlled to be provided through the voice output unit. Smart pen.

12. In paragraph 10, The above control unit If the mode selected by the user is a translation mode, the voice data is collected through the voice collection unit, the voice data is converted into text data, and the text data is translated into the target language and provided through the voice output unit. Smart pen.

13. In paragraph 10, The above control unit If the mode selected by the user is a summary mode, voice data is collected through a voice collection unit, and then summary data of the collected content is generated and provided to the user terminal. Smart pen.

14. In paragraph 1, The above control unit After the print mode is set by the user, the process of receiving the position coordinates from the scanning unit and operating the ink receiving unit is characterized in that a specific ink among the inks is ejected as the position coordinates change. Smart pen.

15. In paragraph 14, The above print mode is characterized by including at least one of a user-defined color printing mode, a texture printing mode and a handwriting mode; Smart pen.

16. In paragraph 1, The above printed matter It is characterized in that it is created by being printed through a printing program after being written as an electronic document, and the combination of the electronic document and location coordinates is stored in a printed information providing server. Smart pen.

Images