Artificial intelligence-based braille conversion method, and computer program, recorded on recording medium, for executing same

Abstract

The present invention presents a method for converting documents of various formats into braille on the basis of artificial intelligence (AI). The method may comprise the steps of: parsing an electronic document to undergo braille translation, so as to identify one or more elements included in the electronic document; using pre-trained AI to assign a braille attribute to each element identified from the electronic document; and generating, on the basis of the assigned braille attribute, a braille document corresponding to the electronic document. According to the present invention, electronic documents having various file formats can be converted into braille documents.

Description

Artificial intelligence-based Braille conversion method and a computer program recorded on a recording medium for executing the method

[0001] The present invention relates to Braille conversion. It relates to language translation. More specifically, it relates to a method for converting various formats of documents into Braille using artificial intelligence (AI), and a computer program recorded on a recording medium for executing the method.

[0002] Braille is a writing system developed to enable the visually impaired to recognize letters through touch. Specifically, Braille is written using six tactile dots arranged in two rows and three columns, either in relief or sunk relief. Based on the positional relationship of these six dots, Korean letters, English letters, numbers, and symbols can be expressed. In this way, characters expressed in Braille can be read by visually impaired people by tracing them with their fingers or other tools.

[0003] Many countries have enacted Braille literacy laws to protect the rights of the blind or visually impaired. These laws require public institutions and other organizations to provide Braille documents with the same content when distributing printed documents. For example, when distributing documents such as tax guides and bills in printed form, public institutions must also provide Braille documents with the same content for the visually impaired.

[0004] Meanwhile, with the advancement of information and communication technology (IT), a growing number of documents are being distributed electronically, rather than on paper. This trend has led to the provision of electronic Braille documents to the visually impaired. However, electronic documents can contain not only simple text but also user interface (UI) elements such as images, tables, and buttons. Furthermore, the diverse types of elements contained in electronic documents pose significant challenges in converting them into electronic Braille.

[0005] To address these challenges, services capable of translating non-text elements within electronic documents into Braille have emerged. However, currently developed Braille translation services only predefine and translate the Braille characters for each element within the document. Therefore, they are significantly inadequate for translating electronic documents in unauthorized file formats, undefined elements within the document, the perceived order of elements based on their layout, and reference relationships between elements arranged in a discrete manner.

[0006] One object of the present invention is to provide a method for converting various formats of documents into Braille based on artificial intelligence (AI).

[0007] Another object of the present invention is to provide a computer program recorded on a recording medium for executing a method for converting documents of various formats into Braille.

[0008] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] In order to achieve the technical task described above, the present invention proposes a method for converting various formats of documents into Braille based on artificial intelligence (AI).

[0010] The method may include a step of parsing an electronic document to be subjected to braille translation and identifying one or more elements contained in the electronic document; a step of assigning a braille attribute to each element identified from the electronic document using pre-learned artificial intelligence (AI); and a step of generating a braille document corresponding to the electronic document based on the assigned braille attribute.

[0011] Specifically, the step of identifying the element identifies a text-format element included in the electronic document, and if the identified text-format element is composed of two or more texts positioned apart from each other within the electronic document, the two or more texts can be identified as individual multiple elements.

[0012] Additionally, the step of identifying the elements may include identifying one or more non-text format elements included in the electronic document, and performing optical character recognition (OCR) on each identified non-text format element to recognize characters included in each non-text format element.

[0013] And, the step of identifying the above elements may include detecting a layout in which the text-format elements and non-text-format elements are arranged within the electronic document, and assigning a braille order to the text-format elements and non-text-format elements based on the detected layout.

[0014] Meanwhile, the step of assigning the Braille attribute may include splitting the identified element into chunks of a preset size according to the type, embedding the divided chunks to generate a feature vector, and, if the generated feature vector matches a predefined pattern or template, assigning the Braille attribute to the identified element according to the matched pattern or template.

[0015] To this end, the step of assigning the Braille attribute may determine whether or not the pattern or template is matched based on vector similarity with the identified feature vector, targeting an internal database in which a pattern or template is stored, including one or more of a title, text, image, table, combo box, check box, radio button, and callback function-based reference that may be included in an electronic document.

[0016] According to one embodiment, the step of assigning the Braille attribute may include, when the generated feature vector does not match a pattern or template, performing multi-class classification on the feature vector using a pre-trained artificial neural network (ANN), and assigning the Braille attribute to the identified element based on the result of the multi-class classification.

[0017] According to another embodiment, the step of assigning the Braille attribute may include retrieving data having the highest similarity to the generated feature vector from a pre-specified external database related to Braille, if the generated feature vector does not match a pattern or template, and generating the Braille attribute for the identified element based on the retrieved data.

[0018] Meanwhile, the step of generating the above-described Braille document may generate a Braille Text Markup Language (BTML) corresponding to the electronic document according to the assigned Braille properties, and generate a file for a Braille information instrument based on the generated Braille Text Markup Language (BTML).

[0019] In order to achieve the technical task described above, the present invention proposes a computer program recorded on a recording medium to execute a method for converting documents of various formats into Braille.

[0020] The computer program may be combined with a computing device comprising a memory; and a processor that processes instructions residing in the memory. In addition, the computer program may be recorded on a recording medium to cause the processor to perform a step of parsing an electronic document to be transcribed and identifying one or more elements included in the electronic document; a step of assigning a Braille attribute to each element identified from the electronic document using pre-learned artificial intelligence; and a step of generating a Braille document corresponding to the electronic document based on the assigned Braille attribute.

[0021] Specific details of other embodiments are included in the detailed description and drawings.

[0022] According to embodiments of the present invention, it is possible to convert electronic documents in various file formats into Braille documents.

[0023] In particular, it is possible to convert elements contained in an electronic document into Braille in an easy-to-read order by considering the layout, and even if elements that are not defined in advance appear during the process of converting an electronic document, it is possible to convert them into replaceable Braille, and even reference relationships between elements that are spaced apart from each other within an electronic document can be expressed in Braille.

[0024] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description of the claims.

[0025] FIG. 1 is a configuration diagram of a Braille conversion system according to one embodiment of the present invention.

[0026] FIG. 2 and FIG. 3 are exemplary diagrams illustrating the configuration of a Braille conversion system with separated roles according to some embodiments of the present invention.

[0027] Figure 4 is a logical configuration diagram of a Braille conversion server according to one embodiment of the present invention.

[0028] FIG. 5 is a logical configuration diagram of a user device and a Braille conversion server according to another embodiment of the present invention.

[0029] FIG. 6 and FIG. 7 are exemplary diagrams for explaining a process of analyzing an electronic document according to one embodiment of the present invention.

[0030] FIG. 8 is an exemplary diagram illustrating a process of assigning Braille properties according to one embodiment of the invention.

[0031] FIG. 9 is an exemplary diagram illustrating an example of a Braille attribute granted according to one embodiment of the invention.

[0032] Figure 10 is a hardware configuration diagram of a Braille conversion server according to one embodiment of the present invention.

[0033] FIG. 11 and FIG. 12 are flowcharts for explaining a Braille conversion method according to some embodiments of the present invention.

[0034] It should be noted that the technical terms used in this specification are used merely to describe specific embodiments and are not intended to limit the present invention. Furthermore, unless specifically defined otherwise herein, the technical terms used herein should be interpreted as having a meaning generally understood by those skilled in the art to which the present invention pertains, and should not be interpreted in an excessively broad or narrow sense. Furthermore, if a technical term used herein is incorrect and does not accurately express the spirit of the present invention, it should be replaced with a technical term that can be correctly understood by those skilled in the art. Furthermore, general terms used herein should be interpreted according to their dictionary definitions or according to the context, and should not be interpreted in an excessively narrow sense.

[0035] Additionally, the singular expressions used herein include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "consist of" or "have" should not be construed to necessarily include all of the components or steps described in the specification, and should be construed to mean that some of the components or steps may not be included, or that additional components or steps may be included.

[0036] Additionally, terms including ordinal numbers, such as "first," "second," etc., used herein may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0037] When a component is referred to as being "connected" or "connected" to another component, it may be directly connected or connected to that other component, but there may also be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0038] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. In addition, when describing the present invention, if a detailed description of a related known technology is judged to obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the present invention, and the spirit of the present invention should not be construed as being limited by the attached drawings. The spirit of the present invention should be construed to extend to all modifications, equivalents, and substitutes other than the attached drawings.

[0039]

[0040] Meanwhile, as mentioned above, the braille translation services developed to date only predefine and express the Braille characters that correspond to each element within an electronic document. Therefore, they are significantly inadequate for representing electronic documents in unauthorized file formats, undefined elements within documents, the perceived order of elements based on their layout, and reference relationships between elements arranged in a separate location.

[0041] To overcome these limitations, the present invention proposes means for converting various formats of electronic documents into Braille documents based on artificial intelligence (AI).

[0042]

[0043] FIG. 1 is a configuration diagram of a Braille conversion system according to one embodiment of the present invention.

[0044] First, referring to FIG. 1, a Braille conversion system according to one embodiment of the present invention may be configured to include a user device (100a, 100b, …, 100n; 100) and a Braille conversion server (200).

[0045] As such, the components of the Braille conversion system according to one embodiment of the present invention merely represent functionally distinct elements, so two or more components may be implemented integrated with each other in an actual physical environment, or one component may be implemented separately from each other in an actual physical environment.

[0046] As for each component, the user device (100) is a device that can request Braille translation for an electronic document and output a Braille document corresponding to the requested electronic document.

[0047] The user of the user device (100) may be a visually impaired person, but is not limited thereto, and may also be a guardian of a visually impaired person, a person who intends to provide Braille documents to a visually impaired person, etc.

[0048] In addition, an electronic document is a document that can be created, stored, and transmitted / received in an electronic form rather than on paper. An electronic document according to the present invention may be a document that is subject to Braille translation, but is not limited thereto. In addition, an electronic document according to various embodiments of the present invention may be any one of a text-based document such as TXT (TeXT), HTML (Hyper Text Markup Language), Adobe's PDF (Portable Document Format), Microsoft's DOC (DOCument), DOCX (DOCument Xml), PPT (Powerpoint Presentation), PPTX (Powerpoint Presentation Xml), Hangul & Computer's HWP (Hangul Word Processor) and HWPX (Hangul Word Processor Xml) format, or an image-based document such as JPG / JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bit Map Protocol), and GIF (Graphics Interchange Format) format, but is not limited thereto.

[0049] The user device (100) according to various embodiments of the present invention may be any device capable of inputting and outputting data, transmitting and receiving input data, and calculating transmitted and received data. For example, the user device (100) may be any fixed computing device such as a desktop, workstation, or server, or a mobile computing device such as a laptop, tablet, phablet, portable multimedia player (PMP), personal digital assistant (PDA), or e-book reader, but is not limited thereto.

[0050] In the following configuration, the Braille conversion server (200) is a device that can generate a Braille document corresponding to an electronic document according to a request from a user device (100) and provide the generated Braille document to the user device (100).

[0051] In particular, the Braille conversion server (200) according to one embodiment of the present invention has the characteristic of being able to convert various formats of electronic documents into Braille documents based on artificial intelligence (AI). The specific configuration and operation of the Braille conversion server (200) will be described later with reference to FIGS. 4 to 12.

[0052] Meanwhile, the user device (100) and the Braille conversion server (200) of the Braille conversion system described above can transmit and receive data using a network that combines at least one of a secure line, a public wired communication network, and a mobile communication network that directly connects each device.

[0053] For example, public wired telecommunications networks may include, but are not limited to, Ethernet, Digital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). Mobile telecommunications networks may include, but are not limited to, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), High Speed ​​Packet Access (HSPA), Long Term Evolution (LTE), and 5th / 6th generation mobile telecommunications.

[0054]

[0055] Meanwhile, the components of the Braille conversion system described above may have different roles depending on the security, copyright, compatibility, or implementation difficulty required for electronic documents.

[0056] FIG. 2 and FIG. 3 are exemplary diagrams illustrating the configuration of a Braille conversion system with separated roles according to some embodiments of the present invention.

[0057] First, as illustrated in FIG. 2, the user device (100) of the Braille conversion system according to one embodiment of the present invention may be configured to perform only the role of transmitting an electronic document for which Braille translation is requested to the Braille conversion server (200) as is and outputting the Braille document received from the Braille conversion server (200).

[0058] In this case, high compatibility can be secured for various types of user devices (100), and the difficulty of implementation can be reduced because only the Braille conversion server (200) needs to be maintained and repaired.

[0059] In contrast, as illustrated in FIG. 3, a user device (100) of a Braille conversion system according to another embodiment of the present invention may be implemented to directly analyze an electronic document for which Braille has been requested, and then transmit a data exchange file including the analysis results to a Braille conversion server (200). Here, the data exchange file may be a file in the DIF (Data Interchange Format) or JSON (JavaScript Object Notation) format, but is not limited thereto.

[0060] In this case, since the Braille conversion server (200) performs Braille conversion on an electronic document that has undergone primary processing, security for the electronic document can be maintained and the risk of copyright infringement can be reduced.

[0061] As described above, the division of roles between the components of the Braille conversion system according to various embodiments of the present invention can be easily selected by a person having ordinary skill in the art to which the present invention pertains.

[0062]

[0063] Hereinafter, the logical configuration of the Braille conversion system according to various embodiments of the present invention as described above will be described in detail.

[0064] Figure 4 is a logical configuration diagram of a Braille conversion server according to one embodiment of the present invention. Figure 5 is a logical configuration diagram of a user device and a Braille conversion server according to another embodiment of the present invention.

[0065] FIG. 4 is a logical configuration diagram of a Braille conversion server (200) according to an embodiment implemented so that a user device (100) transmits an electronic document for which Braille has been requested to be translated as is to a Braille conversion server (200).

[0066] And, FIG. 5 is a logical configuration diagram of a user device (100) and a Braille conversion server (200) according to an embodiment implemented so that the user device (100) directly analyzes an electronic document for which Braille has been requested and then transmits the analysis result to the Braille conversion server (200).

[0067] However, the logical configurations according to FIGS. 4 and 5 are substantially the same in technical features, except that the only difference is whether the electronic document analysis unit (115, 215) is included in a component of the user device (100) or in a component of the Braille conversion server (200). Therefore, they will be described as one without being described repeatedly.

[0068] First, according to one embodiment of the present invention illustrated in FIG. 4, the Braille conversion server (200) may be configured to include a communication unit (205), an input / output unit (210), an electronic document analysis unit (215), a Braille attribute assignment unit (220), a Braille conversion unit (225), and a new type processing unit (230).

[0069] According to another embodiment of the present invention illustrated in FIG. 5, the user device (100) may be configured to include a communication unit (105), an input / output unit (110), and an electronic document analysis unit (115). In addition, the electronic conversion server (200) may be configured to include a communication unit (205), an input / output unit (210), a Braille attribute assignment unit (220), a Braille conversion unit (225), and a new type processing unit (230).

[0070] As described above, the components of the user device (100) and the Braille conversion server (200) merely represent functionally distinct elements, and therefore, two or more components may be implemented in an integrated manner in an actual physical environment, or one component may be implemented in a separate manner in an actual physical environment.

[0071] When explaining each component, the communication unit (105, 205) is configured for the user device (100) and the Braille conversion server (200) to transmit and receive data to each other.

[0072] Specifically, the communication unit (105) of the user device (100) can transmit an electronic document or data exchange file to the Braille conversion server (200), and the communication unit (205) of the Braille conversion server (200) can receive the electronic document or data exchange file transmitted from the user device (100).

[0073] The communication unit (205) of the Braille conversion server (200) transmits a Braille document to the user device (100), and the communication unit (105) of the user device (100) can receive the Braille document transmitted from the Braille conversion server (200).

[0074] In the following configuration, the input / output unit (110, 210) is configured to input or output various data required for Braille conversion.

[0075] Specifically, the input / output unit (110) of the user device (100) can receive an electronic document. The input / output unit (110) of the user device (100) can receive a request for Braille translation of the electronic document. In addition, the input / output unit (110) of the user device (100) can output a Braille document.

[0076] The input / output unit (210) of the Braille conversion server (200) can receive information regarding Braille attributes. The input / output unit (210) of the Braille conversion server (200) can receive a pattern or template for assigning Braille attributes.

[0077] The input / output unit (210) of the Braille conversion server (200) can receive various parameters that constitute an artificial neural network (ANN). In addition, the input / output unit (210) of the Braille conversion server (200) can receive information about an external database for performing Retrieval Augmented Generation (RAG).

[0078] In the following configuration, the electronic document analysis unit (115, 215) is configured to parse an electronic document that is the target of Braille translation and identify one or more elements included in the electronic document.

[0079] For convenience of explanation, the electronic document analysis unit (115, 215) will be described with reference to FIGS. 6 and 7.

[0080] FIG. 6 and FIG. 7 are exemplary diagrams for explaining a process of analyzing an electronic document according to one embodiment of the present invention.

[0081] First, as illustrated in FIG. 6, the electronic document analysis unit (115, 215) can parse objects included in an electronic document based on individual data values, formats, locations within the document, etc. Then, the electronic document analysis unit (115, 215) can identify one or more elements included in the electronic document based on the parsing results.

[0082] Here, elements are objects that constitute an electronic document. These elements may be, but are not limited to, any of the following: title, text, image, table, combo box, check box, radio button, and reference based on a callback function. Meanwhile, the electronic document analysis unit (115, 215) may be composed of multiple modules for analyzing each element by format.

[0083] According to one embodiment, the electronic document analysis unit (115, 215) can identify text-format elements included in an electronic document. If the identified text-format elements are composed of two or more texts spaced apart within the electronic document, the electronic document analysis unit (115, 215) can identify the two or more texts as individual multiple elements.

[0084] According to another embodiment, the electronic document analysis unit (115, 215) can identify one or more non-text format elements included in the electronic document. Here, the non-text format elements can be, but are not limited to, titles, images, tables, combo boxes, check boxes, radio boxes, and callback function-based references. The electronic document analysis unit (115, 215) can perform optical character recognition (OCR) on each identified non-text format element to recognize characters included in each non-text format element. In addition, the electronic document analysis unit (115, 215) can assign characters recognized by OCR to attribute values ​​of the identified non-text format elements.

[0085] Next, as illustrated in FIG. 7, the electronic document analysis unit (115, 215) can detect a layout in which text-format elements and non-text-format elements are arranged within an electronic document. Based on the detected layout, the electronic document analysis unit (115, 215) can assign a braille order to the text-format elements and non-text-format elements.

[0086] To this end, the electronic document analysis unit (115, 215) can search for a braille order corresponding to a layout detected from an electronic document, based on a pre-established layout-specific braille order database. Furthermore, the electronic document analysis unit (115, 215) can assign a braille order to text-format elements and non-text-format elements based on the search results.

[0087] In addition, the electronic document analysis unit (115, 215) can generate a data exchange file corresponding to the electronic document based on one or more elements and the order of the transliteration identified from the electronic document that is the target of the transliteration. Here, the data exchange file may be a file in DIF or JSON format, but is not limited thereto.

[0088] In the following configuration, the Braille attribute assignment unit (220) is configured to assign Braille attributes to each element identified from an electronic document using pre-trained artificial intelligence (AI). In this case, one or more elements identified from the electronic document may be input via a data exchange file.

[0089] Here, Braille attributes are attributes that must be referenced during the process of converting elements identified from electronic documents into Braille. These Braille attributes may include, but are not limited to, one or more of the following: data constituting the element, the element's type, whether it is eligible for Braille translation, its positional relationship with other elements, its reference relationship with other elements, and whether it can be replaced with other text.

[0090] For convenience of explanation, the Braille attribute assignment unit (220) will be described with reference to FIGS. 8 and 9.

[0091] FIG. 8 is an exemplary diagram illustrating a process for assigning Braille properties according to one embodiment of the invention of money. FIG. 9 is an exemplary diagram illustrating an example of Braille properties assigned according to one embodiment of the invention of money.

[0092] First, as illustrated in FIG. 8, the Braille attribute assignment unit (220) can assign Braille attributes to each element using one or more of a pattern or template, an artificial neural network (ANN), and augmented search generation (RAG).

[0093] More specifically, the Braille attribute assignment unit (220) can split each element identified from an electronic document into chunks of a preset size according to type. The Braille attribute assignment unit (220) can embed one or more of the split chunks to generate a feature vector.

[0094] The Braille attribute assignment unit (220) can determine whether the generated feature vector matches a predefined pattern or template. If there is a pattern or template matching the feature vector, the Braille attribute assignment unit (220) can assign Braille attributes to elements identified from the electronic document according to the matched pattern or template.

[0095] Here, a pattern is a rule defining Braille properties corresponding to representative elements identifiable in electronic documents. A template is a rule defining multiple Braille properties corresponding to combinations of multiple elements frequently identified in electronic documents.

[0096] To this end, the Braille attribute assignment unit (220) can determine whether a pattern or template is matched based on vector similarity with a feature vector, targeting an internal database in which a pattern or template configured to include one or more of a title, text, image, table, combo box, check box, radio button, and callback function-based reference that can be included in an electronic document is stored.

[0097] Here, vector similarity can be calculated based on cosine similarity, but is not limited thereto and can also be calculated based on Euclidean distance or Jaccard similarity.

[0098] According to one embodiment, the Braille attribute assignment unit (220) may perform multi-class classification on the feature vector using a pre-trained artificial neural network (ANN) if the feature vector does not match a pattern or template. Then, the Braille attribute assignment unit (220) may assign Braille attributes to each element identified from the electronic document based on the results of the multi-class classification.

[0099] To this end, the Braille attribute assignment unit (220) can perform multi-class classification using one of a recurrent neural network (RNN), a long short-term model (LSTM), and a gated recurrent unit (GRU), which are a series of artificial neural networks (ANN), but is not limited thereto.

[0100] According to another embodiment, if the feature vector does not match a pattern or template, the Braille attribute assignment unit (220) may retrieve data with the highest similarity to the feature vector from a pre-specified external database related to Braille. Then, the Braille attribute assignment unit (220) may generate Braille attributes for each element identified from the electronic document based on the retrieved data.

[0101] As an example of a variation, the Braille attribute assignment unit (220) can identify other elements located adjacent to the element to which the Braille attribute is assigned within the electronic document. In addition, the Braille attribute assignment unit (220) can perform a multi-query retrieval on an external database using both the feature vector of the element to which the Braille attribute is assigned and the feature vector of the other element.

[0102] As another example of modification, the Braille attribute assignment unit (220) can calculate vector similarity between feature vectors of two or more elements identified from an electronic document. Furthermore, if the calculated vector similarity is greater than a preset threshold value, the Braille attribute assignment unit (220) can additionally assign a Braille attribute that can express the existence of a reference relationship between the two or more identified elements.

[0103] And as illustrated in FIG. 9, the Braille attribute assignment unit (220) can generate a Braille text markup language (Brialle Text Markup Language, BTML) corresponding to an electronic document according to the assigned Braille attribute.

[0104] In the following configuration, the Braille conversion unit (225) is configured to generate a Braille document corresponding to an electronic document that is the target of Braille translation, based on the Braille properties assigned by the Braille properties assigning unit (220).

[0105] Specifically, the Braille conversion unit (225) can generate a file for a Braille information instrument based on the Braille Markup Language (BTML) generated by the Braille attribute assignment unit (220). Here, the file for the Braille information instrument may include, but is not limited to, one or more of an electronic Braille terminal file (Ballistic Research Laboratory, BRL), an electronic print file (Braille-Ready Format, BRF), and a Braille representation file (Braille Best File, BBF).

[0106] And, the Braille conversion unit (225) can transmit the generated Braille information terminal file to the user device (100).

[0107] In the following configuration, the new type processing unit (230) is configured to process a new type when it appears among the elements identified from an electronic document.

[0108] Specifically, the new type processing unit (230) can identify elements that are not assigned Braille properties based on a pattern or template, but rather are assigned Braille properties by an artificial neural network (ANN) or augmented search generation (RAG), in the process of assigning Braille properties to each element identified from an electronic document by the Braille property assigning unit (220).

[0109] In addition, the new type processing unit (230) can define a new pattern or a new template by matching the identified elements with the Braille properties provided by the artificial neural network (ANN) or augmented search generation (RAG).

[0110] As described above, the Braille conversion system according to various embodiments of the present invention can convert electronic documents of various file formats into Braille documents by performing Braille conversion using artificial intelligence (AI). In particular, it can convert elements contained in an electronic document into Braille in an easily readable order by considering the layout in which the elements are arranged. Even if elements not defined in advance appear during the conversion process of an electronic document, it can convert them into replaceable Braille. In addition, it can exhibit the effect of being able to express in Braille even reference relationships between elements arranged separately from each other within an electronic document.

[0111]

[0112] The hardware configuration of the Braille conversion server (200) for realizing the logical configuration described below will be described in detail.

[0113] Figure 10 is a hardware configuration diagram of a Braille conversion server according to one embodiment of the present invention.

[0114] As illustrated in FIG. 10, the Braille conversion server (200) may be configured to include a processor (250), memory (255), transceiver (260), input / output device (265), data bus (bus) 270, and storage (275).

[0115] Specifically, the processor (250) can implement the operation and function of the Braille conversion server (200) based on instructions according to software (280a) implementing the Braille conversion method residing in the memory (355).

[0116] Software (280b) implementing a Braille conversion method stored in storage (275) can be loaded into memory (255).

[0117] The input / output device (265) can input signals necessary for the operation of the Braille conversion server (200) or output operation results to the outside according to the command of the processor (250).

[0118] The data bus (270) is connected to the processor (250), memory (255), transceiver (260), input / output device (265), and storage (275), respectively, and can serve as a passage for transmitting signals between each component.

[0119] The storage (275) may store application programming interfaces (APIs), library files, resource files, etc. required for execution of software (280a) implementing a Braille conversion method according to embodiments of the present invention. The storage (275) may store software (280b) implementing a Braille conversion method according to embodiments of the present invention. In addition, the storage (275) may store a pre-trained artificial neural network (ANN) and a database (285).

[0120] According to an embodiment of the present invention, software (280a, 280b) for implementing a Braille conversion method residing in a memory (255) or stored in a storage (275) may be a computer program recorded on a recording medium to cause the processor (250) to execute a step of parsing an electronic document to be a target of Braille translation to identify one or more elements included in the electronic document, a step of assigning Braille properties to each element identified from the electronic document using pre-learned artificial intelligence, and a step of generating a Braille document corresponding to the electronic document based on the assigned Braille properties.

[0121] More specifically, the processor (250) may be configured to include one or more of a central processing unit (CPU), an application-specific integrated circuit (ASIC), a chipset, and a logic circuit, but is not limited thereto.

[0122] The memory (255) may be configured to include one or more of ROM (Read-Only Memory), RAM (Random Access Memory), flash memory, and memory card, but is not limited thereto.

[0123] The input / output device (260) may be configured to include one or more of input devices such as buttons, switches, keyboards, mice, and joysticks, and output devices such as liquid crystal displays (LCDs), light emitting diodes (LEDs), organic light emitting diodes (OLEDs), active matrix organic light emitting diodes (AMOLEDs), printers, and plotters, but is not limited thereto.

[0124] When the embodiments included in this specification are implemented in software, the above-described method may be implemented as modules (processes, functions, etc.) that each perform the above-described functions. Each module may reside in memory (255) and be executed by the processor (250). The memory (255) may exist inside or outside the processor (250) and may be connected to the processor (250) by various well-known means.

[0125] Each component illustrated in FIG. 10 may be implemented by various means (e.g., hardware, firmware, software, or a combination thereof). When implemented by hardware, an embodiment of the present invention may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0126] Furthermore, when implemented by firmware or software, one embodiment of the present invention may be implemented in the form of modules, procedures, functions, etc. that perform the functions or operations described above, and recorded on a readable recording medium via various computer means. Here, the recording medium may include program commands, data files, data structures, etc., singly or in combination.

[0127] The program commands recorded on the recording medium may be those specifically designed and configured for the present invention, or may be those known and available to those skilled in the computer software industry. For example, the recording medium includes magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands such as ROMs, RAMs, and flash memories.

[0128] Examples of program instructions may include not only machine language code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. Such hardware devices may be configured to operate as one or more software programs to perform the operations of the present invention, and vice versa.

[0129]

[0130] The operation of the Braille conversion system described above will be described in more detail below.

[0131] FIG. 11 and FIG. 12 are flowcharts for explaining a Braille conversion method according to some embodiments of the present invention.

[0132] In explaining the Braille conversion method according to embodiments of the present invention, specific descriptions of the step of analyzing an electronic document to identify elements, the step of assigning Braille properties to each element, and the step of generating a Braille document are the same as those described above with reference to FIGS. 4 to 9, and therefore will not be described again.

[0133] Referring to FIG. 11, one embodiment of the present invention will be described. A user device (100) can receive an electronic document to be transcribed (S110). The user device (100) can transmit the input electronic document to a Braille conversion server (200) (S120).

[0134] The Braille conversion server (200) can parse an electronic document to be transcribed and identify one or more elements contained within the electronic document (S130). The Braille conversion server (200) can assign Braille properties to each element identified from the electronic document using pre-trained artificial intelligence (AI) (S140). Based on the assigned Braille properties, the Braille conversion server (200) can generate a Braille document corresponding to the electronic document to be transcribed (S150). In addition, the Braille conversion server (200) can transmit the generated Braille document to the user device (100) (S160).

[0135] And, the user device (100) can output a Braille document received from the Braille conversion server (200) (S170).

[0136] Referring to FIG. 12, another embodiment of the present invention will be described. A user device (100) may receive an electronic document to be transcribed (S210). The user device (100) may parse the input electronic document, identify one or more elements contained within the electronic document, and then generate a data exchange file by including the identified elements (S220). Furthermore, the user device (100) may transmit the generated data exchange file to a Braille conversion server (200) (S230).

[0137] The Braille conversion server (200) can assign Braille properties to each input element through the received data exchange file (S240). Based on the assigned Braille properties, the Braille conversion server (200) can generate a Braille document corresponding to the electronic document to be Braille transcribed (S250). In addition, the Braille conversion server (200) can transmit the generated Braille document to the user device (100) (S260).

[0138] And, the user device (100) can output a Braille document received from the Braille conversion server (200) (S270).

[0139]

[0140] As described above, although the present specification and drawings have disclosed preferred embodiments of the present invention, it will be apparent to those skilled in the art that other modifications based on the technical idea of ​​the present invention can be implemented in addition to the embodiments disclosed herein. In addition, although specific terms have been used in the present specification and drawings, these have been used only in a general sense to easily explain the technical contents of the present invention and to help the understanding of the invention, and are not intended to limit the scope of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all aspects, but should be considered as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims

1. A step of parsing an electronic document to be subjected to braille translation and identifying one or more elements contained within the electronic document; A step of assigning a braille attribute to each element identified from the electronic document using pre-learned artificial intelligence (AI); and A Braille conversion method comprising the step of generating a Braille document corresponding to the electronic document based on the Braille properties granted above.

2. In the first paragraph, the step of identifying the element is A Braille conversion method characterized in that when a text-type element included in the electronic document is composed of two or more texts positioned apart from each other within the electronic document, the two or more texts are identified as a plurality of individual elements.

3. In the second paragraph, the step of identifying the element is A Braille conversion method characterized by identifying one or more non-text format elements included in the electronic document, and performing optical character recognition (OCR) on each of the identified non-text format elements to recognize characters included in each non-text format element.

4. In the third paragraph, the step of identifying the element is A Braille conversion method characterized by detecting a layout in which text-format elements and non-text-format elements are arranged within the electronic document, and assigning a Braille order to the text-format elements and non-text-format elements based on the detected layout.

5. In the first paragraph, the step of assigning the Braille attribute is A Braille conversion method characterized in that the identified elements are split into chunks of a preset size according to their types, the split chunks are embedded to generate a feature vector, and when the generated feature vector matches a predefined pattern or template, Braille properties are assigned to the identified elements according to the matched pattern or template.

6. In the fifth paragraph, the step of assigning the Braille attribute is A Braille conversion method characterized in that the matching is determined based on vector similarity with the identified feature vector, targeting an internal database in which a pattern or template configured to include one or more of a title, text, image, table, combo box, check box, radio button, and callback function-based reference that can be included in an electronic document is stored.

7. In the fifth paragraph, the step of assigning the Braille attribute is A Braille conversion method characterized in that, when the generated feature vector does not match a pattern or template, multi-class classification is performed on the feature vector using a pre-trained artificial neural network (ANN), and Braille properties are assigned to the identified element based on the result of the multi-class classification.

8. In the fifth paragraph, the step of assigning the Braille attribute is A Braille conversion method characterized in that, if the generated feature vector does not match a pattern or template, data having the highest similarity to the generated feature vector is retrieved from an external database specified in advance in relation to Braille, and Braille properties for the identified element are generated based on the retrieved data.

9. In the first paragraph, the step of creating the Braille document is A braille conversion method characterized in that it generates a braille text markup language (BTML) corresponding to the electronic document according to the above-mentioned assigned braille properties, and generates a file for a braille information instrument based on the generated braille text markup language (BTML).

10. Memory; and Combined with a computing device comprising a processor for processing instructions residing in the above memory, A step in which the processor parses an electronic document to be transcribed and identifies one or more elements included in the electronic document; The step of the processor assigning Braille properties to each element identified from the electronic document using pre-learned artificial intelligence; and A computer program recorded on a recording medium, for causing the processor to execute a step of generating a Braille document corresponding to the electronic document based on the assigned Braille attribute.

Images