Electronic terminal apparatus that provides braille conversion functionality for electronic documents and the operating method thereof

KR103015062B1Active Publication Date: 2026-09-04TJHE HANCOMM INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
KR1020240079397
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-06-19
Publication Date
2026-09-04
Estimated Expiration
2044-06-19

Smart Images

  • Figure 112024066055350-PAT00003_ABST
    Figure 112024066055350-PAT00003_ABST
Patent Text Reader

Abstract

The present invention provides a braille conversion function for electronic documents, and by proposing a technology that supports braille conversion even for images embedded in electronic documents, it can support braille conversion for electronic documents to be performed more accurately.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to an electronic terminal device that provides a Braille conversion function for electronic documents and a method of operation thereof. Background Technology

[0002] Recently, with the widespread adoption of computers, smartphones, and tablet PCs, various types of electronic document-related programs are being released that enable users to view, create, and edit electronic documents using these electronic terminal devices.

[0003] These electronic document-related programs include word processors that support the creation and editing of basic documents, spreadsheets that assist with data input, arithmetic operations, and data management, and presentation programs that assist the presenter's presentation.

[0004] Furthermore, with the recent widespread adoption of Braille printers for the visually impaired, electronic document programs are increasingly providing features that convert text embedded in electronic documents into Braille.

[0005] This Braille conversion function converts text composed of Korean or English embedded in an electronic document using a predetermined text-to-Braille conversion model, ' ', ' It refers to a function that converts into Braille in a form like this. Since electronic documents converted through this Braille conversion function can be printed as Braille paper documents using a Braille printer, visually impaired people can ultimately read electronic documents by looking at the Braille paper documents.

[0006] However, since existing braille conversion functions only converted text embedded in electronic documents into braille, there was a problem where proper braille conversion was not performed for images embedded in the electronic documents.

[0007] In this regard, if explanatory sentences describing what an image is can be generated in Braille when an image is embedded in an electronic document, it could help visually impaired individuals more clearly identify the image.

[0008] Therefore, in providing Braille conversion functions for electronic documents, research is needed on technology that supports Braille conversion for images embedded in the electronic documents as well. Prior art literature

[0009] Joseph Redmon et al., "You Only Look Once: Unified, Real-Time Object Detection", arXiv:1506.02640, May 2016. Ashish Vaswani et al., "Attention Is All You Need", arXiv:1706.03762, June 2017. The problem to be solved

[0010] The present invention aims to support more accurate Braille conversion of electronic documents by proposing a technology that enables Braille conversion even for images embedded in electronic documents, in providing a Braille conversion function for electronic documents. means of solving the problem

[0011] An electronic terminal device providing a Braille conversion function for an electronic document according to an embodiment of the present invention comprises: a verification unit that checks whether an image exists in the electronic document when a conversion command instructing a user to convert text inserted in the electronic document into Braille is received; a sentence generation unit that generates a descriptive sentence indicating the characteristics of each image corresponding to each of the at least one image when the verification result of the verification unit confirms that at least one image exists in the electronic document; a sentence insertion unit that inserts the descriptive sentence corresponding to each image at a point on the electronic document where each of the at least one image was located, after removing the at least one image from the electronic document when the descriptive sentence corresponding to each of the at least one image is generated; and a Braille conversion unit that converts both the text inserted in the electronic document and the descriptive sentence corresponding to each of the at least one image into Braille based on a preset text-to-Braille conversion model when the insertion of the descriptive sentence corresponding to each of the at least one image is completed.

[0012] Additionally, a method of operation of an electronic terminal device providing a Braille conversion function for an electronic document according to an embodiment of the present invention comprises: receiving a conversion command from a user instructing to convert text inserted in an electronic document into Braille, a step of checking whether an image exists in the electronic document; if, as a result of the checking step, it is confirmed that at least one image exists in the electronic document, a step of generating a descriptive sentence indicating the characteristics of each image corresponding to each of the at least one image; when the descriptive sentence corresponding to each of the at least one image is generated, a step of removing the at least one image from the electronic document and inserting the descriptive sentence corresponding to each image at a point on the electronic document where each of the at least one image was located; and when the insertion of the descriptive sentence corresponding to each of the at least one image is completed, a step of converting both the text inserted in the electronic document and the descriptive sentence corresponding to each of the at least one image into Braille based on a pre-set text-to-Braille conversion model. Effects of the invention

[0013] The present invention provides a braille conversion function for electronic documents, and by proposing a technology that supports braille conversion even for images embedded in electronic documents, it can support braille conversion for electronic documents to be performed more accurately. Brief explanation of the drawing

[0014] FIG. 1 is a diagram illustrating the structure of an electronic terminal device that provides a Braille conversion function for an electronic document according to an embodiment of the present invention. FIGS. 2 to 4 are drawings for explaining the operation of an electronic terminal device that provides a Braille conversion function for an electronic document according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating the operation method of an electronic terminal device that provides a braille conversion function for an electronic document according to an embodiment of the present invention. Specific details for implementing the invention

[0015] Embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. This description is not intended to limit the present invention to specific embodiments and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. Similar reference numerals have been used for similar components in describing each drawing, and unless otherwise defined, all terms used in this specification, including technical or scientific terms, have the same meaning as generally understood by a person skilled in the art to which the present invention pertains.

[0016] In this document, when a part is described as "including" a component, it means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, in various embodiments of the present invention, each component, functional block, or means may be composed of one or more sub-components, and the electrical, electronic, or mechanical functions performed by each component may be implemented by various known devices or mechanical elements, such as electronic circuits, integrated circuits, and ASICs (Application Specific Integrated Circuits), and may be implemented separately or two or more may be integrated into one.

[0017] Meanwhile, the blocks in the attached block diagram or the steps in the flowchart may be interpreted as computer program instructions that perform designated functions by being loaded into the processor or memory of data-processing equipment, such as general-purpose computers, specialized computers, portable notebook computers, and network computers. Since these computer program instructions may be stored in memory provided in a computer device or in memory readable by a computer, the functions described in the blocks in the block diagram or the steps in the flowchart may be produced as manufactured products containing means of instruction to perform them. Furthermore, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specific logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may be executed in a different order than the prescribed order. For example, two blocks or steps shown in succession may be performed substantially simultaneously or in reverse order, and in some cases, some blocks or steps may be omitted.

[0018] FIG. 1 is a diagram illustrating the structure of an electronic terminal device that provides a Braille conversion function for an electronic document according to an embodiment of the present invention.

[0019] Referring to FIG. 1, the electronic terminal device (110) according to the present invention includes a verification unit (111), a sentence generation unit (112), a sentence insertion unit (113), and a Braille conversion unit (114).

[0020] When the verification unit (111) receives a conversion command from the user instructing to convert text inserted in the electronic document into Braille, it checks whether an image exists in the electronic document.

[0021] When the sentence generation unit (112) confirms, based on the confirmation result of the confirmation unit (111), that at least one image exists in the electronic document, it generates a descriptive sentence indicating the characteristics of each image corresponding to each of the at least one image.

[0022] For example, if it is confirmed that there are three images called ‘Image 1, 2, and 3’ in the above electronic document, the sentence generation unit (112) can generate an explanatory sentence that indicates the characteristics of each image corresponding to each of ‘Image 1, 2, and 3’.

[0023] At this time, according to one embodiment of the present invention, the sentence generation unit (112) can generate a description sentence corresponding to each of the at least one image by assigning a sequence number for generating a description sentence to each of the at least one image, and then generating a description sentence indicating the features of each image sequentially according to the assigned sequence number.

[0024] At this time, the sentence generation unit (112) may assign a sequence number to each of the at least one image according to the positional order in which the at least one image is inserted on the electronic document, or may assign a sequence number to each of the at least one image in a random manner.

[0025] In this regard, as in the example described above, if at least one image is referred to as 'Image 1, 2, 3', and 'Image 1' is inserted on page 1 of the electronic document, 'Image 2' is inserted on page 4 of the electronic document, and 'Image 3' is inserted on page 6 of the electronic document, the sentence generation unit (112) can assign a sequence number '1' to 'Image 1', a sequence number '2' to 'Image 2', and a sequence number '3' to 'Image 3' according to the order of insertion positions of 'Image 1, 2, 3' in the electronic document. As another example, the sentence generation unit (112) can assign a sequence number in a random manner to each of 'Image 1, 2, 3'. In this regard, the sentence generation unit (112) can assign a sequence number '2' to 'Image 1', a sequence number '3' to 'Image 2', and a sequence number '1' to 'Image 3'.

[0026] At this time, according to one embodiment of the present invention, the sentence generation unit (112) may include an object identification unit (115) and a description sentence generation unit (116) as specific configurations for generating a description sentence for each of the at least one image.

[0027] When the object identification unit (115) has a turn to generate a description sentence for a first image, which is one of the at least one image, it obtains an object name for an object existing on the first image from the first image based on a preset object identification model.

[0028] Here, the object identification model refers to an artificial intelligence-based model that has been pre-machine-learned to identify one or more objects present in an image and output an object name referring to the identified object, and a YOLO (You Only Look Once) based object identification model may be utilized.

[0029] At this time, according to one embodiment of the present invention, when it is time for the object identification unit (115) to generate a description sentence for the first image, which is one of the at least one image, it can generate four divided images of the same size by dividing the first image horizontally and vertically based on the center point of the first image, and additionally generate a center image having the same size as the four divided images located at the center of the first image based on the center point of the first image, and then, for each of the four divided images and the center image, obtain an object name for an object existing on each image by performing object identification based on the object identification model, and then, for results that overlap among the obtained object names, process them as a single object name.

[0030] Hereinafter, with reference to FIGS. 2 and FIGS. 3, the operation of the object identification unit (115) will be explained in detail with an example.

[0031] First, let us assume that the electronic document is the document indicated by reference numeral 210 in FIG. 2, and that 'Image 1, 2, 3' are inserted into the electronic document as in the example described above. At this time, let us assume that 'Image 1' (211) is inserted into a specific page of the electronic document (210) as shown in FIG. 2, and that it is time to generate a descriptive sentence for 'Image 1' (211) among 'Image 1, 2, 3'.

[0032] Then, the object identification unit (115) can generate ‘divided image 1, divided image 2, divided image 3, divided image 4’ (311, 312, 313, 314) into four divided images of the same size by dividing ‘image 1’ (211) horizontally and vertically based on the center point of ‘image 1’ (211), as shown in reference numeral 310 of FIG. 3.

[0033] Then, the object identification unit (115) can additionally generate a center image (321) having the same size as the 'divided image 1, divided image 2, divided image 3, divided image 4' (311, 312, 313, 314), which is located at the center of 'image 1' (211) based on the center point of 'image 1' (211) as shown in reference numeral 320 of FIG. 3.

[0034] After that, the object identification unit (115) can obtain object names for objects existing on each image by performing identification of objects included in each image based on the object identification model for each of the 'divided image 1, divided image 2, divided image 3, divided image 4' (311, 312, 313, 314) and the center image (321).

[0035] Then, the object identification unit (115) can process duplicate results among the acquired object names into a single object name.

[0036] In this regard, if the object identification unit (115) obtains an object name of 'tree' from 'divided image 1' (311) based on the object identification model, obtains an object name of 'person' from 'divided image 2' (312), obtains an object name of 'tree' from 'divided image 3' (313), obtains an object name of 'person' from 'divided image 4' (314), and obtains an object name of 'car' from the center image (321), the object identification unit (115) integrates the duplicate result of 'tree' obtained from 'divided image 1' (311) and 'divided image 3' (313) into a single object name of 'tree', and integrates the duplicate result of 'person' obtained from 'divided image 2' (312) and 'divided image 4' (314) into a single object name of 'person', thereby integrating the object names included in 'image 1' (211) into 'tree, person, It can be acquired in the form of 'tree, person, car' rather than 'tree, person, car'.

[0037] According to the method described so far, when the object identification unit (115) obtains object names from the first image and n (where n is a natural number) object names are obtained from the first image, the description sentence generation unit (116) generates a description sentence indicating the features of the first image based on the n object names.

[0038] At this time, according to one embodiment of the present invention, the description sentence generation unit (116) can generate a description sentence describing the features of the first image by inputting a prompt instructing a preset sentence generation model to generate a sentence describing the features of the image composed of the n object names when the n object names are obtained from the first image as a result of obtaining object names through the object identification unit (115).

[0039] Here, the sentence generation model refers to a generative artificial intelligence model that receives a predetermined prompt and automatically generates a sentence corresponding to the input prompt based on a pre-built Large Language Model (LM), and a Transformer-based language model may be utilized.

[0040] In this regard, as in the example described above, the first image is referred to as ‘Image 1’ (211) of FIG. 2, and the object identification unit (115) obtains object names such as ‘tree, person, car’ from ‘Image 1’ (211). Then, the description sentence generation unit (116) constructs a prompt in the form of ‘Generate a sentence describing the features of the image composed of a tree, person, and car’ based on the object names such as ‘tree, person, car’, and then inputs this as an input to the sentence generation model, thereby obtaining a description sentence indicating the features of ‘Image 1’ (211), such as ‘Cars are speeding along the road, and people are walking leisurely on the sidewalk where large trees are planted,’ from the sentence generation model.

[0041] According to the method described so far, the sentence generation unit (112) can generate a description sentence corresponding to each image by performing the process of obtaining an object name through the object identification unit (115) and the process of generating a description sentence through the description sentence generation unit (116) for each of at least one image existing on the electronic document. That is, as in the example described above, if 'Image 1, 2, and 3' exist on the electronic document, the sentence generation unit (112) can generate a description sentence corresponding to 'Image 1', a description sentence corresponding to 'Image 2', and a description sentence corresponding to 'Image 3'.

[0042] Thus, when an explanatory sentence corresponding to each of the at least one image is generated through the sentence generation unit (112), the sentence insertion unit (113) removes the at least one image from the electronic document and then inserts the explanatory sentence corresponding to each image at the point on the electronic document where each of the at least one image was located.

[0043] In this regard, as in the example above, if 'Image 1, 2, and 3' exist on the electronic document, the sentence insertion unit (113) can remove all 'Image 1, 2, and 3' from the electronic document, and then insert an explanatory sentence corresponding to 'Image 1' at the point on the electronic document where 'Image 1' was located, insert an explanatory sentence corresponding to 'Image 2' at the point on the electronic document where 'Image 2' was located, and insert an explanatory sentence corresponding to 'Image 3' at the point on the electronic document where 'Image 3' was located.

[0044] For example, among ‘Images 1, 2, and 3’, if ‘Image 1’ is inserted as shown in the figure in FIG. 2, and a descriptive sentence corresponding to ‘Image 1’ (211) is generated such as ‘Cars are speeding along the road and people are walking leisurely on the sidewalk where large trees are planted’, the sentence insertion part (113) can remove ‘Image 1’ (211) from the electronic document (210) as shown in FIG. 4, and then insert the descriptive sentence corresponding to ‘Image 1’ (211), ‘Cars are speeding along the road and people are walking leisurely on the sidewalk where large trees are planted’ (412), at the location where ‘Image 1’ (211) was located.

[0045] When the insertion of the explanatory sentence corresponding to each of the at least one image is completed, the Braille conversion unit (114) converts both the text inserted in the electronic document and the explanatory sentence corresponding to each of the at least one image into Braille based on a preset text-to-Braille conversion model.

[0046] Here, the text-to-Braille conversion model is a model that converts text written in Korean or English, etc., into Braille, and may be configured in a form that converts the target text into Braille by referring to a dictionary database that records a predetermined text and its corresponding Braille, or may be configured in the form of an artificial intelligence-based translation model for translating text into Braille.

[0047] In this regard, as in the example above, if 'Image 1, 2, 3' exists on the electronic document and 'Image 1, 2, 3' is replaced with an explanatory sentence corresponding to each image, the Braille conversion unit (114) can convert both the text inserted in the electronic document and the explanatory sentence corresponding to each of 'Image 1, 2, 3' into Braille based on the text-to-Braille conversion model.

[0048] For example, as illustrated in FIG. 4, the process of performing Braille conversion on a page in which an explanatory sentence (412) corresponding to 'Image 1' (211) among 'Images 1, 2, and 3' is inserted is explained, and the Braille conversion unit (114) can convert both the text (411) inserted in the electronic document (210) and the explanatory sentence (412) corresponding to 'Image 1' (211) into Braille. In this way, the Braille conversion unit (114) can convert both the text inserted in the electronic document (210) and the explanatory sentences corresponding to each of 'Images 1, 2, and 3' into Braille.

[0049] In this way, once the Braille conversion of the above electronic document is completed, the user can create a Braille paper document for the visually impaired by printing the converted electronic document as a paper document using a designated Braille printer.

[0050] FIG. 5 is a flowchart illustrating the operation method of an electronic terminal device that provides a braille conversion function for an electronic document according to an embodiment of the present invention.

[0051] In step (S510), when a conversion command is received from the user instructing to convert text inserted in an electronic document into Braille, it is checked whether an image exists in the electronic document.

[0052] In step (S520), if it is confirmed that at least one image exists in the electronic document as a result of the verification in step (S510), a descriptive sentence indicating the characteristics of each image corresponding to each of the at least one image is generated.

[0053] In step (S530), when a descriptive sentence corresponding to each of the at least one image is generated, the at least one image is removed from the electronic document, and then the descriptive sentence corresponding to each image is inserted at the point on the electronic document where each of the at least one image was located.

[0054] In step (S540), when the insertion of the explanatory sentence corresponding to each of the at least one image is completed, the text inserted in the electronic document and the explanatory sentence corresponding to each of the at least one image are all converted into Braille based on a preset text-to-Braille conversion model.

[0055] At this time, according to one embodiment of the present invention, in step (S530), a sequence number for generating a description sentence is assigned to each of the at least one image, and then a description sentence indicating the features of each image is generated sequentially according to the assigned sequence number, thereby generating a description sentence corresponding to each of the at least one image.

[0056] At this time, according to one embodiment of the present invention, in step (S530), when it is time to generate a descriptive sentence for a first image, which is one of the at least one image, the method may include the step of obtaining an object name for an object existing on the first image based on a pre-set object identification model (the object identification model refers to a pre-machine-learned artificial intelligence-based model that identifies one or more objects existing on an image and outputs an object name referring to the identified object) and, if n (n is a natural number) object names are obtained from the first image as a result of obtaining the object name through the step of obtaining the object name, the method may include the step of generating a descriptive sentence indicating the features of the first image based on the n object names.

[0057] At this time, according to one embodiment of the present invention, the step of obtaining the object name can be performed by, when it is time to generate a description sentence for the first image, which is one of the at least one image, dividing the first image horizontally and vertically based on the center point of the first image to generate four divided images of the same size, and additionally generating a center image located at the center of the first image based on the center point of the first image and having the same size as the four divided images, and then performing object identification based on the object identification model for each of the four divided images and the center image to obtain the object name for the object existing on each image, and then integrating any duplicate results among the obtained object names into a single object name.

[0058] In addition, according to one embodiment of the present invention, the step of generating a descriptive sentence indicating the features of the first image can be performed by inputting a prompt instructing a pre-set sentence generation model (the sentence generation model is a generative artificial intelligence model that receives a prompt and automatically generates a sentence corresponding to the input prompt based on a pre-built LLM (Large Language Model)) to generate a sentence describing the features of the image composed of the n object names when the n object names are obtained from the first image as a result of obtaining the object names through the step of obtaining the object names.

[0059] Hereinafter, a method of operation of an electronic terminal device according to an embodiment of the present invention has been described with reference to FIG. 5. Here, since the method of operation of an electronic terminal device according to an embodiment of the present invention may correspond to the configuration of the operation of the electronic terminal device (110) described using FIG. 1 to 4, a more detailed description thereof will be omitted.

[0060] A method of operation of an electronic terminal device according to one embodiment of the present invention can be implemented as a computer program stored in a storage medium for execution through combination with a computer.

[0061] In addition, the method of operation of an electronic terminal device according to one embodiment of the present invention may be implemented in the form of computer program instructions for execution through combination with a computer and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0062] As described above, the present invention has been explained by specific details such as specific components, limited embodiments, and drawings; however, this is provided merely to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments. A person skilled in the art can make various modifications and variations from this description.

[0063] Accordingly, the scope of the present invention is not limited to the described embodiments, and all things equivalent to or having equivalent variations to the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention. Explanation of the symbols

[0064] 110: Electronic terminal device 111: Verification section 112: Sentence generation section 113: Sentence insertion section 114: Braille conversion section 115: Object Identifier 116: Description Sentence Generator

Claims

Claim 1 An electronic terminal device providing a Braille conversion function for an electronic document comprises: a verification unit that checks whether an image exists in the electronic document when a conversion command instructing a user to convert text inserted in the electronic document into Braille is received; a sentence generation unit that, if the verification result of the verification unit confirms that at least one image exists in the electronic document, assigns a sequence number for generating a descriptive sentence to each of the at least one image, and then generates a descriptive sentence indicating the characteristics of each image sequentially according to the assigned sequence number, thereby generating a descriptive sentence indicating the characteristics of each image corresponding to each of the at least one image; and a sentence insertion unit that, when a descriptive sentence corresponding to each of the at least one image is generated, removes the at least one image from the electronic document and then inserts the descriptive sentence corresponding to each image at a point on the electronic document where each of the at least one image was located. The apparatus includes a Braille conversion unit that, when the insertion of a descriptive sentence corresponding to each of the at least one image is completed, converts both the text inserted in the electronic document and the descriptive sentence corresponding to each of the at least one image into Braille based on a preset text-to-Braille conversion model; and when the sentence generation unit becomes the turn to generate a descriptive sentence for a first image, which is one of the at least one image, an object identification unit that obtains an object name for an object existing on the first image from the first image based on a preset object identification model—the object identification model refers to an AI-based model that has been pre-machine-learned to identify one or more objects existing on an image and output an object name referring to the identified object.An electronic terminal device comprising an explanation sentence generation unit that, when n (where n is a natural number) object names are obtained from the first image as a result of obtaining object names through the object identification unit, generates an explanation sentence indicating a feature of the first image based on the n object names. Claim 2 delete Claim 3 delete Claim 4 An electronic terminal device according to claim 1, wherein the object identification unit, when it is time to generate a descriptive sentence for the first image, which is any one of the at least one image, generates four divided images of the same size by dividing the first image horizontally and vertically based on the center point of the first image, and additionally generates a center image having the same size as the four divided images located at the center of the first image based on the center point of the first image, and then, for each of the four divided images and the center image, performs object identification based on the object identification model to obtain an object name for an object existing on each image, and then integrates any duplicate results among the obtained object names into a single object name. Claim 5 An electronic terminal device according to claim 1, wherein, when the above-described sentence generation unit obtains the above-described n object names from the above-described first image as a result of obtaining object names through the above-described object identification unit, the above-described sentence generation unit generates an explanatory sentence indicating the features of the above-described first image by inputting a prompt instructing a preset sentence generation model—the above-described sentence generation model is a generative artificial intelligence model that receives a prompt and automatically generates a sentence corresponding to the input prompt based on a preset LLM (Large Language Model)—to generate a sentence describing the features of the above-described first image composed of the above-described n object names. Claim 6 A method of operation of an electronic terminal device that provides a Braille conversion function for an electronic document, comprising: receiving a conversion command from a user instructing to convert text inserted in an electronic document into Braille, a step of checking whether an image exists in the electronic document; if, as a result of the checking step, it is confirmed that at least one image exists in the electronic document, a step of assigning a sequence number for generating a descriptive sentence to each of the at least one image, and then generating a descriptive sentence indicating the characteristics of each image sequentially according to the assigned sequence number, thereby generating a descriptive sentence indicating the characteristics of each image corresponding to each of the at least one image; and when a descriptive sentence corresponding to each of the at least one image is generated, a step of removing the at least one image from the electronic document, and then inserting the descriptive sentence corresponding to each image at a point on the electronic document where each of the at least one image was located. The method includes the step of converting all text inserted in the electronic document and the explanatory sentences corresponding to each of the at least one image into Braille based on a preset text-to-Braille conversion model when the insertion of the explanatory sentences corresponding to each of the at least one image is completed, and the step of generating the explanatory sentences includes, when it is the turn to generate an explanatory sentence for a first image which is one of the at least one image, the step of obtaining an object name for an object existing on the first image from the first image based on a preset object identification model—the object identification model refers to an AI-based model that has been pre-machine-learned to identify one or more objects existing on an image and output an object name referring to the identified object.A method of operation of an electronic terminal device comprising the step of generating a descriptive sentence indicating a feature of the first image based on the n object names, when, as a result of obtaining object names through the step of obtaining object names, n (where n is a natural number) object names are obtained from the first image. Claim 7 delete Claim 8 delete Claim 9 In claim 6, the step of obtaining the object name is characterized by obtaining the object name for an object existing on the first image in a manner such that, when it is time to generate a descriptive sentence for the first image, which is any one of the at least one image, the first image is divided horizontally and vertically based on the center point of the first image to generate four divided images of the same size, and a center image is additionally generated that is located at the center of the first image based on the center point of the first image and has the same size as the four divided images, and then, for each of the four divided images and the center image, object identification is performed based on the object identification model to obtain the object name for an object existing on each image, and any duplicate results among the obtained object names are integrated into a single object name. Claim 10 In claim 6, the step of generating an explanatory sentence indicating the features of the first image is characterized by generating an explanatory sentence indicating the features of the first image by inputting a prompt instructing to generate a sentence describing the features of the image composed of the n object names to a preset sentence generation model—where n object names are obtained from the first image as a result of obtaining object names through the step of obtaining object names—the sentence generation model is a generative artificial intelligence model that receives a prompt and automatically generates a sentence corresponding to the input prompt based on a preset Large Language Model (LLM). Claim 11 A computer-readable recording medium having a computer program for executing the method of any one of paragraphs 6, 9, or 10 through combination with a computer. Claim 12 A computer program stored on a storage medium for executing the method of any one of paragraphs 6, 9, or 10 through combination with a computer.

Citation Information

Patent Citations

  • Method and system for supporting braille facebook for blind

    KR1020140096724A

  • Braille trainning apparatus and braille translation method using it

    KR1020190023464A

  • System for Sending Documents Containing Private Information for Blind User

    KR1020220115659A

  • Information processor for visually impaired person and information distribution service using the same

    JP2002366024A