Chinese ancient book reading sequence sorting method and device based on adjacent character detection

Abstract

The invention relates to a Chinese ancient book reading sequence sorting method and device based on adjacent character detection. The method for sorting the reading sequence of the Chinese and ancient books comprises the following steps of: selecting an analysis object frame from an OCR (Optical Character Recognition) detection frame of an original image of the Chinese and ancient books; a step for generating a first movement path comprising a frame corresponding to the text and the right detailed note of the main column of the current analysis object and a second movement path comprising a frame corresponding to the text and the left detailed note of the main column by using an adjacent frame continuous detection method on the basis of the coordinate information and the size information of the frame of the analysis object; and a step for obtaining a union of frames included in the first movement path and frames included in the second movement path, and generating a frame list of the main column to be analyzed currently.

Description

Technical Field

[0001] The present invention relates to a technology for automatically generating digital text from a document image. More specifically, the present invention relates to a method and device for automatically sorting the reading order of characters contained in a document image. Background Art

[0002] As of 2023, the total number of ancient Chinese books in Korea is expected to exceed 3 million. Since 1999, the digitization of Korean studies materials has been promoted as part of a national project, but the actual situation is that the scale of digital texts is negligible compared to the total number of ancient documents preserved, due to the inefficiency of the manual computerization process and the individual character identification by a small number of Chinese character experts.

[0003] To overcome this situation, artificial intelligence-based optical character recognition (OCR) technology is emerging as an alternative solution that automatically detects and recognizes text contained in images, thereby being able to extract information on individual Chinese characters up to hundreds of words at a time.

[0004] Figure 1 1 is a diagram showing the process of digital texting of ancient documents based on OCR. First, the original ancient book 11 is scanned or photographed to obtain a digital image 12 of the ancient book. The OCR algorithm can form a bounding box 13 containing the position (x, y) and size (w, h) information of each character in the digital image 12 of the ancient book and infer (recognize) the type of the glyph. The inference result 14 includes the glyph and its position (x, y) and size (w, h). However, the inference result 14 only provides independent information for individual Chinese characters, and cannot reconstruct their continuous set, that is, sentence. The actual situation is that in order to generate a digital text 15 that is directly used in the translation process as a subsequent step, manual reading order sorting based on manpower is required. This manual post-processing process becomes a bottleneck in the entire ancient document digitization process, thereby increasing operation time and cost, so there is a problem that the potential of OCR technology cannot be fully utilized.

[0005] Figure 2a and Figure 2b This is a diagram used to illustrate the reading order of ancient documents. Figure 2a is a text image containing only the main text. Figure 2b This image shows the original text with mixed text and detailed notes. Ancient documents from countries belonging to the Chinese character cultural circle not only left most of the records in Chinese characters, but also Figure 2a That follows the form of writing vertically from right to left, starting from the right column and writing from top to bottom, then moving to the left column 21.

[0006] Figure 2b is the original text image containing the main text 22, 24 and detailed annotations 23-1, 23-2. In Figure 2b , the reference symbol 25 indicates the main column. In the structure of ancient documents, the detailed annotation refers to the part where the smaller characters in a single column of the main text are divided into two columns as Figure 2b . The detailed annotation functions as a note to supplement the previous main text.

[0007] In the existing human-based digitalization process of ancient documents, due to budget and time constraints, it is mostly the case that only the main text excluding the detailed annotations is translated. However, there are also cases where the detailed annotations convey important information in historical research. Therefore, it is necessary to develop a technology for effective complete translation including the detailed annotations.

[0008] On the other hand, different from the original text image that only contains the main text ( Figure 2a ), for the original text image mixed with detailed annotations ( Figure 2b ), the direction of the reading order is somewhat ambiguous like an arrow. Generally speaking, when detailed annotations appear during the reading order in the format of vertical writing from right to left, within the cluster of these detailed annotations 23, the right detailed annotation column (right detailed annotation, 23-1) should be read first, and then move to the left detailed annotation column (left detailed annotation, 23-2). In Figure 2b the example of, the reading order should be "Jihai Sir Thirty-nine Years October" (in the order of 22, 23-1, 23-2, 24).

[0009] However, in the case of simply sorting based on the principle of vertical writing from right to left along the right-to-left direction (x-axis) or top-to-bottom direction (y-axis), the results are respectively shown as "Sir Three Jihai October Nineteen Years" and "Jihai Ten First Nine Sir Years Thirty Months", resulting in misreading. Even more, in the case where the sorting of the rows and columns of the characters in the image is not perfect, the recognition result is more difficult to predict, and the same problem may also occur in the case of Figure 2a without detailed annotations. For these reasons, research has been conducted on the algorithm for line segmentation of the original text image (in the case of horizontal writing) or column segmentation (in the case of vertical writing) and the reading order of characters within each segmented element.

[0010] The line segmentation of the original text image can be roughly divided into top-down and bottom-up methods. In the former case, it is a method of setting regions for the entire image, and in the latter case, it is a method of clustering based on the position information of individual characters detected in advance.

[0011] The pixel projection profile technique (non-patent document 1), which is an early top-down line segmentation method, analyzes the histogram of pixels in a binary image projected onto one axis and forms a boundary line in the part where the pixel density is lower than a threshold. In the above technique, the closer the distance between rows or columns, the higher the boundary surface threshold must be set, thereby increasing the error rate. In addition, when pixel projection is applied to the main text-detailed annotation composite structure, the information of the main text part and the detailed annotation part overlaps, making it difficult to separate the constituent elements (see Figure 3 C2, C4).

[0012] In order to overcome the problem of overlapping projections between the main text and the detailed notes in the pixel projection technique, a technique (point projection profile) (non-patent document 2) is proposed, which only projects the center coordinates of the characters detected by OCR instead of projecting all the pixels that make up the characters. The above technique successfully separates the main text and the detailed notes for ancient Chinese books with high row and column ordering.

[0013] Figure 3 This is a diagram for explaining the results of comparing the line segmentation performance of the pixel projection technique and the center coordinate projection technique.

[0014] exist Figure 3 In the figure, P1 represents the pixel projection direction, P2 represents the center coordinate projection direction, G1 represents the pixel projection contour, and G2 represents the center coordinate projection contour.

[0015] In the right Figure 3 When the pixel projection technique is applied to the image, low-density pixel areas are clearly seen in the three sections between the four main columns C1 to C4 and in the center of the third column C3 consisting only of detailed notes. However, in the case where the main text and detailed notes are mixed, such as in the second column C2 and the fourth column C4, the projections of the elements of each detailed note column (right detailed note, left detailed note) overlap with the projections of the main text elements, making it difficult to separate them.

[0016] In contrast, the center coordinate projection technique can identify clear clustering by increasing the resolution of the character box coordinate projection volume, and can perform line segmentation when the distance between clusters is greater than a certain threshold.

[0017] However, due to the characteristics of the center coordinate projection contour technique based on the top-down line segmentation method, it attempts to sort all the characters from the top to the bottom of the column together, so it has the disadvantage of not being able to withstand the skew or warping that occurs in the process of ancient book production or digital image acquisition.

[0018] Since then, although research on the development of line segmentation models based on deep learning has been actively conducted (non-patent document 3), some people have pointed out the inefficiency of the human-based labeling required in the process of pre-learning database construction. In the case of unsupervised learning-based models developed to circumvent this (non-patent document 4), there is a disadvantage that the accuracy of line segmentation will be greatly reduced even if the standardization of the original text is slightly reduced.

[0019] As a representative example of bottom-up reading order sorting, there has been an attempt in Korea to determine the reading order by comparing the geometric distance from the lower left end of the document based on the center coordinates of each character detected by OCR (Non-Patent Document 5), but it is unable to cope with non-standard structures such as detailed notes.

[0020] The bottom-up reading order sorting method attempts to mathematically analyze the two coordinate information (x, y) of the characters, so there is a problem of overcoming the uncertainty of the two degrees of freedom. That is, for each character, there is a problem of difficulty in determining the comprehensive priority of the two position information on the x-axis and the y-axis. For example, assuming that the upper rightmost box detection is the target, in the bottom-up reading order sorting, it is usually easy to assume that it is enough to find the "uppermost of the rightmost" or the "rightmost of the uppermost". However, the coordinates of the detection box are expressed by two values, x and y. At the moment of taking the first extreme condition (rightmost or uppermost), the correct answer has been determined as a rightmost or uppermost box, and there is a problem that the second extreme condition (uppermost or rightmost) cannot be applied.

[0021] Prior Art Literature

[0022] Non-patent literature

[0023] Non-patent literature 1: R. Ptak, et al., “Projection-based text line segmentation with a variable threshold”, Int. J. Appl. Math. Comput. Sci., vol. 27, pp. 195, 2017.

[0024] Non-patent document 2: Lee Ya-lan, et al., "Line segmentation and reading order sorting algorithm for digitizing ancient Korean documents", JDCS, vol.23, no.11, pp.2239, 2022.

[0025] Non-patent literature 3: O.Mechi, et al., "Text line segmentation in historical document images using an adaptive U-Net architecture", 2019ICDAR, pp.369, 2019.

[0026] Non-patent literature 4: BKBarakat, et al., "Unsupervised deep learning for textline segmentation", 25th ICPR, 2021.

[0027] Non-patent literature 5: Liu Junhuan, et al., "Detection result numbering for detecting and translating ancient document cursive script", ICROS2019, pp.140, 2019. Summary of the invention

[0028] Problem to be solved

[0029] The purpose of the present invention is to provide a method and device for sorting the reading order of ancient Chinese books based on adjacent character detection for automatically converting ancient documents into digital texts.

[0030] In addition, the present invention aims to provide a method and device for sorting the reading order of ancient Chinese books based on neighboring character detection, which has high processing capabilities for skew and warping that occur during the initial production process of the ancient books or the digital image acquisition process.

[0031] Specifically, according to the method and device for sorting the reading order of ancient Chinese books based on adjacent character detection of the present invention, in order to solve the problems existing in the bottom-up reading order sorting method, based on the coordinates and size of each character detected by OCR, column detection is performed by continuously moving (scanning) toward adjacent characters from a specific starting point similar to human cognition. That is, the above method and device are to determine the reading order of each glyph in the original text image of the ancient book (ancient book) by line segmentation based on adjacent character detection.

[0032] The present invention aims to realize the automatic textualization of the original image of ancient Chinese books by distinguishing between the main text and detailed notes and line segmentation which cannot be achieved in the existing line segmentation based on pixel projection contour and geometric analysis without relying on a model based on deep learning.

[0033] The objects of the present invention are not limited to the above-mentioned objects, and those skilled in the art will clearly understand other objects not mentioned from the following description.

[0034] Solutions to the problem

[0035] According to an embodiment of the present invention, a method for sorting the reading order of ancient Chinese books comprises: a step in which a device for sorting the reading order of ancient Chinese books selects an analysis object frame from an OCR detection frame of an original image of an ancient Chinese book; a multi-scanning step in which the device for sorting the reading order of ancient Chinese books generates, based on coordinate information and size information of the analysis object frame, a first moving path consisting of frames corresponding to the main text and right detailed notes of a main column of a current analysis object and a second moving path consisting of frames corresponding to the main text and left detailed notes of the main column using a continuous detection method of adjacent frames; and a step in which the device for sorting the reading order of ancient Chinese books calculates the union of frames included in the first moving path and frames included in the second moving path to generate a frame list of the main column of the current analysis object.

[0036] According to an embodiment of the present invention, a device for sorting the reading order of ancient Chinese books comprises: a memory storing an OCR result of an original image of an ancient Chinese book and a computer-readable command; and at least one processor, which is specifically implemented to read the OCR result and execute the command. The at least one processor is configured to: select an analysis object frame from the OCR detection frame of the original image of the ancient Chinese book contained in the OCR result by executing the command, and based on the coordinate information and size information of the analysis object frame, generate a first moving path composed of frames corresponding to the main text and right detailed notes of the main column of the current analysis object and a second moving path composed of frames corresponding to the main text and left detailed notes of the main column by using a continuous detection method of adjacent frames, and calculate the union of the frames contained in the first moving path and the frames contained in the second moving path to generate a frame list of the main column of the current analysis object.

[0037] Effects of the Invention

[0038] In order to sort the reading order of individual Chinese characters in the digital image of ancient documents, the present invention first applies OCR to the entire original image, and then performs line segmentation and text / note category discrimination based on the coordinates and size of each bounding box through quadruple adjacent character detection scanning. This bottom-up line segmentation can further fully utilize the potential of character detection and recognition information provided by existing OCR and increase the efficiency of producing the original text of ancient documents.

[0039] According to the present invention, lines can be successfully segmented in an ancient book image that contains not only a main text-detailed annotation composite structure but also noises such as skew and warping, and the main text / detailed annotation of individual glyphs can be distinguished and assigned a reading order.

[0040] The digital text technology proposed in the present invention acts as a connection point between the existing OCR Chinese character detection / recognition and the translation model based on the corpus database, thereby realizing the fully automatic translation of ancient documents.

[0041] Effects obtainable from the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a diagram showing the process of digitizing ancient documents based on OCR.

[0043] Figure 2a and Figure 2b This is a diagram used to illustrate the reading order of ancient documents.

[0044] Figure 3 This is a diagram for explaining the results of comparing the line segmentation performance of the pixel projection technique and the central coordinate projection technique.

[0045] Figure 4 It is a flowchart for illustrating a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0046] Figure 5 This is a schematic diagram for quadruple scanning based on neighboring character detection.

[0047] Figure 6 is a diagram about an applicable example of quadruple scanning based on neighboring character detection.

[0048] Figure 7a and Figure 7b It is a diagram for explaining a method for continuous detection of neighboring boxes.

[0049] Figure 8a It is a flowchart for illustrating a first upward scanning process included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0050] Figure 8b is a diagram showing a pseudo code of an algorithm for the first upward scanning process.

[0051] Figure 9a It is a flowchart for illustrating a first downward scanning process included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0052] Figure 9b is a diagram showing a pseudo code of an algorithm for the first scan-down process.

[0053] Fig.10aIt is a flowchart for illustrating the second downward scanning process included in the method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0054] Fig.10b is a diagram showing a pseudo code of an algorithm for the second scan-down process.

[0055] Fig.11a It is a flowchart for illustrating a second upward scanning process included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0056] Fig.11b is a diagram showing a pseudo code of an algorithm for the second upward scanning process.

[0057] Fig.12 This is a diagram of the category discrimination conditions for the main text, left detailed notes, and right detailed notes.

[0058] Fig.13a It is a flowchart for illustrating the reading order sorting and digital text generation process included in the reading order sorting method of Chinese ancient books according to an embodiment of the present invention.

[0059] Fig.13b is a diagram showing pseudo code of an algorithm for the reading order sorting and digital text generation process.

[0060] Fig.14 FIG. 1 is a diagram showing an example of digital text generation for an ancient Chinese book.

[0061] Fig.15 It is a flow chart for illustrating a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0062] Fig.16 The block diagram shows a computer system for implementing the method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention.

[0063] Description of Reference Numerals

[0064] 10: device for sorting the reading order of ancient Chinese books; 1000: computer system; 1010: processor;

[0065] 1020: communication device; 1030: memory; 1040: storage device; 1050: input interface device;

[0066] 1060: output interface device; 1070: bus. DETAILED DESCRIPTION

[0067] The present invention relates to a reading order sorting method and device based on adjacent character detection developed for the automation of generating digital texts in the process of fully utilizing and preserving ancient documents. The method and device according to the present invention detect the reading order correlation between adjacent characters based on the center coordinates and size of individual Chinese characters in the text detected / recognized by optical character recognition (OCR) based on artificial intelligence. The method and device according to the present invention perform reading order sorting based on the main text-detailed note composite structure in a vertical format from right to left after distinguishing each character into a main text class or a detailed note class by four-fold (up-down-down-up: up-down-down-up) scanning. The original text image digital text technology according to the present invention can support the collation (comparing different versions of the same book and examining the differences) and comparison (putting the contents of two or more objects together for comparison) as the morphological inference and restoration process of the original ancient book, and finally can automatically translate the original text image of the ancient book.

[0068] The advantages and features of the present invention, and the methods for achieving them will become clear by referring to the accompanying drawings and the embodiments described in detail below. However, the present invention is not limited to the embodiments disclosed below but will be specifically implemented in various different ways. The present embodiments are only provided to make the disclosure of the present invention complete and to fully inform the scope of the invention to those of ordinary skill in the art. The present invention is only defined by the scope of the claims. On the other hand, the terms used in this specification are used to illustrate the embodiments and are not intended to limit the present invention. In this specification, unless otherwise specified in the text, the singular also includes the plural. The "comprises" and / or "comprising..." used in the specification does not exclude the existence or addition of one or more other constituent elements, steps, actions and / or elements other than the mentioned constituent elements, steps, actions and / or elements.

[0069] When a certain component is mentioned as being "connected" or "coupled" to another component, it is also possible that it is directly connected or coupled to the other component, but it should be understood that there may be other components between the two. On the contrary, when a certain component is mentioned as being "directly connected" or "directly coupled" to another component, it should be understood that there are no other components between the two. Other expressions that describe the relationship between the components, such as "between" and "just between" or "adjacent to" and "directly adjacent to", should also be interpreted in the same way.

[0070] The terms "first" and "second" can be used to describe a variety of components, but the above components shall not be limited by the above terms. The above terms can be used to distinguish one component from other components. For example, without departing from the scope of the present invention, the first component can be named as the second component, and similarly, the second component can also be named as the first component.

[0071] In describing the present invention, a specific description of a related known technology will be omitted if it is judged that it may make the gist of the present invention unnecessarily obscure.

[0072] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meanings as those generally understood by those of ordinary skill in the art. Terms that are the same as those defined in commonly used dictionaries should be interpreted as having the same meanings as those in the context of the relevant art and should not be interpreted as ideal or excessive formal meanings unless explicitly defined in this specification.

[0073] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the process of describing the present invention, for the sake of overall understanding, the same reference numerals are used for the same components regardless of the number of the drawings.

[0074] Figure 4 It is a flowchart for illustrating a method for sorting the reading order of ancient Chinese books based on adjacent character detection according to an embodiment of the present invention.

[0075] The premise of the method for sorting the reading order of ancient Chinese books based on adjacent character detection (hereinafter referred to as “the method for sorting the reading order of ancient Chinese books”) according to one embodiment of the present invention is that it is executed by the device 10 for sorting the reading order of ancient Chinese books.

[0076] The method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention is developed for the digital text conversion of the original images of ancient documents. The method performs line segmentation and reading order sorting based on OCR results.

[0077] The above OCR result includes the center coordinate information (x, y) and size information (w, h) of the OCR detection box generated by the detection results of each character in the digital image of the ancient book (which can be referred to as the "original image") and the character generated by the recognition result.

[0078] The OCR detection frame is a bounding box that indicates the position and size of characters in the original image. For reference, the OCR detection frame has a vertically straight square shape regardless of the skew or warping of the original image. In this specification, the OCR detection frame may be simply referred to as a "frame".

[0079] The x contained in the center coordinate information of the OCR detection frame (hereinafter referred to as “coordinate information”) indicates the x coordinate, and the y contains the y coordinate. The w contained in the size information of the OCR detection frame indicates the width, and the h contains the height.

[0080] The OCR results can be organized as shown in Table 1.

[0081] Table 1

[0082]

[0083] The Chinese ancient book reading order sorting device 10 receives an input of an OCR result of a digital image of a Chinese ancient book from the outside.

[0084] As another example, the Chinese ancient book reading order sorting device 10 may receive an input of a digital image of a Chinese ancient book from the outside, and generate an OCR result through optical character recognition (OCR) ( S100 ).

[0085] The reading order sorting algorithm M1 based on adjacent character detection applicable to the reading order sorting method of ancient Chinese books according to one embodiment of the present invention includes Algorithm 1 to Algorithm 5. Figure 4 The method for sorting the reading order of ancient Chinese books shown in the figure includes steps S200, S300, S420 and S440.

[0086] After obtaining the OCR result of the original text image through step S100, in order to facilitate the subsequent sorting of the reading order, the Chinese ancient book reading order sorting device 10 sorts the OCR result in the ascending order of the y coordinates of the OCR detection frame. The Chinese ancient book reading order sorting device 10 assigns indexes to each OCR detection frame in ascending order of the y coordinate. Therefore, there is a relationship that the y coordinate distance is close between adjacent indexed frames. However, because there is no correlation with the x coordinate, it does not mean that the actual distance between the two frames is close.

[0087] Step S200 is the main column line segmentation step, which includes steps S220 to S280. In step S200, the Chinese ancient book reading order sorting device 10 determines the OCR detection frame corresponding to the current analysis object main column (hereinafter referred to as "current main column") through a total of 4 scans (UP1, DOWN1, DOWN2, UP2) based on the analysis object OCR detection frame other than the OCR detection frame in the OCR result that has completed the main column line segmentation, reading order sorting and digital text analysis operation, and forms the boundary (main column line segmentation) between the current main column and the next analysis object main column (hereinafter referred to as "next main column") in the original text image. In other words, the above boundary can be called the left boundary of the current main column.

[0088] The names of the above four scans (which may be referred to as "quadruple scans") used in this specification and the purpose and output of each scan are summarized as shown in Table 2.

[0089] Table 2

[0090]

[0091] Step S300 is a step of sorting the reading order of the current main column and generating digital text, which includes steps S320 to S360. In this step, the Chinese ancient book reading order sorting device 10 determines which category of the main text, left detailed notes, and right detailed notes each OCR detection frame corresponding to the current main column corresponds to (S320), and based on the main text / detailed notes category determination result, forms a boundary between the main text part and the detailed notes part of the current main column in the original text image (S340), and uses the above boundary to determine the reading order of the glyphs corresponding to all OCR detection frames corresponding to the current main column, and sorts the glyphs according to the determined reading order and generates and outputs the digital text for the current main column (S360).

[0092] Thereafter, the Chinese ancient book reading order sorting device 10 determines whether there is a next main column. That is, the Chinese ancient book reading order sorting device 10 confirms whether there is a remaining OCR result in addition to the information corresponding to the current main column in the OCR result (S420). If there is an OCR detection frame in the OCR result that has not been segmented into main column lines and sorted in reading order, the Chinese ancient book reading order sorting device 10 moves to the next main column (S440) and executes steps S200 and S300. If the main column line segmentation and reading order sorting have been completed for all OCR detection frames included in the OCR result, the process ends.

[0093] Step S440 is a step of moving to the next main column, in other words, a step of setting the next main column as the analysis object. The Chinese ancient book reading order sorting device 10 removes the boxes included in the box list of the main column where the digital text has been generated from the OCR detection boxes included in the OCR result and selects the analysis object box (box list) of the next main column.

[0094] Figure 5 This is a schematic diagram for quadruple scanning based on neighboring character detection.

[0095] As in Figure 2b As indicated by reference numeral 25 in the text, the main column refers to the columns in the ancient book. Figure 5 In , a main column is equivalent to the area inside the main column boundary. The main column includes sub-columns. Figure 5 In the text, subcolumns are distinguished by subcolumn boundaries. The subcolumns contained in the main column are divided into the main text section and the detailed notes section. The reading order between the main columns is independent and simply follows the order of the main columns from the right to the left. Therefore, as long as the reading order within each main column is defined, the reading order for the entire original text image will be clear. According to an embodiment of the present invention, a method and device for sorting the reading order of ancient Chinese books performs a quadruple scan for clustering (linesegmentation) of characters (detection boxes) belonging to each main column.

[0096] The names, purposes, and outputs of each scan included in the quadruple scan are summarized in Table 2. The Chinese ancient book reading order sorting device 10 performs quadruple scans in the order of the first upward scan (Upscan#1, hereinafter referred to as "UP1"), the first downward scan (Downscan#1, hereinafter referred to as "DOWN1"), the second downward scan (Downscan#2, hereinafter referred to as "DOWN2"), and the second upward scan (Upscan#2, hereinafter referred to as "UP2"). UP1 and DOWN2 do not directly participate in the detection of the constituent elements of the current main column, but set the starting points of DOWN1 and UP2.

[0097] The Chinese ancient book reading order sorting device 10 performs each scan in a manner of repeatedly moving from a set starting point to an adjacent frame and reaching an end point according to a specified reference, and generates a moving path according to each scan. Here, the starting point and the end point are respectively an OCR detection frame, and the moving path has the form of a list of OCR detection frames.

[0098] like Figure 5As shown, the first upward scan UP1 moves continuously from the rightmost frame B1 of the current main column to the adjacent frame located on the right or upper part, and ends when it reaches the rightmost upper frame B2. The rightmost upper frame B2 is set as the starting point of the first downward scan DOWN1 and the second downward scan DOWN2. That is, the first upward scan UP1 is a scan for detecting the starting point of the first downward scan DOWN1 and the second downward scan DOWN2.

[0099] After executing the first upward scan UP1 , the Chinese ancient book reading order sorting device 10 executes the first downward scan DOWN1 and the second downward scan DOWN2 .

[0100] The first downward scan DOWN1 is a scan for generating a list of boxes corresponding to the main text and right annotation of the current main column (first moving path). The first downward scan DOWN1 moves continuously from the upper rightmost box B2 of the current main column to the adjacent box located on the right or bottom, and ends when reaching the lower rightmost box.

[0101] The second downward scan DOWN2 moves continuously from the upper rightmost frame B2 of the current main column to the adjacent frame located at the left or lower part, and ends when it reaches the lower leftmost frame B3. The lower leftmost frame B3 is set as the starting point of the second upward scan UP2. That is, the second downward scan DOWN2 is a scan for detecting the starting point of the second upward scan UP2.

[0102] The Chinese ancient book reading order sorting device 10 performs the second upward scan UP2 after performing the second downward scan DOWN2.

[0103] The second upward scan UP2 is a scan for generating a list of boxes corresponding to the text and left detailed notes of the current main column (second moving path). The second upward scan UP2 moves continuously from the leftmost lower end box B3 of the current main column to the adjacent box located on the left or upper part, and ends when reaching the leftmost upper end box.

[0104] For the specific content of the quadruple scan, please refer to the following Figures 7a to 11b Narrative.

[0105] Figure 6 is a diagram showing an applicable example of quadruple scanning based on neighboring character detection. Figure 6 In the figure, the main columns are separated by dotted lines. Figure 6 The OCR detection frame generated by the original text image shown in the figure is used as the object to perform quadruple scanning, and each OCR detection frame is divided into a text frame, a left detailed note frame, and a right detailed note frame. For example, the Chinese ancient book reading order sorting device 10 can display the text frame as a pink quadrilateral, the left detailed note frame as a red triangle, and the right detailed note frame as a blue triangle, and then output the result to the screen (refer to Fig.13bAlgorithm 5). Figure 6 In the first main column of the original text image, the Chinese ancient book reading order sorting device 10 performs a first upward scan with the rightmost frame B11 as the starting point to detect the rightmost upper end frame B12, and performs a second downward scan with the rightmost upper end frame B12 as the starting point to detect the leftmost lower end frame B13.

[0106] Figure 7a and Figure 7b 1 is a diagram for explaining a method for continuous detection of neighboring frames used in quadruple scanning. The present invention defines a line detection technique for detecting neighboring frames ( Figure 7a ) and point detection techniques ( Figure 7b ).

[0107] Line detection techniques are used to detect and reference frames (the most recently moved frames, in Figure 7a In the example of FIG. 1 , the adjacent frame (neighboring frame) is a frame adjacent to the upper part (when performing upward scanning) or the lower part (when performing downward scanning) of the reference frame. For reference, in this specification, the adjacent frame (neighboring frame) is a frame near the reference frame, and it does not only refer to the nearest frame. The line detection technique is used to detect a frame having at least a portion of an area corresponding to the x range (xw / 2:x+w / 2) of the reference frame in other frames adjacent to the reference frame.

[0108] In this specification, a frame detected by applying the line detection technique from a reference frame (detection target frame) is referred to as a line contact frame.

[0109] Although the present specification generally refers to the frame (detection target frame) detected by applying the line detection technique from the reference frame as a line contact frame, there may also be a line contact frame relationship between frames that are not reference frames. For example, when selecting the three nearest frames of the reference frame and determining whether there is a line contact between the selected frames, if there is a shared x range based on the area of ​​any two frames in the selected frames, the two frames are in a line contact frame relationship (which can be referred to as a line contact relationship).

[0110] Point detection techniques are used to detect and reference frames (in Figure 7b The box adjacent to the left or right of SB2). Figure 7b In the point detection technique, characters are located side by side on the right side of the y range (yh / 2:y+h / 2) of the reference frame. Specifically, the point detection technique is used to detect the case where the y coordinate of the center of another frame adjacent to the left or right side of the reference frame is included in the y range (yh / 2:y+h / 2) of the reference frame. That is, among the frames adjacent to the reference frame, the frame having the y coordinate included in the y range (yh / 2:y+h / 2) of the reference frame becomes the point contact frame.

[0111] Although the present specification generally refers to the frame (detection target frame) detected by applying the point detection technique from the reference frame as a point contact frame, there may be a point contact frame relationship between frames that are not reference frames. For example, when selecting the three nearest frames of the reference frame and determining whether there is a point contact between the selected frames, if the y range of any one of the selected frames includes the y coordinate of the center of the other frame, the two frames are in a point contact frame relationship (which may be referred to as a point contact relationship).

[0112] For reference, at the beginning of the scanning process, the reference frame is set as the starting point of the scanning, and if it moves to the neighboring frame, the neighboring frame is designated as the new reference frame.

[0113] Figure 8a is a flowchart for illustrating a first upward scanning process (S220) included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention, Figure 8b : is a diagram showing a pseudo code of an algorithm for the first upward scanning process. Step S220 includes steps S221 to S227.

[0114] The first upward scan targets the upper rightmost frame in the original image. First, the Chinese ancient book reading order sorting device 10 needs the coordinate information (x, y) and size information (w, h) of the OCR detection frame to be analyzed in the OCR result of the original image in order to perform the first upward scan (S221).

[0115] The coordinates of the OCR detection frame of the analysis object are arranged in a two-dimensional coordinate system. Therefore, the degree of freedom of detection of the upper rightmost frame is 2, and the upper rightmost frame cannot be detected by the secondary use of a simple min-max operation, that is, the logic of "the uppermost part of the rightmost part" or "the rightmost part of the uppermost part". In order to overcome this, the Chinese ancient book reading order sorting device 10 according to one embodiment of the present invention places the rightmost frame (in Figure 5 The first upward scanning process is performed by setting the starting point (the initial reference frame) as B1, and continuously performing line detection of the nearest frame in the upper right section from the reference frame. In order to give priority to the movement of the detailed notes to the right (movement to the upper right), the Chinese ancient book reading order sorting device 10 further performs right point detection and movement after each line detection and movement to reach the upper rightmost frame (at the branch point from the main text to the detailed notes). Figure 5 In the example shown in FIG. 1 , the Chinese ancient book reading order sorting device 10 sets the index of the reference frame to the index of the upper rightmost frame when there is no line contact frame above the reference frame.

[0116] In this specification, the Chinese ancient book reading order sorting device 10 uses the y-axis as a reference to consider the one-dimensional distance and judge the "nearest frame". However, the judgment reference can be the y-coordinate of the center of the frame, or the upper or lower boundary line of the frame. For each scan, the frame located in the opposite direction of the given scanning direction is excluded from the nearest frame detection. For example, the Chinese ancient book reading order sorting device 10 only uses the frame located above the reference frame as the nearest frame detection object during the upward scans UP1 and UP2, and only uses the frame located below the reference frame as the nearest frame detection object during the downward scans DOWN1 and DOWN2.

[0117] Below, refer to Figure 8a Steps S222 to S227 are described.

[0118] The Chinese ancient book reading order sorting device 10 selects the rightmost frame in the analysis object OCR detection frame, and sets the rightmost frame as the starting point (reference frame) of the first upward scan (S222). For example, the Chinese ancient book reading order sorting device 10 can select the rightmost frame based on the maximum value of the x-coordinate (x+w / 2) of the boundary line of the analysis object frame, or can select the rightmost frame based on the maximum value of the x-coordinate of the analysis object frame.

[0119] The Chinese ancient book reading order sorting device 10 determines whether there is a line contact frame above the reference frame (S223), and if so, sets the upper most adjacent line contact frame as a new reference frame (S224), and if not, executes step S227.

[0120] After step S224, the Chinese ancient book reading order sorting device 10 determines whether there is a point contact frame on the right side of the reference frame (S225), and if so, sets the point contact frame located on the rightmost side of the reference frame as a new reference frame (S226). After step S226 is completed, the Chinese ancient book reading order sorting device 10 returns to step S223.

[0121] In step S223, when it is determined that there is no line contact frame on the upper part of the reference frame, the Chinese ancient book reading order sorting device 10 determines the current reference frame as the upper rightmost frame of the current main column, and sets the index of the current reference frame to the index of the upper rightmost frame (S227).

[0122] Figure 9a is a flowchart for illustrating a first downward scanning process (S240) included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention. Figure 9b : is a diagram showing a pseudo code of an algorithm for the first downward scanning process. Step S240 includes steps S241 to S256.

[0123] The first downward scan is to set the upper rightmost frame of the current main column detected in the first upward scan as the starting point and detect all the text frames and right detailed note frames of the current main column by continuously detecting the adjacent frames in the lower right section. The Chinese ancient book reading order sorting device 10 generates a first moving path (DOWN1 list) composed of the frames detected by performing the first downward scan. That is, the first moving path is an index list of the text frames and right detailed note frames of the current main column.

[0124] First, in order to perform the first downward scan, the coordinate information (x, y) and size information (w, h) of the OCR detection frame to be analyzed and the index of the upper rightmost frame of the current main column derived in step S220 are required (S241).

[0125] Unlike the first upward scan, the first downward scan targets not only the final destination but also the detection of all boxes on the path. Figure 9a In order to conduct continuous detection with high accuracy, exceptions for branching points from the main text to the detailed notes have been added to the flowchart. In particular, Figure 9b The pseudo code of branch 3 (Case 3) is constructed as follows: as a condition for coping with incomplete (detection omission) OCR detection, the pattern of the three nearest boxes is analyzed instead of simply analyzing the pattern of the nearest 1 box ( Figure 9a S249 to S255 of DOWN2 and UP2 also include the same conditions). Generally speaking, the detailed notes follow the principle of being divided into two columns from the main text. Therefore, when the OCR detection of the right detailed note or the left detailed note is omitted, the detailed note box located on the left or right side of the omitted box is identified as a single column if there is no exception condition of Case 3, and the problem of being judged as the main text will occur. If the Chinese ancient book reading order sorting device 10 reaches the rightmost bottom end box of the current main column through continuous detection of the first downward scan, the indexes of all boxes on the moving path are added to the current main column box list and the first moving path (which can be referred to as the first downward scan box list or DOWN1 list).

[0126] Below, refer to Figure 9a Steps S242 to S256 are described.

[0127] The Chinese ancient book reading order sorting device 10 sets the upper rightmost frame determined in step S220 as the starting point (reference frame) of the first downward scan (S242). The Chinese ancient book reading order sorting device 10 determines whether there is a line contact frame below the reference frame (S243), and executes step S244 if there is, and executes step S256 if there is not.

[0128] In step S244, the Chinese ancient book reading order sorting device 10 determines the number of frames that the lower line of the reference frame touches, and branches to one of S245, S246 or S249 according to the number of frames that the lower line touches.

[0129] If the number of the line contact frame at the bottom of the reference frame is 1, the Chinese ancient book reading order sorting device 10 moves the reference frame to the line contact frame. That is, the Chinese ancient book reading order sorting device 10 sets the line contact frame as a new reference frame (S245).

[0130] In addition, when the number of line contact frames at the bottom of the reference frame is 2, the Chinese ancient book reading order sorting device 10 sets the frame closest to the current reference frame as the new reference frame (S246). Furthermore, when there is a point contact frame on the right side of the reference frame (S247), the Chinese ancient book reading order sorting device 10 sets the rightmost point contact frame of the current reference frame as the new reference frame (S248).

[0131] In addition, when the number of frames that the lower line of the reference frame contacts is more than 3, the Chinese ancient book reading order sorting device 10 selects the three frames that are closest to the current reference frame (S249), and determines whether there is point contact between the selected frames (S250). When the judgment result is that there is no point contact, the Chinese ancient book reading order sorting device 10 sets the closest frame among the three frames as a new reference frame (S251). When the judgment result is that the two frames that are closest to the reference frame are in point contact with each other, the Chinese ancient book reading order sorting device 10 sets the frame on the right side of the two closest frames as a new reference frame (S252). When the judgment result is that the remaining two frames (marked as "distant contact" in the accompanying drawings) other than the closest frame of the current reference frame are in point contact with each other, the Chinese ancient book reading order sorting device 10 determines whether the above-mentioned closest frame is in line contact with the right side of the two distant frames (S253). When the above-mentioned nearest frame touches the right frame line of the two distant frames, the Chinese ancient book reading order sorting device 10 sets the nearest frame of the current reference frame as the new reference frame (S254); when there is no line contact, the right frame of the two distant frames is set as the new reference frame (S255).

[0132] After the new reference frame is determined (after S245, the negative branch of S247, S248, S251, S252, S254, and S255), the Chinese ancient book reading order sorting device 10 executes step S243 to re-determine whether there is a line contact frame at the bottom.

[0133] In step S243, when it is determined that there is no line contact frame below the reference frame, the Chinese ancient book reading order sorting device 10 determines the current reference frame as the rightmost lower frame of the current main column, and adds the indexes of all frames on the moving path from the rightmost upper frame to the rightmost lower frame of the current main column to the current main column frame list and the first moving path (DOWN1 list). That is, the Chinese ancient book reading order sorting device 10 generates the first moving path (DOWN1 list) and the current main column frame list based on the first downward scanning result.

[0134] Fig.10a is a flowchart for illustrating a second downward scanning process (S260) included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention, Fig.10b : is a diagram showing a pseudo code of an algorithm for the second down scanning process. Step S260 includes steps S261 to S277.

[0135] The second downward scan is to set the upper rightmost frame of the current main column detected in the first upward scan as the starting point and to detect the lower leftmost frame in the current main column by continuously detecting the adjacent frames in the lower left section. The upper rightmost frame of the current main column, which is the detection object of the first upward scan, is located at the upper right end in the analysis object OCR detection frame, and thus there is no room for confusion with frames located in other main columns. Therefore, the first upward scan does not require high accuracy in continuous detection of adjacent frames. However, the lower leftmost frame of the current main column, which is the detection object of the second downward scan, is located at the boundary of the main column directly adjacent to the left side of the current main column. Therefore, the second downward scan needs to ensure high accuracy in continuous detection of adjacent frames, and therefore, in order to ensure high accuracy, it is necessary as follows Fig.10a Multiple exception handling as shown.

[0136] Below, refer to Fig.10a Steps S261 to S277 are described.

[0137] First, in order to perform the second downward scan, the Chinese ancient book reading order sorting device 10 needs the coordinate information (x, y) and size information (w, h) of the OCR detection box that is the analysis object, as well as the current main column box list exported in step S240 and the index of the upper rightmost box of the current main column (S261).

[0138] The Chinese ancient book reading order sorting device 10 sets the upper rightmost frame determined in step S220 as the starting point (reference frame) of the second downward scan (S262). The Chinese ancient book reading order sorting device 10 determines whether there is a line contact frame below the reference frame (S263), and executes step S264 if there is, and executes step S277 if there is not.

[0139] In step S264, the Chinese ancient book reading order sorting device 10 determines the number of frames that the lower line of the reference frame touches, and branches to one of S265, S266 or S269 according to the number of frames that the lower line touches.

[0140] If the number of the line contact frame at the bottom of the reference frame is 1, the Chinese ancient book reading order sorting device 10 moves the reference frame to the line contact frame. That is, the Chinese ancient book reading order sorting device 10 sets the line contact frame as a new reference frame (S265).

[0141] In addition, when the number of frames that the lower line contacts with the reference frame is 2, the Chinese ancient book reading order sorting device 10 determines the number of frames (undetected frames) that are not included in the current main column frame list in the lower line contact frame (S266). When the number of undetected frames in the lower line contact frame is 1, the Chinese ancient book reading order sorting device 10 sets the undetected frame as a new reference frame (S267), otherwise, sets the frame that is the closest to the current reference frame as a new reference frame (S268).

[0142] In addition, when the number of frames that the lower line of the reference frame touches is more than 3, the Chinese ancient book reading order sorting device 10 selects the three frames that are closest to the current reference frame (S269), and determines the number of frames (undetected frames) that are not included in the current main column frame list among the selected frames (S270). The Chinese ancient book reading order sorting device 10 executes step S271 when there are one or two undetected frames among the three frames that are closest to the current reference frame, otherwise the closest frame among the three frames is set as a new reference frame (S272).

[0143] In step S271, the Chinese ancient book reading order sorting device 10 determines whether there is point contact between the three selected frames. If the result of the judgment is that there is no point contact, the Chinese ancient book reading order sorting device 10 sets the most adjacent frame among the three frames as a new reference frame (S272). If the result of the judgment is that the two frames of the most adjacent reference frame are in point contact with each other, the Chinese ancient book reading order sorting device 10 sets the undetected frame in the two most adjacent frames as a new reference frame (S273). If the result of the judgment is that the remaining two frames (marked as "remote contact" in the accompanying drawings) other than the most adjacent frame of the current reference frame among the three frames are in point contact with each other, the Chinese ancient book reading order sorting device 10 determines whether the above-mentioned most adjacent frame is in contact with the undetected frame line in the two remote frames (S274). When the nearest frame is in line contact with the undetected frame in the two remote frames, the Chinese ancient book reading order sorting device 10 sets the nearest frame of the current reference frame as the new reference frame (S275), and when there is no line contact, sets the undetected frame in the two remote frames as the new reference frame (S276). As described above, in the second downward scanning process, the Chinese ancient book reading order sorting device 10 makes full use of the current main column frame list and continuously detects the lower left adjacent frame in a manner of moving the reference frame to the undetected frame when detecting the detailed annotation part.

[0144] After the new reference frame is determined (after S265, S267, S268, S272, S273, S275, and S276), the Chinese ancient book reading order sorting device 10 executes step S263 to re-determine whether there is a line contact frame at the bottom.

[0145] In step S263, when it is determined that there is no line contact frame at the bottom of the reference frame, the Chinese ancient book reading order sorting device 10 determines the current reference frame as the leftmost lower frame of the current main column, and sets the index of the current reference frame to the index of the leftmost lower frame (S277). The purpose of the second downward scan is to detect the starting point (leftmost lower frame) of the second upward scan, so even if step S277 is completed, the Chinese ancient book reading order sorting device 10 will not add an index to the current main column frame list. Therefore, the current main column frame list before step S280 is the same as the current main column frame list after step S240.

[0146] Fig.11a is a flowchart for illustrating a second upward scanning process (S280) included in a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention, Fig.11b : is a diagram showing a pseudo code of an algorithm for the second upward scanning process. Step S280 includes steps S281 to S297.

[0147] The second upward scan as the last scan is to set the lower leftmost frame of the current main column detected in the second downward scan as the starting point and detect all the text frames and left detailed note frames of the current main column by continuously detecting the upper left adjacent frames. If the Chinese ancient book reading order sorting device 10 performs the second upward scan and reaches the upper leftmost frame of the current main column, it generates a second moving path (UP2 list) composed of the detected frames. That is, the second moving path is an index list of the text frames and left detailed note frames of the current main column. In addition, the Chinese ancient book reading order sorting device 10 appends the indexes of all frames on the second moving path to the current main column frame list.

[0148] The Chinese ancient book reading order sorting device 10 performs clustering and line segmentation of the boxes contained in the current main column with the end of the second upward scan as the starting point ( Figure 5 The major column boundary or Figure 6 In addition, in order to apply the same quadruple scanning algorithm to the analysis of the next (left) main column, the Chinese ancient book reading order sorting device 10 deletes the frame information contained in the current main column from the x, y, w, h list and removes it from the analysis object. That is, the Chinese ancient book reading order sorting device 10 removes the frame contained in the frame list of the current main column from the OCR detection frame and selects the analysis object frame of the next main column.

[0149] Below, refer to Fig.11a Steps S281 to S297 are described.

[0150] First, in order to perform the second upward scan, the Chinese ancient book reading order sorting device 10 needs the coordinate information (x, y) and size information (w, h) of the OCR detection frame to be analyzed, as well as the current main column frame list and the index of the lower leftmost frame of the current main column (S281).

[0151] The Chinese ancient book reading order sorting device 10 sets the leftmost bottom frame determined in step S260 as the starting point (reference frame) of the second upward scan (S282). The Chinese ancient book reading order sorting device 10 determines whether there is a line contact frame on the upper part of the reference frame (S283), and executes step S284 if there is, and executes step S297 if there is not.

[0152] In step S284, the Chinese ancient book reading order sorting device 10 determines the number of upper line contact frames of the reference frame, and branches to one of S285, S286 or S289 according to the number of upper line contact frames.

[0153] If the number of the upper line contact frame of the reference frame is 1, the Chinese ancient book reading order sorting device 10 moves the reference frame to the line contact frame. That is, the Chinese ancient book reading order sorting device 10 sets the line contact frame as a new reference frame (S285).

[0154] In addition, when the number of upper line contact frames of the reference frame is 2, the Chinese ancient book reading order sorting device 10 determines the number of frames (undetected frames) that are not included in the current main column frame list in the upper line contact frame (S286). When the number of undetected frames in the upper line contact frame is 1, the Chinese ancient book reading order sorting device 10 sets the undetected frame as a new reference frame (S287), otherwise, sets the most adjacent frame of the current reference frame as a new reference frame (S288).

[0155] In addition, when the number of the upper line contacting frames of the reference frame is more than 3, the Chinese ancient book reading order sorting device 10 selects the 3 frames closest to the current reference frame (S289), and determines the number of frames (undetected frames) not yet included in the current main column frame list among the selected frames (S290). The Chinese ancient book reading order sorting device 10 executes step S291 when there are 1 or 2 undetected frames among the 3 frames closest to the current reference frame, otherwise, the closest frame among the 3 frames is set as the new reference frame (S292).

[0156] In step S291, the Chinese ancient book reading order sorting device 10 determines whether there is point contact between the three selected frames. If the judgment result is that there is no point contact, the Chinese ancient book reading order sorting device 10 sets the most adjacent frame among the three frames as a new reference frame (S292). If the judgment result is that the two frames of the most adjacent reference frames are in point contact with each other, the Chinese ancient book reading order sorting device 10 sets the undetected frame in the two most adjacent frames as a new reference frame (S293). If the judgment result is that the remaining two frames (marked as "remote contact" in the accompanying drawings) other than the most adjacent frame of the current reference frame among the three frames are in point contact with each other, the Chinese ancient book reading order sorting device 10 determines whether the above-mentioned most adjacent frame is in contact with the undetected frame line in the two remote frames (S294). When the nearest frame is in line contact with the undetected frame in the two remote frames, the Chinese ancient book reading order sorting device 10 sets the nearest frame of the current reference frame as the new reference frame (S295). When the nearest frame is not in line contact with the undetected frame in the two remote frames, the undetected frame in the two remote frames is set as the new reference frame (S296). As described above, in the second upward scanning process, the Chinese ancient book reading order sorting device 10 makes full use of the current main column frame list and continuously detects the upper left adjacent frame in a manner of moving the reference frame to the undetected frame when detecting the detailed annotation part.

[0157] After the new reference frame is determined (after S285, S287, S288, S292, S293, S295, and S296), the Chinese ancient book reading order sorting device 10 executes step S283 to re-determine whether there is a line contact frame at the top.

[0158] In step S283, when it is determined that there is no line contact frame above the reference frame, the Chinese ancient book reading order sorting device 10 determines the current reference frame as the upper leftmost frame of the current main column, and adds the indexes of all frames on the moving path from the lower leftmost frame to the upper leftmost frame of the current main column to the current main column frame list and the second moving path (UP2 list). That is, the Chinese ancient book reading order sorting device 10 generates the second moving path (UP2 list) based on the second upward scanning result, and updates the current main column frame list.

[0159] Fig.12 This is a diagram of the category discrimination conditions for the main text, left detailed notes, and right detailed notes. Fig.12 is with Figure 4 The Chinese ancient book reading order sorting device 10 classifies the OCR detection frames included in the current main column frame list into main text, left detailed note and right detailed note based on the frames included in the first moving path (DOWN1 list) and the frames included in the second moving path (UP2 list) (S320).

[0160] Specifically, the DOWN1 list and UP2 list obtained through the quadruple scanning process contain the box indexes of the text / right detailed notes and the text / left detailed notes of the current main column, respectively. Therefore, Fig.12 As shown in the figure, each box can be identified as a main text box, a right detailed note box, or a left detailed note box by the intersection and difference of the two lists. The main text box can be identified by finding the intersection of the DOWN1 list and the UP2 list; the right detailed note box can be identified by finding the difference of the DOWN1 list and the UP2 list; the left detailed note box can be identified by finding the difference of the UP2 list and the DOWN1 list (refer to Fig.13b For example, the Chinese ancient book reading order sorting device 10 can display the original text image with a pink quadrangle (main text), a blue triangle (right detailed note), and a red triangle (left detailed note) based on the discrimination result (classification result) of the corresponding category (refer to Figure 5 , Figure 6 , Fig.13b ).

[0161] Fig.13a is a flowchart for illustrating a reading order sorting and digital text generation process included in a method for sorting reading order of ancient Chinese books according to an embodiment of the present invention. Fig.13bis a diagram showing pseudo code of an algorithm for the reading order sorting and digital text generation process.

[0162] Fig.13a The specific embodiment shows that the Chinese ancient books reading order sorting device 10 performs Figure 4 The S300 step of the process.

[0163] like Fig.13a As shown, step S300 includes steps S310 to S360, and step S360 includes steps S361 to S367.

[0164] In order to perform the current main column reading order sorting and digital text generation, the Chinese ancient book reading order sorting device 10 requires the first moving path (DOWN1 list), the second moving path (UP2 list) and the character information corresponding to the boxes included in the two lists (S310).

[0165] The Chinese ancient book reading order sorting device 10 classifies the OCR detection frames included in the current main column frame list into main text, left detailed notes, and right detailed notes based on the frames included in the first moving path (DOWN1 list) and the frames included in the second moving path (UP2 list). That is, the Chinese ancient book reading order sorting device 10 determines the main text / left detailed notes / right detailed notes category for each frame included in the current main column frame list based on the first moving path and the second moving path (S320).

[0166] Based on the above-mentioned discrimination result (classification result), the Chinese ancient book reading order sorting device 10 forms the boundary between the main text part and the detailed note part of the current main column in the original text image (S340). Specifically, the Chinese ancient book reading order sorting device 10 can form the boundary between the main text part and the detailed note part (or the boundary between the main text part and the detailed note part) along the vertical direction of the current main column based on the main text / detailed note classification discrimination result. Figure 5 subcolumn boundaries).

[0167] Step S360 is a reading order sorting and digital text outputting step. The Chinese ancient book reading order sorting device 10 uses the boundary between the main text part and the detailed annotation part to determine the reading order of the glyphs corresponding to the boxes included in the box list of the current main column, and sorts the reading order of the glyphs according to the reading order to generate and output the digital text of the current main column.

[0168] As mentioned above, step S360 includes steps S361 to S367.

[0169] Steps S361 to S366 are steps for sorting the reading order of each mutually separated sub-column cluster based on the boundary (or sub-column boundary) between the main text and the detailed note, and generating and outputting digital text for each sub-column cluster. In this specification, the case where the main text box forms one sub-column cluster is referred to as the main text sub-column, and the case where the left detailed note and / or the right detailed note form one sub-column cluster is referred to as the detailed note sub-column.

[0170] The Chinese ancient book reading order sorting device 10 generates and outputs digital texts of each sub-column cluster while advancing (moving) from the uppermost sub-column cluster of the current main column downward.

[0171] First, the Chinese ancient book reading order sorting device 10 generates a character string of the main text part / right detailed note part / left detailed note part in the sub-column cluster as the current analysis object (S361). In addition, the Chinese ancient book reading order sorting device 10 determines whether the current sub-column (the sub-column cluster as the current analysis object) is a main text sub-column or a detailed note sub-column (S362), generates and outputs a digital text corresponding to the main text if it is a main text sub-column (S363), and generates and outputs a digital text corresponding to the right detailed note and the left detailed note if it is a detailed note sub-column (S364).

[0172] The sub-column clusters separated by lines above and below the boundary between the main text and the detailed notes follow the top-to-bottom reading order (from top to bottom, refer to Fig.13b The boxes listed as a single column in the main text subcolumn simply follow the top-bottom reading order, and in the case of the detailed notes subcolumn, follow the reading order from the right detailed notes (top-bottom) to the left detailed notes (top-bottom). The method and device for sorting the reading order of ancient Chinese books based on adjacent character detection according to the present invention can distinguish the categories of main text / detailed notes. Therefore, when the above method and device generate digital text after the final reading order sorting, the main text and detailed notes can be distinguished by using brackets before and after the detailed notes or distinguishing the applicable fonts.

[0173] If the Chinese ancient book reading order sorting device 10 completes the digital text generation operation for the current sub-column, it checks whether the next sub-column exists (S365). If so, it moves to the next (lower) sub-column (S366) and repeatedly performs the reading order sorting and digital text generation and output operations (S361 to S364).

[0174] In the case that the next sub-column does not exist, the generation and output of the digital text for the current main column is completed (S367). As another example, after repeatedly executing steps S361 to S366 to generate digital text for each sub-column cluster, the device 10 for sorting the reading order of ancient Chinese books can integrate all the generated digital texts for each sub-column cluster belonging to the current main column in step S367 to generate the digital text of the current main column and output it.

[0175] A method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention is described with reference to the flowchart shown in the accompanying drawings. For the sake of simplicity, the above method is illustrated and described with a series of blocks, but the present invention is not limited to the order of the above blocks. Some blocks can also be executed with other blocks in an order different from the illustrations and descriptions in this specification or at the same time, and a variety of other branches, process paths and block orders that bring the same or similar results can be specifically implemented. In addition, it may not be necessary to illustrate all the blocks in order to specifically implement the method described in this specification.

[0176] Fig.14 FIG. 1 is a diagram showing an example of digital text generation for an ancient Chinese book.

[0177] Fig.14 It is a practical application example of the method and device for sorting the reading order of ancient Chinese books based on adjacent character detection according to the present invention, expressing the status of digital text generated from the original image of the ancient Chinese book.

[0178] like Fig.14 As shown, according to the present invention, not only can digital text dt1 be extracted from the original image img1 without tilt or distortion, but also the same digital text dt1 can be extracted from the original image img2 with tilt or the original image img3 with distortion by making full use of the adjacent character detection technique. When according to the present invention, a reliability of 95% was shown in the digital text extracted from the original image with a tilt of 2 degrees. About 50,000 ancient book images were analyzed in the AIHub Chinese Ancient Books DB, and the results confirmed that 99% of the ancient books had a tilt of less than 2 degrees. Therefore, the Chinese ancient book reading order sorting method according to the present invention can be said to be a method that can obtain highly reliable outputs from most ancient book images. Therefore, the present invention has the effect of improving the efficiency of producing original texts of ancient documents.

[0179] Fig.15 1 is a flowchart for illustrating a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention. Fig.15 According to an embodiment of the present invention, a method for sorting the reading order of ancient Chinese books includes steps S510 to S600. Fig.15The method for sorting the reading order of ancient Chinese books shown in the figure is a method according to an embodiment. The steps of the method for sorting the reading order of ancient Chinese books according to the present invention are not limited to Fig.15 The embodiments shown in the figure can be added, changed or deleted as needed.

[0180] Step S510 is a step in which the Chinese ancient book reading order sorting device 10 reads the OCR result of the original text image. That is, the OCR result is input into the Chinese ancient book reading order sorting device 10. The information contained in the OCR result is shown in Table 1.

[0181] Step S520 is a step of selecting an analysis target frame.

[0182] The Chinese ancient book reading order sorting device 10 selects the analysis target frame from the OCR detection frame of the Chinese ancient book original text image. The Chinese ancient book reading order sorting device 10 selects the analysis target frame by eliminating the frames included in the frame list of the past analysis target main column from the OCR detection frame.

[0183] Step S530 is a multi-scanning step.

[0184] The Chinese ancient book reading order sorting device 10 generates the first moving path and the second moving path by multiple scanning of the analysis object frame. In this embodiment, the Chinese ancient book reading order sorting device 10 generates the first moving path and the second moving path by quadruple scanning.

[0185] The information generated by the quadruple scan is used as the basic information for main column line segmentation, text / detailed note classification, and sub-column line segmentation. The names of the scans included in the quadruple scan and the purpose and output of each scan are shown in Table 2.

[0186] In this step, the Chinese ancient book reading order sorting device 10 generates a first moving path composed of frames corresponding to the main text and right detailed notes of the main column of the current analysis object and a second moving path composed of frames corresponding to the main text and left detailed notes of the above main column based on the coordinate information and size information of the analysis object frame and using the adjacent frame continuous detection method.

[0187] The above-mentioned continuous detection method of the neighboring frame sets the initial reference frame in the above-mentioned analysis object frame, and uses the line detection technique to detect the neighboring frame in the vertical direction of the reference frame, and uses the point detection technique to detect the neighboring frame in the horizontal direction of the reference frame. The new reference frame is determined repeatedly according to the detection results and the specified benchmark. If the specified conditions are met, the detection is terminated.

[0188] The line detection technique is a technique for detecting a frame having an area corresponding to the x range of the reference frame in the analysis target frame.

[0189] However, the Chinese ancient book reading order sorting device 10 can also determine whether there is a line contact relationship between frames that are not reference frames through line detection techniques.

[0190] The point detection technique is a technique for detecting a frame whose y coordinate of the center of the frame is included in the y range of the reference frame in the analysis target frame.

[0191] However, the Chinese ancient book reading order sorting device 10 can also determine whether there is a point contact relationship between frames that are not reference frames through point detection techniques.

[0192] Although not shown in the drawings, step S530 includes steps S531 to S534.

[0193] The Chinese ancient book reading order sorting device 10 determines the upper rightmost frame of the current analysis object main column based on the coordinate information and size information of the analysis object frame and the rightmost frame in the analysis object frame as a reference by using the first upward scan using the adjacent frame continuous detection method (S531).

[0194] In addition, the Chinese ancient book reading order sorting device 10 generates a first moving path (S532) by performing a first downward scan using the upper rightmost frame in the analysis object frame as the initial reference frame, i.e., the starting point, and the lower rightmost frame in the analysis object frame as the end point, based on the coordinate information and size information of the analysis object frame using the adjacent frame continuous detection method. In the process of generating the first moving path, the Chinese ancient book reading order sorting device 10 detects a frame in the analysis object frame that exists at the bottom of the reference frame and has an area corresponding to the x range of the reference frame. When more than three frames are detected, three frames can be selected from the detected frames in the order of the adjacent reference frames, and an operation of determining a new reference frame based on the y ranges of the selected three frames and the y coordinates of the centers is performed.

[0195] In addition, the Chinese ancient book reading order sorting device 10 sets the above-mentioned upper rightmost frame as the initial reference frame based on the coordinate information and size information of the analysis object frame, and determines the lower leftmost frame of the current analysis object main column through a second downward scan using the adjacent frame continuous detection method (S533).

[0196] Finally, the Chinese ancient book reading order sorting device 10 generates a second moving path (S534) by using the adjacent frame continuous detection method based on the coordinate information and size information of the analysis object frame, by taking the lower leftmost frame as the initial reference frame, i.e., the starting point, and the upper leftmost frame of the current analysis object main column as the end point for a second upward scan. In the process of generating the second moving path, the Chinese ancient book reading order sorting device 10 detects a frame that exists in the upper part of the reference frame and has an area corresponding to the x range of the reference frame in the analysis object frame. When more than three frames are detected, three frames can be selected from the detected frames in the order of the adjacent reference frames, and the operation of determining a new reference frame based on the y range of the selected three frames, the y coordinate of the center, and whether they are included in the first moving path is performed.

[0197] Step S540 is a step of generating a box list of the current main column.

[0198] The Chinese ancient book reading order sorting device 10 obtains the union of the frames included in the first moving path and the frames included in the second moving path to generate a frame list of the main column of the current analysis object.

[0199] Step S550 is a main column line segmentation step.

[0200] In this step, the Chinese ancient book reading order sorting device 10 forms a boundary between the current analysis object main column and the next analysis object main column in the above-mentioned original text image based on the frame list of the current analysis object main column.

[0201] Step S560 is a main text / detailed notes classification step.

[0202] The Chinese ancient book reading order sorting device 10 classifies the boxes included in the box list of the main column of the current analysis object into main text, left detailed notes, and right detailed notes based on the boxes included in the first moving path and the boxes included in the second moving path. The main text box can be determined by finding the intersection of the first moving path box list and the second moving path box list; the right detailed note box can be determined by finding the difference between the first moving path box list and the second moving path box list; the left detailed note box can be determined by finding the difference between the second moving path box list and the first moving path box list (refer to Fig.12 ).

[0203] Step S570 is a sub-column line segmentation step.

[0204] In this step, the Chinese ancient book reading order sorting device 10 forms the boundary between the main text part and the detailed note part of the main column of the current analysis object in the above-mentioned original text image based on the classification result (discrimination result) of step S560.

[0205] Step S580 is a reading order determination step.

[0206] The Chinese ancient book reading order sorting device 10 determines the reading order of the glyphs corresponding to the boxes included in the box list of the current main column based on the boundary between the main text part and the detailed note part of the current main column. The Chinese ancient book reading order sorting device 10 can distinguish the sub-column clusters whose lines are divided above and below the boundary between the main text part and the detailed note part as one of the main text sub-column and the detailed note sub-column based on the classification result of step S560. The Chinese ancient book reading order sorting device 10 applies the top-bottom reading order to the boxes listed as a single column inside the main text sub-column to determine the reading order of the glyphs. In the case of the detailed note sub-column, the reading order of the glyphs is determined by first applying the top-bottom reading order to the right detailed note and then applying the top-bottom reading order to the left detailed note.

[0207] Step S590 is a step of generating digital text of the current main column.

[0208] The Chinese ancient book reading order sorting device 10 sorts the above-mentioned glyphs according to the reading order determined in step S580 to generate the digital text of the current main column of the analysis object. That is, the Chinese ancient book reading order sorting device 10 arranges the glyphs according to the reading order determined for each sub-column cluster, thereby generating the digital text of the current main column.

[0209] Step S600 is a step for determining whether the next main column exists.

[0210] The Chinese ancient book reading order sorting device 10 re-executes steps S520 to S590 if the next main column of the current main column exists, and terminates the process if the next main column does not exist.

[0211] On the other hand, referring to Fig.15 In the description, each step can be further divided into additional steps or combined into fewer steps according to the specific implementation example of the present invention. In addition, some steps can be omitted as needed, and the order of the steps can also be changed. At the same time, even if other omitted contents are Figures 4 to 14 The content can also be applied to Fig.15 In addition, Fig.15 The content can be applied to Figures 4 to 14 content.

[0212] Fig.16 is a block diagram of a computer system for implementing a method for sorting the reading order of ancient Chinese books according to an embodiment of the present invention. Fig.16 The computer system shown in the figure is specifically implemented in the form of.

[0213] Reference Fig.16The computer system 1000 may include at least one of a processor 1010, a memory 1030, an input interface device 1050, an output interface device 1060, and a storage device 1040 that communicate through a bus 1070. The computer system 1000 may also include a communication device 1020 connected to a network. The processor 1010 may be a central processing unit (CPU) or a semiconductor device that executes computer-readable commands stored in the memory 1030 or the storage device 1040. The memory 1030 and the storage device 1040 may include volatile or non-volatile storage media in various forms. For example, the memory may include a ROM (read only memory) and a RAM (random access memory). In the embodiments of the present disclosure, the memory may be located inside or outside the processor, and the memory may be connected to the processor through various known technical means. The memory is a volatile or non-volatile storage medium in various forms, for example, the memory may include a read-only memory (ROM) or a random access memory (RAM).

[0214] Therefore, embodiments of the present invention may be implemented as a method implemented on a computer or as a non-transitory computer-readable medium storing computer-executable instructions. In one embodiment, when executed by a processor, the computer-readable instructions may perform a method according to at least one aspect of the present disclosure.

[0215] The communication device 1020 may transmit or receive a wired signal or a wireless signal.

[0216] In addition, the method according to the embodiment of the present invention can be specifically implemented in the form of program commands that can be executed through various computer technology means and recorded in a computer-readable medium.

[0217] The above-mentioned computer-readable medium may include program commands, data files, data structures, etc. alone or in combination. The program commands recorded in the above-mentioned computer-readable medium may be program commands specially designed and constructed for the embodiments of the present invention, or program commands known and usable by ordinary technicians in the field of computer software. The computer-readable recording medium may include a hardware device configured to store and execute program commands. For example, the computer-readable recording medium may be a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, a magneto-optical medium such as a floptical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, etc. The program command may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer through an interpreter, etc.

[0218] As mentioned above, the memory 1030 and the storage device 1040 store computer-readable commands. The processor 1010 is specifically implemented by executing the above commands. In addition, the memory 1030 and the storage device 1040 can store OCR results of the original text images of Chinese ancient books, and the processor 1010 can read the above OCR results and execute the above commands to process the information contained in the above OCR results.

[0219] In one embodiment of the present invention, the at least one processor 1010 is configured to: execute the above command, thereby: (1) selecting an analysis object frame from the OCR detection frame of the original image of the ancient Chinese book contained in the above OCR result; (2) based on the coordinate information and size information of the above analysis object frame, using the adjacent frame continuous detection method to generate a first moving path composed of frames corresponding to the main text and right detailed notes of the main column of the current analysis object and a second moving path composed of frames corresponding to the main text and left detailed notes of the main column; (3) obtaining the union of the frames included in the first moving path and the frames included in the second moving path to generate a frame list of the main column of the current analysis object.

[0220] The above-mentioned method for continuous detection of neighboring frames involves: (1) setting an initial reference frame in the above-mentioned analysis object frame; (2) using a line detection technique to detect neighboring frames in the longitudinal direction of the reference frame, and using a point detection technique to detect neighboring frames in the lateral direction of the reference frame, and repeatedly executing the detection results to determine a new reference frame according to a specified reference; (3) terminating the detection if the specified conditions are met.

[0221] The line detection technique is a technique for detecting a frame having an area corresponding to the x range of the reference frame in the analysis target frame.

[0222] The point detection technique is a technique for detecting a frame whose y coordinate of the center of the frame is included in the y range of the reference frame in the analysis target frame.

[0223] In one embodiment of the present invention, the at least one processor 1010 is configured to: in the process of generating the first moving path and the second moving path, (1) based on the coordinate information and size information of the analysis object frame, using the neighboring frame continuous detection method, generate the first moving path by taking the upper right end frame of the analysis object frame as the initial reference frame, i.e., the starting point, and taking the lower right end frame of the analysis object frame as the end point through a first downward scan; (2) based on the coordinate information and size information of the analysis object frame, set the upper right end frame as the initial reference frame, and determine the lower left end frame of the current analysis object main column through a second downward scan using the neighboring frame continuous detection method; (3) based on the coordinate information and size information of the analysis object frame, using the neighboring frame continuous detection method, generate the second moving path by taking the lower left end frame as the initial reference frame, i.e., the starting point, and taking the upper left end frame of the current analysis object main column as the end point through a second upward scan.

[0224] In one embodiment of the present invention, the at least one processor 1010 is configured to determine the upper rightmost frame based on the coordinate information and size information of the analysis object frame and the rightmost frame in the analysis object frame as a reference by utilizing the first upward scan using a continuous detection method for adjacent frames.

[0225] In one embodiment of the present invention, the at least one processor 1010 is configured to select the analysis object frame by excluding frames included in the frame list of the main column of past analysis objects from the OCR detection frame.

[0226] In one embodiment of the present invention, the at least one processor 1010 is configured to further execute a task of forming a boundary between a current analysis object main column and a next analysis object main column in the original text image based on the frame list of the current analysis object main column.

[0227] In one embodiment of the present invention, the at least one processor 1010 is configured to: (1) classify the boxes included in the box list of the current analysis object main column into main text, left detailed note, and right detailed note based on the boxes included in the first moving path and the boxes included in the second moving path; (2) form a boundary between the main text part and the detailed note part of the current analysis object main column in the original text image based on the classification result; (3) further perform a task of determining the reading order of the glyphs corresponding to the boxes included in the box list of the current analysis object main column based on the boundary between the main text part and the detailed note part.

[0228] In one embodiment of the present invention, the at least one processor 1010 is configured to further execute a task of sorting the glyphs according to the reading order to generate a digital text of the main column of the current analysis object.

[0229] In one embodiment of the present invention, the at least one processor 1010 detects a frame in the analysis object frame that exists at the bottom of the reference frame and has an area corresponding to the x range of the reference frame during the process of generating the first moving path. When more than three frames are detected, three frames are selected from the detected frames in the order of adjacent reference frames, and an operation of determining a new reference frame based on the y ranges and center y coordinates of the selected three frames is performed.

[0230] In one embodiment of the present invention, the at least one processor 1010 detects a frame in the analysis object frame that exists on the upper part of the reference frame and has an area corresponding to the x range of the reference frame during the process of generating the second moving path. When more than three frames are detected, three frames are selected from the detected frames in the order of adjacent reference frames, and an operation of determining a new reference frame based on the y range of the selected three frames, the y coordinate of the center, and whether they are included in the first moving path is performed.

[0231] Above, refer to Fig.16 The structure of the computer system for implementing the method for sorting the reading order of ancient Chinese books according to the embodiment of the present invention, that is, the device 10 for sorting the reading order of ancient Chinese books is described. Fig.16 Contents omitted from the description of Figures 4 to 15 The content can also be applied to Fig.16 In addition, Fig.16 The content can be applied to Figures 4 to 15 content.

[0232] For reference, each constituent element according to an embodiment of the present invention can be implemented in a hardware manner such as software or a digital signal processor (DSP), a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and can perform specified functions.

[0233] However, the meaning of “each component” is not limited to software or hardware, and each component may be configured to exist in an addressable storage medium or may be configured to be reproduced by one or more processors.

[0234] Thus, as an example, components include components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, programs, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables.

[0235] Each component and the functions provided in the corresponding component may be combined into a smaller number of components, or may be further separated into additional components.

[0236] On the other hand, it will be understood that each block of the flowchart and the combination of each flowchart can be performed by computer program instructions. These computer program instructions can be loaded on a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that those instructions executed by the processor of the computer or other programmable data processing device will generate means for performing the various functions described in the flowchart block (multiple blocks). Computer program instructions can also be loaded on a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable data processing device to generate a process run by the computer and the instructions of the computer or other programmable data processing device can also provide steps for performing the various functions described in the flowchart block (multiple blocks).

[0237] In addition, each block may represent a module, segment or part of a code that contains one or more executable instructions for performing a specific logical function (various functions). In addition, it should be noted that in some alternative execution examples, the various functions mentioned in the block may also be executed out of sequence. For example, two blocks shown in succession may actually be executed simultaneously, or these blocks may sometimes be executed in reverse order according to the corresponding functions.

[0238] Although the present invention has been described above with reference to the preferred embodiments, those skilled in the art will appreciate that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as described in the appended claims.

Claims

1. A method for sorting the reading order of ancient Chinese books, characterized in that: include: The step of selecting an analysis object frame, wherein the Chinese ancient book reading order sorting device selects the analysis object frame from the OCR detection frame of the original Chinese ancient book image; A multiple scanning step, wherein the Chinese ancient book reading order sorting device generates a first moving path consisting of frames corresponding to the main text and right detailed notes of the main column of the current analysis object and a second moving path consisting of frames corresponding to the main text and left detailed notes of the main column based on the coordinate information and size information of the analysis object frame by using a continuous detection method of adjacent frames; and The step of generating a frame list of the main column of the current analysis object, wherein the above-mentioned Chinese ancient book reading order sorting device calculates the union of the frames included in the above-mentioned first moving path and the frames included in the above-mentioned second moving path to generate the frame list of the main column of the current analysis object.

2. The method for sorting the reading order of ancient Chinese books according to claim 1, wherein: The above-mentioned multiple scanning steps include: The step of generating the first moving path, wherein the Chinese ancient books reading order sorting device generates the first moving path by first downward scanning with the upper rightmost frame of the analysis object frame as the starting point of the initial reference frame and the lower rightmost frame of the analysis object frame as the end point based on the coordinate information and size information of the analysis object frame and using the adjacent frame continuous detection method; a step of determining the leftmost lower frame of the main column of the current analysis object, wherein the Chinese ancient books reading order sorting device sets the rightmost upper frame as the initial reference frame based on the coordinate information and size information of the analysis object frame, and determines the leftmost lower frame of the main column of the current analysis object by a second downward scan using a neighboring frame continuous detection method; and The step of generating the above-mentioned second moving path, wherein the above-mentioned Chinese ancient book reading order sorting device generates the above-mentioned second moving path based on the coordinate information and size information of the above-mentioned analysis object frame, using the continuous detection method of adjacent frames, by taking the above-mentioned lower left end frame as the starting point of the initial reference frame and taking the upper left end frame of the current analysis object main column as the end point for a second upward scan.

3. The method for sorting the reading order of ancient Chinese books according to claim 2, wherein: The above-mentioned multiple scanning steps further include: a step of judging the above-mentioned upper rightmost frame, wherein the above-mentioned Chinese ancient book reading order sorting device is based on the coordinate information and size information of the above-mentioned analysis object frame, takes the rightmost frame in the above-mentioned analysis object frame as a reference, and judges the above-mentioned upper rightmost frame by using the first upward scan using the adjacent frame continuous detection method.

4. The method for sorting the reading order of ancient Chinese books according to claim 1, wherein: The step of selecting the analysis object frame is a step in which the Chinese ancient book reading order sorting device removes the frames included in the frame list of the past analysis object main column from the OCR detection frame and selects the analysis object frame.

5. The method for sorting the reading order of ancient Chinese books according to claim 1, wherein: It further includes: a main column line segmentation step, wherein the Chinese ancient book reading order sorting device forms a boundary between the current analysis object main column and the next analysis object main column in the original text image based on the frame list of the current analysis object main column.

6. The method for sorting the reading order of ancient Chinese books according to claim 1, wherein: Further including: a classification step, wherein the Chinese ancient book reading order sorting device classifies the boxes included in the box list of the main column of the current analysis object into main text, left detailed notes and right detailed notes based on the boxes included in the first moving path and the boxes included in the second moving path; a sub-column line segmentation step, wherein the Chinese ancient book reading order sorting device forms a boundary between the main text part and the detailed annotation part of the main column of the current analysis object in the original text image based on the classification result; and The step of determining the reading order of the glyphs, wherein the above-mentioned Chinese ancient book reading order sorting device determines the reading order of the glyphs corresponding to the boxes included in the box list of the main column of the above-mentioned current analysis object based on the boundary between the above-mentioned main text part and the detailed note part.

7. The method for sorting the reading order of ancient Chinese books according to claim 6, wherein: The method further comprises the step of generating a digital text of the main column of the current analysis object, wherein the above-mentioned Chinese ancient book reading order sorting device sorts the above-mentioned character shapes according to the above-mentioned reading order to generate the digital text of the main column of the current analysis object.

8. The method for sorting the reading order of ancient Chinese books according to claim 1, wherein: The above-mentioned neighboring box continuous detection method is as follows: Set the initial reference frame in the above analysis object frame. For the vertical direction of the reference frame, the line detection technique is used to detect the adjacent frame, and for the horizontal direction of the reference frame, the point detection technique is used to detect the adjacent frame. This process is repeated and a new reference frame is determined according to the specified reference based on the detection results. If the specified conditions are met, the detection ends. The line detection technique is a technique for detecting a frame having an area corresponding to the x range of the reference frame in the analysis target frame. The point detection technique is a technique for detecting a frame whose y coordinate of the center of the frame is included in the y range of the reference frame in the analysis target frame.

9. The method for sorting the reading order of ancient Chinese books according to claim 2, wherein: The step of generating the first moving path comprises: The above-mentioned Chinese ancient book reading order sorting device detects a frame that exists in the lower part of the reference frame and has an area corresponding to the x range of the reference frame in the above-mentioned analysis object frame. When more than three frames are detected, three frames are selected from the detected frames in the order of adjacent reference frames, and a new reference frame is determined based on the y ranges and center y coordinates of the selected three frames.

10. The method for sorting the reading order of ancient Chinese books according to claim 2, wherein: The step of generating the second moving path comprises: The above-mentioned Chinese ancient book reading order sorting device detects a frame that exists on the upper part of the reference frame and has an area corresponding to the x range of the reference frame in the above-mentioned analysis object frame. When more than three frames are detected, three frames are selected from the detected frames in the order of adjacent reference frames, and a new reference frame is determined based on the y range of the selected three frames, the y coordinate of the center, and whether they are included in the above-mentioned first moving path.

11. A device for sorting the reading order of ancient Chinese books, characterized in that: include: A memory storing an OCR result of an original image of an ancient Chinese book and computer-readable commands; as well as at least one processor, which is specifically implemented to read the above OCR result and execute the above command, The at least one processor is configured to: by executing the above command, Select an analysis target frame from the OCR detection frame of the original Chinese ancient book image included in the above OCR result, Based on the coordinate information and size information of the analysis object frame, a first moving path consisting of frames corresponding to the main text and right detailed notes of the main column of the current analysis object and a second moving path consisting of frames corresponding to the main text and left detailed notes of the main column are generated by using a neighboring frame continuous detection method. A frame list of a main column currently being analyzed is generated by obtaining a union of the frames included in the first movement path and the frames included in the second movement path.

12. The device for sorting the reading order of ancient Chinese books according to claim 11, wherein: The at least one processor is configured to: in the process of generating the first movement path and the second movement path, Based on the coordinate information and size information of the analysis object frame, the first moving path is generated by using a neighboring frame continuous detection method through a first downward scan with the upper rightmost frame in the analysis object frame as the starting point of the initial reference frame and the lower rightmost frame in the analysis object frame as the end point. Based on the coordinate information and size information of the analysis object frame, the upper rightmost frame is set as the initial reference frame, and the lower leftmost frame of the current analysis object main column is determined by a second downward scan using the adjacent frame continuous detection method. Based on the coordinate information and size information of the above-mentioned analysis object frame, the second moving path is generated by using the continuous detection method of the adjacent frame through a second upward scan with the above-mentioned lower left end frame as the starting point of the initial reference frame and the upper left end frame of the current analysis object main column as the end point.

13. The device for sorting the reading order of ancient Chinese books according to claim 12, wherein: The at least one processor is configured as follows: Based on the coordinate information and size information of the above-mentioned analysis object frame, the above-mentioned upper rightmost frame is determined by a first upward scan using a neighboring frame continuous detection method, taking the rightmost frame in the above-mentioned analysis object frame as a reference.

14. The device for sorting the reading order of ancient Chinese books according to claim 11, wherein: The at least one processor is configured as follows: The analysis target frame is selected by excluding frames included in the frame list of the main column of the past analysis target from the OCR detection frame.

15. The device for sorting the reading order of ancient Chinese books according to claim 11, wherein: The at least one processor is configured as follows: Further, a task of forming a boundary between the current analysis object main column and the next analysis object main column in the original text image based on the frame list of the current analysis object main column is performed.

16. The device for sorting the reading order of ancient Chinese books according to claim 11, wherein: The at least one processor is configured to further perform the following operations: Based on the boxes included in the first moving path and the boxes included in the second moving path, the boxes included in the box list of the current analysis object main column are classified into main text, left detailed note and right detailed note, Based on the classification result, the boundary between the main text part and the detailed annotation part of the main column of the current analysis object is formed in the original text image. Based on the boundary between the main text portion and the detailed note portion, a reading order of the glyphs corresponding to the boxes included in the box list of the main column currently being analyzed is determined.

17. The device for sorting the reading order of ancient Chinese books according to claim 16, wherein: The at least one processor is configured as follows: The operation of sorting the above-mentioned character shapes according to the above-mentioned reading order to generate a digital text of the main column of the current analysis object is further performed.

18. The device for sorting the reading order of ancient Chinese books according to claim 11, wherein: The above-mentioned neighboring box continuous detection method is as follows: Set the initial reference frame in the above analysis object frame. For the vertical direction of the reference frame, the line detection technique is used to detect the adjacent frame, and for the horizontal direction of the reference frame, the point detection technique is used to detect the adjacent frame. This process is repeated and a new reference frame is determined according to the specified reference based on the detection results. If the specified conditions are met, the detection ends. The line detection technique is a technique for detecting a frame having an area corresponding to the x range of the reference frame in the analysis target frame. The point detection technique is a technique for detecting a frame whose y coordinate of the center of the frame is included in the y range of the reference frame in the analysis target frame.

19. The device for sorting the reading order of ancient Chinese books according to claim 12, wherein: The at least one processor performs the following operations in the process of generating the first moving path: A frame that exists in the lower part of the reference frame and has an area corresponding to the x range of the reference frame is detected in the above-mentioned analysis object frame. When more than three frames are detected, three frames are selected from the above-mentioned detected frames in the order of adjacent reference frames, and a new reference frame is determined based on the y ranges and center y coordinates of the selected three frames.

20. The device for sorting the reading order of ancient Chinese books according to claim 12, wherein: The at least one processor performs the following operations in the process of generating the second moving path: A frame that exists on the upper part of the reference frame and has an area corresponding to the x range of the reference frame is detected in the above-mentioned analysis object frame. When more than three frames are detected, three frames are selected from the above-mentioned detected frames in the order of adjacent reference frames, and a new reference frame is determined based on the y ranges of the selected three frames, the y coordinates of the centers, and whether they are included in the above-mentioned first moving path.

Images