Target detection-based string diagram family tree digitization method, system and device

Abstract

The application discloses a kind of based on target detection's hanging line chart pedigree digitalization method, system and equipment, first preprocess pedigree image, including gray scale processing, binaryzation processing;Get hanging line chart horizontal projection chart, and remove title part, then splice two side pictures, and eliminate generation mark;Then to the pedigree image after preprocessing, based on maximum stable extreme value area algorithm MSER, obtain name area;Dissolve out the name area coordinates and name area side length contained in MSER object, according to the upper left corner coordinates of name area and name side length get the coordinates of the upper left and lower right corner of name area coordinates;Based on the coordinates coordinates obtained by MSER object, eliminate the area with overlap rate higher than threshold A;Then based on geometric position fusion biao-pang part, and identify hanging line area;Finally identify name area;The application improves the efficiency of pedigree input in the aspect of pedigree automation input, improves the level of pedigree digitalization.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer image recognition and processing, and relates to a hanging line chart family tree digitization method, system and device, in particular to a hanging line chart family tree digitization method, system and device based on target detection. BACKGROUND

[0002] The hanging line chart is used to represent the kinship between the characters in the family, and is named after the hanging line chart. The characters are arranged from left to right or from top to bottom based on generations, and the characters with kinship are connected by lines, which looks like a string of connected lines. Common hanging line charts include double-column five-generation and nine-generation charts.

[0003] OpenCV is the abbreviation of Open Source Computer Vision Library, which is an open source computer vision algorithm and tool library, and is also the most widely used computer vision library at present.

[0004] OpenCV has the following characteristics:

[0005] (1) A large number of computer vision algorithms are implemented, and many tools and data structures are provided to facilitate the development of computer vision programs.

[0006] (2) Based on Apache 2 license open source, it allows any individual and organization to use and modify the code of OpenCV under relatively loose conditions, and is friendly to commercial applications.

[0007] (3) Developed using C++ language, mainly provides C++ language interface. At the same time, OpenCV also provides interfaces of other programming languages based on C++ language interface, among which the most important is the Python language interface, that is, the opencv-python package. Due to the characteristics of simple and flexible Python language, strong expression ability and easy to learn, opencv-python is widely used in prototype development, demonstration and teaching field, and is also the OpenCV interface used in the scheme described in this paper.

[0008] (4) Compatible with major desktop operating systems such as Linux, Windows, macOS, FreeBSD and OpenBSD, and mainstream mobile operating systems such as Android and iOS.

[0009] OpenCV has a wide range of applications, including various 2D and 3D image tools, face recognition, gesture recognition, human-computer interaction, mobile robot technology, motion understanding, object detection, segmentation and recognition, binocular stereo vision, structure in motion, motion tracking, augmented reality, etc. To support the above application fields, OpenCV provides a machine learning library, which includes boosting methods, decision tree learning, gradient boosting tree, expectation maximization algorithm, K- nearest neighbor algorithm, naive Bayes classifier, artificial neural network, random forest, support vector machine, deep neural network, etc.

[0010] NumPy is a basic library in Python for scientific computing, providing data structures such as multi-dimensional arrays, matrices, and functions for fast operations on these data structures, including mathematics, logic, shape operations, sorting, selection, IO, discrete Fourier transform, basic linear algebra, basic data operations, random simulation, etc. The core data structure of NumPy is ndarray, which encapsulates a multi-dimensional array of the same type and fixed length, which is quite different from the native Python list.

[0011] (1) Many operations on \texttt{ndarray}, such as addition, subtraction, multiplication, and scalar multiplication between two matrices, are compiled into machine code for execution, avoiding the low efficiency of the Python interpreter;

[0012] (2) Operations on multi-dimensional arrays are wrapped in a similar form to operations on ordinary numbers, avoiding the need to write iterative code, reducing code size and the probability of errors. For example, to calculate the multiplication of two matrices, the handwritten iterative code needs to maintain 4 index values, multiply the numbers in the two matrices in turn and add them up. The code size is relatively large and may contain errors. Using NumPy, you can directly use a*b to calculate the multiplication of two matrices, greatly reducing the code size, and the code in NumPy has been tested and applied extensively, making it difficult to contain errors.

[0013] Ndarray has several important properties, as shown in Table 1:

[0014] Table 1

[0015] Attribute Name Type Description ndim int Number of dimensions of the ndarray shape tuple Shape of the ndarray, e.g. a matrix with n rows and m columns has shape (n, m) size int Number of elements in the ndarray, equal to the product of all values in shape dtype np.dtype Type of the elements, e.g. unint32, float64, etc. itemsize int Size of an element in bytes flags np.flagsobj Memory information of the ndarray

[0016] The imread function of OpenCV reads an image from a file into memory, and the image in memory is stored as an ndarray of NumPy. This is because the pixels in the image are generally composed of 3 channels of 8-bit unsigned integers, and the image itself can be regarded as a two-dimensional matrix of pixels. For example, an image read into memory in the sRGB color space is an ndarray with shape (h, w, 3) and dtype uint8, where the first value h is the height of the picture, the second value w is the width of the image, and the third value 3 indicates that each pixel is composed of 3 uint8 integers; the grayscale image converted by cvtColor has a corresponding shape (h, w) and dtype uint8, indicating that each pixel of it is a uint8, i.e., a grayscale value.

[0017] MSER stands for Maximally Stable Extremal Regions, which is a computer image-based target detection algorithm. It is mainly based on the idea of "watershed" to detect blob regions in images. Although the accuracy of MSER algorithm is not as good as that of many algorithms of convolutional neural network (CNN), it can be used as a preliminary stage of target detection to generate as many candidate targets as possible. MSER is very suitable for processing text target detection tasks with large contrast and target region grayscale close to each other.

[0018] The MSER algorithm mainly performs binaryzation processing on grayscale images, but not only uses a few threshold values, but increases from 0 to 255. Intuitively, the binaryzation processed image is like high and low flat land, hills and mountains being flooded by water. From top to bottom, the entire region is divided into two parts: land and water. When the threshold value is 0, no land is flooded, and the entire image is white. As the threshold value increases, more and more land is flooded, and the small black areas gradually connect to form a line and a piece, and the small black areas gradually connect to form a whole. When the threshold value is 255, all the land is flooded, and the entire image is black.

[0019] As the threshold value increases, the area of the connected region corresponding to the text changes at a relatively small rate within a certain threshold value range. They are like those very steep peaks, and the flooded area under high and low water levels does not differ much. Therefore, as long as the threshold value at which the area change rate of the connected region is the smallest can be captured, the region of interest can be found.

[0020] The MSER algorithm has the following advantages:

[0021] (1) Affine invariance for image intensity;

[0022] (2) Stability: only select regions with almost the same credibility within the threshold range;

[0023] (3) Multi-scale detection does not require any smoothing, and both fine structures and large structures are detected.

[0024] (4) Enumerating all extreme value regions also only requires O(n) time complexity in the worst case, where n is the number of pixels in the image.

[0025] The mathematical principle of the MSER algorithm can be simply summarized as:

[0026]

[0027] where i is the threshold, R(i) is a connected region when the threshold is i, Δ is a small increment of i, R(i)+Δ is the corresponding connected region, and |R(i)| is the area of R(i). Therefore, |R(i+Δ)-R(i)| is the change in the area of the connected region when i is slightly increased, and the ratio of |R(i+Δ)-R(i)| to |R(i)| is q(i), which is the change rate of the area of the connected region when the threshold is i. As a function of i, the change of q(i) indicates that the area of the connected region changes at different rates at different thresholds. When q(i) reaches a local minimum, it means that the change amplitude of the corresponding R(i) is the smallest, which can be regarded as the maximum stable extreme value region.

[0028] For text region recognition, the MSER algorithm can be used as the first step, followed by the following processing methods:

[0029] (1) Combine corner detection to select accurate text regions;

[0030] (2) According to the angle, aspect ratio, and other simple geometric relationships, eliminate regions that are obviously not text;

[0031] (3) Perform dilation, erosion, and other operations on the MSER region to obtain the connected domain of the text region and obtain its contour;

[0032] (4) Use the NMS algorithm to obtain non-repeating regions.

[0033] NMS (Non-Maximum Suppression, commonly used in target detection algorithms to remove duplicate regions. The NMS algorithm judges the standard of duplicate regions is IOU (Intersection over Union), which is the overlap degree of duplicate regions. An IOU calculation method for rectangular regions A and B is:

[0034]

[0035] The IOU is the ratio of the area of the overlapping part of the two rectangles to the total area. The larger the IOU, the more the two rectangles overlap.

[0036] The main idea of NMS is to sort all candidate regions according to the score or confidence, keep the region with the highest score as the standard, and calculate the IOU of other regions with the standard region. If the IOU is higher than a certain threshold, it means that the region is repeated with the standard region, and the region is deleted. The next round continues to process the remaining regions in this way until all regions with a high repetition rate higher than the threshold are removed. In commonly used target detection algorithms, such as Faster R-CNN, NMS algorithm is often used to filter a large number of candidate regions detected in the previous step to obtain the most representative candidate regions, thereby speeding up the efficiency of target detection.

[0037] Since entering the era of informatization and digitization, the trend of digitizing genealogy has emerged in China, and many digitized and specialized genealogy software and website platforms have appeared. In 2002, Chen Weibin et al. proposed a digital genealogy website system with the structure of data service layer, transaction logic block and presentation layer, which uses ASP (Active Server Pages) technology to provide fast dynamic web page generation features to support various query functions of genealogy, including surname query, regional query and combined query, etc. Query mode can visualize the generation relationship as a family tree. In 2010, Xiao Yu introduced the National Library's "Global Chinese Roots Network" project based on the concept and method of Lib2.0. The project is committed to collecting and sorting pedigree documents, indexing the characters and articles in them, and generating regular genealogy charts for genealogy with high cultural value. It also provides a website for uploading family trees and online family tree compilation. In 2011, Kezhou realized a genealogy website, mainly based on the WAMP technology of the fast website building tool at that time, which realized the functions of genealogy search, hundred surnames search, user management, etc. In 2017, Xu Mingmin studied the optimization of genealogy system based on in-memory database SQLite and table partitioning technology, and researched the visualization of genealogy based on the analysis of genealogy data. Using database query technology, indexing technology and SSC coding method, an effective method for family root-seeking is proposed. In 2018, He Zilong optimized the generation process of genealogy with a large number of people by using large string internal and external dynamic calling algorithm and parallel processing algorithm, and explored the way of personalized customization of genealogy. In 2020, Peng Xufei realized a personalized management system of genealogy based on SSH (Structs2 framework, Spring web framework and Hibernate ORM framework), JDBC and Python technology, and designed a splicing algorithm for character relationship.

[0038] There are also international platforms to record the family proliferation and development history, among which FamilySearch and Ancestry are the representatives.

[0039] FamilySearch (https: / / www.familysearch.org) is a non-profit organization and website, mainly providing genealogy records, education and software. It provides free access to the public, maintains a series of records, resources and services, aims to help more people understand the history of the family, and is committed to protecting the records of human family activities. The website also provides a variety of personalized genealogy services for users, including free and paid services, including genealogy creation, editing, viewing, uploading, indexing and other functions. In terms of technology, FamilySearch has created the international genealogy industry standard GEDCOM, and standardized the style of genealogy. At present, FamilySearch provides services in more than 5100 family history centers in 140 countries, with more than 1.3 billion people in the genealogy tree and historical records database, and more than 5.7 billion digitized images, books, videos and other data.

[0040] Ancestry (https: / / www.ancestry.com) is the world's largest private for-profit genealogy company. Since 1983, it has developed into the world's most famous genealogy website, with a wealth of genealogy information database. Ancestry provides software FamilyTreeMaker, which can create and manage family trees, and mobile software such as Ancestry and Shoebox to help users understand their origins and explore their ancestors, as well as add genealogy data.

[0041] In summary, the existing genealogy digitization system mainly focuses on providing faster and more convenient input experience, but essentially still relies on manpower to complete repetitive work, which is low in efficiency and can also reduce user experience, leading to user abandonment. SUMMARY

[0042] The present application provides a target detection-based method that combines mature OCR technology to derive the relationship of the characters in the hanging line diagram type genealogy, improves the genealogy input efficiency, and improves the genealogy digitization level.

[0043] The technical solution adopted by the method of the present application is: a hanging line diagram genealogy digitization method based on target detection, comprising the following steps:

[0044] Step 1: preprocessing the genealogy image, including gray processing, binarization processing of the gray image; based on the binarized image, a horizontal projection graph of the hanging line graph is obtained; based on the horizontal projection graph features, the title part is removed, and then the two side images are spliced; based on the horizontal projection graph of the spliced image, the generation identifier is removed;

[0045] Step 2: based on the maximum stable extreme value region algorithm MSER, the approximate position of the text region is obtained for the preprocessed genealogy image; the text region coordinates and the text region length contained in the MSER object are disassembled, and the coordinates of the upper left corner and the lower right corner of the text region are obtained according to the upper left corner coordinates and the text length; based on the coordinates obtained by the MSER object, the region with an overlap rate higher than a threshold A is removed;

[0046] Step 3: fusing the components based on the geometric position;

[0047] The text region obtained in step 2 is taken as a candidate region, and the geometric center of each candidate region is obtained by traversing the candidate region; if the distance between the geometric centers of two candidate regions is less than a threshold B, the two regions are fused according to the maximum range;

[0048] Step 4: removing the hanging line region and the non-text region;

[0049] According to the pixel area size of the text in the image, a threshold C is set, and all candidate regions are traversed; if the area of a candidate region is less than the threshold C, the region is determined as a non-text region; according to the difference between the hanging line and the Chinese character geometric features recognized in the image, a length-width ratio threshold D of the candidate region is set; if the length-width ratio of the detected region is greater than the threshold D, the region is determined as a hanging line region; the coordinates of the non-text region and the hanging line region are set to 0 and are collectively deleted;

[0050] Step 5: fusing the text region into a name region;

[0051] The geometric centers of the candidate regions are obtained by traversing the candidate regions; if the geometric centers of two candidate regions are less than a threshold E, the two texts are considered to belong to one name, and the two regions are fused according to the maximum range to obtain an accurate name region.

[0052] The technical scheme adopted by the system of the application is: a hanging line graph genealogy digitization system based on target detection, comprising the following modules:

[0053] Module 1 is used for preprocessing the genealogy image, including gray processing, binarization processing of the gray image; based on the binarized image, a horizontal projection graph of the hanging line graph is obtained; based on the horizontal projection graph features, the title part is removed, and then the two side images are spliced; based on the horizontal projection graph of the spliced image, the generation identifier is removed;

[0054] Module 2, for obtaining the approximate position of the text region based on the maximum stable extreme region algorithm MSER for the preprocessed family tree image; disassembling the text region coordinates and text region edge length contained in the MSER object, and obtaining the coordinates of the upper left corner and the lower right corner of the text region according to the upper left corner coordinates and the text edge length of the text region; based on the coordinates coordinates obtained by the MSER object, the region with an overlap rate higher than a threshold A is removed;

[0055] Module 3, for fusing the components based on the geometric position;

[0056] fusing the components based on the geometric position;

[0057] The text region obtained in module 2 is taken as a candidate region, the candidate region is traversed, the geometric center of each candidate region is obtained, and if the distance between the geometric centers of two candidate regions is less than a threshold B, the two regions are fused according to the maximum range;

[0058] Module 4, for removing the hanging line region and the non-text region;

[0059] According to the pixel area size occupied by the text in the image, a threshold C is set, all candidate regions are traversed, and if the area of a candidate region is less than the threshold C, the region is judged as a non-text region; according to the fact that the hanging line and the Chinese character have different geometric features in the image, a length-width ratio threshold D of the candidate region is set, and if the length-width ratio of the detected region is greater than the threshold D, the region is judged as a hanging line region; the coordinates of the non-text region and the hanging line region are set to 0 and are collectively deleted;

[0060] Module 5, for fusing the text region into a name region;

[0061] The candidate regions are traversed, the geometric centers of the candidate regions are obtained, and if the geometric centers of two candidate regions are less than a threshold E, the two texts are considered to belong to one name, and the two regions are fused according to the maximum range to obtain an accurate name region.

[0062] The technical scheme adopted by the device of the application is: a hanging line graph family tree digitization device based on target detection, comprising:

[0063] One or more processors;

[0064] A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the target detection-based hanging line graph family tree digitization method.

[0065] Compared with the prior art, the device has the following beneficial effects:

[0066] (1)In the technical level, the present application is only based on target detection, and does not use the method of machine learning or deep learning to identify the name area in the subsequent process. The present application can accurately identify the name area by using the traditional detection method, and the identification accuracy can be more than 90%. Since the present application can accurately identify the character area, it can provide accurate and large data set for future machine learning of Chinese character area recognition or name area recognition, and can avoid time-consuming and error-prone manual annotation. This detection method is simple and reliable, but also because of its simplicity, it is only suitable for name area recognition of hanging line chart family tree.

[0067] (2)In the humanistic level, as the economic society gradually develops, the public and the academic circle gradually pay more attention to the family with long history. The present application can accurately arrange a large number of person relationships in a short time. After establishing the person relationship, the academic field can analyze the person relationship, trace the history of the person, and perform the work of historical data evidence, and the public can greatly facilitate the query and trace the person relationship in the large number of person relationships. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 The method flowchart of the embodiment of the present application is shown in the figure;

[0069] Figure 2 The horizontal projection flowchart of the hanging line chart of the embodiment of the present application is shown in the figure;

[0070] Figure 3 The flowchart of removing the generation mark of the embodiment of the present application is shown in the figure;

[0071] Figure 4 The non-maximum suppression algorithm NMS flowchart of the embodiment of the present application is shown in the figure;

[0072] Figure 5 The fusion of the radical and the head of the embodiment of the present application is shown in the figure;

[0073] Figure 6 The father area detection flowchart of the embodiment of the present application is shown in the figure;

[0074] Figure 7 The person relationship integration flowchart of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0075] In order to facilitate those skilled in the art to understand and implement the present application, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0076] See Figure 1 The present application provides a hanging line chart family tree digitization method based on target detection, which comprises the following steps:

[0077] Step 1: preprocessing family tree image, including gray processing, binarization processing of gray image; based on the binarized image, get the horizontal projection graph of the hanging line graph; based on the horizontal projection graph features, remove the title part, and then splice the two side pictures; based on the horizontal projection graph of the spliced picture, remove the generation identifier affecting the subsequent MSER algorithm;

[0078] See Figure 2 In this embodiment, based on the binarized image, the horizontal projection graph of the hanging line graph is obtained; the specific implementation includes the following sub-steps:

[0079] (1) input the binarized image of numpy type;

[0080] (2) count the non-zero values of each row of the binarized image, denoted as c;

[0081] (3) set each row in the result picture as a pure black value of c units;

[0082] (4) create a matrix, corresponding to each row of c pure black value, output the matrix in the form of picture.

[0083] See Figure 3 In this embodiment, the generation identifier is removed, and the specific implementation includes the following sub-steps:

[0084] (1) input the picture to be removed from the generation identifier, denoted as img; and its binarized image, denoted as bin;

[0085] (2) get the number of non-zero values of each row of bin, denoted as the number of non-zero values of the ith row C i；

[0086] (3) traverse each row of C i , if C i is greater than threshold A1, put it into a container with capacity B, when the container is full of elements, record the minimum row number and empty the container; if C i is less than threshold A1, empty the container;

[0087] (4) for each recorded row number, delete this row and the next preset number of rows from the input img;

[0088] (5) delete the specified row of img, which is the required result.

[0089] Step 2: For the pre-processed family tree image, the approximate position of the text region is obtained based on the maximum stable extreme region algorithm MSER. The coordinates of the text region and the length of the text region are extracted from the MSER object. The coordinates of the upper left corner and the lower right corner of the text region are obtained according to the coordinates of the upper left corner and the length of the text region. Based on the region coordinates obtained by the MSER object, the non-maximum suppression algorithm NMS is used to eliminate regions with an overlap rate higher than a threshold value.

[0090] See Figure 4 In this embodiment, the non-maximum suppression algorithm NMS includes the following sub-steps:

[0091] (1) Traverse all the name regions to obtain the area of the region.

[0092] (2) Sort the regions according to the size of the vertical coordinates of the lower right corner from small to large.

[0093] (3) Unconditionally push the region with the largest vertical coordinate value to the stack C.

[0094] (4) Calculate the overlap rate of the remaining regions with the top element of the stack C, and eliminate the regions with an overlap rate greater than the threshold value.

[0095] (5) Continue to select the region with the largest vertical coordinate value of the lower right corner from the remaining regions and push it to the stack. Calculate the overlap rate of the top element of the stack C with the remaining regions. If the overlap rate is greater than the threshold value, eliminate it. Repeat this process until there is no region with an overlap rate greater than the threshold value with the top element of the stack C.

[0096] In this embodiment, the MSER object is obtained using the interface cv2.MSER_create provided by OpenCV, and its Python signature is as follows:

[0097] defMSER_create(_delta=None,_min_area=None,

[0098] _max_area=None,_max_variation=None,...):

[0099] Pass

[0100] All parameters are optional. The focus of the present application is the first four parameters, and the following parameters are only needed when MSER_create is applied to color images, which will not be described here. The first four parameters are described in Table 2 as follows. Through the study of the rules of genealogy images, the more suitable parameter settings are obtained after multiple experiments: the _delta value is 5, the _min_area value is 10, the _max_variation value is 0.5, and the _max_area value is the default value.

[0101] Table 2

[0102]

[0103]

[0104] Based on the region coordinates coordinates obtained by the MSER object, the NMS algorithm, i.e. the non-maximum suppression algorithm, is used. The coordinate parameter is the coordinates coordinates of the region in the MSER object, and the threshold parameter is the maximum allowed overlap rate threshold of each detected region, which is set to 0.5 here, i.e. the maximum allowed overlap rate is 50%.

[0105] After the NMS algorithm, the regions with high overlap rate are removed. At this time, the remaining detected regions basically complete the detection of the position of the text, but the detected regions may contain radicals, hanging lines, parts of characters, etc. This result is retained for further processing in step 3.

[0106] Step 3: Fusion of radicals based on geometric position

[0107] The regions obtained in step 2 contain complete Chinese characters, incomplete Chinese characters, radicals, strokes, etc., so the parts that cannot be recognized need to be fused into complete Chinese character regions.

[0108] In this embodiment, the text regions obtained in step 2 are taken as candidate regions, and the candidate regions are traversed to obtain the geometric center of each candidate region. If the distance between the geometric centers of two candidate regions is less than a threshold B, the two regions are fused according to the maximum range.

[0109] See Figure 5 The specific implementation of step 3 in this embodiment includes the following sub-steps:

[0110] (1) Traverse all candidate regions;

[0111] (2) Record the geometric center;

[0112] (3) Obtain the larger value of the horizontal distance and the vertical distance between the geometric center of the subject region and the geometric center of the object region, and denote it as dist;

[0113] (4) judging the size relation of dist to threshold B, if dist is smaller than threshold B, then merging the object region to the main region in the maximum range of the object region, if dist is larger than threshold B, then no operation.

[0114] In this embodiment, all candidate regions are traversed, and the regions with small area and obviously not meeting the normal character size are removed, and the detected hanging line regions in the image are removed. That is, according to the pixel area size of the characters in the image, a threshold is set, if the area of the candidate region is smaller than the threshold, then the region is judged as a small area region not belonging to the character. According to the difference between the hanging line and the Chinese character in the image, a threshold of the aspect ratio of the candidate region, MAX CHAR RATIO, is set, if the aspect ratio of the detected region is larger than the threshold, then the region is judged as a hanging line region. The coordinates of the non-character region and the hanging line region are set to 0.

[0115] In combination with the above result, the 0 coordinates are deleted, and the regions obtained at this time can accurately frame the characters. Based on the positional relation of each character, the regions are merged, so that the framed region is the name of the person and not only a single character. The detected regions are traversed, the geometric centers of the regions are obtained, if the geometric centers of two regions are smaller than 2.2 times the half length (AVERATE HALF LENGTH), then the two characters are considered to belong to the same name, then the two regions are merged in the maximum range to obtain the name region. At this time, almost all the target detected regions are the name of the person.

[0116] Step 4: removing the hanging line region and the non-character region;

[0117] In this embodiment, according to the pixel area size of the characters in the image, a threshold C is set, all candidate regions are traversed, if the area of the candidate region is smaller than the threshold C, then the region is judged as a non-character region; according to the difference between the hanging line and the Chinese character in the image, a threshold D of the aspect ratio of the candidate region is set, if the aspect ratio of the detected region is larger than the threshold D, then the region is judged as a hanging line region; the coordinates of the non-character region and the hanging line region are set to 0 and are collectively deleted;

[0118] Step 5: merging the character region to the name region;

[0119] The candidate regions are traversed, the geometric centers of the candidate regions are obtained, if the geometric centers of two candidate regions are smaller than a threshold E, then the two characters are considered to belong to the same name, then the two regions are merged in the maximum range to obtain the accurate name region.

[0120] After obtaining the accurate name region, this embodiment can further judge and find the father or brother region of the main region.

[0121] The specific implementation includes the following steps:

[0122] Step 6: detecting the person relationship according to the physical position relationship between the respective name regions, detecting the father region and the brother region of the subject region; wherein the subject region is the name region obtained in step 5, and the region compared with the subject region is called the object region;

[0123] Detecting the father region of the subject region, first traversing the subject region, judging whether there is an object region on the left side of the subject region, that is, judging whether the difference between the longitudinal coordinate of the geometric center of the subject region and the geometric center of the object region is within the threshold value F (the value of the embodiment is 10 pixels), and the horizontal coordinate value of the geometric center of the object region is less than the horizontal coordinate value of the geometric center of the subject region, then it is indicated that the subject region has a "father"; putting all the object regions meeting the above requirements into a list, finding the object region with the largest horizontal coordinate of the geometric center in the list, that is, the object region closest to the subject region, and regarding it as the "father" region of the subject region; if there is no object region meeting the requirements on the left side of the subject region, then step 7 is executed; if there is no object region meeting the requirements on the left side of the subject region, then no operation is performed.

[0124] Detecting the brother region of the subject region, that is, judging whether the longitudinal coordinate value of the geometric center of the object region is less than the longitudinal coordinate value of the geometric center of the subject region, and whether the difference between the horizontal coordinates of the geometric center of the subject region and the geometric center of the object region is within the threshold value G (the value of the embodiment is 130 pixels), if yes, then it is indicated that the subject region has a "brother"; putting all the object regions meeting the above requirements into a list, finding the object region with the largest longitudinal coordinate of the geometric center in the list, that is, the object region closest to the subject region, and regarding it as the "brother" region of the subject region; if there is no object region meeting the requirements above the subject region, then no operation is performed.

[0125] See Figure 6 The implementation of "detecting the father region" in the embodiment specifically includes the following sub-steps:

[0126] (1) traversing the name region;

[0127] (2) obtaining the horizontal and longitudinal coordinates of the geometric center of the subject region and the horizontal and longitudinal coordinates of the geometric center of the object region;

[0128] (3) putting the object region with the horizontal coordinate less than the horizontal coordinate of the geometric center of the subject region and the difference within the threshold value E from the longitudinal coordinate of the geometric center of the subject region into the container E;

[0129] (4) traversing the regions in the container F, and the region with the largest horizontal coordinate is the father region of the subject region;

[0130] The implementation of "detecting the brother region" specifically includes the following sub-steps:

[0131] (1) traversing the name region;

[0132] (2) get the horizontal and vertical coordinates of the geometric center of the subject region and the geometric center of the object region;

[0133] (3) put the object region whose vertical coordinate of the geometric center is less than the vertical coordinate of the geometric center of the subject region and whose difference from the horizontal coordinate of the geometric center of the subject region is within the threshold G into the container H;

[0134] (4) traverse the region in the container H, and the region with the largest vertical coordinate is the brother region of the subject region.

[0135] Step 7: integrate the "father" relationship and the "brother" relationship into the "father" relationship;

[0136] traverse all subject regions, if the center point coordinate of the subject region is less than the boundary threshold G, i.e. the region is at the leftmost side of the picture, the "father" region cannot be found, and the region is regarded as the root node of a tree; if the subject region has the "father" attribute, no operation is performed; if the subject region does not have the "father" attribute but has the "brother" attribute, it is recursively found whether the "brother" region has the "father" attribute, if the "brother" region has the "father" attribute, the "father" attribute of the subject region is changed to the "father" attribute of the "brother" region, if the "brother" region does not have the "father" attribute but has the "brother" attribute, it is further recursively found the "father" attribute of the "brother" of the "brother" region; if the subject region does not have the "father" attribute and does not have the "brother" attribute, it is indicated that the region is the root region of a tree.

[0137] See Figure 7 The specific implementation of step 7 in the embodiment includes the following sub-steps:

[0138] (1) traverse the name region, detect the father region and the brother region of the subject region;

[0139] (2) if the subject region has the father region, no operation is performed; if the subject region does not have the father region, it is judged whether the subject region has the brother region, if the subject region does not have the brother region, it is determined that the subject region is the root node; if the subject region has the brother region, it is recursively found the father region of the brother region of the subject region, and the father region of the brother region is regarded as the father region of the subject region.

[0140] According to the "parent-child" relationship between the obtained regions, the regions are connected to the text, and the parent-child relationship between the person entities is obtained. Here, the persons in the pedigree in the page diagram are established to have the person relationship.

[0141] It should be understood that the above description is merely a detailed explanation of the preferred embodiments and is not intended to limit the patent protection scope of the present application. Any modification or alternation made by those skilled in the art without departing from the scope of the present application shall fall within the patent protection scope of the present application. The patent protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for digitizing genealogical records using pylon diagrams based on target detection, characterized in that, The method comprises the following steps: Step 1: preprocessing the genealogy image, including gray processing and binarization processing of the gray image; Based on the binarized image, the horizontal projection graph of the hanging line graph is obtained; Based on the horizontal projection graph features, the title part is removed, and then the two side images are spliced; Based on the horizontal projection graph of the spliced image, the generation identifier is removed; Step 2: based on the maximum stable extreme region algorithm MSER, the approximate position of the text region is obtained for the preprocessed genealogy image; The text region coordinates and text region area length contained in the MSER object are disassembled, the coordinates of the upper left corner and the lower right corner of the text region are obtained according to the upper left corner coordinates and the text length of the text region, and the region with an overlap rate higher than a threshold A is removed based on the coordinates coordinates obtained by the MSER object; Step 3: fusing the components and radicals based on the geometric position; The text region obtained in step 2 is taken as a candidate region, the geometric center of each candidate region is obtained by traversing the candidate region, and if the distance between the geometric centers of two candidate regions is less than a threshold B, the two regions are fused according to the maximum range; Step 4: removing the hanging line region and non-text region; According to the pixel area size of the text in the image, a threshold C is set, all candidate regions are traversed, and if the area of a candidate region is less than the threshold C, the region is determined as a non-text region; according to the fact that the hanging line and the Chinese character have different geometric features in the image, a length-width ratio threshold D of the candidate region is set, and if the length-width ratio of the detected region is greater than the threshold D, the region is determined as a hanging line region; the coordinates of the non-text region and the hanging line region are set to 0 and then deleted; Step 5: fusing the text region into a name region; The candidate regions are traversed to obtain the geometric centers of the candidate regions, and if the geometric centers of two candidate regions are less than a threshold E, the two texts are regarded as belonging to one name, and the two regions are fused according to the maximum range to obtain an accurate name region.

2. The target detection based string diagram pedigree digitization method of claim 1, wherein: The horizontal projection graph of the hanging line graph based on the binarized image in step 1 comprises the following sub-steps: (1) input the binarized image of the numpy type; (2) count the non-zero values of each row of the binarized image, and record the count as c; (3) set each corresponding row in the result image as a pure black value of c units; (4) create a new matrix, and set each row as a pure black value of c units, and output the matrix in the form of an image.

3. The target detection based string diagram pedigree digitization method of claim 1, wherein: The removal of the generation identifier in step 1 comprises the following sub-steps: (1) input the image from which the generation identifier is to be removed, denoted as img, and the binarized image thereof, denoted as bin; (2) obtain the number of non-zero values of each row of bin, and record the number of non-zero values of the i-th row as Ci; (3) traverse each row's C i , if C i is greater than threshold A1, put it into a container with capacity B, when the container is full of elements, record the minimum row number and empty all elements in the container; if C i is less than threshold A1, empty all elements in the container; (4) for each recorded row number, delete the row and a predetermined number of rows below the row from the input img; (5) delete the specified row of img to obtain the required result.

4. The target detection based string diagram pedigree digitization method of claim 1, wherein: In step 2, the region coordinates coordinates obtained by the MSER object are removed by using the non-maximum suppression algorithm NMS when the overlap rate is higher than a threshold; The non-maximum suppression algorithm NMS comprises the following sub-steps: (1) traverse all name regions to obtain the area of each region; (2) According to the size of the lower right corner of the region from small to large; (3) The maximum vertical coordinate value of the region is unconditionally stacked into the stack C; (4) The remaining regions are respectively calculated with the stack C top element to obtain the overlap rate, and the region with the overlap rate greater than the threshold value is removed; (5) Continue to select the region with the maximum right lower corner vertical coordinate value from the remaining regions, calculate the overlap rate of the stack C top element and the remaining regions, and if the overlap rate is greater than the threshold value, the region is removed, until there is no region with the overlap rate greater than the threshold value with the stack C top element.

5. The target detection based string diagram pedigree digitization method of claim 1, wherein: The specific implementation of step 3 includes the following sub-steps: (1) Traverse all candidate regions; (2) Record the geometric center; (3) Get the larger value of the horizontal distance and the vertical distance between the geometric center of the main region and the geometric center of the object region, and mark it as dist; (4) Judge the size relationship of dist and the threshold value B, if dist is less than the threshold value B, then the object region is fused according to the maximum range of the main region in the object region, if dist is greater than the threshold value B, then do not operate.

6. The target detection based string diagram family digitization method of any of claims 1-5, wherein: After obtaining the accurate name region in step 5, further judge and find out the father or brother region of the main region; The specific implementation includes the following steps: Step 6: Detect the relationship between the characters according to the physical position relationship between the name regions, and detect the father region and the brother region of the main region; wherein the region compared with the name region obtained in step 5 is called the object region; Said detection of the father region of the main region, first traverse the main region, judge whether there is an object region on the left side of the main region, that is, whether the difference between the vertical coordinates of the geometric center of the main region and the geometric center of the object region is within the threshold value F, and the horizontal coordinate value of the geometric center of the object region is less than the horizontal coordinate value of the geometric center of the main region, then it is proved that the main region has "father"; put all the object regions meeting the above requirements into the list, find the object region with the largest horizontal coordinate of the geometric center in the list, that is, the object region closest to the main region, and regard it as the "father" region of the main region; if there is no object region meeting the requirements on the left side of the main region, then execute the following step 7; if there is no object region meeting the requirements on the left side of the main region, then do not operate; Said detection of the brother region of the main region, that is, to judge whether the vertical coordinate value of the geometric center of the object region is less than the vertical coordinate value of the geometric center of the main region, and whether the difference between the horizontal coordinates of the geometric center of the main region and the geometric center of the object region is within the threshold value G, if yes, it is proved that the main region has "brother"; put all the object regions meeting the above requirements into the list, find the object region with the largest vertical coordinate of the geometric center in the list, that is, the object region closest to the main region, and regard it as the "brother" region of the main region; if there is no object region meeting the requirements above the main region, then do not operate; Step 7: Integrate the "father" relationship and "brother" relationship into "father" relationship; Traverse all subject regions, if the subject region center point coordinate is less than the boundary threshold G, that is, the region is on the left side of the picture, and the "father" region cannot be found, it is regarded as the root node of a branch; if the subject region has a "father" attribute, no operation is performed; if the subject region does not have a "father" attribute but has a "brother" attribute, recursively find whether the "brother" region has a "father" attribute; if the "father" attribute exists, the "father" attribute of the subject region is changed to the "father" attribute of the region; if the "father" attribute does not exist but the "brother" attribute exists, further recursively find the "father" attribute of the "brother"; if the subject region does not have a "father" attribute and does not have a "brother" attribute, it is indicated that the region is a tree root region of a branch.

7. The target detection based string diagram pedigree digitization method of claim 6, wherein, The implementation of "detecting the father region" in step 6 specifically includes the following sub-steps: (1) Traverse the name region; (2) Obtain the geometric center horizontal and vertical coordinates of the subject region and the geometric center horizontal and vertical coordinates of the object region; (3) Place the object region with a geometric center horizontal coordinate less than the geometric center horizontal coordinate of the subject region and a difference from the geometric center vertical coordinate of the subject region within a threshold value E in container E; (4) Traverse the regions in container F, and the region with the largest horizontal coordinate is the father region of the subject region. The implementation of "detecting the brother region" specifically includes the following sub-steps: (1) Traverse the name region; (2) Obtain the geometric center horizontal and vertical coordinates of the subject region and the geometric center horizontal and vertical coordinates of the object region; (3) Place the object region with a geometric center vertical coordinate less than the geometric center vertical coordinate of the subject region and a difference from the geometric center horizontal coordinate of the subject region within a threshold value G in container H; (4) Traverse the regions in container H, and the region with the largest vertical coordinate is the brother region of the subject region.

8. The target detection based string diagram pedigree digitization method of claim 6, wherein, The specific implementation of step 7 includes the following sub-steps: (1) Traverse the name region to detect the father region and the brother region of the subject region; (2) If the subject region has a father region, no operation is performed; if the subject region does not have a father region, it is determined whether the subject region has a brother region; if the subject region does not have a brother region, it is determined that the subject region is a root node; if the subject region has a brother region, recursively find the father region of the brother region, and regard the father region of the brother region as the father region of the subject region.

9. A target detection based string diagram family digitization system, comprising: The following modules are included: Module 1 is used for preprocessing a genealogy image, including grayscale processing, binaryzation processing of the grayscale image, obtaining a hanging line graph horizontal projection graph based on the binaryzation image; Based on the horizontal projection graph features, removing the title part, and then splicing the two side images; Based on the horizontal projection graph of the spliced image, removing the generation identifier; Module 2 is used for obtaining the approximate position of a text region based on a maximum stable extreme region algorithm MSER for the genealogy image after preprocessing; Decomposing the text region coordinates and text region side length contained in the MSER object, obtaining the coordinates of the upper left corner and the lower right corner of the text region according to the upper left corner coordinates and the text side length of the text region; based on the coordinates obtained by the MSER object, removing the regions with an overlap rate higher than a threshold value A; Module 3 is used for fusing radicals and components based on geometric positions; Fusing the radicals and the components based on the geometric positions; The text area obtained in the module 2 is taken as a candidate area, and the candidate area is traversed to obtain the geometric center of each candidate area. If the distance between the geometric centers of two candidate areas is less than a threshold value B, the two areas are fused according to the maximum range. The module 4 is used for removing the hanging line area and the non-text area. According to the pixel area size of the text in the image, a threshold value C is set, and all candidate areas are traversed. If the area of a candidate area is less than the threshold value C, the candidate area is determined as a non-text area. According to the fact that the hanging line and the Chinese character have different geometric features, a length-width ratio threshold value D of the candidate area is set. If the length-width ratio of a detected area is greater than the threshold value D, the detected area is determined as a hanging line area. The coordinates of the non-text area and the hanging line area are set to 0 and are collectively removed. The module 5 is used for fusing the text area into a name area. The candidate areas are traversed to obtain the geometric centers of the candidate areas. If the geometric centers of two candidate areas are less than a threshold value E, the two texts are considered to belong to one name, and the two areas are fused according to the maximum range to obtain an accurate name area.

10. A target detection based string diagram family digitalization apparatus, comprising: Comprise: One or more processors; A storage device is configured to store one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the target detection-based hanging line graph family spectrum digitization method according to any one of claims 1 to 8.

Images