Character processing and character recognition method, storage medium and terminal device

Abstract

A character processing and character recognition method, storage medium and terminal device. The character processing method comprises: identifying and obtaining the coordinates of characters in an image; using a kernel density function to perform clustering calculation on the difference between the first coordinate values ​​of two adjacent characters to determine the characters belonging to the same row.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, word processing technology, and in particular to a character processing and character recognition method, a storage medium, and a terminal device. Background Art

[0002] After patients undergo physical examinations or tests at physical examination institutions or hospitals, paper test reports are not easy to preserve. Moreover, when users go to another hospital for examination, the current hospital cannot make a good assessment of the patient's condition due to a series of problems such as the inability to structure the data on paper test reports, the inability to use data between physical examination institutions and hospitals, or between hospitals, etc. It often happens that patients have to go to another hospital for reexamination, wasting a lot of time, money, and manpower. Therefore, a method is needed to structure the data in the patient's paper physical examination report or test report, and to integrate the fragmented test report information, which is of great significance for the establishment of electronic medical records for patients and the interconnection of data between hospitals.

[0003] Forms are an important part of test reports and physical examination reports. When converting paper forms into structured electronic information, the recognition of characters in the forms is a problem that needs to be solved. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] In a first aspect, an embodiment of the present disclosure provides a character processing method, comprising: identifying and obtaining the coordinates of characters in an image; and using a kernel density function to perform clustering calculations on the differences in the first coordinate values ​​of two adjacent characters to determine characters belonging to the same row.

[0006] In a second aspect, an embodiment of the present disclosure provides a character recognition method, comprising: preprocessing an image; identifying the coordinates of characters in the image; and using the aforementioned character processing method to perform recognition processing on the characters in the image to determine characters belonging to the same row.

[0007] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to implement the above method.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above method when executing the program.

[0009] Other aspects will be apparent upon reading and understanding the drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of a text processing method provided by an embodiment of the present disclosure;

[0011] Figure 2 for Figure 1 A flowchart of a method for implementing step 120 in the steps shown;

[0012] Figure 3 for Figure 1 A flowchart of another implementation method of step 120 in the steps shown;

[0013] Figure 4 A flowchart of a text recognition method provided by an embodiment of the present disclosure;

[0014] Figure 5 A flowchart of another text recognition method provided by an embodiment of the present disclosure;

[0015] Figure 6 is a general flow chart of an algorithm of an exemplary embodiment of the present disclosure;

[0016] Figure 7 is a flowchart of picture preprocessing in an exemplary embodiment of the present disclosure;

[0017] Figure 8 An example of a table image with a tilted state according to an exemplary embodiment of the present disclosure is provided;

[0018] Fig. 9 is an image after image preprocessing according to an embodiment of the present disclosure;

[0019] Fig.10 A flowchart for reconstructing a table of exemplary embodiments of the present disclosure;

[0020] Fig.11 A density curve graph obtained by fitting a discrete sequence using kernel density estimation in an exemplary embodiment of the present disclosure;

[0021] Fig.12 This is a diagram showing table recognition results of an exemplary embodiment of the present disclosure;

[0022] Fig.13a and 13b The effect display diagram of the application program to which the method of the embodiment of the present disclosure is applied;

[0023] Fig.14 A schematic diagram of the structure of a terminal device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The present disclosure describes multiple embodiments, but the description is exemplary rather than restrictive, and it is apparent to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0025] The present disclosure includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features, and elements disclosed in the present disclosure may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present disclosure may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other limitations. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0026] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be appreciated by those of ordinary skill in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art can easily understand that these orders can be changed and still remain within the spirit and scope of the disclosed embodiments.

[0027] Unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0028] One technology for converting paper forms into structured electronic information is to take a photo of the form data and then use optical character recognition (OCR) technology to recognize the form. One method of OCR table recognition is to identify the frame lines in the table, divide the text in the frame lines into text blocks, recognize the content of each text block separately, and then form the text blocks and the frame lines into a table. However, most medical physical examination test forms are frameless forms, which makes it impossible to distinguish each text block in the table. In addition, when affected by the user's shooting angle or shooting technology, the photo may be deformed to a certain extent. Although the perspective transformation in the open source computer vision library (Open Source Computer Vision Library, referred to as opencv) can be used to correct the image to a certain extent, the text originally on the same horizontal line will be offset to a certain extent after the perspective transformation processing, which will have a huge impact on the OCR table recognition results. The text that should be in the same line may be recognized as multiple lines, affecting the recognition accuracy and precision.

[0029] To this end, the disclosed embodiment provides a character processing and character recognition method, which can correct the error of recognizing text that belongs to one row as multiple rows due to image deformation. It can be applicable to images including characters, or images including frames without lines, or images including frames.

[0030] The following embodiments provided by the present disclosure may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0031] Figure 1 A flowchart of a character processing method provided by an embodiment of the present disclosure, as shown in the figure, includes the following steps 110 to 120.

[0032] Step 110, identifying the coordinates of the characters in the image;

[0033] For example, an OCR text recognition method may be used to recognize the coordinates of characters in a table image to obtain the coordinate positions of the characters in the table image.

[0034] Step 120 , using a kernel density function to perform clustering calculation on the difference between the first coordinate values ​​of two adjacent characters, to determine the characters belonging to the same row.

[0035] The same row includes: the same line or the same column, which is determined according to the arrangement of the characters. When the characters are arranged horizontally (usually written horizontally), the "row" includes the row, that is, the characters belonging to the same line are determined; when the characters are arranged vertically (usually written vertically), the "row" includes the column, that is, the characters belonging to the same column are determined.

[0036] In an exemplary embodiment, the row includes rows, and the step 120 uses a kernel density function to perform clustering calculations on the difference in first coordinate values ​​of two adjacent characters to determine the characters belonging to the same row, including: using a kernel density function to perform clustering calculations on the difference in first vertical coordinates of two horizontally adjacent characters to determine a first horizontal spacing, and determining the characters belonging to the same row based on the first horizontal spacing.

[0037] In this example, the two horizontally adjacent characters include characters that are adjacent in position, such as characters that are adjacent in the x-axis direction, and characters that are adjacent in content. In the horizontal writing mode, the last character in a line and the first character in the next line can be regarded as characters that are adjacent in content, that is, characters that are adjacent in the horizontal direction. Taking horizontal writing and the writing order of writing text from left to right and writing lines from top to bottom as an example, when calculating, after identifying the coordinates of all the characters in the image, the coordinates of one or more preset points in all the characters can be summarized in a list in the order of characters from left to right and lines from top to bottom. Each character in the list is an item, and the two adjacent items in the list represent two adjacent characters.

[0038] Taking the first character and the second character adjacent to each other horizontally as an example, the first ordinate refers to the ordinate of one or more preset points. The difference between the first ordinates of the two characters adjacent to each other horizontally may be the difference between the first ordinates of one or more points in the first character and the first ordinates of one or more points in the second character, including one of the following situations:

[0039] Case 1: the difference between the ordinates of a point in the first character and the corresponding point in the second character, for example, the difference between the ordinates of the topmost point of the first character and the topmost point of the second character, or the difference between the ordinates of the bottommost point of the first character and the bottommost point of the second character, or the difference between the ordinates of the center point of the first character and the center point of the second character; the topmost point, the bottommost point and the center point are only for illustration, and in other embodiments, other points may be selected for calculation, as long as the standard for selecting the points is unified;

[0040] Case 2: The difference between the mean ordinate of multiple points in the first character and the mean ordinate of multiple points in the second character, for example, the mean ordinate of multiple upper endpoints in the first character and the mean ordinate of multiple upper endpoints in the second character, or the mean ordinate of multiple lower endpoints in the first character and the mean ordinate of multiple lower endpoints in the second character; the above-mentioned multiple upper endpoints and multiple lower endpoints are only for illustration. In other embodiments, they can be selected as needed as long as the selection criteria are unified.

[0041] For any two laterally adjacent first and second characters, calculate the difference between the first longitudinal coordinate of one or more points in the first character and the first longitudinal coordinate of the corresponding one or more points in the second character; use the kernel density function to perform cluster calculation on all the calculated differences, determine the first horizontal spacing according to the calculation result of the kernel density function (minimum value), judge the difference between the first longitudinal coordinates of the two laterally adjacent characters according to the first horizontal spacing, and determine the characters belonging to the same character row.

[0042] The kernel density function is a method for estimating the basic distribution of data. By setting a kernel function (such as a Gaussian kernel) at each sample point in the data set and adding all the kernel functions, the kernel density estimate of the data set is obtained. The density function obtained will be different if the kernel function bandwidth parameters used are different. Mean shift (a gradient ascent algorithm) is used to move the data points in the direction of the fastest density increase and finally converge at the local maximum point to form a cluster. Points that converge to the same maximum are members of the same cluster. By setting appropriate bandwidth parameters, one or more clusters that meet the image conditions can be obtained. Each cluster represents a type of spacing, and the boundary of the cluster (the minimum value of the kernel density function) is also the boundary of the spacing of this type. Therefore, the minimum value of the kernel density function can be found by calculation to determine the corresponding spacing.

[0043] All calculated differences are clustered, and the calculation results represent one or more types of spacing that may appear in the graph. A minimum value represents the maximum value of a horizontal spacing. In this embodiment, the first horizontal spacing represents the maximum value of the character row offset, that is, the deviation of two adjacent characters in the same row in the x-axis direction in the y-axis direction.

[0044] In another exemplary embodiment, the row includes columns, and the step 120 uses a kernel density function to perform clustering calculations on the difference in first coordinate values ​​of two adjacent characters to determine characters belonging to the same row, including: using a kernel density function to perform clustering calculations on the difference in first horizontal coordinates of two vertically adjacent characters to determine a first vertical spacing, and determining characters belonging to the same column based on the first vertical spacing.

[0045] In this example, the two vertically adjacent characters include characters that are adjacent in position, such as characters that are adjacent in the y-axis direction, and characters that are adjacent in content. In the vertical writing mode, the last character in a column and the first character in the next column can be regarded as characters that are adjacent in content, that is, they are vertically adjacent characters. Taking vertical writing and the writing order of writing text from top to bottom and writing columns from right to left as an example, when calculating, after identifying the coordinates of all the characters in the image, the coordinates of one or more preset points in all the characters can be summarized in a list in the order of characters from top to bottom and columns from right to left. Each character in the list is an item, and the two adjacent items in the list represent two adjacent characters.

[0046] Taking the third and fourth characters adjacent to each other in the vertical direction as an example, the first horizontal coordinate refers to the horizontal coordinate of one or more preset points. The difference between the first horizontal coordinates of the two characters adjacent to each other in the vertical direction may be the difference between the first horizontal coordinates of one or more points in the third character and the first horizontal coordinates of one or more corresponding points in the fourth character, including one of the following situations:

[0047] Case 3: the difference between the horizontal coordinates of a point in the third character and the corresponding point in the fourth character, for example, the difference between the horizontal coordinates of the leftmost point of the third character and the leftmost point of the fourth character, or the difference between the horizontal coordinates of the rightmost point of the third character and the rightmost point of the fourth character, or the difference between the horizontal coordinates of the center point of the third character and the center point of the fourth character; the leftmost point, the rightmost point and the center point are only for illustration, and in other embodiments, other points may be selected for calculation, as long as the standard for selecting the points is unified;

[0048] Case 4: The difference between the mean horizontal coordinate of multiple points in the third character and the mean horizontal coordinate of multiple points in the fourth character, for example, the mean horizontal coordinate of multiple left endpoints in the third character and the mean horizontal coordinate of multiple left endpoints in the fourth character, or the mean horizontal coordinate of multiple right endpoints in the third character and the mean horizontal coordinate of multiple right endpoints in the fourth character; the above-mentioned multiple left endpoints and multiple right endpoints are only for illustration. In other embodiments, they can be selected as needed as long as the selection criteria are unified.

[0049] For any two vertically adjacent third characters and fourth characters, calculate the difference between the first horizontal coordinates of one or more points in the third character and the first horizontal coordinates of the corresponding one or more points in the fourth character; use the kernel density function to perform clustering calculation on all the calculated differences, determine the first vertical spacing based on the calculation result of the kernel density function (minimum value), judge the difference between the first horizontal coordinates of the two vertically adjacent characters based on the first vertical spacing, and determine the characters belonging to the same character column.

[0050] As mentioned above, the kernel density function performs cluster calculation on all calculated differences, and the calculation result represents one or more types of spacing that may appear in the graph. A minimum value represents the maximum value of a longitudinal spacing. In this embodiment, the first longitudinal spacing represents the maximum value of the character column offset, that is, the deviation of two adjacent characters in the same column in the y-axis direction in the x-axis direction.

[0051] The offset of the character row in the image can be determined by performing a clustering calculation on the difference in the first coordinate values ​​of two adjacent characters. For example, one or more minima can be obtained by using a kernel density function, where the smallest first minimum value represents the maximum value of the character row offset, which can be used to determine the characters belonging to the same character row or character column. When the difference in the first coordinate values ​​of two adjacent characters is less than or equal to the first minimum value, it can be determined that the two adjacent characters belong to the same character row or character column.

[0052] The implementation method of the present disclosure can be used to correct characters that have been shifted due to tilted shooting angles, etc. These shifts can be classified into their correct categories through a kernel density algorithm with clustering properties, that is, the shifts can be removed and characters that originally belonged to the same row or column can be recognized as being in the same row or column.

[0053] As mentioned above, clustering the coordinate differences using a kernel density function may result in a minimum value, which can be used to correct the offset of the character row. Clustering the coordinate differences using a kernel density function may also result in multiple minima, wherein the smallest minimum value (first minimum value) is used to correct the offset of the character row, and the other one or more minima (second minimum value) reflect the existence of several types of spacing in the image (for horizontal writing, the spacing includes row spacing, and for vertical writing, the spacing includes column spacing), and a second minimum value corresponds to the maximum value of a type of spacing. In other embodiments, one or more minima obtained by clustering the coordinate differences using a kernel density function may all represent spacing, and one minimum value corresponds to the maximum value of a type of spacing.

[0054] In an exemplary embodiment, taking a row as an example, the character processing method further includes a step of determining a line spacing and a step of determining a character column group. Figure 2As shown, the above step 120 may include the following steps 121 to 124.

[0055] Step 121, for any two laterally adjacent first characters and second characters, calculate the difference between the first ordinate of one or more points in the first character and the first ordinate of one or more points in the second character;

[0056] The implementation method is the same as described in the above embodiment and will not be repeated here.

[0057] Step 122, using a kernel density function to perform clustering calculation on all calculated differences, determining a first horizontal spacing according to the calculation result of the kernel density function (e.g., a first minimum value), judging a difference between first ordinates of two horizontally adjacent characters according to the first horizontal spacing, and determining characters belonging to the same character row;

[0058] Step 123, determining a second horizontal spacing according to the calculation result of the kernel density function (such as the second minimum value), judging the difference between the first ordinates of two characters adjacent to each other in the horizontal direction according to the second horizontal spacing, and determining the line spacing;

[0059] The above steps 122 and 123 may be performed together.

[0060] A kernel density function is used to perform clustering calculations on all the calculated differences. Since horizontally adjacent characters include not only positionally adjacent characters but also content-adjacent characters, clustering the coordinate differences of horizontally adjacent characters using a kernel density function can obtain the maximum offset of a character row and the maximum values ​​of one or more line spacings (i.e., the second horizontal spacing). The kernel density function calculation result includes one or more minima, such as a minimum first minimum and a second minimum greater than the first minimum, wherein the first minimum represents the maximum value of the smallest type of spacing (character row offset) that exists, the first minimum represents the maximum offset of two adjacent characters in the x-axis direction in the y-axis direction, the first minimum can be used to correct tilted character rows, and there can be one or more second minima, which can represent one or more line spacings, i.e., the spacing between two adjacent rows.

[0061] When it is determined that the difference between the first ordinates of the two horizontally adjacent characters is less than or equal to the first minimum value, it is determined that the two horizontally adjacent characters belong to the same character row. When it is determined that the difference between the first ordinates of the two horizontally adjacent characters is greater than the first minimum value and less than the second minimum value, it is determined that the two horizontally adjacent characters belong to two character rows respectively, that is, there is a line spacing between the two characters. Through the above steps, one or more line spacings existing in the image can be determined. When there are multiple second minimum values, for example, including the first minimum value T1 and the second minimum values ​​T2 and T3, when the coordinate difference between the two horizontally adjacent characters S1 and S2 is greater than T1 and less than T2, it means that the two characters S1 and S2 are located in two rows, character S1 is the last character in one row, and character S2 is the first character in the next row, and there is a first line spacing between the two characters. When the coordinate difference between the two horizontally adjacent characters S3 and S4 is greater than T2 and less than T3, it means that the two characters S3 and S4 are located in two rows, there is a second line spacing between the two characters, and the first line spacing is less than the second line spacing. If there is a table in the figure (not limited to whether it has a frame), there may be a table line at the position with the second line spacing (if it is a frameless table, it is an implicit table line).

[0062] The case where the coordinate difference is equal to the minimum value can be classified into the case corresponding to the smaller minimum value.

[0063] Step 124, using a kernel density function to perform clustering calculation on the difference in the second coordinate values ​​of two horizontally adjacent characters, determine a third horizontal spacing, judge the difference in the second coordinate values ​​of two horizontally adjacent characters based on the third horizontal spacing, judge whether the two horizontally adjacent characters belong to the same character column group, and finally determine the characters that belong to the same character column group.

[0064] In this embodiment, due to the horizontal writing mode, the second coordinate value is the second horizontal coordinate, which can be the horizontal coordinate of one or more preset points. Taking the fifth and sixth characters adjacent to each other in the horizontal direction as an example, the difference between the second horizontal coordinates of the two characters adjacent to each other in the horizontal direction can be the difference between the second horizontal coordinates of one or more points in the fifth character and the second horizontal coordinates of one or more points corresponding to each other in the sixth character, including one of the following situations:

[0065] Case 5: the difference between the horizontal coordinates of a point in the fifth character and the corresponding point in the sixth character, for example, the difference between the horizontal coordinates of the leftmost point of the fifth character and the leftmost point of the sixth character, or the difference between the horizontal coordinates of the rightmost point of the fifth character and the rightmost point of the sixth character, or the difference between the horizontal coordinates of the center point of the fifth character and the center point of the sixth character; the leftmost point, the rightmost point and the center point are only for illustration, and in other embodiments, other points may be selected for calculation, as long as the standard for selecting the points is unified;

[0066] Case 6: the difference between the abscissa of a point in the fifth character and the corresponding point in the sixth character, for example, the difference between the abscissa of the rightmost point of the fifth character and the abscissa of the leftmost point of the sixth character, the abscissa of the sixth character is greater than that of the fifth character;

[0067] Case 7: The difference between the mean horizontal coordinate of multiple points in the fifth character and the mean horizontal coordinate of multiple points in the sixth character, for example, the mean horizontal coordinate of multiple left endpoints in the fifth character and the mean horizontal coordinate of multiple left endpoints in the sixth character, or the mean horizontal coordinate of multiple right endpoints in the fifth character and the mean horizontal coordinate of multiple right endpoints in the sixth character; the above-mentioned multiple left endpoints and multiple right endpoints are only for illustration. In other embodiments, they can be selected as needed as long as the selection criteria are unified.

[0068] Case 8: The difference between the abscissas of multiple points in the fifth character and the corresponding multiple points in the sixth character, for example, the difference between the average abscissas of multiple right endpoints in the fifth character and the average abscissas of multiple left endpoints in the sixth character, the abscissa of the sixth character is greater than the abscissa of the fifth character;

[0069] The above-mentioned "corresponding" endpoints refer to endpoints with the same position or the same rules for selecting positions. The above-mentioned "corresponding" endpoints refer to endpoints with mirror symmetry.

[0070] It can be known from the above difference calculation method that the third horizontal spacing represents the coordinate difference between two adjacent characters in the x-axis direction, where "adjacent" includes the situation where the aforementioned contents are adjacent.

[0071] The difference values of all the calculated second coordinate values are clustered using a kernel density function. The calculation result of the kernel density function includes one or more minimum values, and each minimum value represents a type of third horizontal spacing in the figure. For example, it includes the smallest third minimum value and a fourth minimum value greater than the third minimum value. The smallest third minimum value represents the spacing between normal characters. For example, the distance between the characters "字" and "符" in the two characters "字符". There may be one or more fourth minimum values greater than the third minimum value, which can represent various spacings. For example, in the same row, the spacing between the last character in the first list table and the first character in the second list table is one type of spacing, and the spacing between the last character in the second list table and the first character in the third list table may be another type of spacing. Since the writing method in this embodiment is horizontal writing, the part of the difference value less than the smallest third minimum value can be not considered, and only the difference value between the third minimum value and the fourth minimum value is considered. When it is determined that the difference value of the second coordinate values of two horizontally adjacent characters is greater than the third minimum value and less than the fourth minimum value, or when it is determined that the difference value of the second coordinate values of two horizontally adjacent characters is between two fourth minimum values, it is determined that the two horizontally adjacent characters belong to two character column groups respectively. After judging the difference values of the second coordinates of all adjacent characters in the figure using the above method, one or more character column groups existing in the image can be determined, and finally the characters belonging to the same character column group can be determined. If there is a table in the figure (regardless of whether there is a frame line), there may be a table line between the two characters, and one table column is a character column group.

[0072] For example, the judgment can be made for each row of character column groups. After making the character column group judgment for all the characters in the first character row, multiple character column groups can be obtained. Then, make the character column group judgment for all the characters in the second character row, and at the same time compare with the abscissas of the characters in the previously obtained character column groups to determine which character column group to divide them into. If the character column group is a table column, and combined with the judgment of the line spacing, the table in the figure can be completely recognized.

[0073] In an exemplary embodiment, taking a row including columns as an example, the character processing method further includes the steps of determining the column spacing and determining the character row group. As Figure 3 shown, the above step 120 may include steps 121' to 124'.

[0074] Step 121', for any two vertically adjacent third character and fourth character, calculate the difference value between the first abscissa of one or more points in the third character and the first abscissa of the corresponding one or more points in the fourth character;

[0075] As described in the foregoing embodiment, it will not be elaborated here.

[0076] Step 122', using a kernel density function to perform clustering calculation on all calculated differences, determining a first longitudinal spacing according to the calculation result of the kernel density function (e.g., the fifth minimum), judging the difference between the first horizontal coordinates of two vertically adjacent characters according to the first longitudinal spacing, and determining characters belonging to the same character column;

[0077] Step 123', determining a second longitudinal spacing according to the calculation result of the kernel density function (for example, the sixth minimum value), judging the difference between the first horizontal coordinates of two characters adjacent to each other in the longitudinal direction according to the second longitudinal spacing, and determining the column spacing;

[0078] The above steps 122' and 123' may be combined and performed.

[0079] A kernel density function is used to perform clustering calculation on all the calculated differences. Since vertically adjacent characters include not only positionally adjacent characters but also content-adjacent characters, clustering the coordinate differences of vertically adjacent characters using a kernel density function can obtain the maximum offset of the character column and the maximum value of one or more column spacings (second longitudinal spacings). The kernel density function calculation result includes one or more minima, for example, the smallest fifth minimum and a sixth minimum greater than the fifth minimum, wherein the fifth minimum represents the maximum value of the smallest type of spacing (character column offset) that exists, and the fifth minimum represents the maximum offset of two characters adjacent in the y-axis direction in the x-axis direction, and the fifth minimum can be used to correct the tilted character column. There can be one or more second minima, which can represent one or more column spacings, that is, the spacing between two adjacent columns.

[0080] When it is determined that the difference between the first horizontal coordinates of the two characters adjacent in the longitudinal direction is less than or equal to the fifth minimum value, it is determined that the two characters adjacent in the longitudinal direction belong to the same character column. When it is determined that the difference between the first horizontal coordinates of the two characters adjacent in the longitudinal direction is greater than the fifth minimum value and less than the sixth minimum value, it is determined that the two characters adjacent in the longitudinal direction belong to two character columns respectively, that is, there is a column spacing between the two characters. Through the above steps, one or more column spacings existing in the image can be determined. When there are multiple sixth minimum values, for example, including the fifth minimum value T4 and the sixth minimum values ​​T5 and T6, when the coordinate difference between the two characters S5 and S6 adjacent in the longitudinal direction is greater than T4 and less than T5, it means that the two characters S5 and S6 are located in two columns, character S5 is the last character in one column, character S6 is the first character in the next column, and there is a first column spacing between the two characters. When the coordinate difference between the two characters S7 and S8 adjacent in the longitudinal direction is greater than T5 and less than T6, it means that the two characters S7 and S8 are located in two columns, there is a second column spacing between the two characters, and the first column spacing is less than the second column spacing. If there is a table in the figure (not limited to whether it has a frame), there may be vertical table lines at the position with the second column spacing (if it is a frameless table, it is an implicit table line).

[0081] Step 124', uses a kernel density function to perform clustering calculation on the difference in the second coordinate values ​​of two vertically adjacent characters, determines a third vertical spacing, and judges the difference in the second coordinate values ​​of two vertically adjacent characters based on the third vertical spacing to determine whether the two vertically adjacent characters belong to the same character row group, and finally determines the characters that belong to the same character row group.

[0082] In this embodiment, due to the vertical writing mode, the second coordinate value is the second ordinate, which can be the ordinate of one or more preset points. Taking the seventh and eighth characters adjacent to each other in the vertical direction as an example, the difference between the second ordinates of the two characters adjacent to each other in the vertical direction can be the difference between the second ordinates of one or more points in the seventh character and the second ordinates of one or more points corresponding to each other in the eighth character, including one of the following situations:

[0083] Case 9: the difference between the ordinates of a point in the seventh character and the corresponding point in the eighth character, for example, the difference between the ordinates of the topmost point of the seventh character and the topmost point of the eighth character, or the difference between the ordinates of the bottommost point of the seventh character and the bottommost point of the eighth character, or the difference between the ordinates of the center point of the seventh character and the center point of the eighth character; the topmost point, the bottommost point and the center point are only for illustration, and in other embodiments, other points may be selected for calculation, as long as the standard for selecting the points is unified;

[0084] Case 10: the difference between the ordinates of a point in the seventh character and the corresponding point in the eighth character, for example, the difference between the ordinate of the bottom point of the seventh character and the ordinate of the top point of the eighth character, the ordinate of the eighth character is greater than the ordinate of the seventh character;

[0085] Case 11: The difference between the mean ordinate of multiple points in the seventh character and the mean ordinate of multiple points in the eighth character, for example, the mean ordinate of multiple upper endpoints in the seventh character and the mean ordinate of multiple upper endpoints in the eighth character, or the mean ordinate of multiple lower endpoints in the seventh character and the mean ordinate of multiple lower endpoints in the eighth character; the above-mentioned multiple upper endpoints and multiple lower endpoints are only for illustration. In other embodiments, they can be selected as needed as long as the selection criteria are unified.

[0086] Case 12: the difference between the ordinates of multiple points in the seventh character and the corresponding multiple points in the eighth character, for example, the difference between the average ordinates of multiple lower endpoints in the seventh character and the average ordinates of multiple upper endpoints in the eighth character, the ordinate of the eighth character is greater than the ordinate of the seventh character;

[0087] The above-mentioned "corresponding" endpoints refer to endpoints with the same position or the same rules for selecting positions. The above-mentioned "corresponding" endpoints refer to endpoints with mirror symmetry.

[0088] It can be known from the above difference calculation method that the third longitudinal spacing represents the coordinate difference between two adjacent characters in the y-axis direction, where "adjacent" includes the situation where the aforementioned contents are adjacent.

[0089] The step of determining whether the characters belong to the same character row group can be performed with reference to the step of determining whether the characters belong to the same character column group in step 124, which will not be described in detail here. The character row group is, for example, a table row.

[0090] Figure 4 A flowchart of a character recognition method provided in an embodiment of the present disclosure includes steps 310 to 330 as shown in the figure.

[0091] Step 310, preprocessing the image;

[0092] The preprocessing includes one or more of the following processes: color image to grayscale image conversion, Gaussian filtering, background extraction, contrast compensation, binarization and perspective transformation.

[0093] Step 320, identifying the coordinates of the characters in the image;

[0094] Step 330, using the character processing method in the above embodiment (eg Figure 1 , Figure 2 or Figure 3 )Recognize the characters in the image and determine the characters belonging to the same row.

[0095] In an exemplary embodiment, if Figure 5 As shown, after step 330, the character recognition method further includes step 340, displaying part or all of one or more rows of characters determined after the recognition process.

[0096] In step 330, after the characters in the image are recognized, the following recognition results can be determined: which characters belong to the same row, and thus the number of rows in the image can be determined. Taking the example that a row includes rows, the characters belonging to the same row and how many rows there are in the image can be determined. Taking the example that a row includes columns, the characters belonging to the same column and how many columns there are in the image can be determined. When the image includes a table, in addition to the above content, the table rows and / or table columns can also be determined. After obtaining the above recognition results, part or all of the recognition results can be selected to be displayed on the interface of the terminal. Taking the example that the image includes a table, it can be one or more of the following two situations:

[0097] Select part or all of the content in one or more table rows (part or all of all characters in the table rows) for display;

[0098] Select part or all of the content in one or more specified table columns (part or all of all characters in the table columns) for display.

[0099] When selecting, you can judge based on the character content and select the rows or columns to be displayed. The display style can be pre-set as needed.

[0100] By adopting the method provided in the embodiment of the present disclosure, the noise in the image is removed by preprocessing the image, and the obtained table recognition result is more accurate.

[0101] The character processing and recognition method provided by the present disclosure has fast recognition speed and high recognition accuracy. The offset distance of the result after character recognition and the distance between characters in the character block are estimated by the kernel density estimation function, and the line reconstruction and character block reconstruction are performed. The method has high robustness and anti-interference ability, and high recognition accuracy.

[0102] The method of the disclosed embodiment can be used not only for text recognition and frameless table recognition, but also for framed table recognition. Because the disclosed embodiment recognizes and analyzes the row spacing and column spacing, the presence or absence of a frame does not affect the recognition result. The method of the disclosed embodiment can process table photos where the text position is offset due to tilted shooting angles, uneven shooting lighting, etc., and the kernel density algorithm with clustering properties can classify these offsets into their correct classes (rows, columns).

[0103] The method of the embodiment of the present disclosure is described below in conjunction with an exemplary embodiment. In this exemplary embodiment, a test form without frame lines is used as an example for description. Figure 6 4 is an algorithm flow chart of the table recognition method of this exemplary embodiment, which includes the following steps 410 to 430.

[0104] Step 410, image preprocessing;

[0105] The image preprocessing process may include one or more of the following steps as needed: color image to grayscale image, filtering (such as Gaussian filtering), background extraction, contrast compensation, binarization and perspective transformation. The following is an example of including all the above processes. When all the processes are included, the accuracy of subsequent OCR text recognition of the image can be guaranteed under extreme conditions (such as tilted shooting angle, uneven lighting, etc.). Figure 7 As shown, the following steps 411 to 416 are included.

[0106] Step 411: converting the input color image to be recognized into a grayscale image;

[0107] Converting a color image to a grayscale image can be achieved using opencv.

[0108] Step 412, using Gaussian filtering to remove noise in the grayscale image;

[0109] Gaussian filtering to remove noise can be implemented using opencv.

[0110] Step 413, performing background extraction on the grayscale image after noise removal;

[0111] Because the background of photos taken in poor light is relatively dark, and the characters in the image are usually black, the background is equivalent to interference, making the characters unclear. Therefore, background extraction can be used to estimate the grayscale of the background and separate the main body of the character from the background.

[0112] The background of a certain point in the image can be estimated by the set of brighter points in the w*w neighborhood of the point, that is, some whiter points in a region can be used to represent the background of the region. w is an empirical value, and the value range of w is greater than 0 and less than the number of pixels of the side length. For example, it can be about one tenth of a certain side length (such as the shortest side length).

[0113] Take w as 31 as an example. First, compress the image proportionally to 300 pixels on the shortest side, then scan the image row by row or column by column, select the 7 points with the highest brightness in the 31*31 neighborhood of each pixel, remove the brightest pixel to reduce the influence of white noise points, and then average the remaining 6 points as the background value of the point. After calculating the background values ​​of all pixels, enlarge the background image to the original size. The image can be reduced or enlarged by interpolation.

[0114] The processing speed can be increased by reducing the size of the image before processing. If the pixels of an image are several thousand by several thousand, the processing speed will be greatly increased after proportional compression without affecting the background extraction results.

[0115] The values ​​of the neighborhood range and the number of brightness samples are empirical values ​​determined according to the image size, and other parameters can be tried according to actual conditions.

[0116] Step 414, compensating the contrast of the grayscale image after noise removal according to the background extraction result;

[0117] The uneven lighting background can be removed by contrast compensation, which can be calculated using the following formula:

[0118]

[0119] Where y is the gray value of any pixel after compensation; p s is the grayscale value of the original image at the same pixel position obtained in step 102, pb is the grayscale value of the background image at the same pixel position extracted in step 103.

[0120] The above contrast compensation method is a contrast compensation implementation method in the open source tool ScanTailor, which has a small amount of calculation and a fast speed. In other embodiments, other contrast compensation methods can also be selected, and the present disclosure does not limit this.

[0121] Step 415, binarization processing is performed on the contrast-compensated image;

[0122] The local threshold Sauvola algorithm (an image binarization method that considers local mean brightness) can be used for binarization. The input of the Sauvola algorithm is a grayscale image. It takes the current pixel as the center and dynamically calculates the threshold of the pixel based on the grayscale mean and standard deviation in the neighborhood of the current pixel. It can be implemented using scikit-image. scikit-image (skimage for short) is a collection of algorithms for image processing and computer vision, including the Sauvola algorithm.

[0123] In addition to using the Sauvola algorithm, you can also use the adaptive binarization algorithm provided by opencv.

[0124] Step 416, performing perspective transformation on the binarized image.

[0125] You can use the Canny algorithm for edge detection, and then use the opencv built-in contour detection findContours function to find the largest contour with four corner points, and perform perspective transformation on the image circled by these four corner points. Perspective transformation is used to solve the problem that affine transformation cannot change the relative position relationship inside the shape. Similar to the "free transform" function in Photoshop, or the "perspective" function in the GNU (an operating system) image processing program (GNU Image Manipulation Program, referred to as GIMP), both can be implemented using the perspective transformation matrix. Perspective transformation uses the four-vertex perspective transformation method. By finding the perspective transformation matrix, a tilted quadrilateral can be converted into a rectangle.

[0126] Several implementation methods of the steps are given above, and in other embodiments, other methods may be used to implement the steps.

[0127] Step 420, performing general OCR text recognition on the pre-processed image;

[0128] Compared with OCR table recognition, general OCR text recognition technology is more mature. It only focuses on text recognition. The result of recognition is each word and its top, bottom, left, and right coordinates. There are many ways to implement general text OCR recognition. This article uses Google's open source Tesseract-OCR implementation, and other general text recognition services can also be called to implement it.

[0129] Figure 8 An example of a medical examination report taken at an oblique angle for the user. Fig. 9 This is an image after the above image preprocessing and general text OCR recognition.

[0130] Step 430: reconstruct the table of the content recognized by OCR.

[0131] The table reconstruction process of the disclosed embodiment uses the Gaussian kernel density estimation method to reconstruct the results of the general OCR text recognition process to form a table. The reconstruction process includes text row reconstruction, table row reconstruction and table column reconstruction. Through table row reconstruction and table column reconstruction, the cells in the table can be determined, and then the text blocks inside can be reconstructed. The reconstruction process does not depend on the frame lines inside and outside the table, and has a strong anti-interference ability for tilt. Fig.10 Reconstruct the table process. Use the kernel density estimation algorithm to reconstruct the table to achieve OCR table recognition, which improves the previous shortcomings of table recognition relying on table frame lines and low accuracy.

[0132] Kernel density estimation is a method used in probability theory to estimate unknown density functions. It is one of the non-parametric test methods. The kernel density estimation method does not use prior knowledge about data distribution and does not make any assumptions about data distribution. It is a method to study data distribution characteristics from the data sample itself.

[0133] In an exemplary embodiment, in addition to using the above-mentioned clustering method to determine the table boundary portion, before reconstructing the table, the table to be identified can be compared with tables in a pre-established table format library, for example, by comparing and determining through the table header and / or table footer. For example, when the table header portion is the same as the pre-saved table format, the table border is determined according to the pre-saved table format, the table range is determined, and the table is reconstructed within the determined table range.

[0134] like Fig.10 As shown, the table reconstruction process includes the following steps 801 to 806. This embodiment is described by taking the example of first determining the rows and then determining the columns. In other embodiments, the columns may be determined first and then the rows.

[0135] Step 801, after the pre-processed image is recognized by general text recognition, the top, bottom, left and right coordinates (hereinafter referred to as coordinates) of each text in the image are obtained, and the top coordinate of each text is recorded;

[0136] If the coordinate axis is established with the upper left corner of the image as the origin, the horizontal axis is the x-axis and the vertical axis is the y-axis. Each character can be regarded as a rectangle. A rectangle contains at least one character. A rectangle includes four sides: the top side (or top side), the bottom side (or bottom side), the left side and the right side. The upper coordinate of the character is the distance from the upper edge of the rectangle to the x-axis, the left coordinate of the character is the distance from the left side of the rectangle to the y-axis, the lower coordinate of the character is the distance from the lower edge of the rectangle to the x-axis, and the right coordinate of the character is the distance from the right side of the rectangle to the y-axis. For the convenience of calculation, the upper coordinates of all characters can be inserted into an upper coordinate list, which is recorded as list1;

[0137] Step 802, traverse list1, calculate the absolute value of the upper coordinate difference between two adjacent characters in list1, and insert the obtained absolute value into a new list - the upper coordinate difference list, which is recorded as list_top;

[0138] This embodiment is described by taking the calculation of the upper coordinate difference as an example, and in other embodiments, the lower coordinate difference of two adjacent characters may also be calculated. The subsequent method for finding the spacing is the same.

[0139] The purpose of calculating the upper coordinate difference of two adjacent characters is to obtain the data distribution, which includes the following possible situations: the first horizontal spacing (i.e., text line offset or offset spacing, such as text line offset caused by tilted shooting angle), the second horizontal spacing (including text line spacing, table line spacing or other line spacing, other line spacing, such as the large distance line spacing between the last examination item and the signature at the end of the table in the physical examination form or the test form, and there may be one or more other line spacings). According to the test form in this example, it has the three horizontal spacings as described above - offset spacing, table line spacing (in this example, the text line spacing is the same as the table line spacing) and other spacings.

[0140] Step 803, for the list list_top, use the Gaussian kernel density estimation fitting density curve to find the minimum point of the discrete sequence list_top, and set the initial bandwidth (bandwidth is the smoothing parameter) to 1 during the search. In this embodiment, if the number of minimum values ​​of the discrete sequence list_top is greater than 3 (the number of horizontal spacings), the preset step size (for example, 0.1) is increased on the basis of the initial bandwidth, and the Gaussian kernel density estimation fitting density curve is recalculated until the number of minimum values ​​is 3. The minimum point of the discrete sequence is the dividing point of multiple types of horizontal spacings. In other embodiments, the number of horizontal spacings may also be different depending on the table. There may be a type of horizontal spacing, namely the first horizontal spacing. When the upper coordinate difference (or lower coordinate difference) of two characters is less than the first horizontal spacing, the two characters are considered to be in the same row, so the first horizontal spacing can be used to correct the text offset caused by the shooting angle.

[0141] Fig.11 The density curve is fitted by using kernel density estimation to the coordinate difference of adjacent characters. The X-axis is the value of the input data, that is, the upper coordinate difference, and the Y-axis is the estimated logarithmic kernel density estimate under a certain upper coordinate difference. For the kernel density estimation curve of multiple data points, since waveform synthesis occurs between adjacent peaks, the final curve shape is not closely related to the selected kernel function. Considering the ease of use of the function in waveform synthesis calculation, this embodiment uses the Gaussian kernel function (normal distribution curve) as the kernel function of the kernel density estimation. Fig.11 In the kernel density estimation, the logarithm of the estimated value of the ordinate can be taken (lg10(f)) to compress the size of the estimated value. i The kernel functions of are superimposed together to form a density curve.

[0142] The kernel density estimate calculation formula is as follows:

[0143]

[0144] in: is the kernel density estimation function, n is the sample size (i.e., the total number of coordinate differences), h is the bandwidth, also known as the smoothing parameter, h>0, K() is the kernel function, and this embodiment uses the Gaussian kernel function. The Gaussian kernel function calculation formula is as follows:

[0145]

[0146] Various lateral spacings can be distinguished by kernel density estimation.

[0147] Fig.11 It is only used to illustrate the density curve waveform and the corresponding minimum point in this example. The minimum value of the kernel density function can be directly calculated by derivation during calculation.

[0148] Step 804, determining characters belonging to the same row and characters belonging to the same table row according to the found minimum value;

[0149] In this embodiment, the number of lateral spacings is 3, that is, there are 3 minimum values ​​(such as Fig.11 ), the X-axis coordinate corresponding to the first minimum value is the maximum value of the text offset spacing (hereinafter referred to as the text offset spacing), the X-axis coordinate corresponding to the second minimum value is the maximum value of the table row spacing (hereinafter referred to as the table row spacing), and the X-axis coordinate corresponding to the third minimum value is other spacing, which is the large distance row spacing in this example (for example, the spacing between the last examination item in the physical examination form or the test form and the signature at the end of the table). Text offset spacing < table row spacing < large distance row spacing. In this embodiment, when the difference in the upper coordinates between two characters is less than or equal to the text offset spacing, it is considered that the two characters are in the same text row, and when the difference in the upper coordinates between the two characters is greater than the text offset spacing and less than or equal to the table row spacing, it is considered that the two characters are in different table rows, that is, there is a table line between the two characters. When the difference in the upper coordinates between the two characters is greater than the table row spacing and less than or equal to the large distance row spacing, it is considered that there is a large distance row spacing between the two characters.

[0150] During implementation, the first minimum horizontal coordinate value x can be recorded as mi, where mi is the maximum text offset spacing. The upper coordinate difference less than mi is considered to be the same text line. Take the first character as the first line, and traverse the remaining characters in the list list1. If the absolute value of the difference between its upper coordinate and the average value of the upper coordinates of the characters on the line that have been determined is less than mi, then the character is determined as the character on the line. If it is greater than mi, it is recorded as an independent new text line. After all the texts have been traversed, the characters in each line are sorted according to the left coordinate, and the reconstruction of the text line is completed. In this example, the upper coordinate of each character is compared with the average value of the upper coordinates of the characters on the determined line to determine whether they belong to the same text line. In other exemplary embodiments, the upper coordinate difference between two adjacent characters can also be directly compared with mi to determine whether the two characters belong to the same text line.

[0151] The table row reconstruction is performed in the same way. The x value of the second minimum point can be recorded as ni, where ni is the maximum table row spacing. The upper coordinate difference less than ni is considered to be the same table row. The first character is taken as the first table row, and the remaining characters in the list list1 are traversed. If the absolute value of the difference between its upper coordinate and the average value of the upper coordinate of the characters in the determined table row is greater than mi and less than ni, it is determined that there is a table line between the two characters, that is, the two characters are in different table rows. In an exemplary embodiment, the upper coordinate difference of two adjacent characters can also be directly compared with ni to determine the position of the table line. In another exemplary embodiment, the table line can also be determined based on the determined text row, and the table line position can be determined by traversing the upper coordinate difference of each character to find a coordinate difference greater than mi and less than ni, or the average upper coordinate difference of the two text rows is determined respectively, and the absolute value of the difference between the average upper coordinate difference of the two text rows is calculated. If the absolute value is greater than mi and less than ni, it is determined that there is a table line between the two text rows. In an exemplary embodiment, the determined table line can also be verified. For example, the above-mentioned method for determining the table line can be a verification method for each other. At this point, the table row reconstruction is complete.

[0152] Step 805, for each text row, calculate the absolute value of the difference between the right coordinate of each character and the left coordinate of the character located on the right side of the character and adjacent to the character (hereinafter referred to as the left and right coordinate difference of two adjacent characters), and use all the differences as discrete sequences to find the minimum point using the Gaussian kernel density estimation fitting curve, similar to step 803. For the table in this embodiment, three extreme points (number of column spacings) will be obtained by Gaussian kernel density estimation, and the x-axis coordinates of the three extreme points represent the text column spacing, table column spacing and other spacings (large distance column spacing greater than the table column spacing, such as the spacing on both sides of the table), where the text column spacing can be regarded as the maximum column spacing between characters in the same cell (the smallest unit of a table consisting of one row and one column), the text column spacing is less than the table column spacing, and the table column spacing is less than other large distance column spacings. There may be one or more other spacings. In this embodiment, when the left and right coordinate difference of two adjacent characters is greater than the text column spacing and less than or equal to the table column spacing, it is considered that the two characters are in different table columns.

[0153] In this embodiment, the writing habit is from left to right as an example. If the writing habit is from right to left, the absolute value of the difference between the left coordinate of each character in each text line and the right coordinate of the character located on the left side of the character and adjacent to the character should be calculated. Another adjacent situation is that the last character in one line and the first character in another line are also considered to be adjacent characters.

[0154] Step 806 , reorganize the text according to the table rows determined in step 804 and the table columns determined in step 805 , and reconstruct the table.

[0155] The use of the method of the embodiment of the present disclosure does not depend on the presence or absence of frame lines in the table, because even if there are no frame lines, the row spacing and column spacing still exist in reality.

[0156] The above method can be used to identify characters that originally belonged to the same row, as well as characters that belonged to the same table row and the same table column. Fig.12 When the above method is applied to the recognition of tables such as physical examination reports, the display styles of different types of tables in the terminal interface can be preset, and the display style corresponding to the current table can be determined according to the table header and / or table footer recognition. Fig.12 After the recognition result is shown, the recognized content is displayed to the user in a preset display style. Fig.13a and Fig.13b The display effect of some contents is shown. Fig.13a and Fig.13b In the example shown, since the project name and inspection results are the contents that the user is concerned about, the display style is set to a large black font. The unit and reference range are reference contents, so the display style is set to a small gray font. The serial number and English abbreviation are not the contents that the user is concerned about and are set not to be displayed in this example. When displaying, the contents in the table can be selected row by row in the order of the table rows and filled into the display page. In another embodiment, when the table is a standard table, the standard contents, such as the project name, unit and reference range, etc., can be filled in advance when the style is preset. By comparing the contents such as the project name or serial number in the preset style, the corresponding inspection results are selected from the recognition results of the image table and filled into the corresponding positions in the display page. Fig.12 a and Fig.12 b is only a display example, and in other embodiments, it can be set to other styles as needed. The disclosed technical solution can achieve better results on medical scenarios, such as physical examination reports and test reports without frame lines.

[0157] The method described in the embodiment of the present disclosure has higher robustness and anti-interference ability, as well as higher recognition speed and accuracy compared to the traditional table recognition method. The method of the embodiment of the present disclosure still has the characteristic of good effect when the frame line in the table is missing, and is particularly suitable for the recognition of frameless tables such as medical examination reports and test sheets in medical scenarios.

[0158] In an exemplary embodiment of the present disclosure, a terminal device (referred to as a terminal) is also provided. The terminal device may include a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor may implement the character processing method or character recognition method in the embodiment of the present disclosure when executing the computer program.

[0159] like Fig.14 As shown, in one example, the terminal device 1300 may include: a processor 1310, a memory 1320, a bus system 1330 and a transceiver 1340, wherein the processor 1310, the memory 1320 and the transceiver 1340 are connected via the bus system 1330, the memory 1320 is configured to store instructions, and the processor 1310 is configured to execute the instructions stored in the memory 1320 to implement the character processing or character recognition method as described above. In an exemplary embodiment, the transceiver 1340 is configured to receive the coordinates of the characters in the image obtained by recognition and send them to the processor, and is also configured to output the character processing results of the processor. In other exemplary embodiments, the transceiver 1340 is configured to obtain the image to be processed and receive the character recognition results of the processor.

[0160] The terminal may be, for example, a mobile phone, a tablet computer or other mobile handheld device. Taking the mobile phone as an example, the transceiver 1340 may include a camera, which may obtain the image to be processed by shooting an image, or the transceiver 1340 may obtain the image to be processed from other applications installed in the mobile phone (e.g., an application with an image transmission function), or the transceiver 1340 may obtain the coordinates of the characters in the image obtained after recognition from other applications installed in the mobile phone (e.g., an application with a text recognition function). In addition, the transceiver 1340 may also include a display module, which is configured to receive and display the character processing result (the result obtained by the processor executing the character processing method) or the character recognition result (the result obtained by the processor executing the character recognition method) output by the processor; or the transceiver 1340 includes other applications, which are configured to receive the character processing result or the character recognition result output by the processor and process the received result.

[0161] It should be understood that the processor 1310 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0162] The memory 1320 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1310. A portion of the memory 1320 may also include a nonvolatile random access memory. For example, the memory 1320 may also store information on the device type.

[0163] In addition to the data bus, the bus system 1330 may also include a power bus, a control bus, and a status signal bus. Fig.14 The various buses are labeled as bus system 1330 .

[0164] In the implementation process, the processing performed by the terminal device can be completed by the hardware integrated logic circuit in the processor 1310 or the instructions in the form of software. That is, the steps of the method disclosed in the embodiment of the present disclosure can be embodied as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1320, and the processor 1310 reads the information in the memory 1320 and completes the steps of the above method in combination with its hardware. To avoid repetition, it is not described in detail here.

[0165] Those skilled in the art should understand that the technical solutions of the embodiments of the present disclosure may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present disclosure, and should all be included in the scope of the claims of the present application.

[0166] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A character processing method, comprising: Identify and obtain the coordinates of characters in the image; Using a kernel density function to perform clustering calculation on the difference between the first coordinate values ​​of two adjacent characters to determine the characters belonging to the same row; when the row is a line, using a kernel density function to perform clustering calculation on the difference between the first longitudinal coordinates of two horizontally adjacent characters to determine a first horizontal spacing; and determining the characters belonging to the same line according to the first horizontal spacing; Determine a second horizontal spacing according to the calculation result of the kernel density function, and determine a line spacing by judging a difference between first ordinates of two characters adjacent to each other in the horizontal direction according to the second horizontal spacing; A kernel density function is used to perform clustering calculation on the difference in the second coordinate values ​​of two horizontally adjacent characters to determine a third horizontal spacing, and the difference in the second coordinate values ​​of two horizontally adjacent characters is judged based on the third horizontal spacing to determine characters belonging to the same character column group.

2. The method according to claim 1, wherein: When the arrangement is a column; the use of a kernel density function to perform clustering calculation on the difference in the first coordinate values ​​of two adjacent characters to determine the characters belonging to the same row includes: using a kernel density function to perform clustering calculation on the difference in the first horizontal coordinates of two vertically adjacent characters to determine a first vertical spacing, and determining the characters belonging to the same column based on the first vertical spacing.

3. The method according to claim 1, wherein: The method of using a kernel density function to perform cluster calculation on the difference between the first ordinates of two characters adjacent to each other in the horizontal direction to determine a first horizontal spacing, and determining characters belonging to the same row according to the first horizontal spacing, comprises: For any two laterally adjacent first and second characters, calculate the difference between the first longitudinal coordinate of one or more points in the first character and the first longitudinal coordinate of the corresponding one or more points in the second character; use the kernel density function to perform cluster calculation on all the calculated differences, determine the first horizontal spacing based on the calculation result of the kernel density function, judge the difference between the first longitudinal coordinates of the two laterally adjacent characters based on the first horizontal spacing, and determine the characters belonging to the same character row.

4. The method according to claim 2, wherein: The method of using a kernel density function to perform clustering calculation on the difference between the first horizontal coordinates of two characters adjacent to each other in the vertical direction to determine a first vertical spacing, and determining characters belonging to the same column according to the first vertical spacing, comprises: For any two vertically adjacent third characters and fourth characters, calculate the difference between the first horizontal coordinates of one or more points in the third character and the first horizontal coordinates of the corresponding one or more points in the fourth character; use the kernel density function to perform clustering calculation on all the calculated differences, determine the first vertical spacing based on the calculation result of the kernel density function, judge the difference between the first horizontal coordinates of the two vertically adjacent characters based on the first vertical spacing, and determine the characters belonging to the same character column.

5. The method according to claim 1, wherein: The difference between the second coordinate values ​​of the two horizontally adjacent characters includes: the difference between the second horizontal coordinates of one or more points in the fifth character and the second horizontal coordinates of one or more corresponding points in the sixth character, and the fifth character and the sixth character are two horizontally adjacent characters.

6. The method according to claim 1, wherein judging the difference between the first ordinates of two characters adjacent to each other in the horizontal direction according to the first lateral spacing to determine the characters belonging to the same character row, and determining the second lateral spacing according to the calculation result of the kernel density function, and judging the difference between the first ordinates of two characters adjacent to each other in the horizontal direction according to the second lateral spacing to determine the row spacing, comprises: The calculation result of the kernel density function includes at least two minimum values, including a smallest first minimum value and a second minimum value greater than the first minimum value. When it is determined that the difference between the first longitudinal coordinates of the two horizontally adjacent characters is less than or equal to the first minimum value, it is determined that the two horizontally adjacent characters belong to the same character row. When it is determined that the difference between the first longitudinal coordinates of the two horizontally adjacent characters is greater than the first minimum value and less than or equal to the second minimum value, it is determined that there is a line spacing between the two horizontally adjacent characters.

7. The method according to claim 4, further comprising: Determine a second longitudinal spacing according to the calculation result of the kernel density function, and determine the column spacing by judging the difference between the first horizontal coordinates of two characters adjacent to each other in the longitudinal direction according to the second longitudinal spacing; The kernel density function is used to perform cluster calculation on the difference in the second coordinate values ​​of two vertically adjacent characters to determine the third vertical spacing, and the difference in the second coordinate values ​​of two vertically adjacent characters is judged based on the third vertical spacing to determine the character rows belonging to the same group.

8. The method according to claim 7, wherein: The difference between the second coordinate values ​​of the two vertically adjacent characters includes: the difference between the second vertical coordinates of one or more points in the seventh character and the second vertical coordinates of one or more corresponding points in the eighth character, and the seventh character and the eighth character are two vertically adjacent characters.

9. The method according to claim 7, wherein judging the difference between the first horizontal coordinates of two characters adjacent to each other in the vertical direction according to the first vertical spacing to determine the characters belonging to the same character column, and determining the second vertical spacing according to the kernel density function calculation result, and judging the difference between the first horizontal coordinates of two characters adjacent to each other in the vertical direction according to the second vertical spacing to determine the column spacing, comprises: The calculation result of the kernel density function includes at least two minimum values, including a smallest fifth minimum value and a sixth minimum value greater than the fifth minimum value. When it is determined that the difference between the first horizontal coordinates of the two vertically adjacent characters is less than or equal to the fifth minimum value, it is determined that the two vertically adjacent characters belong to the same character column. When it is determined that the difference between the first horizontal coordinates of the two vertically adjacent characters is greater than the fifth minimum value and less than or equal to the sixth minimum value, it is determined that there is a column spacing between the two vertically adjacent characters.

10. The method according to any one of claims 1 to 9, wherein: The kernel density function is a Gaussian kernel density function.

11. A character recognition method, comprising: Preprocess the image; identify the coordinates of the characters in the image; use the character processing method described in any one of claims 1 to 10 to identify the characters in the image to determine the characters belonging to the same row.

12. The method according to claim 11, wherein: The image preprocessing includes one or more of the following processes: color image to grayscale image conversion, Gaussian filtering, background extraction, contrast compensation, binarization and perspective transformation.

13. The method according to claim 11, after determining the characters belonging to the same row, the method further comprises: Display part or all of one or more rows of characters determined after recognition processing.

14. A computer-readable storage medium storing computer-executable instructions for executing the method of any one of claims 1 to 10 or any one of claims 11 to 13.

15. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 10 or any one of claims 11 to 13 are implemented.

Images