A hand-drawn bracketing diagram recognition method and system
By integrating object detection and clustering algorithms, and combining deep neural networks to identify the hierarchical logical relationships in hand-drawn bracket diagrams, the problem of digitizing hand-drawn bracket diagrams was solved, achieving efficient digitization and recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2023-03-17
- Publication Date
- 2026-04-28
AI Technical Summary
Existing technologies lack effective methods for digitizing hand-drawn bracket diagrams, and deep learning models lack sufficient data in this field, resulting in poor recognition performance.
By integrating object detection algorithms, pixel-domain clustering algorithms, and object counting algorithms, and combining deep neural networks to identify left curly braces and text blocks, a hierarchical logic tree is generated and mapped to LaTeX code.
It enables accurate identification of the hierarchical logical relationships in hand-drawn bracket diagrams without requiring a large amount of hand-drawn bracket diagram data, and digitizes them, improving the accuracy and portability of the identification.
Smart Images
Figure CN116311261B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pattern recognition, specifically to a method and system for recognizing hand-drawn bracket diagrams. Background Technology
[0002] Mind mapping is an effective graphical thinking tool for expressing divergent thinking. It is simple yet efficient and widely used in learning, daily life, and work. Traditional text notes often contain large amounts of text, making it difficult to identify key points. Mind maps, however, break down large amounts of content by identifying hierarchical relationships, focusing on recording key information, reducing text length, and facilitating understanding and memorization. Mind maps often use bracket diagrams to represent the whole and the relationships between parts, providing a clearer understanding of the microscopic structure of a subject. For example... Figure 1 As shown, primary school subjects are divided into Chinese, mathematics, and English. Chinese includes sentence construction and picture description, while English includes memorizing words and learning phonetics.
[0003] There are two ways to create bracket diagrams: one is to create electronic mind maps using software, which is easy to edit and share, but has significant device limitations; the other is to hand-draw them on paper, which enhances memory while understanding the internal logic of knowledge, but has poor editability, portability, aesthetics, and interactivity. Currently, methods exist for recognizing electronic mind maps and converting them into hand-drawn versions, but there is a lack of methods for digitizing hand-drawn bracket diagrams, which is what this invention aims to achieve. Figure 1 The demonstration demonstrates the following functions. With the development of deep learning, pattern recognition has been applied to numerous fields. Among them, OCR recognition and table recognition have achieved good results. However, there is still no relevant research and practice in the area of hand-drawn bracket recognition. At the same time, the amount of hand-drawn bracket data available for training deep learning models in this field is limited, which prevents deep learning models from being fully applied in solving this problem. Summary of the Invention
[0004] To address the aforementioned problems, this invention provides a method and system for recognizing hand-drawn bracket diagrams. This invention integrates object detection algorithms, pixel-domain clustering algorithms, and object counting algorithms, which can compensate for the shortcomings of pattern recognition in handling this problem without requiring a large amount of hand-drawn bracket diagram data.
[0005] In a first aspect, the present invention provides a method for recognizing hand-drawn bracket diagrams, comprising the following steps:
[0006] S1. The hand-drawn image to be processed is preprocessed using the image preprocessing module to obtain a preprocessed image;
[0007] S2. Input the preprocessed image into the counting module to obtain the number of text blocks and the number of left curly braces in the preprocessed image;
[0008] S3. Input the preprocessed image into the left curly brace recognition module, and combine it with the counting results of step S2 to obtain the pixel position information of all left curly braces;
[0009] S4. Remove all left curly braces from the preprocessed image using pixel position information to obtain the text image, and input it into the text block recognition module;
[0010] S5. Use a text block recognition module to obtain the pixel boundary information and recognition content of all text blocks in the text image;
[0011] S6. Based on the pixel boundary information and recognized content of the text block, as well as the pixel position information of the left curly brace, a hierarchical logic generation module is used to generate a hierarchical logic tree between the left curly brace and the text block.
[0012] S7. Map the hierarchical logic tree to LaTeX code using the Map function, and display the electronic bracket image on the experimental platform by compiling the LaTeX code.
[0013] Furthermore, the preprocessing in step S1 includes:
[0014] S11. Compress the hand-drawn image to be processed using an image compression algorithm;
[0015] S12. Denoise the compressed hand-drawn image to be processed.
[0016] Furthermore, the denoising process for the compressed hand-drawn image to be processed includes:
[0017] Set a grayscale threshold to obtain the grayscale value of each pixel in the compressed hand-drawn image to be processed. If the grayscale value of a pixel is greater than the grayscale threshold, the pixel is determined to be an invalid handwriting and its grayscale value is directly set to 255.
[0018] Furthermore, the counting process of the counting module in step S2 includes:
[0019] S21. Input the preprocessed image into the counting module and perform clustering processing on all its pixels;
[0020] S22. The number of left curly braces and the number of text blocks are obtained by describing the shape of the clusters after clustering.
[0021] Furthermore, the operation process of the left curly brace recognition module includes:
[0022] S31. A left curly brace recognition model is built using a YOLOv7 network and trained using a labeled handwritten left curly brace dataset;
[0023] S32. Input the preprocessed image into the trained left curly brace recognition model to obtain multiple recognition results and their corresponding similarity values. Sort all recognition results in descending order according to their similarity values. Introduce the number of left curly braces K output by the counting module and select the top K recognition results from the descending order as the true recognition results.
[0024] S33. Extract and save the pixel position information of each left curly brace in the real recognition result. The pixel position information includes the vertical maximum value, vertical minimum value, horizontal maximum value, and horizontal minimum value of the predicted box of each left curly brace.
[0025] S34. Remove all left curly braces from the preprocessed image based on pixel position information to obtain the text image.
[0026] Furthermore, the operation process of the text block recognition module includes:
[0027] S41. Clustering algorithm is used to cluster the content in the text image, and the number of text blocks output by the counting module is combined to obtain several text blocks;
[0028] S42. Obtain and save the pixel boundary information of each text block. The pixel boundary information includes the vertical highest value, vertical lowest value, horizontal highest value, and horizontal lowest value of each text block.
[0029] S43. Use a handwritten character recognition algorithm to recognize the content of each character block and save the recognized content.
[0030] Furthermore, the process of generating a hierarchical logic tree by the hierarchical logic generation module includes:
[0031] S51. Sort all left curly braces in ascending order of their left boundary values in pixel position information, and sort all text blocks in ascending order of their left boundary values in pixel boundary information;
[0032] S52. Match a text block to the head of each left curly brace in sequence;
[0033] S53. Match one or more text blocks sequentially to the end of each left curly brace;
[0034] S54. Sort the text blocks matched by the tail of each left curly brace to obtain the hierarchical logic tree.
[0035] In a second aspect, based on the method proposed in the first aspect, the present invention also provides a hand-drawn bracket recognition system, comprising:
[0036] The experimental platform is used as the input for the hand-drawn image to be processed, and as the output display of the conversion result of the hand-drawn image to be processed;
[0037] The image preprocessing module is used to compress and denoise the hand-drawn image to be processed, so as to obtain a preprocessed image.
[0038] The counting module is used to obtain the number of expanding brackets and text blocks in the preprocessed image;
[0039] The left curly brace recognition module is used to combine the counting results of the counting module, use the left curly brace recognition model to obtain the pixel position information of all left curly braces in the preprocessed image, and remove all left curly braces in the preprocessed image.
[0040] The text block recognition module is used to detect text blocks in the image output by the left curly brace recognition module, and to obtain the pixel boundary information and recognition content of all text blocks;
[0041] The hierarchical logic generation module is used to generate a hierarchical logic tree using the pixel position information of the left curly brace and the pixel boundary information of the text block.
[0042] The beneficial effects of this invention are:
[0043] This invention proposes a hand-drawn bracket image recognition method for hand-drawn images containing curly braces. It integrates target detection algorithm, pixel domain clustering algorithm and target recognition algorithm to make up for the shortcomings of current methods in processing hand-drawn images containing curly braces. That is, it makes up for the lack of current technology for hand-drawn bracket image recognition, and further studies the recognition method for left curly braces of different sizes.
[0044] This invention can identify hierarchical logical relationships based on the spatial positional relationship of handwriting pixels, eliminating the need for a large amount of hand-drawn bracket diagram data; the introduction of a target counting mechanism enhances the accuracy of hierarchical logical generation in the target recognition module; in addition, a mapping relationship between a tree structure and a LaTeX formula is established to enable modification of the content and display of the electronic bracket diagram. Attached Figure Description
[0045] Figure 1 This is a diagram illustrating the effects of the present invention;
[0046] Figure 2 This is a flowchart of the hand-drawn bracket recognition method of the present invention;
[0047] Figure 3 This is a diagram illustrating the experimental platform of the present invention;
[0048] Figure 4 This is a flowchart illustrating the operation of the left curly brace recognition module according to an embodiment of the present invention.
[0049] Figure 5 This is a flowchart illustrating the operation of the text block recognition module according to an embodiment of the present invention.
[0050] Figure 6This is a flowchart of the hierarchical logic generation algorithm according to an embodiment of the present invention;
[0051] Figure 7 This is a schematic diagram of head matching according to an embodiment of the present invention;
[0052] Figure 8 This is a schematic diagram of tail matching according to an embodiment of the present invention;
[0053] Figure 9 This is a diagram of electronic brackets according to an embodiment of the present invention;
[0054] Figure 10 This is a system flowchart of an embodiment of the present invention. Detailed Implementation
[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0056] This invention provides a method for recognizing hand-drawn bracket images. The method first obtains a hand-drawn image to be processed and inputs it into an experimental platform. The experimental platform then outputs and displays the electronic bracket image after conversion and recognition of the hand-drawn image. The experimental platform is as follows: Figure 3 As shown, it consists of two parts: "Upload Image" and "Recognition Result". The usage process is: Upload Image -> Start Conversion -> Recognition Result. Users can choose to "Enter the number of text blocks in the image" to improve the recognition accuracy.
[0057] Specifically, you can use PyQt5 to build a... Figure 3 The simplified GUI interface shown serves as the demonstration interface for the experimental platform. Other application platforms can also be developed to serve as the demonstration interface for the experimental platform.
[0058] In one embodiment, the conversion and recognition process of the hand-drawn image to be processed is as follows: Figure 2 As shown, it includes the following steps:
[0059] S1. The image preprocessing module preprocesses the hand-drawn image to be processed to obtain a preprocessed image.
[0060] Specifically, the preprocessing process includes:
[0061] S11. Compress the hand-drawn image to be processed using an image compression algorithm; because the size of the scanned image and the captured image are different, and the image is too large, it will affect the efficiency of deleting and clustering elements in the image. Therefore, the hand-drawn image to be processed is compressed using an image compression algorithm while ensuring accurate recognition.
[0062] S12. Denoising the compressed hand-drawn image to be processed; the noise sources of the hand-drawn image to be processed include paper quality, invalid handwriting, etc. The denoising method used in this embodiment is as follows: First, a grayscale threshold is set and compared with the grayscale value of the pixels in the compressed hand-drawn image to be processed; if the grayscale value of a pixel is greater than the grayscale threshold, the pixel is determined to be invalid handwriting, and the grayscale value of the pixel is directly set to 255; if the grayscale value of a pixel is not greater than the grayscale threshold, no processing is performed on it.
[0063] S2. Input the preprocessed image into the counting module to obtain the number of text blocks and the number of left curly braces in the preprocessed image.
[0064] Specifically, the counting process of the counting module includes:
[0065] S21. Input the preprocessed image into the counting module; if the user selects "Input the number of text blocks in the image" when uploading the hand-drawn image to be processed to the experimental platform, the counting module will receive the number of text blocks in the preprocessed image, and it is no longer necessary to execute a counting algorithm to obtain the number of text blocks; similarly, if the user selects "Input the number of left curly braces in the image", the counting module will receive the number of left curly braces in the preprocessed image, and it is no longer necessary to execute a counting algorithm to obtain the number of left curly braces;
[0066] S22. The number of left curly braces and the number of text blocks are obtained by describing the shape of the clusters after clustering; describing the shape of the clusters after clustering means obtaining the number of left curly braces and the number of text blocks by judging the shape of the pixel set after clustering. For example, the shape of the pixel set of the left curly brace has certain special characteristics. Its leftmost end is raised, and the upper and lower ends of the rightmost end have values. The vertical gray value exists continuously and the length is variable.
[0067] Specifically, in addition to describing the shape of the clusters after clustering, the number of left braces and the number of text blocks can be obtained through an attention encoder-decoder mechanism. This mechanism primarily acquires features from the preprocessed image through an encoder, then decodes the features using convolutional kernels of multiple sizes (3x3 and 5x5), and finally obtains the counting result through channel attention, 1x1 convolution, counting mapping, and aggregation operators. Different writing habits result in characters of varying sizes in the image, and a single kernel size cannot effectively handle scale variations; therefore, multiple convolutional kernel sizes are used.
[0068] S3. Input the preprocessed image into the left curly brace recognition module, and combine it with the counting results of step S2 to obtain the pixel position information of all left curly braces.
[0069] S4. Remove all left curly braces from the preprocessed image using pixel location information to obtain the text image, and input it into the text block recognition module.
[0070] Specifically, the operation process of the left brace recognition module is as follows: Figure 4 As shown, it includes:
[0071] S31. A left curly brace recognition model is built using a deep neural network model and trained using an annotated handwritten left curly brace dataset; the annotated handwritten left curly brace dataset includes various hand-drawn left curly braces; the deep neural network used in this embodiment is the YOLOv7 network.
[0072] S32. Input the preprocessed image into the trained left curly brace recognition model to obtain multiple recognition results and their similarity values. Sort all recognition results in descending order of their similarity values. Introduce the number K of left curly braces output by the counting module, and select the K highest similarity recognition results from the descending sorted results as the true recognition results; Figure 4 In the embodiment shown, the number of left curly braces output by the counting module is K = Brace_Num = 3, which ultimately yields 3 left curly braces connected to "Primary School Subjects", "Chinese", and "English" respectively.
[0073] S33. Extract and save the pixel position information of each left curly brace in the real recognition result. The pixel position information includes the vertical highest value top, vertical lowest value bottom, horizontal highest value right, and horizontal lowest value left of the predicted box of each left curly brace. Figure 4 The pixel position information of the three left curly braces obtained in the embodiment shown is represented as: Brace_1(left,right,top,bottom), Brace_2(left,right,top,bottom), Brace_3(left,right,top,bottom);
[0074] S34. Remove all left curly braces from the preprocessed image based on pixel position information to obtain the text image.
[0075] Specifically, the left curly brace recognition model combines image feature matching algorithms with the characteristics of the left curly brace itself (protrusion at the leftmost end, values at the top and bottom of the rightmost end, and continuous vertical grayscale values), which can improve recognition accuracy.
[0076] S5. Use a text block recognition module to obtain the pixel boundary information and recognition content of all text blocks in the text image;
[0077] Directly using object detection algorithms to detect text blocks has certain drawbacks. Specifically, when the characters within the same text block have a large interval, they are likely to be divided into two text blocks; similarly, when the interval between two text blocks is small, they are also likely to be recognized as one text block. To address the above drawbacks, the present invention is based on a clustering algorithm and introduces a text block counting mechanism to improve the accuracy of text block segmentation and the accuracy of text block boundary division.
[0078] Specifically, as Figure 5 shown, the operation process of the text block recognition module includes:
[0079] S41. Use a clustering algorithm to cluster the content in the text image, and fuse the number of text blocks output by the counting module to obtain a number of text blocks;
[0080] Specifically, the fusion counting module mainly uses the number of text blocks output by the counting module as the termination condition for clustering:
[0081] (1) If the partitioning clustering algorithm is used, the number of text blocks output by the counting module can be directly used as the number of initial clusters of the partitioning clustering algorithm;
[0082] (2) If the density clustering algorithm is used, the size of the density parameter can be optimized and updated according to the number of text blocks output by the counting module. For example, when using the DBSCAN algorithm, if the number of clusters after the current clustering is n1 and the result of the counting module is n2, if n1 < n2, it proves that the neighborhood is too large and the neighborhood range needs to be appropriately reduced; if n1 > n2, it proves that the neighborhood is too small and the neighborhood range needs to be appropriately enlarged.
[0083] (3) If the hierarchical clustering algorithm is used, the number of text blocks output by the counting module can be used as the termination condition for agglomeration or splitting, that is, when the number of clusters after agglomeration or splitting is exactly the result of the counting module, agglomeration or splitting will no longer be performed.
[0084] S42. Obtain and save the pixel boundary information of each text block. The pixel boundary information includes the vertical maximum value top, the vertical minimum value bottom, the horizontal maximum value right, and the horizontal minimum value left of each text block; as Figure 5 shown, the pixel position information of the text block can be expressed as Text_1(left, right, top, bottom), Text_2(left, right, top, bottom);
[0085] S43. Use a handwritten text recognition algorithm to recognize the content of each text block and save the recognized content.
[0086] S6. Based on the pixel boundary information and recognized content of the text block, as well as the pixel position information of the left curly brace, a hierarchical logic generation module is used to generate a hierarchical logic tree between the left curly brace and the text block.
[0087] Specifically, the pseudocode for the algorithm used by the hierarchical logic generation module to generate the hierarchical logic tree is as follows: Figure 6 As shown, the process includes:
[0088] S51. Sort all left curly braces in ascending order of their left boundary values in pixel position information, and sort all text blocks in ascending order of their left boundary values in pixel boundary information;
[0089] S52. Match the text blocks at the beginning of each left curly brace in sequence; Figure 6 Line 5 in the code represents `min{distance(right_center(tj),left_center(bi))}`, where `left_center(bi)` represents the midpoint of the left boundary of the i-th left curly brace `bi`, and `right_center(tj)` represents the midpoint of the right boundary of the j-th text block `tj`. The distance between the left curly brace `bi` and the text block `tj` is calculated using these two midpoints. The text block corresponding to the minimum distance for the left curly brace `bi` is selected for header matching. Each left curly brace has only one header, and one header matches one text block. Once a text block is matched, it no longer participates in the header matching of other left curly braces. Figure 1 Taking handwriting input as an example, after the preceding processing, we obtain... Figure 7 The results are shown.
[0090] S53. Match the text blocks to the end of each left curly brace in sequence; Figure 6 Line 9 represents `min{distance(left_center(ti),right_same_y(bj)}`, where `left_center(ti)` is the midpoint of the left boundary of the i-th text block `ti`, and `right_same_y(bj)` indicates that all or more of the content of text block `ti` is within the left curly brace `bj`. The algorithm calculates the vertical distance from the midpoint of the left boundary of text block `ti` to the right boundary of the left curly brace `bj`, and selects the left curly brace corresponding to the minimum vertical distance for tail matching. Figure 7 After further processing of the results, we obtain Figure 8 The results are shown.
[0091] S54. At this point, the hierarchical logic tree has been constructed, but the order of the text blocks matched at the end of the left curly brace may not be correct. Therefore, traverse each left curly brace and sort the text blocks matched at the end of the left curly brace according to the upper boundary value of the text blocks.
[0092] S7. The hierarchical logic tree is mapped to LaTeX code using the Map function. The LaTeX code is then compiled and displayed as an electronic bracket image on the experimental platform. Figure 9 As shown.
[0093] This invention also provides a hand-drawn bracket recognition system, such as... Figure 10 As shown, it includes:
[0094] The experimental platform is used as the input for the hand-drawn image to be processed, and as the output display of the conversion result of the hand-drawn image to be processed;
[0095] The image preprocessing module is used to compress and denoise the hand-drawn image to be processed, so as to obtain a preprocessed image.
[0096] The counting module is used to obtain the number of expanding brackets and text blocks in the preprocessed image;
[0097] The left curly brace recognition module is used to combine the counting results of the counting module, use the left curly brace recognition model to obtain the pixel position information of all left curly braces in the preprocessed image, and remove all left curly braces in the preprocessed image.
[0098] The text block recognition module is used to detect text blocks in the image output by the left curly brace recognition module, and to obtain the pixel boundary information and recognition content of all text blocks;
[0099] The hierarchical logic generation module is used to generate a hierarchical logic tree using the pixel position information of the left curly brace and the pixel boundary information of the text block.
[0100] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0101] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for recognizing hand-drawn bracket diagrams, characterized in that, The experimental platform receives the hand-drawn image to be processed and inputs it into the experimental platform. The experimental platform outputs and displays the electronic bracket image after the hand-drawn image to be processed has been converted and recognized. The conversion and recognition process of the hand-drawn image to be processed includes the following steps: S1. The hand-drawn image to be processed is preprocessed using the image preprocessing module to obtain a preprocessed image; S2. Input the preprocessed image into the counting module to obtain the number of text blocks and the number of left curly braces in the preprocessed image; S3. Input the preprocessed image into the left curly brace recognition module, and combine it with the counting results of step S2 to obtain the pixel position information of all left curly braces; The operation process of the left curly brace recognition module includes: S31. A left curly brace recognition model is built using a YOLOv7 network and trained using a labeled handwritten left curly brace dataset; S32. Input the preprocessed image into the trained left curly brace recognition model to obtain multiple recognition results and their corresponding similarity values. Sort all recognition results in descending order according to their similarity values. Introduce the number of left curly braces K output by the counting module. Select the top K recognition results from the descending order as the true recognition results. S33. Extract and save the pixel position information of each left curly brace in the real recognition result. The pixel position information includes the vertical maximum value, vertical minimum value, horizontal maximum value, and horizontal minimum value of the predicted box of each left curly brace. S34. Remove all left curly braces from the preprocessed image based on pixel position information to obtain the text image; S4. Remove all left curly braces from the preprocessed image using pixel position information to obtain the text image, and input it into the text block recognition module; S5. Use a text block recognition module to obtain the pixel boundary information and recognition content of all text blocks in the text image; S6. Based on the pixel boundary information and recognized content of the text block, as well as the pixel position information of the left curly brace, a hierarchical logic generation module is used to generate a hierarchical logic tree between the left curly brace and the text block. The process of generating a hierarchical logic tree by the hierarchical logic generation module includes: S51. Sort all left curly braces in ascending order of their left boundary values in pixel position information, and sort all text blocks in ascending order of their left boundary values in pixel boundary information; S52. Match a text block to the head of each left curly brace in sequence; S53. Match one or more text blocks sequentially to the end of each left curly brace; S54. Sort the text blocks matched at the end of each left curly brace to obtain the hierarchical logic tree; S7. Map the hierarchical logic tree to LaTeX code using the Map function, and display the electronic bracket image on the experimental platform by compiling the LaTeX code.
2. The method for recognizing hand-drawn bracket diagrams according to claim 1, characterized in that, The preprocessing process in step S1 includes: S11. Compress the hand-drawn image to be processed using an image compression algorithm; S12. Denoise the compressed hand-drawn image to be processed.
3. The method for recognizing hand-drawn bracket diagrams according to claim 2, characterized in that, Denoising of the compressed hand-drawn image to be processed includes: Set a grayscale threshold to obtain the grayscale value of each pixel in the compressed hand-drawn image to be processed. If the grayscale value of a pixel is greater than the grayscale threshold, the pixel is determined to be an invalid handwriting and its grayscale value is directly set to 255.
4. The method for recognizing hand-drawn bracket diagrams according to claim 1, characterized in that, The counting process of the counting module in step S2 includes: S21. Input the preprocessed image into the counting module and perform clustering processing on all its pixels; S22. The number of left curly braces and the number of text blocks are obtained by describing the shape of the clusters after clustering.
5. The method for recognizing hand-drawn bracket diagrams according to claim 1, characterized in that, The operation process of the text block recognition module includes: S41. Clustering algorithm is used to cluster the content in the text image, and the number of text blocks output by the counting module is combined to obtain several text blocks; S42. Obtain and save the pixel boundary information of each text block. The pixel boundary information includes the vertical highest value, vertical lowest value, horizontal highest value, and horizontal lowest value of each text block. S43. Use a handwritten character recognition algorithm to recognize the content of each character block and save the recognized content.
6. A hand-drawn bracket recognition system, characterized in that, The hand-drawn bracket recognition method as described in any one of claims 1-5 includes: The experimental platform is used as the input for the hand-drawn image to be processed, and as the output display of the conversion result of the hand-drawn image to be processed; The image preprocessing module is used to compress and denoise the hand-drawn image to be processed, so as to obtain a preprocessed image. The counting module is used to obtain the number of expanding brackets and text blocks in the preprocessed image; The left curly brace recognition module is used to combine the counting results of the counting module, use the left curly brace recognition model to obtain the pixel position information of all left curly braces in the preprocessed image, and remove all left curly braces in the preprocessed image. The text block recognition module is used to detect text blocks in the image output by the left curly brace recognition module, and to obtain the pixel boundary information and recognition content of all text blocks; The hierarchical logic generation module is used to generate a hierarchical logic tree using the pixel position information of the left curly brace and the pixel boundary information of the text block.