A water writing character recognition method and system based on target detection

Abstract

The present application relates to the technical field of computer vision, and especially relates to a water writing character recognition method and system based on target detection. The method comprises the following steps: obtaining a water writing image, and performing image preprocessing to obtain a water writing image to be processed; recognizing a character region of the water writing image to be processed; correcting character geometric deformation based on the character region, and recognizing water writing character features; classifying characters of the water writing image by using the water writing character features, and determining character semantic information; reconstructing a character sequence based on the character semantic information; recognizing a water writing phrase structure of the water writing image according to the character sequence, and generating a complete water writing recognition sample; calculating a water writing confidence based on the complete water writing recognition sample, and correcting an incorrect water writing character by using the water writing confidence, and storing the corrected water writing character. The present application improves the accuracy and integrity of water writing character recognition based on computer vision technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a water script character recognition method and system based on target detection. BACKGROUND

[0002] At present, as a unique pictographic character system of China's ethnic minorities, water script has been widely used in literature arrangement, ethnic culture research and digital storage scenarios, but its character recognition method still has many limitations in dealing with complex literature background, diversified writing style and high-precision academic research requirements. Most existing technologies rely on template matching or manual feature extraction, lack joint processing capability for large-scale character sets and complex semantic structures, resulting in insufficient recognition compatibility under different writing bodies and mixed layout conditions. Only relying on sequential segmentation and linear matching method ignores the characteristics of large differences in water script character form and complex stroke connection, which is easy to produce errors in character positioning and phrase reconstruction process, and lacks a global correction mechanism for character spatial arrangement and semantic order, which makes the overall recognition efficiency and reliability defective. The existing general OCR-based scheme focuses on the detection and single character recognition of the text area, adopts a fixed recognition process, and is difficult to adapt to dynamic reconstruction under multi-character combination, phrase structure level and complex context, and lacks high robustness recognition and semantic correction capability for special application scenarios of water script. SUMMARY

[0003] Therefore, it is necessary to provide a water script character recognition method and system based on target detection to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a water script character recognition method based on target detection comprises the following steps:

[0005] Step S1: obtaining a water script image and performing image preprocessing to obtain a water script image to be processed; recognizing the text area of the water script image to be processed;

[0006] Step S2: correcting the text geometric deformation based on the text area, recognizing the water script character features; using the water script character features to classify the characters of the water script image and determine the character semantic information;

[0007] Step S3: reconstructing the character order based on the character semantic information; recognizing the water script phrase structure of the water script image according to the character order to generate a complete water script recognition sample;

[0008] Step S4: calculating the water script confidence based on the complete water script recognition sample, and correcting the error water script character using the water script confidence, and storing the corrected water script character.

[0009] Preferably, the present specification also provides a target detection-based water writing character recognition system for executing the target detection-based water writing character recognition method as described above, which comprises:

[0010] An image preprocessing module is configured to acquire a water writing image, perform image preprocessing on the water writing image, and obtain a to-be-processed water writing image; and identify a character region of the to-be-processed water writing image.

[0011] A character classification module is configured to correct character geometric deformation based on the character region, identify water writing character features, perform character classification on the water writing image based on the water writing character features, and determine character semantic information.

[0012] A character order reconstruction module is configured to reconstruct a character order based on the character semantic information, identify a water writing phrase structure of the water writing image according to the character order, and generate a complete water writing recognition sample.

[0013] A confidence calculation module is configured to calculate water writing confidence based on the complete water writing recognition sample, correct an incorrect water writing character based on the water writing confidence, and store the corrected water writing character.

[0014] The present application has the following beneficial effects:

[0015] (1) The present application realizes accurate positioning and feature extraction of water writing characters by accurately identifying and segmenting the character region of a water writing image and combining character feature correction technology, thereby ensuring the reliability of subsequent character classification and semantic information determination.

[0016] (2) In the character semantic information processing link, the present application improves the integrity and logical consistency of water writing phrase structure generation by performing semantic combination and structure level analysis on adjacent characters through a multi-level semantic labeling and phrase unit reconstruction mechanism, and is suitable for diversified water writing layout and writing styles.

[0017] (3) In the character order reconstruction and water writing phrase structure identification process, the present application realizes automatic correction of incorrect recognition characters by establishing a complete recognition sample using character semantic information and combining confidence calculation and weighted correction mechanisms, thereby ensuring the order accuracy and structural integrity of water writing recognition results.

[0018] (4) In the system security and data processing link, the present application ensures the integrity, reliability and verifiability of recognition data and improves the processing efficiency and system stability of the entire recognition process by performing safe storage and traceable recording of recognition results through a water writing confidence dynamic evaluation and storage management mechanism. BRIEF DESCRIPTION OF DRAWINGS

[0019] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments made with reference to the accompanying drawings.

[0020] Figure 1 A step flow diagram of a water writing character recognition method based on target detection according to the present application is shown in the figure;

[0021] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0022] The technical method of the present application will be described clearly and completely below in combination with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0023] In addition, the accompanying drawings are only schematic drawings of the present application, and are not necessarily drawn to scale. The same reference signs in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily have to correspond to physically or logically independent entities. The functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0024] It should be understood that although the terms "first", "second" and the like can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, a first element can be referred to as a second element, and similarly a second element can be referred to as a first element. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0025] To achieve the above-mentioned object, please refer to Figure 1 The present application provides a water writing character recognition method based on target detection, which comprises the following steps:

[0026] Step S1: Obtain a water writing image and perform image preprocessing to obtain a to-be-processed water writing image; and identify the character region of the to-be-processed water writing image;

[0027] In an embodiment, the high-definition scanner collects water manuscript image as input, with a resolution of 300 dpi. First, the image is processed to grayscale, and then the adaptive threshold segmentation is used to eliminate background interference. Then, the median filter is used to remove noise, and the histogram equalization is used to enhance the contrast of the text. Subsequently, the target detection algorithm based on convolutional neural network (such as YOLOv5) is used to locate and identify the candidate bounding box of the text area, and the rectangular area containing single characters is obtained.

[0028] In another embodiment, assuming that a 1200x800 pixel water manuscript image is collected, after preprocessing, 180 candidate character regions are detected, of which 160 are valid character regions after threshold filtering, with an average size of 40x40 pixels, basically covering the entire page of text.

[0029] Step S2: Correct the text geometry based on the text area, identify the water manuscript text features; use the water manuscript text features to classify the water manuscript image and determine the semantic information of the characters;

[0030] In an embodiment, geometric correction is performed on each candidate text area, and perspective transformation is used to eliminate the tilt distortion caused by the shooting angle. Then, the stroke shape is adjusted through contour extraction and affine transformation. After that, the Histogram of Oriented Gradients (HOG), edge skeleton features and local texture features of the water manuscript text are extracted, and the convolutional neural network is used for feature fusion, and the classifier is input to complete character classification, and finally the semantic information corresponding to the character is output.

[0031] In another embodiment, assuming that 160 character regions are corrected, of which 30 regions have obvious tilt distortion. After geometric correction, the recognition accuracy of the character features is improved from 82% to 93%. In the classification stage, the feature vector dimension extracted by the three-layer CNN is 256, and after classification by the full connection layer, 90 common water manuscript characters are recognized, with a correct rate of 91%.

[0032] Step S3: Reconstruct the character order based on the semantic information of the characters; identify the water manuscript phrase structure of the water manuscript image according to the character order, and generate a complete water manuscript recognition sample;

[0033] In an embodiment, the recognized character set is sorted by position, and the scanning rule from left to right and top to bottom is used to reconstruct the character sequence. Combined with the language rule library, the combination relationship between consecutive characters is analyzed to restore the semantic structure of the phrase or sentence.

[0034] In another embodiment, assuming that 10 characters in the 160 characters identified have position deviation, after sorting and phrase reconstruction, 25 phrase structures are successfully parsed, with an average of 6.4 characters per phrase, and the overall semantic coherence after reconstruction reaches 95%.

[0035] Especially important is that the water writing phrase structure of the water writing image is identified according to the character order in step S3, and the complete water writing recognition sample is generated, including:

[0036] The water writing image is semantically segmented according to the character order to obtain a segmentation result; and adjacent characters in the water writing image are combined into a water writing phrase unit according to the segmentation result;

[0037] In an embodiment, the input water writing image is first converted into a grayscale image, and linear grayscale mapping is adopted for grayscale processing, and the image pixel value is normalized to the range of 0 to 255. Subsequently, two-dimensional Gaussian smoothing processing is performed on the grayscale image, the filter kernel size is 5*5 pixels, and the standard deviation is set to 1.2 to suppress image noise and subtle interference. The adaptive threshold algorithm is used for binarization processing on the smoothed grayscale image, the local threshold window size is set to 25*25 pixels, the offset is set to 10, the pixel value greater than the threshold is set to the foreground, and the pixel value lower than the threshold is set to the background, thereby generating a preliminary character outline binary image. The connected domain analysis is performed on the binary image, the minimum bounding rectangle of each connected region is extracted, the length-width ratio (the length of the rectangle / the width) and the area are calculated, the length-width ratio threshold is set to 0.3 to 5, and the area threshold is set to 50 to 2000 pixels, and the noise regions that do not meet the conditions are removed. The remaining connected domains are sorted in ascending order of the center point Y coordinate to realize row segmentation, and then the order index in the row is arranged in ascending order of the X coordinate. Subsequently, the horizontal distance between adjacent center points of characters in the row is calculated, the average distance value of each row is counted, and the adjacent characters with a distance less than a set threshold of 20 pixels are combined into a water writing phrase unit, and the number of each phrase unit is assigned from left to right according to the row. After the combination is completed, the minimum bounding rectangle of each phrase unit is calculated as the boundary box, and the character index sequence, the row number and the phrase unit number are recorded, which are used for subsequent structure analysis and sample generation.

[0038] The structure hierarchical relationship is annotated based on the water writing phrase unit, the complete water writing phrase structure is generated according to the structure hierarchical relationship, and the complete water writing phrase structure is converted into a sample format to output the complete water writing recognition sample.

[0039] In an embodiment, the center point and the horizontal and vertical projection positions of the boundary box of each phrase unit are calculated to construct a spatial relationship matrix between the phrase units. For each phrase unit in the matrix, the horizontal offset and the vertical offset between the upper and lower neighbors and the left and right neighbors thereof are calculated, respectively, wherein , The hierarchical relationship judgment rule is as follows: if , it is marked as an upper and lower hierarchical relationship; if , it is marked as a left and right hierarchical relationship. According to this rule, all phrase units are traversed to form a complete hierarchical relationship tree. Subsequently, the hierarchical tree is converted into a structured coding sample, and the coding of each phrase unit includes unit number, character sequence number, hierarchical relationship identifier (0 for up and down, and 1 for left and right), and boundary box coordinates . All coding information is saved in a JSON file for subsequent call of character recognition or system analysis.

[0040] Especially important is that the hierarchical relationship of the water writing phrase unit is marked based on the water writing phrase unit, including:

[0041] The unique number of the water writing phrase unit is assigned based on the water writing phrase unit, the dependent relationship of the water writing phrase unit is determined, the hierarchical structure is established according to the dependent relationship, and the semantic attribute of each water writing phrase unit is added in the hierarchical structure;

[0042] In an embodiment, the water writing phrase units generated in the water writing image are sequentially assigned unique integer numbers from top to bottom and from left to right, and are numbered from 1. The center point coordinates and the size of the circumscribed rectangle of the boundary box of each phrase unit are calculated, and the average width and the average height of all phrase units are counted. The dependent relationship between the phrase units is determined according to the horizontal offset and the vertical offset between the center points: if and , the phrase unit is marked as a lower dependent relationship; if and , it is marked as a parallel relationship in the same layer. Based on the above dependent relationship, the hierarchical structure of the water writing phrase unit is constructed, the root node corresponds to the uppermost phrase unit of the image, and the lower nodes are the units having a dependent relationship with the upper and lower or left and right. In the hierarchical structure, the semantic attribute of each water writing phrase unit is added, including the character sequence index, line number, boundary box coordinates and the character type identifier, such as independent word, superimposed word or connected word. Through this operation, the number, dependent relationship and semantic attribute record of each water writing phrase unit are completed, providing a data basis for subsequent structure analysis and recognition.

[0043] The hierarchical relationship of the structure is marked according to the semantic attribute.

[0044] In an embodiment, the semantic attribute of each water writing phrase unit in the hierarchical structure is read, including the character sequence index, line number, boundary box coordinates and character type. For each node, the spatial position relationship with the adjacent node is calculated, including the horizontal distance vertical distance and the bounding box overlap area OverlapArea. If the lower layer dependency is marked; if and and the parallel layer relationship is marked. The structure annotation results of all phrase units are recorded in the structure coding table, and each record contains the phrase unit number, parent node number, child node number list, hierarchical depth and semantic attribute fields. After the annotation is completed, the structure coding table is exported as a JSON or XML format file, and the fields include number, parent node, child node list, hierarchical depth and semantic attribute, which provides complete sample input for the water book recognition system.

[0045] Step S4: Calculate the water book confidence based on the complete water book recognition sample, and correct the wrong water book characters using the water book confidence. Store the corrected water book characters.

[0046] In an embodiment, the system assigns a confidence value to each recognized character, which is quantified based on the Softmax output probability. For example, when the confidence is lower than 0.7, the character is marked as to be corrected, and then replaced by a more semantic candidate character through the language model of the context phrase. Finally, the corrected result is stored in the database for subsequent calling or comparison.

[0047] In another embodiment, assuming that the system calculates the average confidence of the generated 160-character sample to be 0.87, and there are 20 characters below the threshold value 0.7. Through the error correction algorithm based on the context of the phrase, 15 characters are correctly corrected, and the remaining 5 characters remain the original result, and finally the overall recognition accuracy is improved from 91% to 96%.

[0048] Especially important is that the calculation of the water book confidence based on the complete water book recognition sample in step S4 includes:

[0049] character-by-character matching of the complete water book recognition sample to calculate the matching probability; calculating the phrase unit confidence in the complete water book recognition sample according to the matching probability;

[0050] In an embodiment, the characters in each phrase unit in the complete water book recognition sample are processed in sequence according to their bounding boxes in the image. First, the image area of each character is converted into a grayscale image, and a fixed threshold of 128 is used for binaryzation processing, with pixel points higher than the threshold being regarded as foreground and the rest of the pixel points being regarded as background. Then, a pixel-level skeletonization algorithm is used to thin the strokes in the character image into a skeleton of single-pixel width, and the coordinate information of the skeleton pixels is extracted. On this basis, each skeleton pixel is compared with the corresponding skeleton of the standard template character point by point to determine whether the pixel position offset is within two pixels, and if the offset meets the requirements, it is considered that the pixel matches successfully, while the gray difference is checked to ensure the matching under the condition of similar gray. After the processing of each character is completed, the matching probability of the character is determined according to the proportion of the number of matching pixels to the total number of skeleton pixels of the character. Then, the matching probabilities of all characters in the phrase unit are comprehensively analyzed to obtain the phrase unit confidence, and the corresponding number and matching result of each phrase unit are recorded to provide basic information for subsequent processing. In operation, the width and height of the bounding box of each character are in pixel-level precision, and the pixel offset range and gray threshold are strictly limited to ensure that the matching process is determined completely according to the image pixel features.

[0051] The phrase unit confidence is normalized to obtain a to-be-processed confidence, and the to-be-processed confidence is used for weighted accumulation to determine a water book confidence.

[0052] In an embodiment, all phrase unit confidences are sorted as a whole to find the maximum and minimum values in the sequence, and the confidence of each phrase unit is mapped to the interval of 0 to 1. Then, weights are assigned according to the number of characters contained in the phrase unit in the image, with the phrase unit having more characters obtaining a higher weight and the phrase unit having fewer characters obtaining a lower weight, while ensuring that the sum of all weights is equal to the total number. In the weighting process, the normalized confidence of each phrase unit is multiplied by its weight, and the results of all phrase units are accumulated to obtain the total confidence of the entire water book recognition sample. The entire operation is strictly performed according to the number of phrase units and the distribution of characters, without using any external model, and is calculated entirely through pixel-level matching results and character number weights.

[0053] Preferably, the step S1 of identifying the text region of the to-be-processed water book image comprises:

[0054] generating a text candidate box using a preset convolution feature extractor; calculating a response value of each text candidate box and performing character confidence prediction to output a confidence of each text candidate box;

[0055] In an embodiment, a convolutional neural network-based feature extractor (such as a ResNet-50 backbone network) is used to generate candidate regions by sliding a window over the input water document image. A 512-dimensional convolutional feature is extracted for each candidate region, and a region proposal network (RPN) is used to generate a text candidate box. For each candidate box, a response value is calculated and input to a fully connected layer for classification prediction, outputting a class probability of the corresponding character as the confidence score of the candidate box.

[0056] In another embodiment, assuming that a 1024x768 pixel water writing image is processed, 500 candidate boxes are generated after convolutional feature extraction. Through screening, there are 320 candidate boxes with a response value greater than a threshold value of 0.6, and the confidence is output by the fully connected layer, with an average value of 0.82, the highest confidence being 0.97 and the lowest being 0.45. Finally, candidate boxes with a confidence score below 0.6 are discarded, and only 300 valid candidate boxes are retained for subsequent recognition.

[0057] According to the confidence score, potential text regions of the water writing image to be processed are screened and recognized; and the potential text regions are corrected in rotation to output text regions.

[0058] In an embodiment, the confidence score is used to determine that candidate boxes with a confidence score greater than a set threshold value are potential text regions. For these regions, a minimum bounding rectangle method is used to calculate the rotation angle, and an affine transformation is used to complete the rotation correction, so that the text line is aligned with the horizontal axis of the image. After correction, the regularized text regions are output for subsequent character recognition.

[0059] In another embodiment, assuming that among the 300 candidate boxes, there are 240 candidate boxes with a confidence score greater than 0.75, which are determined to be potential text regions. Further detection finds that 60 of these regions are inclined, with an average rotation angle of 8.5° and a maximum rotation angle of 17°. After correction by affine transformation, the inclination angles of all the text regions are compressed to within ±1°, thereby ensuring high geometric consistency of the characters in the subsequent recognition stage.

[0060] Preferably, the rotation correction of the potential text region to output the text region comprises:

[0061] The center point coordinates of the potential text region are used as a rotation reference point to establish an affine transformation matrix; the text direction angle of the potential text region is determined, and the text direction angle is written into the affine transformation matrix as a rotation parameter, while the scaling ratio and the aspect ratio are kept unchanged;

[0062] In an embodiment, a bounding box of the detected potential text region is obtained, a reference point in rotation is obtained by calculating the center position of the bounding box. Then, the text direction of the region is analyzed, for example, using the long side direction of the minimum circumscribed rectangle, to determine the inclination angle of the current text relative to the horizontal line. The angle is written into the affine transformation matrix as a rotation parameter, and the final rotation mapping relationship is constructed under the premise of keeping the scaling ratio and aspect ratio of the original region unchanged, which is used for subsequent text region correction.

[0063] In another embodiment, assuming that the bounding box of the potential text region is located at the upper left corner (100, 200) and the lower right corner (260, 280) of the image coordinates, the center point position is (180, 240). Direction analysis shows that the direction angle of the text region is about 12 degrees. After constructing the affine transformation matrix with the center point as the reference, it can be ensured that the text region will not be stretched or compressed during rotation, and the length-width ratio is stable.

[0064] The original pixel points of the potential text region are mapped according to the affine transformation matrix, and the original pixel points are mapped one by one to the rotated position to output the text region.

[0065] In an embodiment, the affine transformation matrix is used to map the pixels of the potential text region point by point, and the pixel points in the original position are converted to new rotated positions. For the pixel points falling on non-integer coordinates after rotation, an interpolation method is used for compensation to ensure that the corrected text edge is clear and the transition is smooth. The final output is a regular text region image that has been rotated to a standard direction.

[0066] In another embodiment, assuming that the size of the text region is 160x80 pixels, after coordinate mapping, the overall pixel average offset is about 5 pixels, and the maximum offset reaches 12 pixels. Through interpolation compensation, the output image still maintains a resolution range of 160x80 pixels, and the inclination of the text line direction is reduced from 12 degrees to less than 1 degree, ensuring the accuracy of the text region in subsequent segmentation and recognition links.

[0067] Preferably, the step S2 of identifying the water writing text features comprises:

[0068] A preset neural network model is used to identify the water writing text features.

[0069] In an embodiment, the collected water writing text image is preprocessed, including grayscale, normalization and denoising. Then, the processed image is input into a preset neural network model, and the forward propagation process of the model is used to extract features of the potential text region in the image. The model has learned the stroke structure, writing habits and common forms of water writing text during training, so it can effectively identify the main feature points and boundaries of water writing text.

[0070] In another embodiment, assuming the input water writing image size is 256x256 pixels, containing about 30 characters. After preprocessing, the number of noise points is reduced by about 40%, and the character contour edge is clearer. After inputting it into the neural network, the model detects about 500 low-level feature points (such as edges, corners) in the first layer, and about 120 stable high-level structure feature points are screened after deep convolution. These feature points provide a reliable input basis for subsequent convolution layers and classification layers.

[0071] The neural network model comprises an input layer, a plurality of convolution feature extraction layers, a feature fusion layer, and a classification decision layer; the input layer is used to receive a water writing image of a text region; the convolution feature extraction layer is used to extract multi-scale text features from the water writing image; the feature fusion layer is used to up-sample and weight integrate text features of different levels to obtain high-dimensional comprehensive features; and the classification decision layer is used to perform character-by-character feature matching on the high-dimensional comprehensive features.

[0072] In an embodiment, the input layer of the neural network receives a water writing text region image and converts it into a tensor input. The convolution feature extraction layer is composed of multiple convolution and pooling units, which are used to capture the detail features and overall morphology of strokes layer by layer. Subsequently, the feature fusion layer up-samples and weight integrates the feature maps extracted by different convolution layers, so that local detail features and global structure features are uniformly represented. Finally, the classification decision layer inputs the high-dimensional comprehensive features into a fully connected network to complete character-by-character classification and recognition, and outputs the water writing text category corresponding to each character.

[0073] In another embodiment, assuming that the neural network includes 5 convolution extraction layers, 1 feature fusion layer, and 2 classification decision layers. After inputting a water writing image containing 30 characters, the convolution layer outputs feature maps with a total of 64, 128, 256, 512, and 1024 channels, and forms a 2048-dimensional high-dimensional feature vector after feature fusion. The feature vector is distributed character by character in the classification layer, and finally outputs the category prediction of 30 characters, with an average recognition accuracy of more than 92%. In single character classification, the recognition accuracy of characters with complex strokes is about 89%, and the recognition accuracy of characters with simple strokes can reach more than 95%.

[0074] Preferably, the multi-scale text features extracted from the water writing image include:

[0075] The first convolution layer of the neural network model performs a primary convolution operation on the input water writing image to extract low-level edge features; the low-level edge features are input into the second convolution layer to extract stroke features;

[0076] In one embodiment, after obtaining the input Shui script image, it is first normalized to a fixed size (such as 128×128 pixels) and noise is filtered. Subsequently, it is input into the first convolutional layer of the neural network, and the character edge information is extracted through a set convolutional kernel (such as 3×3 size) to obtain a low-level edge feature map. This feature map mainly reflects the outline and basic orientation of the text. On this basis, the low-level edge features are input into the second convolutional layer to further capture local features such as the direction, thickness, and intersection points of the strokes, so as to lay a foundation for the subsequent extraction of shape features.

[0077] In another embodiment, assume that the input Shui script image is a character map of 128×128 pixels, which contains about 15 characters in total. After the operation of the first convolutional layer with 32 convolutional kernels, the output feature map size is 64×64, and about 131,000 low-level edge feature points are obtained; after the operation of the second convolutional layer with 64 convolutional kernels, the feature map is reduced to 32×32, and about 65,0k stroke-related feature points are extracted. The low-level edge features are mainly concentrated in the contour area of the characters, while the stroke features highlight the local structures such as vertical strokes, horizontal strokes, and folding strokes in the characters, providing distinguishable information for subsequent shape analysis.

[0078] The stroke features are input into the third convolutional layer to extract character shape features; the character shape features are input into the fourth convolutional layer to extract high-level semantic features.

[0079] In one embodiment, after the stroke features are output by the second convolutional layer, they are input into the third convolutional layer, and a convolutional kernel with a larger receptive field is used to capture the overall stroke combination, thereby obtaining character shape features that reflect the overall outline and structural layout of the characters. Subsequently, the character shape features are input into the fourth convolutional layer for further abstraction processing to extract high-level features that can represent the semantic units of the Shui script, such as common character radicals or compound stroke patterns, to support subsequent character recognition and phrase construction.

[0080] In another embodiment, assume that the third convolutional layer uses 128 convolutional kernels to perform convolution and pooling operations on the input 32×32 feature map, obtaining a character shape feature map of 16×16 size, with a total of about 32,768 high-level nodes; then it is input into the fourth convolutional layer, with 256 convolutional kernels, and the output feature map is 8×8 size, with a total of 16,384 nodes. It is found in the experiment that the third convolutional layer can distinguish about 90% of the overall character shapes, such as distinguishing "日" and "目" through shape features; the fourth convolutional layer further captures the semantic composite structures in the Shui script on this basis, enabling characters with high similarity to be effectively distinguished, and its classification accuracy is improved by about 7%.

[0081] Preferably, the step S2 utilizes the water writing character features to classify the water writing image characters, and the determination of the character semantic information comprises:

[0082] The water writing character features are used to extract character structure features, sample labeling is performed according to the character structure features, and the character structure of the water writing image is divided to determine the independent stroke area and the combined stroke area;

[0083] In an embodiment, the input water writing character image is preprocessed, including grayscale, binarization and noise point removal, to obtain clear stroke contours. Then, the edge and local morphological features of the character are extracted by using a convolution feature extraction operator, and a connected component analysis method is used to obtain preliminary stroke area division. According to the spatial adjacency between strokes, the strokes are divided into independent stroke areas (such as single horizontal, vertical and dot strokes) and combined stroke areas (such as "cross" or "mouth" shapes formed by the connection of horizontal and vertical strokes). On this basis, sample labeling is completed in a manual or semi-automatic manner to provide a basis for subsequent stroke pattern recording and combination analysis.

[0084] In another embodiment, it is assumed that the size of the input water writing image is 256x256 pixels, and a total of 320 connected stroke segments are detected, of which about 210 are identified as independent stroke areas and 110 are divided into combined stroke areas. When further labeling, in the independent stroke areas, 75 "horizontal" type strokes, 68 "vertical" type strokes, 41 "dot" type strokes and 26 "hook and radical" type strokes are recorded; in the combined stroke areas, 24 "cross" intersection structures, 18 "mouth" frames and 5 "product" upper and lower superimposed structures are identified. The labeling set obtained in this way can provide stable data support for subsequent stroke attribute and character semantic extraction.

[0085] In the independent stroke area, the isolated stroke pattern is recorded; in the combined stroke area, the connection relationship between adjacent strokes is identified; and the stroke attribute label is determined according to the isolated stroke pattern;

[0086] In an embodiment, for the independent stroke area, its geometric morphological features (such as stroke length, width, inclination angle and end point position) are extracted one by one and recorded as isolated stroke patterns. For the combined stroke area, the connection mode (such as intersection, parallel and closed) between adjacent strokes is identified by calculating the Euclidean distance and angle relationship between stroke end points. On this basis, an attribute label is assigned to each isolated stroke, such as "horizontal-short", "vertical-long", "dot-oblique right-down", etc.

[0087] In another embodiment, it is assumed that 200 isolated strokes are collected in the independent stroke area, with a length range of 5 - 40 pixels and an inclination distribution within the range of ±75°. After attribute classification, 82 horizontal strokes (about 60% of which are short horizontal strokes), 70 vertical strokes (about 55% of which are long vertical strokes), 28 dot strokes, and 20 oblique strokes are obtained. For the combined stroke area, 95 groups of stroke connection relationships are detected in total, among which the cross type (such as the "plus" structure) accounts for about 38%, the closed type (such as the "square" structure) accounts for about 22%, the parallel type (such as the "double vertical" structure) accounts for about 15%, and the rest are complex mixed types. This attribute and connection information provide constraint conditions for the subsequent character structure fitting.

[0088] Match the stroke types in the preset Shui script stroke library according to the adjacent stroke connection relationship; use the stroke attribute tags and stroke types to gradually fit the complete character structure and determine the character semantic information.

[0089] In one embodiment, the stroke attributes and connection relationships are matched with the preset Shui script stroke library. Common stroke types and their combination rules are pre-stored in the stroke library. For example, "horizontal + vertical" can form "plus", and "vertical + square" can form "middle". After the matching is completed, use the dynamic fitting algorithm to gradually synthesize the complete character structure, and output the corresponding character semantic information according to the semantic rules of the Shui script characters.

[0090] In another embodiment, it is assumed that the stroke library contains about 50 common stroke types, covering basic units such as horizontal, vertical, left-falling, right-falling, dot, and fold and their common combinations. In the test samples, 120 stroke combinations are identified in total. After gradual fitting, 85 complete characters are formed. Among them, the characters directly formed by independent strokes account for about 65%, and the characters parsed from complex combination areas account for about 35%. Among the finally recognized 85 characters, the semantic matching accuracy rate is about 92%, and typical Shui script symbols such as "sacrifice" and "water" can be completely recognized. This method proves that the fitting strategy combining stroke attributes and library matching has high accuracy and adaptability.

[0091] Preferably, in the independent stroke area, record the isolated stroke forms including:

[0092] In the independent stroke area, determine the starting coordinate and ending coordinate of the stroke; calculate the stroke direction angle according to the starting coordinate and ending coordinate, and determine the stroke curvature;

[0093] In one embodiment, after preprocessing the input Shui script image, first extract the stroke contour lines in the independent stroke area. In this area, determine the starting coordinate of the stroke through the contour endpoint detection method and the ending coordinate In the extracted stroke sequence, each independent stroke has a unique pair of end points. For example, in a test sample, 50 independent strokes were detected, with the start point and end point coordinates ranging from to , respectively. Then, the stroke direction angle was calculated based on the end point coordinates

[0094] In another embodiment, assume that 40 independent strokes were detected in a watercolor image, with the start point coordinates averagely distributed in pixels, and the end point coordinates averagely distributed in pixels. The calculated stroke direction angle ranged from , among which about 22 strokes were straight, and about 18 strokes were curved. The further calculated bending values had a minimum value of 0.02, a maximum value of 0.35, and an average value of about 0.12. It can be seen that there are many strokes with obvious curvature in this sample, and these features can provide criteria for subsequent stroke attribute classification.

[0095] According to the stroke bending value, the end point features were recorded, the pen stroke closure state was identified, the closure gap size was calculated based on the pen stroke closure state, the closure gap direction was identified, and the isolated stroke shape was recorded.

[0096] In an embodiment, based on the obtained stroke geometric features, first, the geometric features of the stroke end points (such as the distance between the end points and the local curvature of the end points) were recorded according to the bending value and the difference between the end point coordinates. When it was detected that the strokes were connected at the beginning and the end and the distance was less than a preset threshold (for example, 5 pixels), it was determined that the pen stroke was in a closed state; otherwise, it was determined to be not closed. For the closed stroke, the closure gap size, that is, the residual distance between the end points, was further calculated, and the closure gap direction was identified through the direction vector of the difference between the end point coordinates. For example, in a group of 30 closed strokes, the gap size was distributed in [1.5, 4.8] pixels, and the gap direction was mainly concentrated in the horizontal direction and the oblique direction. Finally, according to the closure features and the gap parameters, the isolated stroke shape was labeled as “straight closed”, “arc closed”, or “unclosed isolated stroke”.

[0097] In another embodiment, assume that the detected independent strokes are 50 in total, of which 28 are in closed state and 22 are in unclosed state. In the closed strokes, the average gap size is 3.2 pixels, and the direction distribution ratio is about 40% horizontal, 30% vertical, and 30% oblique. In the unclosed strokes, the average distance between endpoints is 15 pixels, and some of the partial isolated strokes have a clear long straight line shape, and others show slight bending. By quantitatively recording the closed gap size and direction, auxiliary feature basis can be provided for subsequent stroke library matching and character structure splicing.

[0098] Preferably, in the combined stroke region, identifying the connection relationship between adjacent strokes includes:

[0099] In the combined stroke region, detecting the endpoint coordinates of the combined strokes, calculating the endpoint distance between the endpoint coordinates, and when the endpoint distance is lower than a preset distance threshold, recording the connection endpoint position and determining the connection mode of the endpoint position as end-to-end connection;

[0100] In an embodiment, after preprocessing the combined stroke region in the input water writing image, the endpoint coordinates of each stroke are first extracted. Then, the Euclidean distance between the endpoints of adjacent strokes is calculated. When the distance is less than a preset threshold (such as 5 pixels), the endpoint pair is recorded as end-to-end connection, and the connection mode is marked. For example, in a certain image, 20 pairs of combined strokes are detected, of which 12 pairs satisfy the end-to-end connection condition, and the endpoint distance is between [2.1, 4.8] pixels. Recording the coordinate information of these endpoints can provide preliminary connection basis for subsequent stroke library matching and character structure splicing.

[0101] In another embodiment, assume that 25 pairs of adjacent stroke endpoints are detected in the combined stroke region, and the calculated endpoint distance range is [1.5, 6.0] pixels, of which 18 pairs are less than the threshold of 5 pixels and are marked as end-to-end connection, and the remaining 7 pairs do not satisfy the condition and are not marked as connection. In this way, the endpoint connection features of the combined strokes can be quantified, and reference can be provided for the next step of cross or surrounding relationship judgment.

[0102] If two strokes of the combined strokes have a crossing point at a non-endpoint, record the coordinates and crossing angle of the crossing point, and mark it as cross connection;

[0103] In an embodiment, for the strokes in the combined stroke region that have not been connected end-to-end, the intersection of the stroke lines at non-endpoints is analyzed. When a crossing point is detected, its coordinates (x, y) and the angle α of the stroke intersection are recorded. If the crossing angle is within a reasonable range (such as 20°~160°), the stroke connection mode is marked as cross connection. For example, in a water writing image, 10 groups of cross strokes are detected, the crossing point coordinates are in the pixel interval , and the crossing angle range is .

[0104] In another embodiment, it is assumed that 12 groups of intersection points are detected in the combined stroke, with the coordinate range of pixels, and the intersection angle distribution is , in which 8 groups of intersection angles are in the range of 30°~150°, marked as intersection connection, and the remaining 4 groups are not marked due to too small or too large intersection angles. By recording the intersection point features, the spatial topological relationship between strokes can be further constructed to provide constraint information for character structure recovery.

[0105] Detect whether the strokes in the combined stroke form an inside-outside surrounding relationship, identify the closed region, and mark it as a surrounding connection;

[0106] In an embodiment, by analyzing the topological structure of the combined stroke region, it is determined whether the strokes form a closed region (inside-outside surrounding relationship). For the closed region, record its boundary coordinates and the sequence of strokes contained, and mark it as a surrounding connection. For example, in a water writing image, 3 closed regions are detected, each of which is composed of 2~4 strokes, and the boundary coordinates are in the range of pixels.

[0107] In another embodiment, it is assumed that 4 closed regions are detected in the combined stroke region, and the number of strokes in the closed regions is 2, 3, 3, and 5 respectively, and the closed boundary coordinate range is pixels. By marking the surrounding connection, the part of the combined stroke that forms a complete character structure can be determined, providing topological constraints for the step-by-step combination of strokes.

[0108] The end-to-end connection, intersection connection and surrounding connection are used as the connection relationship between adjacent strokes.

[0109] In an embodiment, the end-to-end connection, intersection connection and surrounding connection are integrated to construct the complete adjacent connection relationship of the combined stroke. The connection relationship can be represented as a graph structure, in which the strokes are nodes and the connection types are edge attributes, which are used for subsequent stroke library matching and character structure combination.

[0110] In another embodiment, it is assumed that in a certain water writing image, there are 18 pairs of end-to-end connections, 8 pairs of intersection connections, and 4 surrounding connections, and the constructed adjacent stroke connection relationship graph has 30 nodes and 30 edges (part of the edges are multiple edges), and the types of the edges include "end-to-end", "intersection" and "surrounding". Through this graph structure, accurate stroke connection information can be provided for subsequent step-by-step character structure recovery.

[0111] Preferably, the step S3 of reconstructing the character order based on the character semantic information comprises:

[0112] Extract the character position index based on the character semantic information, and perform spatial coordinate sorting to obtain the sorting result; decompose the strokes in the sorting result to obtain the character stroke set;

[0113] In one embodiment, semantic analysis is performed on the characters in the input text or image to extract the spatial position index (such as line number, column number, or pixel coordinates) of each character. Then, the characters are sorted spatially according to the character position index, and the sorting result is generated from top to bottom and from left to right. For each sorted character region, the strokes inside the character are decomposed to obtain the stroke set of the character. Taking the character "水" as an example, the extracted character position index is , and the decomposed stroke set is {dot, horizontal, left-falling stroke, right-falling stroke}. This stroke set can be used for subsequent stroke order analysis and reconstruction.

[0114] In another embodiment, assume that the input document contains 5 characters, and the extracted character position indexes are respectively . After sorting by spatial coordinates, the sorting sequence obtained is the 1st, 2nd, 3rd, 4th, and 5th characters. The number of strokes decomposed for each character region is [4, 5, 3, 6, 4] respectively, forming the corresponding character stroke set. Through this sorting and decomposition, a complete stroke set can be established for each character, providing a preliminary data basis for order reconstruction.

[0115] Align the character stroke set with the preset character semantic labels, and correct the stroke order of the character stroke set according to the standard stroke order in the character semantic labels to reconstruct the character order.

[0116] In one embodiment, the character stroke set is matched with the previously established character semantic label library. When matching, features such as stroke type, start and end coordinates, stroke length, and relative position relationship are considered. After alignment, according to the standard stroke order defined in the character semantic labels, the stroke order of the character stroke set is corrected. For example, for the character "水", the standard stroke order is {dot, horizontal, left-falling stroke, right-falling stroke}. If the original stroke set order is {horizontal, dot, left-falling stroke, right-falling stroke}, then the correct order {dot, horizontal, left-falling stroke, right-falling stroke} is obtained after correction. After this step, the character stroke set not only maintains integrity but also conforms to semantic and writing norms, providing a basis for subsequent character reconstruction.

[0117] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the application document are intended to be encompassed within the present invention.

[0118] The foregoing is considered as illustrative only of the principles of the application. Numerous modifications and changes will readily occur to those skilled in the art, and it is intended to embrace all such modifications and changes that fall within the scope of the application. Accordingly, the application is not to be restricted in scope to the specific embodiments disclosed herein but is to be accorded the full scope that the principles and novel features request appropriately granted.

Claims

1. A water-based character recognition method based on target detection, characterized in that, The method comprises the following steps: Step S1: obtaining a water book image and performing image preprocessing to obtain a to-be-processed water book image; Step S2: correcting the geometric deformation of the text based on the text region, and identifying water book text features, wherein identifying water book text features comprises: using a preset neural network model to identify water book text features; The neural network model comprises an input layer, a plurality of convolutional feature extraction layers, a feature fusion layer, and a classification decision layer; the input layer is used to receive the water book image of the text region; the convolutional feature extraction layer is used to extract multi-scale text features from the water book image; the feature fusion layer integrates the text features of different levels by upsampling and weighting to obtain high-dimensional comprehensive features; the classification decision layer performs character-by-character feature matching on the high-dimensional comprehensive features; using water book text features to classify characters in the water book image and determining character semantic information comprises: extracting text structure features using water book text features, annotating samples according to text structure features, and dividing the text structure of the water book image to determine independent stroke regions and combined stroke regions; in the independent stroke region, record the isolated stroke form; in the combined stroke region, identify the connection relationship between adjacent strokes; determine the stroke attribute label according to the isolated stroke form; match the stroke type of the preset water book stroke library according to the adjacent stroke connection relationship; use the stroke attribute label and the stroke type to piece together the complete character structure step by step to determine the character semantic information; Step S3: reconstructing the character order based on the character semantic information; identifying the water book phrase structure of the water book image according to the character order to generate a complete water book recognition sample, wherein reconstructing the character order based on the character semantic information in step S3 comprises: extracting character position indexes based on character semantic information and performing spatial coordinate sorting to obtain a sorting result; decompose the strokes in the sorting result to obtain a character stroke set; align the character stroke set with the preset character semantic label, and correct the stroke order of the character stroke set according to the standard stroke order in the character semantic label to reconstruct the character order; Step S4: calculating the water book confidence based on the complete water book recognition sample, and correcting the wrong water book character using the water book confidence, and storing the corrected water book character. In step S1, the text region of the to-be-processed water book image comprises: 2.The target detection based water script character recognition method according to claim 1, characterized in that, using a preset convolutional feature extractor to generate a text candidate box; calculating the response value of each text candidate box and performing character confidence prediction to output the confidence of each text candidate box; screening and identifying the potential text region of the to-be-processed water book image according to the confidence; performing rotation correction on the potential text region to output the text region. The rotation correction on the potential text region to output the text region comprises: 3.The target detection based water script character recognition method according to claim 2, characterized in that, taking the center point coordinates of the potential text region as the rotation reference point to establish an affine transformation matrix; determining the text direction angle of the potential text region, writing the text direction angle as the rotation parameter into the affine transformation matrix, and keeping the scaling ratio and the aspect ratio unchanged; mapping the original pixel points of the potential text region to the rotated positions one by one according to the affine transformation matrix to output the text region. The multi-scale text features extracted from the water book image comprise: 4.The target detection based water script character recognition method according to claim 1, characterized in that, ​ The first convolutional layer of the neural network model performs a primary convolution operation on the input water book image to extract low-level edge features; the low-level edge features are input to the second convolutional layer to extract stroke features; The stroke features are input to the third convolutional layer to extract character shape features; and the character shape features are input to the fourth convolutional layer to extract high-level semantic features. 5.The target detection based water script character recognition method according to claim 1, characterized in that, In the independent stroke area, the isolated stroke shape is recorded, including: In the independent stroke area, the starting point coordinates and the ending point coordinates of the stroke are determined; the stroke direction angle is calculated according to the starting point coordinates and the ending point coordinates to determine the stroke curvature; According to the stroke curvature, the end point feature is recorded, the state of the pen stroke closure is identified; the size of the closure gap is calculated based on the state of the pen stroke closure, the direction of the closure gap is identified, and the isolated stroke shape is recorded. 6.The target detection based water script character recognition method according to claim 1, wherein, In the combined stroke area, the adjacent stroke connection relationship is identified, including: In the combined stroke area, the end point coordinates of the combined stroke are detected, the end point distance between the end point coordinates is calculated, when the end point distance is lower than the preset distance threshold, the connection end point position is recorded, and the connection mode of the end point position is determined as end-to-end connection; If there is a cross point between the two strokes of the combined stroke at a non-end point, the coordinates and the cross angle of the cross point are recorded, and are marked as cross connection; The inside and outside surrounding relationship between the strokes of the combined stroke is detected, the closed area is identified, and is marked as surrounding connection; The end-to-end connection, the cross connection and the surrounding connection are taken as the adjacent stroke connection relationship.

7. A water-based script recognition system based on object detection, characterized by, The target detection-based water book character recognition system for executing the target detection-based water book character recognition method of claim 1 comprises: An image preprocessing module, configured to acquire a water book image, and perform image preprocessing to obtain a water book image to be processed; and identify a character region of the water book image to be processed; A character classification module, configured to correct character geometric deformation based on the character region, identify water book character features, classify characters of the water book image based on the water book character features, and determine character semantic information; A character order reconstruction module, configured to reconstruct a character order based on the character semantic information, identify a water book phrase structure of the water book image according to the character order, and generate a complete water book recognition sample; A confidence calculation module, configured to calculate a water book confidence based on the complete water book recognition sample, correct an incorrect water book character using the water book confidence, and store the corrected water book character.

Images