Multi-modal intelligent dictation and correction system and method based on dynamic calibration

Abstract

The present application relates to the field of character image recognition, and more particularly to a multi-modal intelligent dictation and correction system and method based on dynamic calibration, which acquires handwriting images based on voice feedback through a set acquisition module and performs character recognition, a feature splitting module constructs a feature observation group and labels structural units, a layout analysis module sets fuzzy recognition labels based on the prior probability of spatial distribution characteristics by sliding window frame selection and restructuring, an identification module determines the difference type by comparing contour features through regional mapping, and a calibration module restores the context and dynamically calibrates by eliminating fuzzy blocks, mapping and comparing with a sample database, and replacing contour features. The present application improves the recognition accuracy of complex handwriting such as messy handwriting and stroke adhesion, and on this basis, the local image content is calibrated through a replacement mechanism based on context restoration, thereby improving the accuracy and reliability of the correction process.

Description

Technical Field

[0001] This invention relates to the field of text image recognition, and in particular to a multimodal intelligent dictation and correction system and method based on dynamic calibration. Background Technology

[0002] In current language teaching and learning assessment practices, dictation, as a fundamental and important method of skills training, is widely used in classroom teaching and after-class exercises. Traditional grading mainly relies on manual work by teachers, which suffers from problems such as low grading efficiency, long feedback cycles, and difficulty in providing personalized guidance. Currently, assisted grading systems based on optical character recognition or speech recognition have emerged.

[0003] For example, Chinese Patent Publication No. CN113486786A discloses an automatic homework correction system, which relates to the field of homework correction technology. The system includes the following steps: (1) Teacher-side setting operation: The teacher uploads the homework image and sets various parameters. Uploading the homework image mainly refers to taking pictures of the homework page by page and uploading them to the system. After receiving the uploaded homework sample, the system enters the parameter setting module, where the teacher sets the specific parameters; (2) Student-side homework upload operation: Students upload the completed homework image. They only need to upload clear, complete images with neat and upright text according to the page number of the current homework; (3) System homework correction operation: The system corrects the homework images uploaded by students by comparing them with the homework sample. This automatic homework correction system adopts electronic and image processing, which allows teachers to correct homework through mobile terminals. This method solves the problem of students answering questions on computers and avoids the drawbacks of paperless homework.

[0004] However, the following problems still exist in the existing technology:

[0005] 1. Existing technologies do not take into account individual differences in students' handwriting habits and cannot dynamically calibrate according to writing characteristics, resulting in low recognition accuracy for complex situations such as illegible handwriting and strokes sticking together.

[0006] 2. In the existing technology, no replacement mechanism based on context restoration is considered, and dynamic calibration of ambiguous written content cannot be achieved. Summary of the Invention

[0007] To address this, the present invention provides a multimodal intelligent dictation and correction system and method based on dynamic calibration, which overcomes the problems in the prior art that do not consider individual differences in students' handwriting habits, cannot perform dynamic calibration according to writing characteristics, resulting in low recognition accuracy for complex situations such as illegible handwriting and strokes sticking together, and do not consider a replacement mechanism based on context restoration, thus failing to achieve dynamic calibration for fuzzy writing content.

[0008] To achieve the above objectives, the present invention provides a multimodal intelligent dictation and correction system based on dynamic calibration, comprising:

[0009] The acquisition module is used to acquire the pen image data based on the user's output voice feedback, and to perform character recognition on the pen image data.

[0010] A feature segmentation module, which is connected to the acquisition module, is used to construct several feature observation groups based on the text recognition results of the pen image data and to label several structural units in the feature observation groups.

[0011] The layout analysis module, which is connected to the feature splitting module, is used to construct a sliding window to locally select structural units in the layout observation group in order to identify several recombination structures with a tendency to combine. Based on the prior probability of the spatial distribution features of the recombination structure in the corpus database, a fuzzy identification label is set for the recombination structure.

[0012] The recognition module, which is connected to the layout analysis module, is used to read the contour features in the reconstructed structure corresponding to the fuzzy recognition label, divide the local image in the reconstructed structure corresponding to the fuzzy recognition label into regions, compare the contour features in each local image with the corpus in the sample database, and determine the difference type of the local image based on the region mapping comparison result.

[0013] The calibration module, connected to the recognition module, is used to remove unmatched regions in local images of local difference types, compare the remaining regions with the corpus in the sample database for region mapping, extract a predetermined number of corpus data based on sample similarity to replace the contour features in the local images one by one to determine the calibration contour features, and identify the calibration contour features.

[0014] Furthermore, the feature segmentation module is used to construct several feature observation groups based on adjacent contour features in the pen image data, and the labeled structural units in the feature observation groups include,

[0015] Used to obtain character recognition results to identify unrecognized local images;

[0016] Used to identify horizontally adjacent local images as feature observation groups;

[0017] Used to perform pixel clustering on local images, and the resulting pixel clusters are labeled as structural units.

[0018] Furthermore, the layout analysis module performs local bounding selection on the structural units in the layout observation group to identify several recombination structures with a tendency to combine, including,

[0019] Used to construct a sliding window based on the size features of the recognized characters;

[0020] Used to move the sliding window horizontally to select different areas of the feature observation group and determine several structural units within the selected area as the reorganized structure.

[0021] Furthermore, the layout analysis module is used to identify several recombination structures with a tendency to combine, and to set fuzzy identification labels for the recombination structures based on the prior probability of the spatial distribution features corresponding to the recombination structures in the corpus database, including...

[0022] Determine the spatial distribution characteristics of several recombinant structures, and determine the prior probability of the spatial distribution characteristics in the corpus database;

[0023] If the prior probability corresponding to the recombined structure is greater than or equal to the predetermined prior probability threshold, then no fuzzy recognition label is set for the recombined structure.

[0024] If the prior probability corresponding to the recombined structure is less than the predetermined prior probability threshold, then a fuzzy recognition label is set for the recombined structure.

[0025] Furthermore, the layout analysis module is also used to determine the spatial distribution characteristics of the reorganized structure, including,

[0026] Used to divide the sliding window into several spatial regions and determine the mapping relationship between the spatial regions and the serial numbers;

[0027] Used to determine the spatial region where each structural unit is located in the reorganized structure, and to determine the sequence numbering based on the mapping relationship;

[0028] This is used to determine the spatial distribution characteristics of the sequence number arrangement.

[0029] Furthermore, the identification module determines the type of difference based on the region mapping comparison results, including:

[0030] Perform region mapping comparison to determine whether each region matches the corpus in the sample database;

[0031] If the local matching condition is met, it is determined to be a local difference type;

[0032] If the local matching condition is not met, it is determined to be a non-local difference class;

[0033] The local matching condition is that there exists a single region that does not match, while all remaining regions match.

[0034] Furthermore, the identification module is used for region mapping comparison, including:

[0035] The contour features are divided into several regions, and the corpus in the sample database is divided into several corresponding corpus regions.

[0036] The similarity between the region and the corresponding corpus region is compared to determine the contour similarity.

[0037] If the contour similarity of the regions is greater than a preset threshold, then the regions are determined to match.

[0038] Furthermore, the calibration module is used to perform region mapping comparison between the remaining region and the corpus in the sample database, and extract a predetermined number of corpora based on sample similarity to replace the contour features in the local image one by one, including,

[0039] The remaining regions are compared with the corresponding regions in the corpus of the sample database to obtain the sample similarity.

[0040] The corpora are sorted from largest to smallest based on sample similarity, and a predetermined number of corpora are selected as candidate corpora in descending order.

[0041] The candidate corpus is used sequentially to replace the contour features in the local image.

[0042] Furthermore, the calibration module is used to determine calibration profile features, including:

[0043] After the replacement of candidate texts is completed, the texts are identified, and contextual understanding is performed based on the texts to determine the semantic matching degree.

[0044] The candidate corpus with the highest semantic matching degree is selected as the calibration contour feature.

[0045] On the other hand, a method for dynamic calibration-based multimodal intelligent dictation and correction applied to a dynamic calibration-based multimodal intelligent dictation and correction system is also provided, including:

[0046] Acquire pen image data based on user input voice feedback, and perform text recognition on the pen image data;

[0047] Based on the character recognition results of the pen image data, several feature observation groups are constructed, and several structural units in the feature observation groups are labeled.

[0048] A sliding window is constructed to locally select structural units in the layout observation group to identify several recombination structures with a tendency to combine. Fuzzy identification labels are set for the recombination structures based on the prior probability of the spatial distribution features of the recombination structures in the corpus database.

[0049] Read the contour features in the reconstructed structure corresponding to the fuzzy recognition label, divide the local image in the reconstructed structure corresponding to the fuzzy recognition label into regions, compare the contour features in each local image with the corpus in the sample database, and determine the difference type based on the region mapping comparison result.

[0050] The corpus samples required for context reconstruction are determined based on the difference type, including:

[0051] Blurred areas in the local image are removed, and the remaining areas are compared with the corpus in the sample database. Based on the sample similarity, a predetermined number of corpus data are extracted to replace the contour features in the local image one by one to determine the calibration contour features. The calibration contour features are then identified.

[0052] Compared with existing technologies, this invention acquires handwriting images based on voice feedback through an acquisition module and performs character recognition; a feature segmentation module constructs feature observation groups and labels structural units; a layout analysis module reconstructs structures by selecting and recombining them through a sliding window and sets fuzzy recognition labels based on prior probabilities of spatial distribution features; a recognition module compares contour features through region mapping to determine the type of difference; and a calibration module achieves context restoration and dynamic calibration by removing fuzzy blocks, comparing them with a sample database, and replacing contour features. This invention improves the recognition accuracy for complex handwriting such as illegible handwriting and overlapping strokes, and further enhances the accuracy and reliability of the correction process by calibrating local image content through a context-based replacement mechanism.

[0053] In particular, this invention considers a feature decomposition mechanism based on structural units and feature observation groups. In practice, the continuity and individual differences in handwriting pose challenges to feature extraction, and traditional methods struggle to effectively separate and represent complex strokes. This invention constructs feature observation groups based on adjacent contour features in the stroke image data through a feature decomposition module, and processes local images using a pixel clustering algorithm, labeling the resulting pixel clusters as structural units. This achieves a systematic decomposition and standardized representation of complex handwriting features. This mechanism not only accurately captures the detailed features of strokes but also maintains the spatial correlation between features, laying the foundation for subsequent layout analysis and recognition calibration, and improving the system's adaptability to various writing styles.

[0054] In particular, this invention considers a layout analysis mechanism based on a sliding window and prior probabilities. In reality, the spatial distribution characteristics of handwritten text are complex and varied, making it difficult for traditional methods to effectively utilize layout information for recognition. This invention constructs a sliding window through a layout analysis module to locally select structural units in the feature observation group. After determining the recombined structure, fuzzy recognition labels are set based on its prior probabilities in the corpus database, thereby achieving quantitative analysis and intelligent recognition of the spatial distribution characteristics of text.

[0055] In particular, this invention considers a dynamic calibration mechanism based on region mapping comparison and blurred block processing. In practice, handwritten images often suffer from localized smudges, blurriness, or missing features, affecting recognition accuracy. Traditional correction methods often struggle to achieve accurate repair while maintaining semantic coherence. This invention divides contour features into several regions through a recognition module, performing region mapping comparison with corpora in a sample database to determine the type of difference. The calibration module then removes blurred blocks, performs multiple rounds of region mapping comparison based on the remaining regions, and replaces contour features one by one using candidate corpora ranked by sample similarity, achieving accurate calibration of locally blurred writing. This hierarchical processing mechanism ensures both the accuracy of the correction process and semantic coherence, solving the problem of recognition errors caused by missing or interfered local features.

[0056] In particular, this invention considers a contour feature replacement mechanism based on sample similarity and semantic matching. In practice, a single feature comparison is insufficient to guarantee the accuracy of the replacement result. This invention uses a calibration module to perform region mapping comparison between the corpus in the sample database and the remaining regions to obtain sample similarity. Candidate corpora are selected for contour feature replacement based on similarity ranking. Then, the calibration contour features are determined through semantic matching calculation. This constructs a synergistic mechanism between character shape comparison and semantic analysis, ensuring the reliability and accuracy of the correction result. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the structural connection of the multimodal intelligent dictation and correction system based on dynamic calibration according to an embodiment of the present invention;

[0058] Figure 2 A logic block diagram for setting fuzzy identification tags according to an embodiment of the invention;

[0059] Figure 3 A logic block diagram for determining the difference type in an embodiment of the invention;

[0060] Figure 4 This is a logic block diagram for determining region matching in an embodiment of the invention. Detailed Implementation

[0061] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0062] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0063] It should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0064] Please see Figure 1 As shown, Figure 1 This is a schematic diagram of the structural connection of the multimodal intelligent dictation and correction system based on dynamic calibration according to an embodiment of the present invention. The multimodal intelligent dictation and correction system based on dynamic calibration of the present invention includes:

[0065] The acquisition module is used to acquire the pen image data based on the user's output voice feedback, and to perform character recognition on the pen image data.

[0066] A feature segmentation module, which is connected to the acquisition module, is used to construct several feature observation groups based on the text recognition results of the pen image data and to label several structural units in the feature observation groups.

[0067] The layout analysis module, which is connected to the feature splitting module, is used to construct a sliding window to locally select structural units in the layout observation group in order to identify several recombination structures with a tendency to combine. Based on the prior probability of the spatial distribution features of the recombination structure in the corpus database, a fuzzy identification label is set for the recombination structure.

[0068] The recognition module, which is connected to the layout analysis module, is used to read the contour features in the reconstructed structure corresponding to the fuzzy recognition label, divide the local image in the reconstructed structure corresponding to the fuzzy recognition label into regions, compare the contour features in each local image with the corpus in the sample database, and determine the difference type of the local image based on the region mapping comparison result.

[0069] The calibration module, connected to the recognition module, is used to remove unmatched regions in local images of local difference types, compare the remaining regions with the corpus in the sample database for region mapping, extract a predetermined number of corpus data based on sample similarity to replace the contour features in the local images one by one to determine the calibration contour features, and identify the calibration contour features.

[0070] Specifically, the output speech can be dictation speech, and students can write based on the output speech. The pen image data containing the students' handwriting can be collected through a user application.

[0071] There are no restrictions on the structure of the acquisition module, feature segmentation module, layout analysis module, recognition module, and calibration module. They can be composed of logic components, including field-programmable processors, computers, or microprocessors in computers.

[0072] Specifically, there are no restrictions on the method of text recognition for image data. Existing open-source text recognition models can be used, or an image processing model that can recognize text can be trained independently. This will not be elaborated further.

[0073] Specifically, the feature segmentation module is used to construct several feature observation groups based on adjacent contour features in the pen image data, and the labeled structural units in the feature observation groups include,

[0074] Used to obtain character recognition results to identify unrecognized local images;

[0075] Used to identify horizontally adjacent local images as feature observation groups;

[0076] Used to perform pixel clustering on local images, and the resulting pixel clusters are labeled as structural units.

[0077] In practice, the local image is an image containing unrecognized characters. The purpose of pixel clustering is to identify the various parts of the contour in the local image. For example, a single stroke of a Chinese character or a continuously written radical will be classified as a cluster, and the corresponding single cluster will be labeled as a single structural unit.

[0078] Specifically, the layout analysis module performs local bounding selection on the structural units in the layout observation group to identify several recombination structures with a tendency to combine, including,

[0079] Used to construct a sliding window based on the size features of the recognized characters;

[0080] Used to move the sliding window horizontally to select different areas of the feature observation group and determine several structural units within the selected area as the reorganized structure.

[0081] Specifically, the layout analysis module is configured to construct a sliding window based on the average size characteristics of the recognized characters. The average height and average width are obtained by statistical analysis of the height and width of the recognized characters. Based on this, the size of the sliding window is determined by enlarging it according to a preset magnification ratio.

[0082] Understandably, the preset magnification ratio should not be too large to avoid the sliding window covering too many irrelevant structural units, introducing unnecessary feature interference, and affecting the effective identification and analysis of the target reconstructed structure. In practice, the preset magnification ratio is typically set within the range of [115%, 130%].

[0083] Please see Figure 2 As shown, Figure 2 This is a logic block diagram of setting fuzzy identification labels according to an embodiment of the invention. The layout analysis module is used to determine several recombination structures with a tendency to combine. Setting fuzzy identification labels for the recombination structures based on the prior probability of the spatial distribution features corresponding to the recombination structures in the corpus database includes...

[0084] Determine the spatial distribution characteristics of several recombinant structures, and determine the prior probability of the spatial distribution characteristics in the corpus database;

[0085] If the prior probability corresponding to the recombined structure is greater than or equal to the predetermined prior probability threshold, then no fuzzy recognition label is set for the recombined structure.

[0086] If the prior probability corresponding to the recombined structure is less than the predetermined prior probability threshold, then a fuzzy recognition label is set for the recombined structure.

[0087] Specifically, the predetermined prior probability threshold is set by those skilled in the art based on the statistical characteristics of the occurrence probability of commonly used characters. In practice, the spatial distribution characteristics of commonly used characters in the modern Chinese character list are determined, and the average occurrence probability of each spatial distribution characteristic in the Chinese dictionary is determined. The product of the average occurrence probability and a correction coefficient ranging from 0.8 to 0.95 is used as the prior probability threshold. This setting method ensures that the threshold effectively covers most common writing structures while retaining appropriate adjustment space through the correction coefficient, allowing the system to fine-tune the recognition sensitivity according to the needs of actual application scenarios. In practice, the correction coefficient is preferably 0.85.

[0088] Specifically, the layout analysis module is also used to determine the spatial distribution characteristics of the reorganized structure, including,

[0089] Used to divide the sliding window into several spatial regions and determine the mapping relationship between the spatial regions and the serial numbers;

[0090] Used to determine the spatial region where each structural unit is located in the reorganized structure, and to determine the sequence numbering based on the mapping relationship;

[0091] This is used to determine the spatial distribution characteristics of the sequence number arrangement.

[0092] In implementation, the sliding window is divided into grids to obtain several spatial regions, each of which corresponds to a serial number. Preferably, the sliding window is divided into 16 spatial regions, each corresponding to a serial number, which can be the number 1-16.

[0093] During implementation, when structural units exist in a spatial region, the corresponding serial number of the spatial region is recorded; when they do not exist, the serial number is not recorded. The serial numbers are arranged in ascending order to obtain the serial number arrangement, which will not be elaborated further.

[0094] It is understandable that spatial distribution features reflect the distribution of various parts in Chinese characters. For blocks with messy layouts or missing features, the spatial distribution features will change and deviate from the norm, thus making the prior probability of the spatial distribution features in the corpus database low.

[0095] Please see Figure 3 As shown, Figure 3 This is a logic block diagram illustrating the determination of the difference type according to an embodiment of the invention. The identification module determines the difference type based on the region mapping comparison result, including:

[0096] Perform region mapping comparison to determine whether each region matches the corpus in the sample database;

[0097] If the local matching condition is met, it is determined to be a local difference type;

[0098] If the local matching condition is not met, it is determined to be a non-local difference class;

[0099] The local matching condition is that there exists a single region that does not match, while all remaining regions match.

[0100] Please see Figure 4 As shown, Figure 4 This is a logic block diagram for determining region matching according to an embodiment of the invention. The identification module is used to perform region mapping comparison, including:

[0101] The contour features are divided into several regions, and the corpus in the sample database is divided into several corresponding corpus regions.

[0102] The similarity between the region and the corresponding corpus region is compared to determine the contour similarity.

[0103] If the contour similarity of the regions is greater than a preset threshold, then the regions are determined to match.

[0104] Specifically, the purpose of setting a preset threshold is to characterize the blocks that are similar to the contour features and the corpus. The preset threshold is pre-set. Several correctly identified contour features are manually selected, and the contour similarity between the contour features and the corresponding corpus is determined. The mean contour similarity is calculated, and the 85th percentile of the mean contour similarity is set as the preset threshold.

[0105] The corpus in the sample database consists of templates for single Chinese characters.

[0106] Specifically, the calibration module is used to perform region mapping comparison between the remaining region and the corpus in the sample database, and to extract a predetermined number of corpus data based on sample similarity to replace the contour features in the local image one by one, including,

[0107] The remaining regions are compared with the corresponding regions in the corpus of the sample database to obtain the sample similarity.

[0108] The corpora are sorted from largest to smallest based on sample similarity, and a predetermined number of corpora are selected as candidate corpora in descending order.

[0109] The candidate corpus is used sequentially to replace the contour features in the local image.

[0110] Specifically, the preset number should not be too large in order to reduce the number of times context understanding is required later. In practice, the preset number is selected within the range [3, 5], preferably 4.

[0111] Specifically, sample similarity is the contour similarity when the remaining region is compared with the corresponding region in the corpus.

[0112] Specifically, the calibration module is used to determine calibration profile features, including:

[0113] After the replacement of candidate texts is completed, the texts are identified, and contextual understanding is performed based on the texts to determine the semantic matching degree.

[0114] The candidate corpus with the highest semantic matching degree is selected as the calibration contour feature.

[0115] Specifically, in context understanding, the semantic matching degree is calculated based on the overall semantic match between the text corresponding to the replaced candidate text and the already recognized text.

[0116] There are no restrictions on the method for calculating semantic matching degree. Word segmentation can be performed to determine a number of keywords. After the keywords are vectorized, the cosine similarity between the keywords is calculated, and the mean cosine similarity is determined as the semantic matching degree.

[0117] When vectorizing the model, existing open-source models can be used, such as the BERT context-aware model, which can obtain the vector representation of keywords in the context. Of course, other forms can also be used, which will not be elaborated here.

[0118] Specifically, after recognizing all the text, the system can correct the correspondence between the text and the output speech, which will not be elaborated further.

[0119] This embodiment also provides a dynamic calibration-based multimodal intelligent dictation and correction method applied to a dynamic calibration-based multimodal intelligent dictation and correction system, including:

[0120] Step S1: Obtain the pen image data based on the user's output voice feedback, and perform text recognition on the pen image data;

[0121] Step S2: Construct several feature observation groups based on the text recognition results of the pen image data, and label several structural units in the feature observation groups;

[0122] Step S3: Construct a sliding window to locally select structural units in the layout observation group to identify several recombination structures with a tendency to combine. Based on the prior probability of the spatial distribution features of the recombination structures in the corpus database, set fuzzy recognition labels for the recombination structures.

[0123] Step S4: Read the contour features in the reconstructed structure corresponding to the fuzzy recognition label, divide the local image in the reconstructed structure corresponding to the fuzzy recognition label into regions, compare the contour features in each local image with the corpus in the sample database for region mapping, and determine the difference type of the local image based on the region mapping comparison result.

[0124] Step S5: Remove unmatched regions in local images of local difference type, compare the remaining regions with the corpus in the sample database for region mapping, extract a predetermined number of corpus based on sample similarity, replace the contour features in the local images one by one to determine the calibration contour features, and identify the calibration contour features.

[0125] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A multimodal intelligent dictation and correction system based on dynamic calibration, characterized in that, include: The acquisition module is used to acquire the pen image data based on the user's output voice feedback, and to perform character recognition on the pen image data. A feature segmentation module, which is connected to the acquisition module, is used to construct several feature observation groups based on the text recognition results of the pen image data and to label several structural units in the feature observation groups. The layout analysis module, which is connected to the feature splitting module, is used to construct a sliding window to locally select structural units in the layout observation group in order to identify several recombination structures with a tendency to combine. Based on the prior probability of the spatial distribution features of the recombination structure in the corpus database, a fuzzy identification label is set for the recombination structure. The recognition module, which is connected to the layout analysis module, is used to read the contour features in the reconstructed structure corresponding to the fuzzy recognition label, divide the local image in the reconstructed structure corresponding to the fuzzy recognition label into regions, compare the contour features in each local image with the corpus in the sample database, and determine the difference type of the local image based on the region mapping comparison result. A calibration module, connected to the recognition module, is used to remove unmatched regions in local images of local difference types, perform region mapping comparison between the remaining regions and the corpus in the sample database, extract a predetermined number of corpus data based on sample similarity, replace the contour features in the local images one by one to determine the calibration contour features, and identify the calibration contour features; The feature segmentation module is used to construct several feature observation groups based on adjacent contour features in the pen image data. The labeled structural units within the feature observation groups include... Used to obtain character recognition results to identify unrecognized local images; Used to identify horizontally adjacent local images as feature observation groups; Used to perform pixel clustering on local images, and the resulting pixel clusters are labeled as structural units.

2. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 1, characterized in that, The layout analysis module performs local bounding selection on the structural units in the layout observation group to identify several recombination structures with a tendency to combine, including, Used to construct a sliding window based on the size features of the recognized characters; Used to move the sliding window horizontally to select different areas of the feature observation group and determine several structural units within the selected area as the reorganized structure.

3. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 2, characterized in that, The layout analysis module is used to identify several recombination structures with a tendency to combine. Based on the prior probability of the spatial distribution features of the recombination structures in the corpus database, fuzzy identification labels are set for the recombination structures, including... Determine the spatial distribution characteristics of several recombinant structures, and determine the prior probability of the spatial distribution characteristics in the corpus database; If the prior probability corresponding to the recombined structure is greater than or equal to the predetermined prior probability threshold, then no fuzzy recognition label is set for the recombined structure. If the prior probability corresponding to the recombined structure is less than the predetermined prior probability threshold, then a fuzzy recognition label is set for the recombined structure.

4. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 3, characterized in that, The layout analysis module is also used to determine the spatial distribution characteristics of the recombined structure. include, Used to divide the sliding window into several spatial regions and determine the mapping relationship between the spatial regions and the serial numbers; Used to determine the spatial region where each structural unit is located in the reorganized structure, and to determine the sequence numbering based on the mapping relationship; This is used to determine the spatial distribution characteristics of the sequence number arrangement.

5. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 1, characterized in that, The identification module determines the type of difference based on the region mapping comparison results, including: Perform region mapping comparison to determine whether each region matches the corpus in the sample database; If the local matching condition is met, it is determined to be a local difference type; If the local matching condition is not met, it is determined to be a non-local difference class; The local matching condition is that there exists a single region that does not match, while all remaining regions match.

6. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 5, characterized in that, The identification module is used for region mapping comparison, including: The contour features are divided into several regions, and the corpus in the sample database is divided into several corresponding corpus regions. The similarity between the region and the corresponding corpus region is compared to determine the contour similarity. If the contour similarity of the regions is greater than a preset threshold, then the regions are determined to match.

7. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 1, characterized in that, The calibration module is used to perform region mapping comparison between the remaining region and the corpus in the sample database, and extracts a predetermined number of corpora based on sample similarity to replace the contour features in the local image one by one, including... The remaining regions are compared with the corresponding regions in the corpus of the sample database to obtain the sample similarity. The corpora are sorted from largest to smallest based on sample similarity, and a predetermined number of corpora are selected as candidate corpora in descending order. The candidate corpus is used sequentially to replace the contour features in the local image.

8. The multimodal intelligent dictation and correction system based on dynamic calibration according to claim 1, characterized in that, The calibration module is used to determine the calibration profile features, including: After the replacement of candidate texts is completed, the texts are identified, and contextual understanding is performed based on the texts to determine the semantic matching degree. The candidate corpus with the highest semantic matching degree is selected as the calibration contour feature.

9. A method applied to the multimodal intelligent dictation and correction system based on dynamic calibration as described in any one of claims 1-8, characterized in that, include, Acquire pen image data based on user input voice feedback, and perform text recognition on the pen image data; Based on the character recognition results of the pen image data, several feature observation groups are constructed, and several structural units in the feature observation groups are labeled. A sliding window is constructed to locally select structural units in the layout observation group to identify several recombination structures with a tendency to combine. Fuzzy identification labels are set for the recombination structures based on the prior probability of the spatial distribution features of the recombination structures in the corpus database. Read the contour features in the reconstructed structure corresponding to the fuzzy recognition label, divide the local image in the reconstructed structure corresponding to the fuzzy recognition label into regions, compare the contour features in each local image with the corpus in the sample database, and determine the difference type of the local image based on the region mapping comparison result. Unmatched regions in local images with local differences are removed. The remaining regions are then compared with the corpus in the sample database through region mapping. Based on the sample similarity, a predetermined number of corpus data are extracted to replace the contour features in the local images one by one to determine the calibration contour features. The calibration contour features are then identified. The feature segmentation module is used to construct several feature observation groups based on adjacent contour features in the pen image data. The labeled structural units within the feature observation groups include... Used to obtain character recognition results to identify unrecognized local images; Used to identify horizontally adjacent local images as feature observation groups; Used to perform pixel clustering on local images, and the resulting pixel clusters are labeled as structural units.

Images