Identification method and device based on semantic space consistency, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses an identification method, device and equipment based on semantic space consistency, and a medium, and the method comprises the steps: obtaining and preprocessing a target image, a positioning text region and an object region, and generating region positioning information; identifying and correcting the text region to generate a target text vector, extracting an object region image feature and normalizing to generate a target image vector, and mapping the two to the same semantic space for consistency analysis to generate a fusion confidence coefficient and a preliminary identification result; and retrieving, comparing and outputting final identification information and the fusion confidence coefficient in a preset knowledge base based on the fusion confidence coefficient and the preliminary identification result. According to the method, complementary fusion of the text and the image is realized through semantic space mapping, and standardized verification is completed by combining with knowledge base comparison, so that the problem that a single recognition mode is easy to make mistakes is effectively solved, and the accuracy and reliability of a result are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a recognition method, apparatus, device, and medium based on semantic space consistency. Background Technology

[0002] In the healthcare sector, drug identification is a crucial step in clinical prescription review, drug management, and intelligent medication recommendation. Existing technologies primarily rely on two methods: OCR-based text recognition, which uses text information on prescriptions or medicine box labels to identify drug names, dosages, and usage instructions; and image retrieval-based appearance recognition, which extracts features from medicine box packaging images and compares them with a standard image database. However, these methods have significant shortcomings in practical applications. Text recognition methods are highly sensitive to the quality of the prescription writing, image clarity, and shooting environment, especially when faced with handwritten prescriptions, blurry images, or complex backgrounds, easily leading to misidentification or garbled text, and their accuracy in recognizing drug-specific terms is insufficient. Image recognition methods that solely rely on the appearance of the medicine box are prone to confusion due to packaging differences between different manufacturers or batches. For example, the same drug may correspond to multiple different appearances, while different drugs may be very similar in color and size, increasing the risk of misjudgment. More importantly, existing technologies generally lack an effective fusion mechanism for textual information and image features, and cannot improve the accuracy and robustness of recognition through cross-validation of multimodal information. They also lack integration with drug knowledge bases. Therefore, when the recognition results are not standardized and corrected and reasonableness reviews are not carried out, there are potential drug safety risks.

[0003] In the fintech sector, drug identification technology is frequently applied to scenarios such as intelligent insurance claims, health insurance risk control, and drug expense settlement. However, existing OCR text recognition and image retrieval methods also have significant shortcomings. Text recognition performs poorly on low-quality prescriptions, non-standardized reimbursement documents, or scanned images, easily leading to errors in drug name and dosage information, and consequently, errors in expense verification. Image-based medicine box recognition methods are also prone to matching errors when faced with different drugs with similar appearances or diverse packaging of the same drug, posing risks to claims decisions and compliance reviews. Furthermore, existing methods lack comprehensive utilization of multimodal features and cannot perform consistency verification between text and images, thus limiting the reliability of the recognition results. More importantly, the recognition results are not fully integrated with a standard drug knowledge base, lacking unified coding and reasonableness verification, making it easy for inconsistencies in drug classification or abnormal expense calculations to occur during claims, review, and settlement processes, posing challenges to compliance and security in financial business scenarios. Summary of the Invention

[0004] The main objective of this invention is to provide a recognition method, apparatus, device, and storage medium based on semantic space consistency. This invention aims to address the technical problem that the existing technology lacks a mechanism to uniformly map data from different modalities to the same semantic space and perform consistency analysis, which makes it difficult to effectively fuse and cross-verify multi-source data such as text and image information, thereby limiting the accuracy of recognition results.

[0005] To achieve the above objectives, the present invention provides a recognition method based on semantic space consistency, comprising:

[0006] Acquire the target image to be identified, preprocess the target image, and generate a preprocessed target image;

[0007] In the preprocessed target image, locate the text region and the object region to generate region positioning information;

[0008] Based on the region positioning information, text recognition is performed on the text region to obtain initial text information. The initial text information is then corrected according to a preset dictionary to generate a target text vector.

[0009] Based on the region positioning information, image features of the object region are extracted, and the image features are normalized to generate a target image vector.

[0010] The target text vector and the target image vector are mapped to the same semantic space, and consistency analysis is performed to generate fusion confidence and preliminary recognition results.

[0011] Based on the fusion confidence score and the preliminary identification result, a search and comparison are performed in a preset knowledge base to obtain the final identification information, and the final identification information and the fusion confidence score are output.

[0012] Furthermore, to achieve the above objectives, the present invention provides a recognition device based on semantic space consistency, comprising:

[0013] The image preprocessing module is used to acquire the target image to be identified, preprocess the target image, and generate a preprocessed target image.

[0014] The region positioning module is used to locate text regions and object regions in the preprocessed target image and generate region positioning information.

[0015] The text recognition and correction module is used to perform text recognition on the text region based on the region positioning information, obtain initial text information, correct the initial text information according to a preset dictionary, and generate a target text vector;

[0016] The image feature extraction module is used to extract image features of the object region based on the region positioning information, normalize the image features, and generate a target image vector.

[0017] The cross-modal consistency analysis module is used to map the target text vector and the target image vector to the same semantic space, perform consistency analysis, and generate fusion confidence and preliminary recognition results.

[0018] The knowledge base retrieval and comparison module is used to search and compare in a preset knowledge base based on the fusion confidence score and the preliminary identification result, obtain the final identification information, and output the final identification information and the fusion confidence score.

[0019] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a semantic space consistency-based identification program stored in the memory and executable on the processor, wherein the semantic space consistency-based identification program, when executed by the processor, implements the steps of the semantic space consistency-based identification method as described above.

[0020] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a semantic space consistency-based identification program, wherein the semantic space consistency-based identification program, when executed by a processor, implements the steps of the semantic space consistency-based identification method as described above.

[0021] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses a recognition method, apparatus, device, and medium based on semantic space consistency, comprising: acquiring a target image to be recognized and preprocessing it to generate a preprocessed target image; locating text regions and object regions in the preprocessed target image to generate region positioning information; performing text recognition on the text regions based on the region positioning information and correcting the initial text information according to a preset dictionary to generate a target text vector; extracting image features of the object regions based on the region positioning information and performing normalization processing to generate a target image vector; mapping the target text vector and the target image vector to the same semantic space for consistency analysis to generate a fusion confidence score and preliminary recognition result; and retrieving and comparing the fusion confidence score and preliminary recognition result in a preset knowledge base to obtain final recognition information and outputting the final recognition information and fusion confidence score. This invention achieves complementary and cross-validation of multimodal data by fusing text information and image features in the same semantic space and comparing and standardizing them with a knowledge base. This effectively overcomes the shortcomings of single character recognition being easily affected by handwriting quality and single image recognition being easily interfered with by appearance differences, thereby improving the accuracy and robustness of the recognition results and outputting standardized recognition information with confidence, providing reliable support for subsequent intelligent processing and applications. Attached Figure Description

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0023] Figure 1 This is a schematic diagram of an application environment for a recognition method based on semantic space consistency in one embodiment of the present invention;

[0024] Figure 2 This is a flowchart illustrating an embodiment of the semantic space consistency-based identification method of the present invention;

[0025] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the semantic space consistency-based recognition device of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0027] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0028] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0029] The semantic space consistency-based identification method provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can acquire the target image to be recognized from the client and perform preprocessing to generate a preprocessed target image. Within the preprocessed target image, text regions and object regions are located to generate region location information. Based on the region location information, text recognition is performed on the text regions, and the initial text information is corrected according to a preset dictionary to generate a target text vector. Based on the region location information, image features of the object regions are extracted and normalized to generate a target image vector. The target text vector and the target image vector are mapped to the same semantic space for consistency analysis to generate a fusion confidence score and preliminary recognition results. Based on the fusion confidence score and preliminary recognition results, a search and comparison are performed in a preset knowledge base to obtain the final recognition information, and the final recognition information and fusion confidence score are output. This invention achieves complementary and cross-validation of multimodal data by fusing text information and image features within the same semantic space and combining this with a knowledge base for comparison and standardized verification. This effectively overcomes the shortcomings of single-character recognition, which is susceptible to handwriting quality, and single-image recognition, which is susceptible to interference from appearance differences. This improves the accuracy and robustness of the recognition results and outputs standardized recognition information with confidence levels, providing reliable support for subsequent intelligent processing and applications. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0030] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the semantic space consistency-based identification method provided by the present invention. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0031] like Figure 2 As shown, the identification method based on semantic space consistency proposed in this invention includes the following steps:

[0032] S10, acquire the target image to be identified, preprocess the target image, and generate a preprocessed target image;

[0033] In this embodiment, acquiring the target image to be identified is the starting point of the entire process. The target image can come from various acquisition channels, such as images captured by a mobile terminal's camera, images input from a scanner, or direct uploads from a business system. Target images typically contain content under different lighting, resolution, and contrast conditions. This necessitates considering the compatibility of image file formats, including JPEG, PNG, and TIFF, during the data input stage, and supporting unified processing for different resolutions and color spaces.

[0034] The purpose of preprocessing the target image is to improve the stability and accuracy of subsequent recognition and analysis. The preprocessing stage can be broken down into several technical steps. First, image quality assessment needs to be performed, generating quality analysis results by analyzing brightness distribution, noise level, and edge sharpness. This process can be implemented using algorithms based on gradient statistics or frequency domain energy distribution to quantify the degree of image blur and noise intensity.

[0035] After obtaining the quality analysis results, the appropriate denoising module needs to be selected based on different noise characteristics. For example, Gaussian filtering or nonlocal mean filtering can be used for Gaussian noise; median filtering can be used for impulse noise; and for complex noise, a denoising model trained by a convolutional neural network can be combined to perform multi-scale noise filtering, ensuring that interference is removed while preserving structural texture. The denoised image data will then serve as input for subsequent geometric feature analysis.

[0036] Geometric distortion detection is another crucial step. During image acquisition, the target image may be affected by viewpoint tilt, lens distortion, or perspective distortion. Distortion needs to be identified by detecting features such as line offset and geometric proportion distortion. Hough transform can be used to detect line features, or keypoint matching can be used to analyze distortion patterns. The detected geometric distortion features then drive adaptive geometric correction operations.

[0037] Adaptive geometric correction relies on the distortion detection results from the previous step. It uses a projection transformation matrix or a camera model inverse method to correct the image, remapping the originally tilted or distorted object regions into rectangular shapes with near-regular proportions. The correction method can be either a global affine transformation or local mesh deformation correction, ensuring the visual proportions of the object are restored.

[0038] After correction, the image needs to undergo feature fusion enhancement. The focus of enhancement is on improving contrast, sharpening edges, and strengthening texture features to facilitate subsequent text recognition and image feature extraction. This can be achieved through multi-scale convolutional feature fusion, combining local detail enhancement with global contour enhancement, or by employing a self-attention mechanism to enhance information representation in both spatial and channel dimensions. The resulting enhanced image data serves as the preprocessed target image, providing a reliable data foundation for subsequent region localization and feature recognition.

[0039] In practical implementations, preprocessing can be performed using traditional image processing techniques, such as filtering-based noise removal and affine matrix-based geometric correction. This approach is advantageous on devices with limited hardware resources due to its lower computational cost. Alternatively, deep learning models can be used for end-to-end preprocessing, such as using convolutional neural networks for denoising and enhancement, while incorporating spatial transformation networks for distortion correction. This approach offers higher performance and robustness in server environments with ample computing resources.

[0040] Technical parameters need to be adjusted in different scenarios. For example, in the image processing of drug instructions in the healthcare field, noise mainly comes from shooting in low-light environments. Therefore, it is necessary to enhance the contrast of low-light areas and combine color constancy to maintain the true colors of the drug packaging. In the image processing of invoices in the financial field, distortion mainly comes from the deviation of the scanning angle. Therefore, geometric correction needs to focus on ensuring the straightness of text lines so that subsequent OCR can recognize them correctly.

[0041] A dynamic adjustment mechanism can also be introduced in the preprocessing stage, such as adjusting the denoising intensity based on image complexity, reducing the denoising weight when the image texture is dense, and increasing the denoising weight when the background noise is strong, thereby achieving better adaptability.

[0042] This embodiment performs quality assessment, noise filtering, geometric correction, and feature enhancement on the target image, which can effectively eliminate interference caused by low-quality input before entering text recognition and object feature extraction, and avoid recognition errors caused by blurring, distortion, or excessive noise, thereby improving the stability and accuracy of the overall processing flow.

[0043] S20, locate the text region and the object region in the preprocessed target image and generate region positioning information;

[0044] In this embodiment, the text and object regions need to be located in the preprocessed target image, and region location information needs to be generated. The preprocessed target image is the image data after noise filtering, geometric correction, and feature enhancement in the previous step, and its quality meets the requirements of subsequent region segmentation. The first step in region localization is to perform multi-scale convolutional feature extraction on the image. Convolutional neural networks extract features such as edges, textures, and shapes at different scales to ensure that both large-scale contour information is captured and local details are preserved. This process can be implemented using convolutional layers of different depths. Shallow convolutions focus on low-level features such as lines and edges, while deep convolutions extract high-level semantic features such as text blocks and object shapes.

[0045] Based on multi-scale convolutional features, a feature pyramid needs to be constructed. The feature pyramid fuses feature maps of different resolutions through layer-by-layer downsampling and lateral connections, enabling the simultaneous detection of both small text regions and large object regions. This structure improves adaptability to regions with significant scale differences, avoiding the omission of small text or the misjudgment of large objects.

[0046] After obtaining the feature pyramid, initial region proposals need to be generated. Region proposals are candidate regions of interest, which are detected by a sliding window method on the feature map using a Region Proposal Network (RPN) to predict rectangular boundaries that may contain text or objects. During the generation of initial region proposals, a confidence score is calculated for each candidate box to measure the probability that the region contains the target.

[0047] The initial region proposals suffer from inaccurate positioning, necessitating boundary regression optimization. Boundary regression learns the deviation between the true boundary and the candidate boundary using a regression model, correcting the coordinates and aspect ratios of the candidate boxes to generate optimized region boundaries that better fit the target. This optimization process ensures that text character boundaries are not over-truncated and that the object region is fully covered.

[0048] Building upon optimized region boundaries, it's necessary to further distinguish between text and object regions. This distinction can be achieved using a classifier to predict the category of features within each region. The classifier's output determines whether the region better matches text or object features. For example, text regions typically exhibit regularly arranged lines, while object regions tend to resemble continuous shapes. Combining the classification results with boundary information generates region localization information with coordinate attributes. This information includes not only the region's category label (text or object) but also its position coordinates, width, and height within the preprocessed target image, used for subsequent text recognition and image feature extraction.

[0049] In implementation, lightweight convolutional neural networks can be used to extract multi-scale features to reduce computational complexity, making them suitable for deployment on mobile devices. Alternatively, deep residual networks or feature pyramid networks can be used to enhance feature representation capabilities, making them more suitable for cloud or server environments. During region proposal generation, anchor-based detection methods or anchor-free detectors can be used to directly generate candidate boxes by predicting keypoints and boundary dimensions. In the boundary regression optimization stage, IoU loss functions or generalized IoU loss functions can be used to optimize localization accuracy.

[0050] To distinguish between text regions and object regions, a dual-channel classification network can be used. One channel learns the texture and character arrangement features of the text, while the other channel learns the shape and appearance features of the object. The classifier can be adjusted according to the needs of different scenarios. For example, in the healthcare field, more emphasis is placed on recognizing small-font drug instructions, while in the financial field, more emphasis is placed on the complete coverage of invoice stamps and signatures.

[0051] This embodiment generates accurate region localization information by performing multi-scale convolutional feature extraction, feature pyramid construction, and region proposal box generation and optimization on the preprocessed target image, and distinguishing between text regions and object regions at the optimized region boundaries. This process avoids the problem of missing small targets caused by single-scale processing and reduces the accumulation of candidate region errors, enabling subsequent text recognition and image feature extraction to be based on more accurate input regions, thereby improving the stability and reliability of the overall recognition chain.

[0052] S30, based on the region positioning information, perform text recognition on the text region to obtain initial text information, correct the initial text information according to a preset dictionary, and generate a target text vector;

[0053] In this embodiment, text recognition can be performed on text regions segmented from an image based on region localization information. The region localization information provides the coordinate range and boundary information of each text region, which allows for the cropping of independent text region images from the preprocessed target image. Once the text region images enter the recognition process, both printed and handwritten characters need to be supported. Printed character recognition can utilize an optical character recognition structure combining convolutional neural networks and recurrent neural networks, capable of extracting character features line by line and concatenating them. Handwritten character recognition requires the introduction of a connection-time classification (CTC) structure based on sequence modeling or an encoding-decoding structure based on an attention mechanism to handle issues such as uneven character spacing and continuous strokes.

[0054] The identified result is called the initial text information. This initial text information may contain erroneous characters or terms, such as incorrect character substitutions, missing characters, or confusion with similar words. To improve the accuracy of the text expression, dictionary correction is necessary. The key step in dictionary correction is detecting suspected erroneous terms, which can be achieved by comparing the similarity between the strings in the initial text information and the standard terms already included in the dictionary. A common similarity measurement method is edit distance, which measures the difference through the number of insertion, deletion, or replacement operations. Once a suspected erroneous term is identified, the standard term with the smallest edit distance needs to be found, and the erroneous text is replaced with this standard term, thus generating corrected text information.

[0055] After obtaining the corrected text information, it needs to be further converted into a numerical representation; this process is called semantic feature encoding. Semantic feature encoding maps strings into multi-dimensional vector representations through a text encoder, typically using word vector-based models or semantic embedding methods based on pre-trained language models. Semantic feature encoding not only preserves the surface information of words but also captures the semantic relationships between contexts. The final result is the target text vector, which serves as the input for subsequent cross-modal consistency analysis and knowledge base comparison.

[0056] Text recognition and correction can be implemented in different ways depending on the scenario. Lightweight convolutional recurrent network structures can be used for fast recognition on mobile devices; alternatively, large-scale models based on deep attention mechanisms can be employed to achieve high-precision recognition on the server side. Dictionary correction can be done using a static dictionary or combined with a dynamic expansion mechanism, gradually adding newly emerging domain terms to adapt to different business scenarios. In the semantic feature encoding stage, bag-of-words models and traditional word vectors can be used, or context-based deep language models, such as the Transformer structure, can generate more expressive vectors.

[0057] To address differences in input languages, multilingual training models can be loaded to enable recognition and correction across language scenarios. For handwriting recognition, stroke order features can be introduced, or graph convolutional networks can be combined to handle character cursive writing issues. For editing distance calculation, weighted rules can be introduced during the calculation process, such as assigning higher penalties to character substitution errors, to adapt to accuracy requirements in specific scenarios.

[0058] Example Explanation: In the healthcare business, the text on drug packaging labels can be identified. First, the printed text in the label area is detected. When a misidentification of a similar drug name occurs, it is corrected through dictionary correction. For example, "Loxatang" is automatically corrected to "Loxatan". Then, the target text vector is generated through semantic feature encoding, providing accurate input for subsequent identification and comparison with the knowledge base.

[0059] In the financial sector, text recognition can be performed on the amount fields and signature areas in images of bills or contracts. When a numerical recognition error occurs, such as "1000.0" being misrecognized as "10000", a dictionary correction process is used to replace and correct the error using a standard amount dictionary. The result is then converted into a vector representation for input into subsequent verification steps, thereby improving the accuracy and compliance of automated bill processing.

[0060] This embodiment effectively reduces misidentification in the initial text information by performing text recognition and dictionary correction under the constraint of regional positioning information. Furthermore, semantic feature encoding transforms the corrected text information into a unified vector representation. This provides more stable and accurate text input in subsequent multimodal fusion and knowledge base comparison stages, thereby improving the robustness and reliability of the overall recognition process.

[0061] S40, Based on the region positioning information, extract the image features of the object region, normalize the image features, and generate a target image vector;

[0062] In this embodiment, based on region positioning information, an object region image can be cropped from the preprocessed target image. The region positioning information includes the boundary coordinates and size of the object region, enabling accurate location of the object region to be analyzed across the entire image. After acquiring the object region image, feature extraction is required. The goal of feature extraction is to extract high-dimensional features that represent the shape, texture, and structure of the object region from pixel-level input.

[0063] In the feature extraction stage, both global and local visual features can be extracted simultaneously. Global visual features emphasize the overall appearance information of the object, such as shape outline, color distribution, and overall texture pattern. Local visual features focus on fine-grained local areas, such as text edges, texture blocks, and pattern details. The combination of global and local features can cover different levels of object representation, avoiding information loss caused by relying on a single feature.

[0064] After global and local features are extracted, a fusion operation is required to generate a unified feature representation. Feature fusion can be achieved by constructing multi-level feature maps, with different levels corresponding to feature representations at different resolutions. Lower-level features preserve edge details and textures, while higher-level features express abstract semantics. After the multi-level feature maps are constructed, attention-based feature enhancement can be further performed. The attention mechanism calculates the importance weights of feature channels or spatial regions, highlighting key feature regions and suppressing redundant or noisy features, thereby improving the discriminative power of the feature representation.

[0065] The enhanced original image features need to undergo normalization. The goal of normalization is to eliminate scale and distribution differences, ensuring consistency in the feature space for object regions from different sources or batches. Normalization can be achieved through contrastive learning, bringing the feature distributions of similar objects closer together and distancing the feature distributions of dissimilar objects further apart, ultimately generating normalized image features.

[0066] Finally, the normalized image features are encoded into target image vectors. The encoding process compresses the normalized features into a fixed-dimensional vector representation, ensuring that it can be matched and compared with the target text vector in the same vector space. The target image vector is a crucial input for subsequent cross-modal consistency analysis.

[0067] In practical applications, convolutional neural networks can be used to extract global and local features, and residual structures can be used to maintain the integrity of features at different depths. Alternatively, self-attention networks based on visual Transformers can be used to model image features in a wide range of contexts. Regarding feature fusion methods, feature pyramids can be constructed through layer-by-layer upsampling, or multi-scale fusion can be achieved through lateral connection mechanisms.

[0068] During the attention enhancement stage, channel attention mechanisms can be used to redistribute the weights of different channels, or spatial attention mechanisms can be used to select key region features. For the normalization process, standardization can be used, or contrastive learning methods can be employed, constructing positive and negative sample pairs during training and achieving feature normalization through loss function constraints. In the vector encoding process, linear projection mapping can be used to compress features into fixed-length vectors, or multilayer perceptrons can be used to enhance vector representation capabilities.

[0069] In terms of device deployment, lightweight convolutional structures can be chosen to support embedded environments, or large-scale deep networks can be used to run on the server side to achieve higher accuracy. When adapting to different scenarios, the window size for local feature extraction can be adjusted according to the complexity of the object region, or the normalization strategy can be dynamically adjusted according to the image sharpness.

[0070] This embodiment extracts image features of the target region based on regional positioning information and performs fusion, enhancement, and normalization processing. This ensures that the final generated target image vector retains both overall appearance features and highlights key local features, while maintaining strong comparability in the vector space. This vector representation significantly reduces confusion caused by differences in sources, batches, or shooting conditions, providing a stable and reliable image representation for subsequent cross-modal matching and knowledge base comparison.

[0071] S50, map the target text vector and the target image vector to the same semantic space, perform consistency analysis, and generate fusion confidence and preliminary recognition results;

[0072] In this embodiment, the target text vector and the target image vector are generated independently in the numerical space, originating from the text recognition and image feature extraction processing chains, respectively. Since the two types of vectors differ in their distribution in the original space, they must be mapped into the same semantic space. The semantic space is a trained high-dimensional vector space that maintains semantic alignment across modalities. For example, textual descriptions and image features of the same object should be represented as adjacent locations in this space, while representations of different objects should remain separated by distance.

[0073] The mapping is implemented using separate text encoders and image encoders. The text encoder performs an embedding transformation on the target text vector to obtain a text semantic vector; the image encoder projects the target image vector onto the same embedding space to obtain an image semantic vector. During training, the two encoders learn to maintain consistency through comparative learning, ensuring that the text and image of the same object are as close as possible in space.

[0074] After generating text semantic vectors and image semantic vectors, consistency analysis is required. This is typically achieved by calculating similarity values, such as using cosine similarity to measure the angle between two vectors, thus determining their proximity in the semantic space. A higher similarity value indicates greater consistency between the text description and image features in semantic expression.

[0075] The consistency analysis results are not used directly, but rather weighted and fused using dynamic weight parameters. These dynamic weight parameters are determined by the source types of the target text vector and the target image vector. For example, when the text originates from a highly regular prescription text, the weight of the text portion can be increased; when the image originates from a clear object region, the weight of the image portion can be increased. The weighted fusion yields a fusion confidence score, which serves as a quantitative indicator of the consistency result.

[0076] After generating the fusion confidence score, a preliminary recognition result needs to be output based on this confidence score. The preliminary recognition result corresponds to the most likely category or object identifier of the current input data. It is the recognition output generated when the text semantic vector and the image semantic vector have a high degree of consistency. The preliminary recognition result not only includes candidate categories but is also associated with the fusion confidence score, which is used to indicate the reliability of the result.

[0077] In implementation, a dual-tower encoder framework can be adopted, training the text tower and the image tower separately, and ensuring cross-modal alignment through a loss function that shares a semantic space. The text encoder can use a Transformer-based structure to fully model contextual semantics; the image encoder can use convolutional neural networks or visual Transformers to capture multi-scale visual features.

[0078] In consistency analysis, different similarity functions can be selected, such as Euclidean distance, dot product similarity, or improved attention-weighted similarity, to enhance the robustness of the analysis. Regarding the dynamic weight determination method, it can be based on preset rules, such as assigning fixed weights after identifying the source type, or a weight prediction module can be trained based on the confidence distribution to adaptively adjust in different scenarios.

[0079] In practical applications, the parameter scale of text encoders and image encoders can be limited to adapt to embedded devices, thereby reducing latency. Larger-scale networks can also be deployed in the cloud to achieve higher accuracy and fault tolerance in financial or medical applications. Under different data quality conditions, the threshold for similarity calculation can be adjusted, and a confidence decay mechanism can be used to reduce the risk of erroneous outputs when the input modality quality is poor.

[0080] This embodiment maps target text vectors and target image vectors to a unified semantic space and generates fusion confidence scores and preliminary recognition results based on consistency analysis. This establishes an effective correspondence between multimodal information, enabling cross-modal complementary verification. This approach avoids the limitations of single-modality susceptibility to interference, improves the robustness and accuracy of recognition, and provides reliable intermediate results for subsequent comparison with the knowledge base.

[0081] S60, based on the fusion confidence score and the preliminary identification result, perform retrieval and comparison in the preset knowledge base to obtain the final identification information, and output the final identification information and the fusion confidence score.

[0082] In this embodiment, the fusion confidence score and preliminary identification result, as key outputs generated in the previous stage, need to be further searched and compared in the knowledge base to obtain the final identification information. The fusion confidence score reflects the degree of consistency between text vectors and image vectors in the semantic space, while the preliminary identification result corresponds to the most likely object category or entity identifier. Introducing these two data points into the knowledge base retrieval process serves a dual purpose: on the one hand, it uses the confidence score to determine the stringency of the retrieval; on the other hand, it uses the preliminary identification result to determine the target entries for the retrieval.

[0083] A knowledge base is a pre-built collection of standardized information, storing unique codes, standardized names, attribute parameters, and validity rules for various objects. During the retrieval process, the scope of the search is first determined based on the fusion confidence level. For example, exact matching is used with high confidence, while fuzzy matching or searches including synonyms and similar entries are expanded to lower confidence levels. This process is achieved through dynamic adjustment of the retrieval strictness level, enabling the system to flexibly respond to different data quality conditions.

[0084] Once candidate entries are retrieved, a standardization mapping process needs to be performed. The purpose of standardization mapping is to map the preliminary identification results to a unique standard code in the knowledge base. This process avoids inconsistencies in object names across different sources; for example, the same object may have differences between a product name and a generic name. After mapping, all names are unified to the standard code.

[0085] After completing the standardized mapping, a reasonableness check needs to be performed on the standard information corresponding to the unique standard code. Dimensions of the reasonableness check can include quantity range, attribute matching, and applicable conditions. For example, for objects in the medical field, the check can examine whether the dosage range or contraindications match; for objects in the financial field, the check can examine whether the amount range or transaction rules are reasonable. The verification result, combined with the unique standard code, is used to generate the final identification information.

[0086] The final identified information, after verification by the knowledge base, possesses standardized identifiers and complete attribute descriptions. The output process requires simultaneously returning the final identified information and the fusion confidence score, enabling subsequent applications to obtain standardized information while also referencing the reliability indicators of the identification process.

[0087] Different knowledge base structures can be used in different implementations. Relational databases can be used to perform exact and fuzzy matching through standard field indexes; vector databases can be used to encode the initial identification results into vector form and perform similarity retrieval with knowledge base entry vectors. A hybrid structure can also be used, combining rule-based matching and semantic retrieval to improve matching accuracy.

[0088] Regarding the implementation of reasonableness checks, a rule engine can be used to apply predefined check rules to information corresponding to a unique standard code; alternatively, a machine learning model can be used to predict and classify the verification results using historical data. In different scenarios, a lightweight rule system or a complex model-driven system can be selected based on the characteristics of the data.

[0089] When adapting to different business needs, the sensitivity levels of retrieval and reasonableness checks can be adjusted by configuring parameter thresholds. For example, in the healthcare field, a higher dose range sensitivity can be set to reduce the risk of misjudgment; in the financial field, a higher transaction amount sensitivity can be set to ensure that abnormal behavior can be identified in a timely manner.

[0090] This embodiment incorporates the fusion confidence score and preliminary recognition results into knowledge base retrieval and comparison, generating final recognition information and fusion confidence score output. This introduces a standardized verification mechanism into the recognition results, ensuring that the output relies not only on model predictions but also on confirmation through authoritative data. This approach improves the standardization and reliability of the recognition results, reduces the risk of confusion caused by diverse expressions and data differences, and ultimately enhances the stability and security of the overall recognition system.

[0091] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses a recognition method, apparatus, device, and medium based on semantic space consistency, comprising: acquiring a target image to be recognized and preprocessing it to generate a preprocessed target image; locating text regions and object regions in the preprocessed target image to generate region positioning information; performing text recognition on the text regions based on the region positioning information and correcting the initial text information according to a preset dictionary to generate a target text vector; extracting image features of the object regions based on the region positioning information and performing normalization processing to generate a target image vector; mapping the target text vector and the target image vector to the same semantic space for consistency analysis to generate a fusion confidence score and a preliminary recognition result; and retrieving and comparing the fusion confidence score and the preliminary recognition result in a preset knowledge base to obtain final recognition information and outputting the final recognition information and the fusion confidence score. This invention achieves complementary and cross-validation of multimodal data by fusing text information and image features in the same semantic space and comparing and standardizing them with a knowledge base. This effectively overcomes the shortcomings of single character recognition being easily affected by handwriting quality and single image recognition being easily interfered with by appearance differences, thereby improving the accuracy and robustness of the recognition results and outputting standardized recognition information with confidence, providing reliable support for subsequent intelligent processing and applications.

[0092] In one embodiment, step S10 includes:

[0093] S101, acquire the target image to be identified, perform image quality analysis on the target image to be identified, and generate quality analysis results;

[0094] S102, based on the quality analysis results, a denoising module is selected, and the selected denoising module is used to perform multi-scale noise filtering on the target image to be identified to generate denoised image data;

[0095] S103, detect geometric distortion features in the denoised image data;

[0096] S104, perform adaptive geometric correction on the denoised image data according to the geometric distortion features to generate corrected image data;

[0097] S105, perform feature fusion enhancement processing on the corrected image data to generate enhanced image data, and use the enhanced image data as the preprocessed target image.

[0098] In this embodiment, the target image to be identified can originate from a terminal camera, still photograph, screen capture, or scan input. Its color space may be grayscale, three-channel, or four-channel, and its resolution and compression method may vary. The input stage completes format unification and metadata organization, including color space conversion, channel order correction, bit depth recognition, and image orientation marking, ensuring that subsequent stages can access the data with a consistent structure. The target image to be identified serves as the sole original carrier throughout the entire process. Each subsequent intermediate result is based on the coordinate system of this image and records the mapping relationship, facilitating downstream location and backtracking.

[0099] Image quality analysis measures sharpness, noise, exposure, color cast, and compression artifacts. Sharpness is estimated through edge energy distribution and local frequency components; noise is estimated through spatial residuals and intra-block brightness fluctuations; exposure is evaluated through the offset and clustering of the brightness histogram; color cast is evaluated through gray-scale world deviation and neutral block consistency; and compression artifacts are evaluated through repetitive patterns at block edges. The quality analysis results are presented in a structured description, including problem type, severity level, spatial distribution mask, and suggested processing strategies, serving as the basis for subsequent module selection and parameter settings. For example, underexposure caused by backlighting in mobile devices will trigger an exposure enhancement strategy, while high ISO noise in night scenes will trigger a strong noise suppression strategy.

[0100] The denoising module is selected from a module library based on quality analysis results. The library includes three categories: spatial adaptive filtering, frequency domain denoising, and deep residual networks. Multi-scale noise filtering is implemented using an image pyramid as a framework. In the high-resolution layer, edge-preserving suppression is performed on high-frequency fine noise; in the mid-resolution layer, random noise and compressed patterns are processed; and in the low-resolution layer, large-scale banded interference and mean shift are addressed. To avoid excessive smoothing of details, edge-sensitive weights are constrained by a sharpness mask, with reduced denoising intensity in textured regions and increased denoising intensity in flat regions. The output generates denoised image data while retaining the noise residual map and denoising weight map, providing a constraint on the gain upper limit of subsequent enhancement stages.

[0101] Geometric distortion feature detection identifies perspective distortion, radial distortion, tilt, and local curvature. Perspective distortion is estimated by using the consistent intersection relationship of line sets to determine the vanishing point and planar orientation; radial distortion is estimated by using concentric arc deviation and grid line deformation to determine the distortion coefficient; tilt is estimated by using text line direction statistics and principal direction spectrum energy; and local curvature is estimated by using row and column direction curvature field estimation. The detection output includes distortion type, directional parameters, intensity level, and reference control point, providing direct input for geometric correction. Examples of sources include pitch perspective caused by desktop photography, barrel distortion caused by short focal length lenses, and skewing caused by handheld camera shake.

[0102] Adaptive geometric correction selects the mapping method and determines the mapping intensity based on geometric distortion characteristics. Planar projection mapping is used when significant perspective is present; radial model correction is used when lens aberration is dominant; affine mapping is used for slight tilt; and meshed deformation is used for local curvature. Control points are derived from boundary lines and corner points. During mapping, priority is given to ensuring the straightness of text lines and object edges to avoid cropping the effective area. To maintain coordinate consistency, a bidirectional mapping table from original coordinates to corrected coordinates is established and stored in the image metadata for reuse by subsequent localization modules. The output generates corrected image data and provides a correction quality metric, which is fed back to the enhancement stage to adjust the gain range.

[0103] Feature fusion enhancement performs integrated enhancement of contrast, sharpness, and color on the corrected image data, constrained by quality analysis results and noise residual maps. For contrast, a combination of constrained adaptive histogram stretching and region-level gain is used to improve local contrast in text and fine-texture areas, while limiting gain in large flat areas to suppress noise backflow. For sharpness, edge-guided high-frequency backinjection is employed, with the injection intensity controlled by edge confidence to avoid halos. For color, common color gamut steady-state corrections such as gray balance and skin tone are performed to prevent oversaturation. The entire process is performed simultaneously across multiple scales, and the enhanced results at each scale are weighted according to confidence level and backinjected to the original resolution. The output generates enhanced image data and retains the enhancement mask and gain log for easy anomaly rollback and quality auditing.

[0104] The enhanced image data is identified as the preprocessed target image. The corresponding data object contains a unified color space, unified dynamic range, stable geometry, and complete coordinate mapping and processing records. This object becomes the sole input for subsequent region localization and recognition. All spatial references maintain consistent coordinate relationships with the original image, avoiding positioning shifts caused by geometric changes, and providing a stable input benchmark for cross-device and cross-scene processing. Examples of sources include various acquisition methods such as warehouse label photography, invoice scanning, and manual page number extraction. The preprocessed target image can be entered into subsequent processes in a unified format.

[0105] This embodiment avoids the loss of detail and noise residue caused by a one-size-fits-all approach through quality analysis-driven module selection. Multi-scale noise filtering suppresses graininess and compressed patterns while preserving texture under edge-sensitive constraints. Geometric distortion feature detection and adaptive correction restore text lines and object edges to a geometrically consistent state. Feature fusion enhancement improves contrast and readability and suppresses overshoot under constraints of limited gain and noise residuals, ultimately forming a stable input carrier—the pre-processed target image. Subsequent region localization can directly reuse coordinate mapping and reduce the probability of false detection. Text recognition and image representation obtain more reliable vector representations under the conditions of guaranteed sharpness, contrast, and geometric consistency, thereby reducing recognition errors caused by cross-device shooting, lighting changes, and lens distortion, and improving the stability and transferability of the overall process.

[0106] In one embodiment, step S20 above includes:

[0107] S201, Extract the multi-scale convolutional features of the preprocessed target image;

[0108] S202, Construct a feature pyramid based on the multi-scale convolutional features;

[0109] S203, Generate an initial region proposal box based on the feature pyramid;

[0110] S204, Perform boundary regression optimization on the initial region proposal box to generate an optimized region boundary;

[0111] S205, distinguish between the text region and the object region in the boundary of the optimized region, and generate region positioning information containing coordinate attributes.

[0112] In this embodiment, when the preprocessed target image enters the localization stage, it first extracts the image's feature representation through multi-scale convolution operations. The convolution kernel size and stride are combined at different scales to simultaneously capture local texture and overall structure, thus preserving character details while also representing the object's shape. The multi-scale convolutional features can originate from the outputs of different layers of a deep convolutional neural network, where shallow layers focus on edges and textures, while deeper layers focus on semantics and contours. By fusing these convolutional features within a spatial pyramid structure, input data that is more sensitive to regional differences in the image can be generated.

[0113] A feature pyramid is constructed based on multi-scale convolutional features, and feature maps of different resolutions are generated by layer-by-layer downsampling. High-resolution layers can finely describe small characters, while low-resolution layers can effectively describe the global contours of larger objects. Within the pyramid's hierarchy, cross-scale information transfer is achieved through lateral connections and layer-by-layer fusion, enabling the target region to be stably captured in multi-resolution space.

[0114] Initial region proposals are generated using a feature pyramid. A region candidate network then slides anchor boxes across each feature map layer, covering potential target locations with varying aspect ratios and sizes. These initial region proposals are filtered and ranked based on classification scores and boundary confidence, resulting in a candidate set covering potential text and object regions.

[0115] Initial region proposals often suffer from inaccurate boundaries, necessitating boundary regression optimization. This optimization process utilizes the regression branch of a convolutional network to refine the four boundary coordinates of the candidate boxes, ensuring a closer fit to the target region's edge. This optimization, based on regression training using predicted offsets, effectively reduces the inclusion of redundant background and improves localization accuracy.

[0116] After generating the optimized region boundaries, it is necessary to distinguish between text regions and object regions. Text regions typically appear as elongated strips or have character arrangement patterns, while object regions tend to have regular geometric shapes or complex textures. The classifier combines convolutional features and geometric features to classify each region and label it as either a text region or an object region.

[0117] Finally, the optimized region boundaries are combined with category labels to generate region positioning information. This information includes not only the geometric coordinates of the boundaries but also the region's category identifier. Coordinate attributes are stored as pixel positions or relative proportions, ensuring that subsequent text recognition and object feature extraction can be performed under a unified spatial reference, avoiding the accumulation of geometric errors.

[0118] This embodiment achieves stable representation of targets of different sizes and scales by combining multi-scale convolutional features and feature pyramids, avoiding the problems of missing small targets or inaccurately locating large targets at a single resolution. The generation of initial region proposal boxes provides the ability to comprehensively cover potential targets, while boundary regression optimization further refines the localization accuracy, making the edges of text and object regions closer together. By classifying the optimized regions, clear distinction between text and objects is achieved, ensuring that subsequent steps can perform different processing tasks for different regions. The final generated region localization information contains accurate coordinates and category labels, providing reliable input for text recognition and image feature extraction.

[0119] In one embodiment, step S30 above includes:

[0120] S301, extract the text region image from the preprocessed target image based on the region positioning information;

[0121] S302, Perform printed and handwritten character recognition on the text region image to generate initial text information;

[0122] S303, Detect suspected erroneous terms in the initial text information;

[0123] S304, determine the edit distance between the suspected erroneous term and the preset dictionary term;

[0124] S305, replace the suspected erroneous terms in the initial text information with preset dictionary terms that have the minimum edit distance to generate corrected text information;

[0125] S306, Semantic feature encoding is performed on the corrected text information to generate a target text vector.

[0126] In this embodiment, text region images are extracted from the preprocessed target image based on the region positioning information. The original coordinates of the positioning boundary are used preferentially, and the geometric mapping relationship with the original image is preserved. When there is a deviation between the positioning boundary and the corrected mapping, coordinate transformation is first performed according to the mapping table, and then cropping is completed within the target coordinate system. To avoid damaging character strokes during cutting, the cropping boundary is extended outwards by a safety band, and the extension amount is recorded for subsequent backtracking. Before entering the recognition process, the extracted text region images undergo lightweight normalization, including brightness range consistency, proportional scaling of long and short sides, line orientation correction, foreground and background contrast balancing, and channel normalization. The normalization parameters are bound to the region positioning information to ensure that the same region maintains a consistent input form when repeatedly accessed.

[0127] Printed and handwritten character recognition are performed within a unified recognizer. The visual front end of the recognizer extracts serialized features from the text region image, and the temporal decoding end infers the character sequence from the feature sequence. To adapt to different character shapes and writing styles, a multi-scale receptive field and stroke-sensitive feature channel are introduced at the visual front end, and a context gating unit is used at the temporal end to enhance inter-character dependencies. When there are bends or misalignments within lines, the attention alignment module guided by line orientation correction rearranges the position of the feature sequence to avoid mixing adjacent lines into the same sequence. The recognition output directly forms the initial text information, along with the alignment trajectory from position to confidence, for reference in subsequent error correction stages.

[0128] The detection of suspected erroneous terms uses initial text information as input and is determined through multi-source consistency and morphological constraints. Multi-source consistency includes consistency in repeated identification within the same region, consistency in context between adjacent regions, and consistency with historical similar samples; morphological constraints include filtering of legal character sets, distribution of character categories within words, length range, and positional patterns. When the initial text information exhibits anomalies in the above dimensions, suspected erroneous term labeling is triggered, and the start and end positions of the specific substring and the triggering reason label are marked in the text. To reduce the false positive rate, the labeling process is linked to the recognition confidence: when the recognition confidence is high and the triggering reason is singular and weak, the labeling threshold is increased; when the recognition confidence is low and multiple triggering reasons occur simultaneously, the labeling threshold is decreased.

[0129] Edit distance is determined by comparing suspected erroneous terms with pre-defined dictionary terms within a standardized character field. The standardization process includes unifying full-width and half-width characters, capitalization, merging similar-looking characters, and unifying connectors to reduce distance bias caused by format differences. To accommodate abbreviations, prefixes / suffixes, and similar-looking substitutions, the distance model distinguishes the costs of replacement, insertion, and deletion, assigning higher replacement costs to easily confused characters. For each suspected erroneous term, several pre-defined dictionary terms are retrieved from the candidate word space, and the edit distance is determined for each, outputting the minimum distance and its corresponding candidate. To avoid incorrectly replacing correctly correct short words, the distance result, word length, word frequency, and contextual consistency are all considered in the scoring. The replacement process only begins when the overall score reaches a threshold.

[0130] The process of determining edit distance is based on a quantitative measurement of the differences between two strings. Its core lies in transforming one string into another through a series of editing operations. Common editing operations include inserting a character, deleting a character, and replacing a character. In practice, a matrix is ​​first constructed to represent the degree of difference between different prefixes of the source and target strings. The first row and column of the matrix are typically initialized in character order, representing the number of operations required to transform an empty string into the target prefix or from the source string into an empty string. During calculation, the value of each element in the matrix represents the minimum edit distance between the source and target string prefixes. When the two characters at the current position are equal, the edit distance is equal to the value of its top-left element; when the characters are unequal, the minimum value among the results of insertion, deletion, and replacement operations is taken and incremented by one. By dynamically recursively filling the matrix, the edit distance between the complete strings is finally obtained at the bottom-right corner of the matrix. Taking "kitten" and "sitting" as an example, replacing the first letter k with s, replacing e with i, and inserting g at the end requires three operations to complete the transformation, therefore the edit distance is three. In practical applications, when a suspected erroneous term is detected in the initial text information, the edit distance of the term can be calculated one by one with the candidate terms in the preset dictionary, and the dictionary term with the smallest edit distance can be selected to replace the initial term, thereby achieving text correction.

[0131] Taking the Chinese scenario as an example: The initial text information is "Amoxicillin Tablets", and the candidate terms in the preset dictionary include "Amoxicillin Tablets", "Amoxicillin Capsules", and "Amoxicillin". Before entering the distance calculation, the character forms are unified, spaces are removed, and the full and half-width forms and the simplified and traditional forms are unified to obtain the aligned comparison strings. The comparison with "Amoxicillin Tablets" only requires one replacement ("Lin" instead of "Ling"), and the edit distance is one; the comparison with "Amoxicillin Capsules" requires one replacement ("Lin" instead of "Ling") and one operation of changing "Tablets" to "Capsules", which is equivalent to one replacement plus one insertion, and the total distance is three; the comparison with "Amoxicillin" requires one replacement ("Lin" instead of "Ling") and one deletion (removing "Tablets"), and the distance is two. The minimum edit distance is one, and the corresponding term is "Amoxicillin Tablets". Accordingly, the incorrect fragment in the initial text information is replaced with "Amoxicillin Tablets" to generate the corrected text information, and this corrected text information is mapped to the target text vector in the subsequent semantic encoding process.

[0132] The replacement process covers the suspected incorrect term with the preset dictionary term corresponding to the minimum edit distance to produce the corrected text information; when a text fragment contains multiple overlapping suspected incorrect terms, they are merged according to the principle of non-overlapping maximum coverage to ensure that the replacement order does not interfere with each other; after the replacement, a structure check is performed, including punctuation coherence, adhesion of numbers and dimensions, word-spacing norms, and left-right pairing of special symbols. After passing the check, the final corrected text information is generated, and the replacement track from the initial text information to the corrected text information is recorded for subsequent review or rollback.

[0133] Semantic feature encoding converts the corrected text information into the target text vector. The encoding process includes text segmentation, sub-word mapping, semantic embedding, and sequence aggregation; text segmentation first performs standard word segmentation to细分 the boundaries of Chinese and English, numbers, and symbols; sub-word mapping disassembles out-of-vocabulary words into sub-words to reduce sparsity; semantic embedding introduces position encoding and word-form encoding within the context window to take into account word order and morphology; sequence aggregation uses attention-based weight aggregation to assign higher weights to sub-fragments related to the object identifier to strengthen category discrimination information; the target text vector corresponds one-to-one with the corrected text information, with a fixed dimension and a range restricted by normalization, facilitating alignment with the image vector at the same scale; when the text contains multiple segments of information, vector encoding aggregates by segment and outputs the weighted result within the group, while retaining the weight distribution within the segment to support interpretability.

[0134] To ensure end-to-end consistency, a strict data lineage chain is established between the text region image, initial text information, suspected erroneous terms, preset dictionary terms, corrected text information, and target text vector. Each link's input, output, and key intermediate quantities are bound to a unique identifier, and the generation time and parameter summary are recorded. During offline evaluation or online backtracking, the source of error can be gradually located along the chain, such as stroke truncation caused by improper cropping boundaries, misaligned splicing caused by insufficient line direction correction, excessive replacement caused by insufficient dictionary coverage, and semantic decay caused by unbalanced encoding weights, thereby supporting steady-state optimization and continuous iteration.

[0135] This embodiment extracts text region images based on regional positioning information and completes normalization and line orientation correction, compressing the morphological differences of the initial input within a controllable range; the unified recognizer achieves co-domain decoding of printed and handwritten characters under multi-scale and stroke-sensitive representation, and the initial text information obtains stable temporal alignment; the detection of suspected erroneous terms combines consistency signals and morphological constraints, making error correction more targeted; the edit distance is determined within the normalized character domain and comprehensive constraints of length, word frequency, and context are introduced, so that the replacement process can both correct deviations and suppress erroneous replacements; semantic feature encoding compresses the corrected text information into a target text vector through sub-word mapping and attention aggregation, carrying semantic clues closely related to category discrimination.

[0136] In one embodiment, step S40 above includes:

[0137] S401, Extract the object region image from the preprocessed target image based on the region positioning information;

[0138] S402, Extract global visual features and local visual features from the object region image;

[0139] S403, The global visual features and local visual features are fused to construct a multi-level feature map;

[0140] S404, perform attention feature enhancement on the multi-level feature map to generate original image features;

[0141] S405, Perform contrastive learning normalization processing on the original image features to generate normalized image features;

[0142] S406, the normalized image features are encoded into a target image vector.

[0143] In this embodiment, the region positioning information provides a mapping between the object boundary and the coordinate system. Based on this information, the object region image is cropped and geometrically aligned within the preprocessed target image. The cropping process preserves a boundary buffer zone to accommodate slight positioning deviations; geometric alignment uses the previously established mapping table to restore the correct shape, avoiding the influence of perspective or curvature on subsequent representations. The object region image undergoes normalization preprocessing to unify the color space and dynamic range, scale along the long side while maintaining the aspect ratio, generate a stable input resolution, and record the scaling factor for coordinate backtracking.

[0144] Global and local visual features are extracted separately for information at different scales. Global visual features are obtained through deep semantic representation, focusing on contours, shapes, layout, and color schemes. Single-frame vectors can be generated by pooling deep features or equivalent aggregation operations. Local visual features are obtained through shallow and medium-level representation, depicting textures, edges, text cutouts, logos, and anti-counterfeiting patterns. Local descriptive subsets can be formed on a dense grid. To suppress background interference, object region masks and edge confidence maps are introduced to weight feature responses, resulting in low background response and high response of key textures. For areas with reflections or shadows, brightness-stabilized masks are used to limit the contribution of features at those locations.

[0145] Multi-level feature maps are obtained by fusing global and local visual features across multiple scales. Top-down paths carry semantic abstraction, bottom-up paths supplement detailed textures, and lateral connections align information at the same resolution. During fusion, scale, direction, and texture density are weighted; dense text regions increase the proportion of fine-grained layers, while regularly shaped regions increase the proportion of semantic layers. To maintain spatial consistency, all fusion operations are performed within a unified coordinate grid, and the weight mapping and source markers for each layer are preserved for easy tracing and visual review.

[0146] Attention feature enhancement performs a combined spatial and channel attention operation on multi-level feature maps. Spatial attention generates attention maps based on saliency, edge confidence, and region masking, emphasizing key areas and weakening background patterns. Channel attention redistributes channel weights based on statistical distribution and mutual information strength, retaining highly discriminative feature channels and suppressing redundant channels. For objects with significant deformation or discrete key textures, a deformable sampling attention mechanism is introduced, adaptively selecting sampling points within the local neighborhood to improve the representation ability of non-rigid regions. After enhancement, the original image features are output, including dense feature maps and corresponding attention weight records.

[0147] Contrastive learning normalization establishes consistency in representation scale and distribution within the feature space. First, it constructs positive pairs of the same class and negative pairs of different classes: positive pairs originate from multi-view enhancements of images of the same object region or different appearance samples of the same category, while negative pairs originate from samples with similar appearances but different categories, and interference samples with similar textures. The original image features are projected into the contrast space, and normalized image features are formed through target constraints of bringing positive pairs closer and negative pairs further apart. To address fine-grained differences, difficult negative samples are added to the training through online mining to improve boundary resolution. To reduce sampling bias, a balanced sampling and class frequency reweighting strategy is introduced to ensure the comparability and stability of the normalized distribution across different objects.

[0148] Normalized image features are ultimately encoded into target image vectors. Encoding involves three stages: dimensionality reduction, scale normalization, and orientation normalization. Dimensionality reduction compresses redundant dimensions and improves inference efficiency; scale normalization unifies vector magnitudes; and orientation normalization maintains the measurability of the angular space. Each target image vector corresponds one-to-one with the object region image, possessing traceable metadata, including source coordinates, scaling factors, enhanced weight distribution, and a contrast-normalized sampling summary. To support subsequent alignment with text vectors within the same semantic space, a cross-modal anchor interface is reserved during the encoding stage. This can be achieved through shared category prototypes or semantic pivots for initial alignment initialization, without altering the independence and discriminativeness of the output from this stage.

[0149] This embodiment uses region positioning information and geometric alignment to ensure that the object region image has a stable shape before entering the representation, reducing the uncertainty caused by pose and distortion; the separate extraction of global and local visual features takes into account both shape and detail, and the bidirectional fusion of multi-level feature maps converges semantics and texture in a unified coordinate system; attention feature enhancement improves the response of key regions and suppresses background interference with masking and saliency constraints, and the original image features obtain a higher discriminative density; contrastive learning normalization pulls the representation distribution back to a stable scale, and the participation of difficult samples constrains fine-grained boundaries, and the normalized image features maintain comparability under cross-scene and cross-device conditions; the encoded output target image vector has consistency in direction and scale and carries a complete lineage record, which facilitates direct entry into the same domain measurement with the target text vector in subsequent multimodal alignment.

[0150] In one embodiment, step S50 above includes:

[0151] S501, The target text vector is converted to a semantic space by a text encoder to generate a text semantic vector;

[0152] S502, the target image vector is converted to the same semantic space as the text semantic vector by an image encoder to generate an image semantic vector;

[0153] S503, determine the similarity value between the text semantic vector and the image semantic vector;

[0154] S504, determine dynamic weight parameters based on the source types of the target text vector and the target image vector;

[0155] S505, The similarity values ​​are weighted and fused using the dynamic weight parameters to generate a fused confidence score;

[0156] S506, Generate preliminary identification results based on the fusion confidence level.

[0157] In this embodiment, the target text vector undergoes semantic mapping after entering the text encoder, outputting a text semantic vector. During the mapping process, the vector scale and orientation are first unified, then positional and lexical markers are introduced to maintain segment boundaries and lexical information. Subsequently, weighted aggregation is performed near semantic pivots to form a stable semantic representation. The target image vector undergoes co-domain mapping after entering the image encoder, outputting an image semantic vector. During the mapping process, scale and orientation standardization are first performed, then weights are redistributed between local and global representations guided by multi-layered semantic anchors, ensuring that shape, texture, and layout information are consistently presented in a unified coordinate system. To avoid cross-modal shift, the two encoders share a set of semantic anchors and constrain the scale boundaries after mapping, making the text semantic vector and image semantic vector comparable.

[0158] Text and image semantic vectors are fed into a metric unit to determine their similarity values. The metric unit operates only within the normalized vector space, providing a single metric through two dimensions: directional proximity and energy distribution consistency. The output is constrained to a stable range and possesses traceable labeling. Dynamic weight parameters are then generated based on the source type. The source type is provided by the preceding link: on the text side, it distinguishes between prescription text and packaging text; on the image side, it distinguishes between whole objects and local regions. The weight generator encodes the source type as a weight cue and incorporates scene cue and shooting condition cue to jointly determine the weight coefficient, outputting both text-side and image-side weights, along with a confidence band to indicate weight reliability. The weight cue and similarity value undergo weighted fusion within the fusion unit to obtain the fusion confidence score. The fusion process maintains monotonicity and interpretability; that is, changes in source type will affect the fusion confidence score in a predictable direction, and each fusion records the weight source, cue content, and metric source.

[0159] Preliminary identification results are generated based on the fusion confidence score. The generation unit receives the fusion confidence score and a set of candidate identifiers, and employs a segmented judgment strategy within the threshold band: when the fusion confidence score is in the high confidence band, the first candidate identifier is directly given; when it is in the intermediate band, narrow-domain re-evaluation is triggered, and the candidate with the highest consistency among adjacent candidates is selected; when it is in the low confidence band, a temporary identifier with an uncertain marker is output according to the expanded candidate list. The generation unit also outputs accompanying information, including source type records, weight hint snapshots, metric tracking markers, and candidate ranking, ensuring that the results are traceable and verifiable. A one-to-one mapping and timestamp binding are established between text semantic vectors, image semantic vectors, similarity values, dynamic weight parameters, fusion confidence scores, and preliminary identification results, allowing any review to pinpoint the specific source type and weight hints affecting the results.

[0160] This embodiment performs co-domain mapping between target text vectors and target image vectors within a unified semantic space. It then uses dynamic weight parameters driven by source type to perform weighted fusion of similarity values ​​and outputs the fusion confidence score. A segmented judgment strategy is then used to generate preliminary recognition results. This allows textual and image clues to work together under consistent measurement, weakening the uncertainty caused by single-modal distortion, reducing the risk of offset under cross-scene and cross-device conditions, and forming a traceable and interpretable intermediate quantity link, providing stable input for subsequent knowledge retrieval and standardized verification.

[0161] In one embodiment, step S60 above includes:

[0162] S601, determine the strictness level of knowledge base retrieval based on the fusion confidence level;

[0163] S602, based on the strictness level of the knowledge base retrieval, retrieve standard entries in the preset knowledge base that match the preliminary identification results;

[0164] S603, performs a normalization mapping on the matched standard entries to generate a unique standard code;

[0165] S604, Perform a rationality check based on the standard information corresponding to the unique standard code, and generate a verification result;

[0166] S605, Generate final identification information based on the unique standard code and the verification result;

[0167] S606, output the final identification information and the fusion confidence level.

[0168] In this embodiment, the fusion confidence level and preliminary identification results are simultaneously input into the retrieval control unit. First, the retrieval strictness level is generated by the confidence level segmentation rules. Based on the confidence band, the segmentation rules classify the input into a high-confidence band, intermediate band, or low-confidence band, and internally register the triggering reason, source type record, and timestamp. The high-confidence band is bound to a precise retrieval strategy, the intermediate band to an extended retrieval strategy, and the low-confidence band to a wide-area retrieval strategy, while retaining level labels for subsequent comparisons and backtracking. The retrieval control unit then assembles the retrieval template and field weights accordingly, avoiding over-limitation under low-quality clues or amplifying ambiguity under high-quality clues.

[0169] The retrieval execution unit initiates queries against a pre-defined knowledge base. The knowledge base uses standard entries as basic units, each containing a standard name, unique code, a list of aliases, hierarchical relationships, expiration status, and version information. The precise retrieval strategy prioritizes matching standard names and unique codes, with intermediate bands adding matches for aliases, abbreviations, language variations, and common misspellings. The low-confidence band introduces neighborhood expansion for similar entries, allowing retrieval of candidates with similar forms or semantic proximity within category boundaries. To reduce false hits, the retrieval execution unit appends a source field matching score and an expiration confidence score to each candidate, and records the location of the matching document, the matching field, and the source of the weights, forming a traceable set of matching evidence.

[0170] After matching is complete, the standardization mapping process begins, transforming candidate entries into unique standard codes. The mapping table supports many-to-one merging and cross-version inheritance. When product names and generic names coexist, conflict resolution is based on mapping priority and expiration status. For cases of different items with the same name, the merging direction is limited by combining category hierarchy and source field matching. Successful mapping generates a unique standard code, along with the output mapping path, basis fields, conflict resolution records, and version tags, ensuring that subsequent verification can trace back to specific evidence.

[0171] Reasonableness checks perform consistency verification based on standard information associated with a unique standard code. The check dimensions include name consistency, category consistency, parameter range constraints, one-to-many dependencies, compatibility restrictions, expiration status, and version consistency. Name consistency identifies deviations caused by similar shapes or sounds; category consistency prevents misclassification across categories; parameter range constraints cover key parameters carried in specifications, formats, or unit text; one-to-many dependencies handle scenarios requiring collaborative verification with other entries; compatibility restrictions identify mutually exclusive or discouraged combinations; expiration status and version consistency ensure that entries are not expired and that the referenced version is consistent with the current baseline. Each check generates local conclusions and evidence pointers, which are ultimately summarized into a verification result and provided with readable structured tags.

[0172] The results generation unit synthesizes the unique standard code and verification results into the final identification information. The synthesis process includes three stages: field integration, evidence aggregation, and risk labeling. Field integration produces the standard code, standard name, mapping path summary, version, and timeliness tags; evidence aggregation retains the most critical matching and verification evidence; risk labeling adds uncertainty markers or collaborative verification suggestions based on the verification results. The fusion confidence level and the final identification information are output together, bound by the same processing serial number, allowing for traceability of the strictness level, matching evidence, and verification evidence at any review. The entire process completes the transformation from clues to standard entities without altering the upstream semantic vector, and maintains the auditability and maintainability of the results through a rigorous chain of evidence.

[0173] Example Description: In the healthcare field, the system first receives an image of medicine packaging taken by a home caregiver. This image may have quality issues due to insufficient lighting or motion blur. The system performs quality analysis on the image, identifying noise density, brightness uniformity, and sharpness level. Based on the analysis results, it selects an appropriate denoising module. For example, under low-light conditions, a multi-scale wavelet filter module is prioritized to perform multi-scale noise filtering on the image, resulting in clearer denoised image data. Subsequently, the system detects geometric distortion features of the image, such as tilt angle or perspective distortion, and performs adaptive geometric correction to generate corrected image data that maintains normal proportions and angles. Based on this, the corrected image data is enhanced by fusing color, texture, and structural features, making the boundaries between text and objects more prominent, thus generating the preprocessed target image.

[0174] After obtaining the preprocessed target image, the system further performs region localization. First, multi-scale convolutional features are extracted from the image, and a feature pyramid is constructed to capture detailed information at different levels on the drug packaging. Initial region proposals are generated based on the feature pyramid, and then boundary regression is used to fine-tune the boundaries of these regions. Finally, the system distinguishes and labels text regions and object regions, generating region localization information containing specific coordinates, laying the foundation for subsequent text recognition and image feature extraction.

[0175] Guided by regional positioning information, the system extracts text regions from images and performs printed and handwritten character recognition to generate initial text information. For example, text extracted from the surface of a medicine box might contain the drug name "Amoxicillin Capsules" and dosage information "0.25g." During recognition, blurring errors due to lighting or ambiguity caused by handwritten annotations may occur. The system detects suspected erroneous terms in the recognition results and calculates the edit distance between these terms and standard terms in a pre-defined dictionary. For example, if the recognition result is "Amoxicillin," the system automatically compares it and finds that its edit distance to "Amoxicillin" in the dictionary is 1, which is the minimum replacement distance. Therefore, it replaces it with the correct term from the dictionary, generating corrected text information. Subsequently, the system performs semantic feature encoding on the corrected text information to obtain the target text vector.

[0176] Simultaneously, the system extracts image features of the drug packaging from the target region. First, based on regional positioning information, the image of the target region is cropped, extracting global visual features such as overall shape, packaging color, and layout. Simultaneously, local visual features such as the dosage label area and local patterns in the production batch number area are extracted. By fusing global and local features, a multi-level feature map is constructed. Then, an attention enhancement operation is performed on the feature map to increase the weight of key region features and reduce the influence of interfering regions. The obtained raw image features are then normalized through contrastive learning to generate more discriminative and uniformly scaled normalized image features, which are then encoded into the target image vector.

[0177] After generating text and image vectors, the system maps the target text vector and the target image vector to the same semantic space. Text and image semantic vectors are generated separately using a text encoder and an image encoder, respectively, and their similarity values ​​are calculated. The system dynamically adjusts weight parameters based on the type of input source; for example, it increases the weight of text when dealing with prescription labels and increases the weight of images when dealing with packaging images. Finally, the system fuses the similarity value with the dynamic weights to generate a fused confidence score, and uses this confidence score to generate a preliminary recognition result.

[0178] After initial identification, the system uses the fusion confidence score and the initial identification results to enter the knowledge base retrieval stage. The fusion confidence score determines the rigor of the search; a high confidence score employs an exact match strategy, while a low confidence score expands the search scope to include aliases, generic names, and possible synonyms. The system finds the standard entry in the knowledge base corresponding to the initial identification results and maps it to a unique standard code, such as the approval number from the National Medical Products Administration. Subsequently, the system performs a rationality check based on the standard information corresponding to this unique standard code, such as whether the dosage range conforms to regulations, whether there are contraindications in the applicable population, or whether the packaging label is consistent with the approval information. After the rationality check generates a verification result, the system combines the unique standard code with the verification result to form the final identification information. Finally, the system outputs the final identification information and the fusion confidence score, providing an accurate reference for subsequent health management.

[0179] In the fintech business, users often upload various document files, including invoices, contract images, and reimbursement vouchers. The quality of these files is easily affected by factors such as shooting equipment, lighting conditions, and angle deviations, resulting in blurriness, noise, and distortion. After receiving the target image, the system first preprocesses it, judging its sharpness and integrity through quality analysis. Then, it selects an appropriate denoising module to perform multi-scale noise filtering and performs adaptive geometric correction on distorted areas to maintain image regularity. Finally, it performs feature fusion enhancement to ensure that the preprocessed target image has a stable visual representation, facilitating subsequent processing.

[0180] In the preprocessed image, the system identifies and distinguishes between text regions and object regions through feature extraction and region localization. For example, in a value-added tax invoice, the text region may correspond to the amount, invoice number, and taxpayer identification number, while the object region may correspond to a QR code, watermark, or layout structure. Through multi-scale convolutional features and feature pyramids, the system generates region proposal boxes and optimizes them using boundary regression to ensure that key information on the invoice is located completely and accurately, ultimately generating region localization results containing coordinate information.

[0181] Based on regional location information, text regions are extracted and identified individually. During this process, the system can process both printed and handwritten text, thereby extracting initial text information. For any erroneous characters or incomplete words that may appear in the recognition results, the system compares them against a pre-set dictionary, uses edit distance calculation to determine the closest standard term, and automatically replaces suspected errors, thus obtaining a corrected text representation. The corrected result is then semantically encoded into a target text vector, ensuring that subsequent processing stages can use a consistent and high-quality semantic representation.

[0182] In processing the object region, the system extracts global and local features, such as the layout framework, seal distribution, and the geometric structure of the QR code, and fuses them into a multi-level feature map. It then uses an attention mechanism to enhance the weights of important regions, thereby generating more discriminative original image features. Subsequently, the system performs normalization through contrastive learning, ensuring consistency in the representation of tickets from different sources, and finally encodes them into a target image vector.

[0183] Once the target text vector and target image vector are ready, the system maps them to a unified semantic space and performs consistency analysis. Similarity calculation measures the degree of matching between text and image, while the introduction of dynamic weight parameters allows for flexible adjustments in different scenarios. For example, when an image is detected as a clear scan, the system increases the text weight; conversely, when a clear stamp or QR code is detected, the image weight is appropriately increased. In this way, the system outputs a fused confidence score and preliminary recognition result, ensuring both accuracy and robustness.

[0184] Finally, the system searches and compares data within a pre-set financial knowledge base based on the fusion confidence level and preliminary identification results. This knowledge base includes standardized invoice codes, contract item specifications, tax rate rules, and other content. The system dynamically adjusts the search scope according to different confidence levels; at high confidence levels, it uses exact matching, while at low confidence levels, it expands to fuzzy matching and synonym search. Matched standard items are mapped to unique standard codes, and reasonableness checks are performed based on the rules corresponding to these codes. For example, the system checks whether invoice amounts exceed reasonable limits, whether contract terms are consistent with regulatory standards, and whether reimbursement vouchers are within their stipulated validity period. The results of these reasonableness checks, along with the standard codes, are used to generate the final identification information and output along with the fusion confidence level, providing reliable compliance data for financial business systems.

[0185] This embodiment transforms fusion confidence into a retrieval strictness level, allowing the retrieval scope and field weights to adaptively switch according to input quality, avoiding excessive diffusion of high-confidence clues and excessive tightening of low-confidence clues. Through a hierarchical strategy of precise retrieval, expanded retrieval, and wide-area retrieval, candidate coverage and false positive risk are balanced within the same mechanism. Standardized mapping outputs unique standard codes, eliminating inconsistencies caused by alias coexistence and version drift. Multidimensional consistency verification jointly constrains name, category, parameters, compatibility, and timeliness, reducing erroneous merging and cross-class confusion. Through evidence aggregation and risk labeling in result synthesis, the output not only includes standard entities but also traceable evidence and clear risk warnings.

[0186] In one embodiment, a semantic space consistency-based identification device is provided, which corresponds one-to-one with the semantic space consistency-based identification method described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the semantic space consistency-based recognition device of the present invention. The modules include an image preprocessing module 10, a region localization module 20, a text recognition and correction module 30, an image feature extraction module 40, a cross-modal consistency analysis module 50, and a knowledge base retrieval and comparison module 60. Detailed descriptions of each functional module are as follows:

[0187] Image preprocessing module 10 is used to acquire the target image to be identified, preprocess the target image, and generate a preprocessed target image;

[0188] The region positioning module 20 is used to locate the text region and the object region in the preprocessed target image and generate region positioning information;

[0189] The text recognition and correction module 30 is used to perform text recognition on the text region based on the region positioning information, obtain initial text information, correct the initial text information according to a preset dictionary, and generate a target text vector;

[0190] The image feature extraction module 40 is used to extract image features of the object region based on the region positioning information, normalize the image features, and generate a target image vector.

[0191] The cross-modal consistency analysis module 50 is used to map the target text vector and the target image vector to the same semantic space, perform consistency analysis, and generate fusion confidence and preliminary recognition results.

[0192] The knowledge base retrieval and comparison module 60 is used to perform retrieval and comparison in a preset knowledge base based on the fusion confidence and the preliminary identification result, obtain the final identification information, and output the final identification information and the fusion confidence.

[0193] In one embodiment, the image preprocessing module 10 is specifically used for:

[0194] Acquire the target image to be identified, perform image quality analysis on the target image to be identified, and generate quality analysis results;

[0195] Based on the quality analysis results, a denoising module is selected, and the selected denoising module is used to perform multi-scale noise filtering on the target image to be identified, generating denoised image data.

[0196] Detect geometric distortion features in the denoised image data;

[0197] Adaptive geometric correction is performed on the denoised image data based on the geometric distortion features to generate corrected image data;

[0198] The corrected image data is subjected to feature fusion enhancement processing to generate enhanced image data, which is then used as the preprocessed target image.

[0199] In one embodiment, the area positioning module 20 is specifically used for:

[0200] Extract the multi-scale convolutional features from the preprocessed target image;

[0201] Construct a feature pyramid based on the multi-scale convolutional features;

[0202] An initial region proposal box is generated based on the feature pyramid;

[0203] Boundary regression optimization is performed on the initial region proposal box to generate an optimized region boundary;

[0204] Distinguish between text regions and object regions within the optimized region boundary, and generate region positioning information containing coordinate attributes.

[0205] In one embodiment, the text recognition and correction module 30 is specifically used for:

[0206] Based on the region positioning information, extract the text region image from the preprocessed target image;

[0207] Perform printed and handwritten character recognition on the text region image to generate initial text information;

[0208] Detect suspected erroneous terms in the initial text information;

[0209] Determine the edit distance between the suspected erroneous terms and the preset dictionary terms;

[0210] The suspected erroneous terms in the initial text information are replaced with preset dictionary terms that have the minimum edit distance to generate corrected text information.

[0211] The corrected text information is semantically encoded to generate a target text vector.

[0212] In one embodiment, the image feature extraction module 40 is specifically used for:

[0213] Based on the region positioning information, extract the object region image from the preprocessed target image;

[0214] Extract global and local visual features from the image of the object region;

[0215] By fusing the global and local visual features, a multi-level feature map is constructed.

[0216] Attention feature enhancement is performed on the multi-level feature maps to generate the original image features;

[0217] Perform contrastive learning normalization processing on the original image features to generate normalized image features;

[0218] The normalized image features are encoded into a target image vector.

[0219] In one embodiment, the cross-modal consistency analysis module 50 is specifically used for:

[0220] The target text vector is transformed into a semantic space using a text encoder to generate a text semantic vector.

[0221] The target image vector is converted to the same semantic space as the text semantic vector by an image encoder to generate an image semantic vector.

[0222] Determine the similarity value between the text semantic vector and the image semantic vector;

[0223] Based on the source types of the target text vector and the target image vector, determine the dynamic weight parameters;

[0224] The similarity values ​​are weighted and fused using the dynamic weight parameters to generate a fusion confidence score.

[0225] Preliminary identification results are generated based on the fusion confidence level.

[0226] In one embodiment, the knowledge base retrieval and comparison module 60 is specifically used for:

[0227] The retrieval strictness level of the knowledge base is determined based on the fusion confidence level.

[0228] Based on the strictness level of the knowledge base retrieval, standard entries matching the preliminary identification results are retrieved from the preset knowledge base.

[0229] Perform normalization mapping on the matching standard entries to generate unique standard codes;

[0230] A rationality check is performed based on the standard information corresponding to the unique standard code, and a verification result is generated;

[0231] The final identification information is generated based on the unique standard code and the verification result;

[0232] Output the final identification information and the fusion confidence score.

[0233] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a semantic space consistency-based identification method on the server side.

[0234] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a semantic space consistency-based identification method.

[0235] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0236] Acquire the target image to be identified, preprocess the target image, and generate a preprocessed target image;

[0237] In the preprocessed target image, locate the text region and the object region to generate region positioning information;

[0238] Based on the region positioning information, text recognition is performed on the text region to obtain initial text information. The initial text information is then corrected according to a preset dictionary to generate a target text vector.

[0239] Based on the region positioning information, image features of the object region are extracted, and the image features are normalized to generate a target image vector.

[0240] The target text vector and the target image vector are mapped to the same semantic space, and consistency analysis is performed to generate fusion confidence and preliminary recognition results.

[0241] Based on the fusion confidence score and the preliminary identification result, a search and comparison are performed in a preset knowledge base to obtain the final identification information, and the final identification information and the fusion confidence score are output.

[0242] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0243] Acquire the target image to be identified, preprocess the target image, and generate a preprocessed target image;

[0244] In the preprocessed target image, locate the text region and the object region to generate region positioning information;

[0245] Based on the region positioning information, text recognition is performed on the text region to obtain initial text information. The initial text information is then corrected according to a preset dictionary to generate a target text vector.

[0246] Based on the region positioning information, image features of the object region are extracted, and the image features are normalized to generate a target image vector.

[0247] The target text vector and the target image vector are mapped to the same semantic space, and consistency analysis is performed to generate fusion confidence and preliminary recognition results.

[0248] Based on the fusion confidence score and the preliminary identification result, a search and comparison are performed in a preset knowledge base to obtain the final identification information, and the final identification information and the fusion confidence score are output.

[0249] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0250] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0251] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0252] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A recognition method based on semantic space consistency, characterized in that, The method comprises the following steps: obtaining a target image to be recognized, pre-processing the target image to generate a pre-processed target image; locating a text region and an object region in the pre-processed target image to generate region positioning information; based on the region positioning information, performing text recognition on the text region to obtain initial text information, correcting the initial text information according to a preset dictionary to generate a target text vector; based on the region positioning information, extracting image features of the object region, and performing normalization processing on the image features to generate a target image vector; mapping the target text vector and the target image vector to the same semantic space, performing consistency analysis to generate a fusion confidence and a preliminary recognition result; based on the fusion confidence and the preliminary recognition result, searching and comparing in a preset knowledge base to obtain final recognition information, and outputting the final recognition information and the fusion confidence.

2. The recognition method based on semantic space consistency according to claim 1, characterized in that, obtaining a target image to be recognized, pre-processing the target image to generate a pre-processed target image, comprising: obtaining a target image to be recognized, performing image quality analysis on the target image to be recognized to generate a quality analysis result; selecting a denoising module based on the quality analysis result, and using the selected denoising module to perform multi-scale noise filtering on the target image to be recognized to generate denoised image data; detecting geometric distortion features in the denoised image data; performing adaptive geometric correction on the denoised image data according to the geometric distortion features to generate corrected image data; performing feature fusion enhancement processing on the corrected image data to generate enhanced image data, and taking the enhanced image data as the pre-processed target image.

3. The recognition method based on semantic space consistency according to claim 1, characterized in that, locating a text region and an object region in the pre-processed target image to generate region positioning information, comprising: extracting multi-scale convolution features of the pre-processed target image; constructing a feature pyramid based on the multi-scale convolution features; generating initial region proposal boxes based on the feature pyramid; performing boundary regression optimization on the initial region proposal boxes to generate optimized region boundaries; distinguishing text regions and object regions in the optimized region boundaries to generate region positioning information containing coordinate attributes.

4. The recognition method based on semantic space consistency according to claim 1, wherein, based on the region positioning information, performing text recognition on the text region to obtain initial text information, correcting the initial text information according to a preset dictionary to generate a target text vector, comprising: extracting a text region image from the pre-processed target image based on the region positioning information; performing printed and handwritten character recognition on the text region image to generate initial text information; detecting suspected error terms in the initial text information; determining the edit distance of the suspected error terms and the preset dictionary terms; using the preset dictionary terms with the smallest edit distance to replace the suspected error terms in the initial text information to generate corrected text information; performing semantic feature coding on the corrected text information to generate a target text vector.

5. The recognition method based on semantic space consistency according to claim 1, characterized in that, based on the region positioning information, extracting image features of the object region, and performing normalization processing on the image features to generate a target image vector, comprising: extract an object region image from the preprocessed target image based on the region positioning information; extract global visual features and local visual features from the object region image; fuse the global visual features and the local visual features to construct a multi-level feature map; perform attention feature enhancement on the multi-level feature map to generate an original image feature; perform contrast learning normalization processing on the original image feature to generate a normalized image feature; encode the normalized image feature into a target image vector.

6. The recognition method based on semantic space consistency according to claim 1, wherein, map the target text vector and the target image vector to the same semantic space to perform consistency analysis, generate a fusion confidence and a preliminary recognition result, including: convert the target text vector to a semantic space through a text encoder to generate a text semantic vector; convert the target image vector to the same semantic space as the text semantic vector through an image encoder to generate an image semantic vector; determine a similarity value of the text semantic vector and the image semantic vector; determine a dynamic weight parameter based on the source type of the target text vector and the target image vector; use the dynamic weight parameter to perform weighted fusion processing on the similarity value to generate a fusion confidence; generate a preliminary recognition result based on the fusion confidence.

7. The recognition method based on semantic space consistency according to claim 1, wherein, based on the fusion confidence and the preliminary recognition result, search and compare in a preset knowledge base to obtain final recognition information, output the final recognition information and the fusion confidence, including: determine a knowledge base search strict level based on the fusion confidence; search for a standard item matching the preliminary recognition result in a preset knowledge base according to the knowledge base search strict level; perform standardized mapping on the matched standard item to generate a unique standard code; perform a reasonableness check based on the standard information corresponding to the unique standard code to generate a verification result; generate final recognition information based on the unique standard code and the verification result; output the final recognition information and the fusion confidence.

8. A recognition apparatus based on semantic space consistency, characterized by, The recognition device based on semantic space consistency includes: an image preprocessing module configured to obtain a target image to be recognized, and perform preprocessing on the target image to generate a preprocessed target image; a region positioning module configured to locate a text region and an object region in the preprocessed target image to generate region positioning information; a text recognition and correction module configured to perform text recognition on the text region based on the region positioning information to obtain initial text information, and correct the initial text information according to a preset dictionary to generate a target text vector; an image feature extraction module configured to extract image features of the object region based on the region positioning information, and perform normalization processing on the image features to generate a target image vector; a cross-modal consistency analysis module configured to map the target text vector and the target image vector to the same semantic space to perform consistency analysis, and generate a fusion confidence and a preliminary recognition result. The knowledge base searching and comparing module is configured to search and compare in a preset knowledge base based on the fusion confidence and the preliminary identification result, obtain final identification information, and output the final identification information and the fusion confidence.

9. A computer device, comprising: The computer device comprises a memory, a processor, and a semantic space consistency based identification program stored in the memory and executable on the processor, and the semantic space consistency based identification program, when executed by the processor, implements the steps of the semantic space consistency based identification method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a semantic space consistency based identification program, and the semantic space consistency based identification program, when executed by the processor, implements the steps of the semantic space consistency based identification method according to any one of claims 1-7.

Images