Document visual scanning processing method and system fusing semantic understanding and artificial intelligence

Abstract

The application discloses a document visual scanning processing method and system fusing semantic understanding and artificial intelligence, comprising: performing irregular deformation correction on a pretreated image, wherein the irregular deformation correction corrects paper deformation through coordinate mapping and does not modify the gray value of the mapped pixels, to obtain a corrected image; performing indentation detection on the corrected image to generate a mask representing an indentation area; performing text recognition processing on the area outside the mask in the corrected image with the mask as a constraint to obtain text content and text position information of the text content; and establishing a corresponding association between the indentation area and the text content according to the position information of the indentation area and the text position information. The complete retention of colorless indentation and the analysis and recognition of text are simultaneously achieved, and the spatial corresponding association between the indentation position and the text content is established, so that the automatic binding of physical traces and digital semantic information is realized, thereby providing a unified data basis for bidirectional traceability search and compliance audit of warehouse documents.

Description

Technical Field

[0001] This application belongs to the field of visual scanning technology, specifically relating to a document visual scanning processing method and system that integrates semantic understanding and artificial intelligence. Background Technology

[0002] In unattended warehousing scenarios where temperatures fluctuate between -20℃ and 45℃, paper documents often exhibit irregular elastic deformations such as dampness, drying, wrinkling, and warping. Additionally, these documents tend to adhere closely to metal shelves, creating high gloss reflections. During warehousing operations, staff use a hard tool to press into designated verification cells on the documents, creating colorless indentations. These indentations represent permanent plastic deformation of the paper, with no ink residue, making them tamper-proof and crucial physical evidence for verification and accountability in warehousing. In accordance with corporate internal control and compliance standards, digital archiving of documents must fully preserve the location, outline, and quantity of these indentations. Furthermore, it must complete image correction, text enhancement, optical character recognition, and text semantic structure analysis to link and archive text data with indentation information.

[0003] Existing technologies generally employ a single-link serial image processing architecture, with a typical process as follows: original image → global Gaussian blur noise reduction → global contrast enhancement → global binarization → four-point perspective correction → optical character recognition. This architecture, with optimizing text display as its core objective, presents the following significant technical challenges: global smoothing and enhancement processes smooth out subtle pixel-level grayscale differences, directly eliminating colorless indentations and resulting in the permanent loss of compliant traceability documents; conversely, abandoning smoothing and enhancement operations leads to reflections and wrinkles causing broken character strokes and a significant decrease in recognition accuracy, thus creating a core technical contradiction between text enhancement and indentation retention. Furthermore, traditional four-point perspective transformation is only applicable to standard rectangular planar documents and cannot correct local warping and irregular wrinkles caused by temperature variations, resulting in simultaneous coordinate shifts between text and indentations; traditional archiving systems store image files and plain text data independently, failing to establish positional associations and business logic binding between the two, making bidirectional traceability difficult.

[0004] Therefore, existing technologies suffer from defects such as mutually exclusive core feature processing, insufficient non-standard deformation correction capabilities, and fragmented multi-source data, which cannot meet the comprehensive business needs of high and low temperature warehousing documents that require both indentation retention and text parsing. Summary of the Invention

[0005] The purpose of this application is to provide a document visual scanning processing method and system that integrates semantic understanding and artificial intelligence, which can solve the defects of related technologies such as mutual exclusion of core feature processing, insufficient non-standard deformation correction capability, and fragmentation of multi-source data, and cannot meet the comprehensive business needs of high and low temperature warehouse documents that take into account both indentation retention and text parsing.

[0006] On the one hand, this application provides a document visual scanning processing method that integrates semantic understanding and artificial intelligence, including: Obtain the original image of the warehouse document; The original image is subjected to partitioned exposure control and small-neighborhood filtering to obtain a preprocessed image; The preprocessed image is subjected to irregular deformation correction. The irregular deformation correction corrects the paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels, thus obtaining a corrected image. Indentation detection is performed on the corrected image to generate a mask representing the indentation region; Using the mask as a constraint, text recognition processing is performed on the regions in the corrected image located outside the mask to obtain text content and text position information of the text content, wherein the regions located inside the mask are left unprocessed; At least based on the location information of the indentation area and the location information of the text, a corresponding association between the indentation area and the text content is established.

[0007] In one optional embodiment of this application, the filtering process based on small-size neighborhood is a 1×N linear neighborhood median filter, where N is an integer greater than 1 and not greater than 5.

[0008] In one optional embodiment of this application, the 1×N linear neighborhood median filter is a 1×3 linear neighborhood median filter, and its processing formula is as follows:

[0009] in, For pixels in grayscale value at that location This represents the median operation. This represents the filtered pixel grayscale value.

[0010] In one optional embodiment of this application, the step of performing zoned exposure control on the original image includes: The original image is divided into M×N local sub-regions, where M and N are both integers greater than 1; Calculate the average brightness of each local sub-region. Local sub-regions with an average brightness higher than a preset highlight threshold are identified as highlight reflection areas, and the exposure gain of these areas is reduced. Local sub-regions with an average brightness lower than a preset low brightness threshold are identified as dark areas, and the exposure gain of these areas is increased. Local sub-regions with an average brightness between the preset highlight threshold and the preset low brightness threshold retain their original exposure parameters.

[0011] In one optional embodiment of this application, the irregular deformation correction of the preprocessed image includes: Detect table lines in the preprocessed image, and divide the preprocessed image into multiple cell blocks based on the table lines; Using each cell block as a correction unit, thin-plate spline interpolation transformation is performed using control points located within that cell block to obtain the correction coordinates of that cell block; Based on the corrected coordinates, the pixels of the cell block are remapped to the standard position while keeping the pixel grayscale value unchanged.

[0012] In one optional embodiment of this application, the step of performing indentation detection on the corrected image and generating a mask representing the indentation region includes: Perform multi-scale local light and shadow difference operations on the corrected image to obtain the light and shadow difference response value; The texture gradient features of the corrected image are calculated to obtain the gradient value; The indentation pixels are marked according to the joint discrimination condition, wherein the joint discrimination condition is: the absolute value of the light and shadow difference response value is greater than the preset difference threshold, and the gradient value is between the preset lower gradient limit and the preset upper gradient limit; The mask is generated based on the marked indentation pixels, and the position information of the indentation area is extracted.

[0013] In one optional embodiment of this application, the neighborhood size used in the multi-scale local lighting difference operation includes 1×3 and 3×1.

[0014] In one optional embodiment of this application, the step of performing character recognition processing on the region outside the mask in the corrected image, constrained by the mask, includes: Local adaptive threshold binarization is performed on the image outside the mask to obtain a binarized image; Optical character recognition is performed on the binarized image to obtain the text content and the pixel coordinates of the text content; Based on the fixed table format of the warehouse documents, the text content is semantically structured and parsed to obtain structured text data.

[0015] In one optional embodiment of this application, the local threshold calculation formula used for the local adaptive threshold binarization is:

[0016] in, In pixels A local neighborhood window centered on the center, The total number of pixels within the local neighborhood window. Let C be the grayscale value of the corrected image within the local neighborhood window, and C be a grayscale offset constant. The local threshold is defined as follows.

[0017] In one optional embodiment of this application, establishing the correspondence between the indentation area and the text content includes: Using the table cells of the aforementioned warehouse document as the associated units, a cell number is established for each table cell; The text location information, text content, and location information of the indentation area falling within the same table cell are bound to the cell number of the table cell to form a cell-level three-dimensional mapping relationship.

[0018] In one optional embodiment of this application, the method further includes: The cell-level three-dimensional mapping relationship is verified according to the warehousing business rules; wherein, when the table cell is a verification cell and its bound indentation area is not empty, the document is determined to have been verified; when the table cell is a regular business cell and its bound indentation area is not empty, the document is determined to be abnormal and an alarm is output.

[0019] In one optional embodiment of this application, the method further includes: Generate an archived data package, which includes: the original voucher image retaining the indentation features, the business structured data obtained after the structured parsing of the text content, and the association index table containing the corresponding associations; In response to input text keywords, the corresponding cell number is matched from the business structured data, and the corresponding indentation area image is output according to the cell number and the association index table; and / or, in response to the selected indentation area identifier, the corresponding cell number is obtained according to the association index table, and the business structured data corresponding to the cell number is output.

[0020] Secondly, embodiments of this application provide a document visual scanning and processing system that integrates semantic understanding and artificial intelligence, including: The image acquisition module is used to acquire the original images of warehouse documents; The preprocessing module is used to perform zonal exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image; The deformation correction module is used to perform irregular deformation correction on the preprocessed image. The irregular deformation correction corrects the paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels, thus obtaining a corrected image. An indentation detection module is used to perform indentation detection on the corrected image and generate a mask representing the indentation region; The text recognition module is used to perform text recognition processing on the region outside the mask in the corrected image, using the mask as a constraint, to obtain the text content and the text position information of the text content, wherein the region inside the mask is left unprocessed; The association module is used to establish a corresponding association between the indentation area and the text content, based at least on the location information of the indentation area and the location information of the text.

[0021] In one optional embodiment of this application, the filtering process based on small-size neighborhood is a 1×N linear neighborhood median filter, where N is an integer greater than 1 and not greater than 5.

[0022] In one optional embodiment of this application, the 1×N linear neighborhood median filter is a 1×3 linear neighborhood median filter, and its processing formula is as follows:

[0023] in, For pixels in grayscale value at that location This represents the median operation. This represents the filtered pixel grayscale value.

[0024] In one optional embodiment of this application, the step of performing zoned exposure control on the original image includes: The original image is divided into M×N local sub-regions, where M and N are both integers greater than 1; Calculate the average brightness of each local sub-region. Local sub-regions with an average brightness higher than a preset highlight threshold are identified as highlight reflection areas, and the exposure gain of these areas is reduced. Local sub-regions with an average brightness lower than a preset low brightness threshold are identified as dark areas, and the exposure gain of these areas is increased. Local sub-regions with an average brightness between the preset highlight threshold and the preset low brightness threshold retain their original exposure parameters.

[0025] In one optional embodiment of this application, the irregular deformation correction of the preprocessed image includes: Detect table lines in the preprocessed image, and divide the preprocessed image into multiple cell blocks based on the table lines; Using each cell block as a correction unit, thin-plate spline interpolation transformation is performed using control points located within that cell block to obtain the correction coordinates of that cell block; Based on the corrected coordinates, the pixels of the cell block are remapped to the standard position while keeping the pixel grayscale value unchanged.

[0026] In one optional embodiment of this application, the step of performing indentation detection on the corrected image and generating a mask representing the indentation region includes: Perform multi-scale local light and shadow difference operations on the corrected image to obtain the light and shadow difference response value; The texture gradient features of the corrected image are calculated to obtain the gradient value; The indentation pixels are marked according to the joint discrimination condition, wherein the joint discrimination condition is: the absolute value of the light and shadow difference response value is greater than the preset difference threshold, and the gradient value is between the preset lower gradient limit and the preset upper gradient limit; The mask is generated based on the marked indentation pixels, and the position information of the indentation area is extracted.

[0027] In one optional embodiment of this application, the neighborhood size used in the multi-scale local lighting difference operation includes 1×3 and 3×1.

[0028] In one optional embodiment of this application, the step of performing character recognition processing on the region outside the mask in the corrected image, constrained by the mask, includes: Local adaptive threshold binarization is performed on the image outside the mask to obtain a binarized image; Optical character recognition is performed on the binarized image to obtain the text content and the pixel coordinates of the text content; Based on the fixed table format of the warehouse documents, the text content is semantically structured and parsed to obtain structured text data.

[0029] In one optional embodiment of this application, the local threshold calculation formula used for the local adaptive threshold binarization is:

[0030] in, In pixels A local neighborhood window centered on the center, The total number of pixels within the local neighborhood window. Let C be the grayscale value of the corrected image within the local neighborhood window, and C be a grayscale offset constant. The local threshold is defined as follows.

[0031] In one optional embodiment of this application, establishing the correspondence between the indentation area and the text content includes: Using the table cells of the aforementioned warehouse document as the associated units, a cell number is established for each table cell; The text location information, text content, and location information of the indentation area falling within the same table cell are bound to the cell number of the table cell to form a cell-level three-dimensional mapping relationship.

[0032] In one optional embodiment of this application, the method further includes: The cell-level three-dimensional mapping relationship is verified according to the warehousing business rules; wherein, when the table cell is a verification cell and its bound indentation area is not empty, the document is determined to have been verified; when the table cell is a regular business cell and its bound indentation area is not empty, the document is determined to be abnormal and an alarm is output.

[0033] In one optional embodiment of this application, the method further includes: Generate an archived data package, which includes: the original voucher image retaining the indentation features, the business structured data obtained after the structured parsing of the text content, and the association index table containing the corresponding associations; In response to input text keywords, the corresponding cell number is matched from the business structured data, and the corresponding indentation area image is output according to the cell number and the association index table; and / or, in response to the selected indentation area identifier, the corresponding cell number is obtained according to the association index table, and the business structured data corresponding to the cell number is output.

[0034] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-described document visual scanning processing methods that integrate semantic understanding and artificial intelligence.

[0035] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described document visual scanning processing methods that integrate semantic understanding and artificial intelligence.

[0036] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described document visual scanning processing methods that integrate semantic understanding and artificial intelligence.

[0037] The solution provided in this application, through front-end preprocessing combining partitioned exposure control and small-size neighborhood filtering, irregular deformation correction that only performs coordinate mapping without modifying pixel grayscale values, and a dual-branch linkage architecture that uses the mask generated by the indentation detection branch as a mandatory constraint for the text processing branch, ensures that in a single image processing flow, text enhancement and recognition operations are strictly limited to non-indentation areas while the indentation area is completely isolated and protected. This fundamentally overcomes the core contradiction in the traditional single-link global processing architecture where text enhancement operations inevitably erase weak, colorless indentation marks, while retaining indentations leads to a significant decrease in text recognition accuracy. It simultaneously achieves clear and complete preservation of colorless indentations and high-precision text analysis and recognition. Furthermore, by establishing a spatial correspondence between indentation locations and text content, it realizes the automatic binding of physical traces and digital semantic information, providing a unified data foundation for bidirectional traceability retrieval and compliance auditing of warehouse documents. Attached Figure Description

[0038] Figure 1 A schematic diagram of the overall system architecture of a document visual scanning and processing device that integrates semantic understanding and artificial intelligence, provided for embodiments of this application; Figure 2 A flowchart illustrating a document visual scanning processing method that integrates semantic understanding and artificial intelligence, provided for an embodiment of this application; Figure 3 A structural block diagram of a document visual scanning and processing system that integrates semantic understanding and artificial intelligence, provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0040] First, let me explain the meaning of several terms used throughout this instruction manual: "Warehouse documents" refer to various paper vouchers generated during the warehousing business process that need to be stored long-term in a digital archiving system, including but not limited to goods receipt slips, goods release slips, inventory lists, and goods handover confirmation slips. These documents are usually printed in a standard table format, containing multiple rectangular areas (i.e., cells) formed by the intersection of rows and columns. Each cell corresponds to a predefined business field, such as product name, specifications, quantity, storage location number, batch number, inspector's signature field, and release confirmation field.

[0041] "Colorless indentation" refers to a localized, permanent indentation formed on a document by warehouse staff using a hard tool (such as the end of a pen or a special stamping pen) to press into specific verification cells. This type of indentation is essentially an irreversible plastic compression of the paper fibers, causing a micrometer-level decrease in surface height in the deformed area, completely free of any ink, pigment, or dye. Because the indentation conveys information solely by altering the microscopic geometry of the paper surface without the aid of any chemical deposits, it possesses the security characteristics of being tamper-proof and impossible to replicate. It serves as core physical evidence for warehouse verification, business workflow tracking, operational accountability, and internal compliance audits. Under normal indoor lighting conditions, colorless indentations can be normally identified by the human eye through the directional change of reflected light. However, in a two-dimensional digital image, the indentation area only appears as a low-contrast visual feature with a few gray level deviations from the paper background color (usually between 2 and 8 gray level deviations, which is extremely weak relative to the total dynamic range of 0-255 in an 8-bit grayscale image). It is easily filtered out as useless noise by conventional image processing algorithms (especially denoising and smoothing operations).

[0042] The "single-link serial image processing architecture" refers to an image processing workflow commonly used in mainstream document scanning and optical character recognition (OCR) devices. In this architecture, the original image sequentially passes through several functionally independent processing stages, with the output of each stage directly becoming the input of the next. The entire processing link is unidirectional, feedback-free, and branchless. A typical link is: original image acquisition → global Gaussian smoothing and noise reduction → global contrast stretching and enhancement → global fixed threshold binarization → four-point perspective transformation deformation correction → optical character recognition → text output. The core flaw of this architecture lies in the fact that the initial global smoothing and enhancement operations treat all image regions equally, indiscriminately changing the grayscale value of every pixel. This inevitably causes the original low-contrast, colorless indentation features on the paper to be smoothed or stretched away during text enhancement, creating an irreconcilable and mutually exclusive relationship between the text processing effect and the retention of physical indentations.

[0043] A "mask" or "pixel mask" refers to a binary image with the same width and height as the image to be processed, where the value of each pixel is limited to one of two possible values. In this invention, the mask is used to accurately identify which areas in the corrected image belong to colorless indentation areas and which belong to non-indentation areas. It is defined as follows: if a pixel is determined to be part of an indentation area, the corresponding position in the mask is assigned a first predetermined value, such as 255; if a pixel does not belong to an indentation area, the corresponding position is assigned a second predetermined value, such as 0. For intuitive understanding, the mask can be imagined as a black background with white markings of the same size as the original document image—the black background (value 0) represents the valid area where text processing is safe, and the white markings (value 255) represent indentation areas that must be strictly isolated and protected. Any subsequent image enhancement, binarization, feature extraction, or other processing operations will be forcibly skipped when a pixel with a mask value of 255 is encountered.

[0044] The "cell-level three-dimensional mapping association model" refers to a data structure relationship formed by strongly binding multi-dimensional information belonging to the same cell—including the structured text content of the business field corresponding to the cell, the pixel coordinate range of such text content on the corrected image, and the set of pixel coordinates of the colorless indentation falling within the cell's spatial area—under the same coordinate system. "Three-dimensional" means that this model simultaneously associates information in three dimensions: the first dimension is semantic (text content), the second dimension is image space (pixel coordinates of text and indentation), and the third dimension is business logic (an index system established through cell numbers). This mapping model achieves a higher-dimensional association from physical traces to semantic information, enabling each piece of text data to correspond to a specific physical pressing voucher, and each pressing mark to be traced back to the business operation background that generated the mark.

[0045] The "three-in-one archive data package" refers to packaging three originally independent data objects—the original voucher image (a version fully containing the colorless indentation feature without any enhancement processing that would destroy it), the structured business text dataset (a collection of structured data extracted from documents and organized according to business field labels), and the associated index table (a data structure recording the mapping relationship between cell numbers, text content, and indentation coordinates)—into a unified archiving unit for long-term archiving and use. The outstanding value of this archiving format lies in ensuring that all digital evidence generated during a complete warehousing operation cycle—the original physical image voucher, the searchable business data, and the links between the two—is preserved as an indivisible whole, allowing for convenient retrieval and cross-verification in any subsequent audit, review, or dispute resolution scenarios. The technical solutions disclosed in the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0046] The technical solutions disclosed in the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0047] Reference Figure 1 This is a schematic diagram of the overall system architecture of the device of the present invention. The warehouse document scanning and archiving device provided by the present invention, which combines colorless indentation retention and text semantic parsing, consists of five functional modules connected in sequence to form a complete processing pipeline that works collaboratively.

[0048] The front-end image acquisition module 100, serving as the starting point of the data transmission pipeline, is deployed within the intelligent capture device located at the top of the warehouse area. It primarily comprises two components: an image sensor 110 and a zoned dynamic exposure control unit 120. The image sensor 110 can be an industrial-grade global shutter CMOS or CCD camera, with its optical lens aimed at the document placement area below. This document placement area is typically the platform surface of a metal shelf or a separate document scanning table. The zoned dynamic exposure control unit 120 is electrically connected to the control circuit of the image sensor 110, enabling it to read the brightness statistics of different spatial areas on the sensor array in real time during acquisition and dynamically adjust the pixel gain coefficient corresponding to each area accordingly. It is particularly noteworthy that this module is designed with the principle of "preserving the original physical characteristics" in mind.

[0049] The global preprocessing and irregular deformation correction module 200 is the second key link in the data pipeline, undertaking the data purification and standardization functions that connect the preceding and following steps. This module receives the raw digital image output from the front-end image acquisition module 100 at the input end, and internally processes it through two stages: the minimum kernel noise filtering submodule 210 and the table grid partitioning deformation correction submodule 220. The core design concept of the minimum kernel noise filtering submodule 210 is "to remove only isolated noise points and not process continuous texture structures." It specifically filters out salt-and-pepper noise formed by fine dust particles floating in the air on the sensor target surface, as well as small dot-like impurities that may adhere to the paper itself. This submodule adopts an asymmetric minimum kernel median filtering scheme that differs from the conventional 3×3 or 5×5 square kernel, and the coverage span of the filtering window is strictly controlled to include only one to three pixels. This method ensures that isolated noise points on the imaging surface are effectively eliminated by median calculation. However, continuous indentation textures with a width of three to seven pixels remain largely unchanged under median calculation because they exhibit an ordered grayscale gradient rather than isolated abrupt changes. The table grid partitioning deformation correction submodule 220 is specifically responsible for correcting irregular paper wrinkles and local warping caused by temperature and humidity changes. The underlying design principle of this submodule is "to reorganize the spatial arrangement of pixels only through coordinate transformation, and strictly prohibit modification of the original grayscale values ​​of pixels," thereby ensuring from the algorithmic mechanism that no matter how nonlinear or complex the correction process is, the physical detection characteristics of the indentation will not be weakened in any way. The detailed working principle of this module will be elaborated layer by layer in subsequent embodiments in conjunction with specific claim features.

[0050] The dual-branch parallel processing module 300 is the core of the entire device and a key component that distinguishes this invention from the existing single-link serial architecture. This module receives a geometrically normalized but physically intact corrected image from the output of the deformation correction submodule 220. It simultaneously feeds this frame of source data into two parallel processing branches with clear data dependencies: one is the indentation feature localization branch 310, and the other is the text enhancement semantic parsing branch 320. Although both branches use the same corrected image frame as input (i.e., from the same source), they are not completely independent and parallel, nor do they lack data exchange during operation. Specifically, after the indentation feature localization branch 310 completes the indentation region detection task and generates a pixel-level accurate mask image, it must pass this mask image as a mandatory pre-constraint parameter to the text enhancement semantic parsing branch 320. Before receiving the mask image, the text enhancement semantic parsing branch 320 does not perform any enhancement or recognition operations on its input image. After receiving the mask image, all its internal image processing sub-algorithms (including local contrast enhancement, adaptive threshold binarization, and text stroke enhancement) are configured to first query the mask image when performing each pixel operation. If a pixel's corresponding value on the mask image is 255 (i.e., it belongs to the indentation region), the pixel is skipped, and its original grayscale remains unchanged. Only pixels with a mask value of 0 are included in the candidate set for regular processing. This branch linkage mechanism, where "the output of one branch is the operational constraint of another branch," is the fundamental means to resolve the mutual exclusion contradiction between text processing and indentation retention from the perspective of system architecture.

[0051] The cell-level association matching and business verification module 400 obtains the indentation coordinate set from the indentation feature localization branch 310, the text content coordinate set and the structured text dataset from the text enhancement semantic parsing branch 320, and the table grid cell division results completed during the deformation correction process from the deformation correction submodule 220. The internal 3D mapping unit 410, based on the above three data sources and using the cell number as the first index key, establishes a many-to-many mapping relationship from cell to text data, text coordinates, and indentation coordinates. After the 3D mapping relationship is constructed, the business verification unit 420 loads the pre-established warehouse business rule base and automatically determines the compliance of the indentation existence status of each cell.

[0052] The integrated compliance archiving module 500, located at the very end of the data pipeline, is responsible for standardizing, packaging, and storing the processing results of the entire process according to the requirements of enterprise document management and compliance auditing. This module, on the one hand, uses the original voucher archive generation unit 510 to encode and save the filtered original image, retaining complete colorless indentation features, as a legal original voucher in a lossless format (such as PNG or TIFF). On the other hand, through the association index management unit 520, it encapsulates the three-dimensional mapping association table and business verification results into open-format index metadata files such as JSON or XML. Both, along with structured text data, are logically categorized into a storage system with collection timestamps, warehouse area numbers, and document types as hierarchical classifications. Simultaneously, the bidirectional retrieval engine 530 provides two database query interfaces: a forward text retrieval interface receives business keywords as input and returns the corresponding original image region crop and indentation marks; a reverse image retrieval interface receives the coordinates of the target rectangle marked by the user on the image and returns the corresponding business field information.

[0053] Figure 2 A flowchart illustrating a document visual scanning processing method integrating semantic understanding and artificial intelligence, provided as an embodiment of this application, is shown below. Figure 2 As shown, the method may include: Step S201: Obtain the original image of the warehouse document.

[0054] The original image refers to a digital pixel matrix generated by an image sensor after photoelectric conversion of a paper warehouse document placed within the scanning area of ​​the warehouse document, without any subsequent software processing. The original image can be a grayscale image (each pixel is represented by a grayscale value from 0 to 255) or a color image (e.g., RGB format, where each pixel is represented by three color components).

[0055] Specifically, for subsequent processing, if the original image is a color image, it is usually converted to a grayscale image before entering the preprocessing stage. The conversion method can use a standard weighted average method to maintain consistency with the perceptual characteristics of the human eye. The resolution of the acquired original image should at least meet the minimum physical accuracy requirements of subsequent optical character recognition. In the application scenario of warehouse documents, it is usually necessary to ensure that the smallest font size (such as 7-point font) on the document occupies at least 12 to 15 pixels in height in the image, and the corresponding scanning resolution is generally not less than 200 dots per inch (DPI). Acquiring the original image can be achieved by triggered snapshot (automatically triggering the shutter when the sensor detects that a document has been placed in the scanning area) or continuous frame extraction (the camera continuously outputs a video stream at a fixed frame rate, and selects the frame with the best clarity from it), etc. This application does not limit this.

[0056] Step S202: Perform partitioned exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image.

[0057] In this context, zoned exposure control refers to a method that differs from the traditional approach of uniformly adjusting exposure parameters globally on a camera. Instead, it involves artificially dividing a complete image frame into several local sub-regions, treating each sub-region as an independent unit for brightness assessment and fine-tuning of exposure parameters (or gain coefficients). This approach proactively addresses two spatially uneven imaging interferences: strong localized reflections caused by warehouse documents being placed close to metal shelves, and insufficient brightness at the edges of documents due to their distance from the light source. Here, exposure gain can be understood as the signal amplification factor after the image sensor converts the light signal into an electrical signal, but before or after analog-to-digital conversion. When the exposure gain for a specific sub-region is reduced, the overall pixel grayscale value of that region decreases accordingly, preventing the loss of texture information in reflective areas due to sensor saturation. Conversely, when the exposure gain for a dark area is increased, the pixel grayscale value of that area is amplified, revealing details in the shadows. Sub-regions can be the result of rectangular grid division. For example, a 2000×1500 pixel image can be divided into 20 sub-regions, each with a size of 400×375 pixels, with 5 rows and 4 columns. The brightness statistics within each sub-region can approximately represent the local lighting conditions.

[0058] In the filtering process based on small neighborhoods, "small size" refers to the size of the convolution kernel, which is commonly used in conventional image denoising algorithms, such as 3×3, 5×5, or even larger. In this invention, "small size" specifically means that the kernel size of the filtering window is limited to covering only a very limited range of pixels, typically 1×3 (a single-row linear window three pixels wide) or 3×1 (a single-column linear window three pixels high). The reason for deliberately using such an extremely small kernel size is that the feature region width formed by colorless indentations in digital images is usually 3 to 7 pixels. If a filtering window with a width comparable to or even larger than the indentation feature width, such as 3×3 or 5×5, is used, then during the filtering operation, multiple pixels within the window (a significant portion of which are grayscale gradient pixels selected from the indentation slope) will be integrated into a single output value. The unique local grayscale gradient structure of the indentation will be smoothed out, making it impossible for subsequent detection algorithms to distinguish the indented area from the flat paper surface area. Therefore, the size of the filter window must be significantly smaller than the pixel width of the smallest recognizable indentation feature.

[0059] The preprocessed image refers to the image after undergoing the dual processing of the above-mentioned zonal exposure control and minima kernel filtering. It not only eliminates the global brightness imbalance problems such as local overexposure and local underexposure in the original image, but also removes isolated particle noise points on the image surface. At the same time, most importantly, the colorless indentation texture structure of the paper surface and the gray-scale gradient of the surrounding paper background are preserved intact without attenuation, providing a high-quality clean data foundation for subsequent irregular deformation correction and feature detection.

[0060] Specifically, the implementation of zone exposure control and minima filtering is as follows: First, the original image is divided into sub-regions according to a fixed grid layout. The arithmetic mean of the gray values ​​of all pixels in each sub-region is calculated to obtain the average brightness value of that sub-region. Then, this average brightness value is compared with the preset highlight reflection threshold Th and low brightness threshold Tl. If the average brightness value is higher than Th, it is determined that the sub-region is under strong metallic reflection, and the exposure gain of the sub-region is reduced by 20% to reduce the analog-to-digital conversion output value of the signal in that region. If the average brightness value is lower than Tl, it is determined that the sub-region is under-lit, and the exposure gain is increased by 25% to improve the visibility of dark areas. If the average brightness value is between Tl and Th, the exposure parameters of the sub-region remain unchanged. In terms of minima filtering, this embodiment uses a 1×3 linear window for median filtering. That is, for each pixel in the horizontal direction, three consecutive pixels centered on itself and spanning from its left neighbor to its right neighbor are examined, and the median value of these three gray values ​​is taken as the new gray value at that position. Because median filtering has a natural selective suppression capability for isolated poles (i.e. noise) and has almost no effect on continuous texture changes that show a gradual trend (such as the slope of indentation), the processed image is perfectly preserved while the grain noise is effectively removed.

[0061] This step yields a preprocessed image, which simultaneously accomplishes two tasks that are impossible to achieve in a traditional single-link architecture during the front-end processing stage: eliminating reflections and noise interference, while ensuring that the physical grayscale gradient information of the colorless indentation is not weakened in any way.

[0062] Step S203: Perform irregular deformation correction on the preprocessed image. The irregular deformation correction corrects the paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels, thus obtaining a corrected image.

[0063] First, it must be clarified that the deformation to be corrected falls into the category of "irregular deformation." "Regular deformation" refers to deformation that can be uniformly described using a single global two-dimensional transformation matrix. Examples include the affine transformation caused by photographing a rectangular document at an angle, or the trapezoidal distortion resulting from perspective. A common characteristic of these deformations is that the coordinate transformation relationship between any two points in the image follows the same linear or projective mapping. However, in unattended warehousing environments with alternating high and low temperatures (-20℃ to 45℃), the deformation patterns of paper documents far exceed the descriptive capacity of regular deformation. Specific manifestations include: paper fibers swelling in a corner due to localized moisture, causing the corner to warp upwards and detach from its original planar position; or multiple irregular radial wrinkles forming in a localized area due to compression by heavy objects, each wrinkle having a different formation mechanism and degree of deformation. These irregular, nonlinear local deformations cannot be corrected uniformly using a single projection matrix for the entire document; nonlinear, locally adaptive correction algorithms must be employed.

[0064] The core constraint of this step is to correct paper deformation through coordinate mapping without modifying the grayscale values ​​of the mapped pixels. Coordinate mapping is a mathematically constructed mapping function (which can be a data-driven interpolation model). Its input is the original coordinate position (u,v) of a pixel on the deformed image, and its output is the correct position (x,y) of that pixel on a standard flat document without physical deformation. During mapping, each target coordinate (x,y) on the corrected blank image is typically traversed, and its corresponding source coordinate (u,v) on the original deformed image is traced back. Then, the grayscale value of the pixel located at (u,v) is extracted from the original image and copied to the (x,y) position. This process does not require grayscale interpolation or GPU texture resampling. This constraint ensures that after correction, the originally weak, only a few grayscale levels of brightness variation in the colorless indented area will not be "diluted" due to interpolation weighting with surrounding background pixels during the correction process; therefore, it can be called "lossless grayscale correction."

[0065] In this embodiment, the specific algorithm for irregular deformation correction is a thin-plate spline interpolation coordinate transformation based on a table grid partitioning. First, an edge detection operator is used to extract straight line segments from the preprocessed image. Then, a Hough transform is used to filter out the horizontal and vertical lines belonging to the printed form, determining the precise location of each table line. Based on the grid formed by the intersections of these table lines, the form image is divided into K relatively independent cell blocks, denoted as C1, C2, ..., C6. KEach cell's four sides form its natural spatial boundary. Next, n pairs of control points are selected within each cell. These control points include: the intersections of the table lines at the four corners of the cell (usually four), and the location points of text stroke feature points or field labels obtained using a corner detector (such as Harris corner detector). The actual coordinates of these control points in the deformed image are then calculated. and their ideal coordinates in the standard template As training data, a thin-plate spline interpolation model is constructed. The mathematical form of thin-plate spline interpolation is:

[0066] in, , which is the radial basis function for thin plate splines, specifically used to correct local nonlinear wrinkles and warping; The original pixel's two-dimensional coordinate vector under the deformed state; This is the two-dimensional coordinate vector of the pixel at the standard position after correction. This is an affine transformation matrix used to correct the overall skewness of the cells; This is a translation vector, used in conjunction with a matrix to complete the overall position correction; This represents the total number of deformation control points selected within the current cell; For the first Coordinate vectors of control points; The Euclidean distance between the original pixel and the control point is abbreviated as ; For the first The weight coefficients corresponding to each control point are determined by the correspondence between the control points.

[0067] For each pixel within a cell, its corrected coordinates are calculated based on the model, and the original grayscale value of the pixel is copied to the corresponding position in the output image. After traversing all cells, the entire corrected image is stitched together. .

[0068] This step effectively eliminates the coordinate offset between the text area and the crease area caused by irregular deformation of the paper, enabling subsequent association and binding of creases to text to be carried out in a high-precision unified spatial coordinate system. On the other hand, its unique grayscale non-destructive design ensures that the original physical characteristics of the crease are not disturbed during the complex correction process, guaranteeing the accuracy of crease detection.

[0069] Step S204: Perform indentation detection on the corrected image and generate a mask representing the indentation area.

[0070] The reason why a dedicated indentation detection step is needed instead of text recognition at the beginning is that the visual salience of colorless indentation is far lower than that of printed text. It cannot be completed by sharing the same image processing pipeline with text detection. It must be processed in a separate branch with algorithm parameters specifically optimized for it.

[0071] The goal of indentation detection is to distinguish corrected images. For each pixel in the image, determine whether it belongs to a colorless indentation region. Generating a mask representing the indentation region involves creating an image that corresponds to the indentation region. A new binary image Mask of the same size is created. For each pixel identified as belonging to the indentation region, a value of 255 is assigned to the corresponding position in the Mask, while all pixels belonging to non-indentation regions are assigned a value of 0. With this Mask, the spatial information of the indentation region is transformed from a continuous pixel image into a logical mask that facilitates computer traversal and conditional judgment.

[0072] Specifically, indentation detection is achieved through a method combining multi-scale local illumination difference with texture gradient joint judgment. Specifically, the local illumination difference value of each pixel in the corrected image is first calculated. The physical intuition behind local illumination difference calculation is that a recessed area can be recognized by the naked eye in an image because the recessed surface reflects ambient light less effectively than the surrounding smooth surface, thus appearing slightly darker than its surroundings. For pixel (x,y), its illumination difference value D(x,y) is defined as the pixel's gray level minus the average gray level of its eight neighboring pixels (top, bottom, left, right, and four diagonal directions). For colorless indentations, due to their weak reflectivity, their pixel grayscale is typically 2 to 8 gray levels darker than the surrounding average background. Therefore, D(x,y) is a negative value here, and its absolute value is significantly greater than zero. For completely flat paper areas, D(x,y) approaches zero. For printed text ink areas, although their grayscale is also low, the boundary between the ink and the paper is very sharp, and the gradient characteristics of the text area differ from those of the indentation. This difference is distinguished by gradient analysis. The formula for calculating the light and shadow difference is as follows:

[0073] in, The pixel gray level at (x,y) of the corrected standard image; , The summation range is the horizontal and vertical offset of the neighboring pixels relative to the current pixel; the summation range is a 3×3 eight-neighbor area centered at (x,y) (excluding itself); This represents the light and shadow difference response value; the indentation reflects light weakly. Negative values ​​with large absolute values; flat paper surface and text area. Approaching 0.

[0074] Next, the texture gradient magnitude of each pixel in the corrected image is calculated. The horizontal and vertical gradient components can be calculated separately using the Sobel operator, and then their squares and square roots can be taken. The gradient magnitude of a colorless embossed texture falls between the very low gradient of blank paper (approximately 0-5) and the high gradient of sharp text edges (typically >30). The formula for calculating the texture gradient magnitude is as follows: ,

[0075]

[0076] in, This represents the gradient (grayscale change rate) of the image in the horizontal direction. This represents the gradient (grayscale change rate) of the image in the vertical direction. The gradient magnitude at pixel (x,y) is greater than that of blank paper but less than that of printed text in the indented area.

[0077] Combining these two discriminant dimensions, we set the joint discriminant conditions:

[0078] in, The threshold for light and shadow difference; , These are the upper and lower thresholds for the gradient magnitude.

[0079] Pixels that simultaneously meet both conditions are marked as indentation pixels. In this way, printed text that is only dark in grayscale but has sharp edges will not meet the gradient upper limit condition, and blank paper with only gentle edges but no difference in grayscale from the background will not meet the difference threshold condition. Indentations are accurately captured by their unique dual features of "slight darkness + micro-texture".

[0080] This step, through complementary verification of the difference dimension and the gradient dimension, effectively improves the detection rate and positioning accuracy of colorless indentations under extreme low-contrast conditions, and avoids large-area false detections caused by a single criterion when faced with complex paper texture variations.

[0081] Step S205: Using the mask as a constraint, perform text recognition processing on the region outside the mask in the corrected image to obtain the text content and the text position information of the text content, wherein the region inside the mask is left unprocessed.

[0082] Specifically, this step is the core operation in the entire process that truly achieves the two goals of indentation preservation and text parsing simultaneously. Using a mask as a constraint means incorporating the mask image generated in step S204 into the entire text processing chain. Whether it's binarization, contrast adjustment, stroke thinning, or text detection, the value of the corresponding position of the currently operated pixel in the mask image must be used as the first priority criterion before execution: if the mask value at that position is 255, it means that this pixel is part of the indentation, and any form of grayscale modification or feature extraction calculation is prohibited; only non-indentation pixels with a mask value of 0 can be processed by conventional text processing operators in the next step. The most fundamental difference between this mechanism and the traditional single-link architecture is that the traditional architecture does not have any prior knowledge of the indentation area as a constraint, and all enhancement and recognition operations indiscriminately sweep across the entire image; while the method of this invention first uses an independent indentation detection branch to obtain complete and accurate protected area location data, and then injects this set of data into each operation step of the text processing branch in the form of a forced constraint, fundamentally ensuring the physical isolation of indentation and text processing in the image space.

[0083] The text recognition processing includes a series of steps aimed at converting printed character information in an image into editable, searchable, and analyzable digital text data. These steps include, but are not limited to, image binarization, text region detection, single-character or text line recognition, and subsequent possible structured semantic parsing. In this embodiment, a prohibited processing list is first established based on a mask. Then, local adaptive threshold binarization is performed on pixels within the allowed processing area: for each non-indented pixel (x, y), a local neighborhood window (e.g., 31×31 pixels) is taken centered on it. The average grayscale value of all pixels within this window is calculated, and a fixed offset constant C is subtracted from it to obtain the binarization threshold applicable to this local area. If the grayscale value of the current pixel is greater than this threshold, the output is white (background); otherwise, the output is black (text foreground). This local thresholding method effectively overcomes the uneven brightness of the paper surface caused by residual reflection without changing any pixels in the indented area, ensuring complete extraction of text strokes. Subsequently, the binarized text content area is fed into the optical character recognition engine, outputting the text content of each individual character and its bounding box pixel coordinate set in the image.

[0084] This step fundamentally and completely solves the industry problem of not being able to obtain high-quality text recognition results while preserving colorless indentations, enabling the same document image to simultaneously meet two previously considered mutually exclusive requirements in a unified processing pipeline.

[0085] Step S206: At least based on the location information of the indentation area and the location information of the text, establish a corresponding association between the indentation area and the text content.

[0086] The goal of establishing corresponding relationships is to structurally organize and bundle two types of independent data (indentation evidence from the physical world and textual semantics from the digital world) that are scattered across different processing branches, forming a data interconnection that is queryable, searchable, and traceable from one end to the other. At least based on location information, this means that the basic link of the relationship is spatial location, that is, the spatial correspondence between the coordinates of the indentation on the image and the coordinates of the adjacent text.

[0087] In this embodiment, this step utilizes the grid of table cells already divided in the deformation correction stage, establishing correspondences with cells as the smallest unit of association. First, each cell is assigned a globally unique number, following the convention of column letters followed by row numbers (e.g., A1 represents the first cell in the first row of the first column, B3 represents the third cell in the second row of the second column). Next, each identified field block in the structured text is traversed, and its position coordinates on the image are used to determine spatial attribution with the cell. If more than 90% of the pixel area of ​​a text block falls within the coordinate range of a certain cell, then the text and its coordinates are associated with that cell number. Similarly, the indentation coordinate set is traversed; if each indentation rectangle falls within a cell area, then the indentation record is associated with that cell. The final structure is a cell-level three-dimensional association mapping table. This table treats each cell with content as a row record, with fields including: cell number, the name and content string of the structured business field within the cell, an array of pixel coordinate rectangles for the text block, and a list of pixel coordinate rectangles and their quantities for the indentations (the list is empty if there are no indentations).

[0088] This step enables a fully automated closed-loop association between physical vouchers and structured business digital evidence in the field of digital archiving of warehouse documents. This eliminates the need for manual comparison of indentation locations and text content for subsequent accountability and audit verification, greatly improving the efficiency and credibility of data utilization.

[0089] The solution provided in this application, through front-end preprocessing combining partitioned exposure control and small-size neighborhood filtering, irregular deformation correction that only performs coordinate mapping without modifying pixel grayscale values, and a dual-branch linkage architecture that uses the mask generated by the indentation detection branch as a mandatory constraint for the text processing branch, ensures that in a single image processing flow, text enhancement and recognition operations are strictly limited to non-indentation areas while the indentation area is completely isolated and protected. This fundamentally overcomes the core contradiction in the traditional single-link global processing architecture where text enhancement operations inevitably erase weak, colorless indentation marks, while retaining indentations leads to a significant decrease in text recognition accuracy. It simultaneously achieves clear and complete preservation of colorless indentations and high-precision text analysis and recognition. Furthermore, by establishing a spatial correspondence between indentation locations and text content, it realizes the automatic binding of physical traces and digital semantic information, providing a unified data foundation for bidirectional traceability retrieval and compliance auditing of warehouse documents.

[0090] In one optional embodiment of this application, the filtering process based on small-size neighborhood is a 1×N linear neighborhood median filter, where N is an integer greater than 1 and not greater than 5.

[0091] Furthermore, the 1×N linear neighborhood median filter is a 1×3 linear neighborhood median filter, and its processing formula is as follows:

[0092] in, For pixels in grayscale value at that location This represents the median operation. This represents the filtered pixel grayscale value. , and These represent the original grayscale values ​​of three adjacent pixels in a horizontal row of the original image: the left pixel, the current pixel itself, and the right pixel.

[0093] Here, 1×N describes the shape and size of the filtering window: the window height is only 1 pixel (i.e., it occupies only a single line of the image), and the window width is N pixels. This means that when performing filtering, the reference range for replacing the center pixel is limited to its left and right adjacent pixels on the same line, each within (N-1) / 2 pixels. "Linear" emphasizes that this filtering window expands unidirectionally along the horizontal line of the image, unlike traditional square windows (such as 3×3, which require simultaneous values ​​from eight neighboring pixels in two dimensions) or cross-shaped windows. Median filtering refers to the core operation of the filtering algorithm being the statistical median operation, that is, after sorting the gray values ​​of all pixels covered by the window according to their numerical values, the gray value of the middle pixel is selected as the output. The biggest difference between median filtering and mean filtering is that mean filtering causes the new value of the center pixel to incorporate information from all pixels in the window, inevitably resulting in smoothing and blurring of edges and textures. Median filtering, on the other hand, is non-linear; it only selects one pixel value that actually exists within the window for output. For isolated noise (values ​​that differ greatly from their surroundings), it will be placed at the far left or far right of the ordered sequence and discarded. However, for continuously increasing or decreasing ordered gradient trends (such as indentation ramps), the output after taking the median is likely to be the actual pixel value representing the middle of the gradient trend, thus preserving the grayscale gradient structure of the texture. When N=3, the window only covers 3 pixels horizontally, which can filter out isolated noise with a single pixel width. When N=5, the window covers 5 pixels, resulting in stronger noise filtering, but it begins to slightly smooth out the narrowest and finest indentations (extremely fine depressions with a width of 2-3 pixels). Therefore, N should not exceed 5, otherwise the indentation protection performance will decrease.

[0094] Specifically, N can be 3. For example, suppose the pixel change of the concave indentation slope is [148, 145, 143, 145, 148], representing the gradual process of the actual paper surface sinking from a flat background and then rising gently. A 1×3 window median filter [145, 143, 145] is applied to the bottom pixel (143) of the indentation, with a median value of 145. This means that extremely subtle grayscale differences at the bottom of the indentation may be slightly increased, but the overall gradient of the indentation (i.e., the process of gradually decreasing from 148 to 143) continues to exhibit an ordered gradient sequence throughout the neighborhood, without being disrupted into abrupt changes. If a 3×3 square window median filter is used instead, the window will additionally involve the pixels in the upper and lower rows. Those rows may have larger grayscale differences because they are opposite the edge of the indentation, which may affect the median result. Therefore, the 1×3 linear window achieves a good trade-off between noise reduction and texture preservation.

[0095] By using the geometric constraints of a 1×N linear window, the filtering operation is made to make smooth selections only in the horizontal direction without disrupting the pixel continuity of the indentation in the vertical direction.

[0096] In one optional embodiment of this application, the step of performing zoned exposure control on the original image includes: The original image is divided into M×N local sub-regions, where M and N are both integers greater than 1; Calculate the average brightness of each local sub-region. Local sub-regions with an average brightness higher than a preset highlight threshold are identified as highlight reflection areas, and the exposure gain of these areas is reduced. Local sub-regions with an average brightness lower than a preset low brightness threshold are identified as dark areas, and the exposure gain of these areas is increased. Local sub-regions with an average brightness between the preset highlight threshold and the preset low brightness threshold retain their original exposure parameters.

[0097] Specifically, zone exposure control can include the following detailed sub-operations: The first step is to divide the original image into M×N local sub-regions. Here, "M×N" represents the number of rows and columns of the sub-regions. For example, if the original image has a width of Wtotal pixels and a height of Htotal pixels, and M=5 and N=4, then the entire image is divided into 20 local sub-regions with 5 rows and 4 columns. The width of each sub-region is Wtotal / N pixels, and the height is Htotal / M pixels. When dividing, if the width or height of the original image is not divisible by N or M, pixel padding can be applied to the edge sub-regions, or the edge regions can be allowed to be slightly smaller than the standard size. The purpose of sub-region division is to decompose the image from a single whole into multiple local units with relatively independent evaluation value of lighting conditions—the reflection or shadow conditions within each unit are usually approximately consistent.

[0098] The second step is to calculate the average brightness of each local sub-region. The average brightness is calculated by arithmetically summing the grayscale values ​​of all pixels within the sub-region, then dividing the sum by the total number of pixels in the sub-region. The resulting floating-point or integer value is the average brightness L_{i,j} of that sub-region. The average brightness objectively reflects the relative level of average illumination received by the sub-region. For example, in a highly reflective area, most pixels in the sub-region have grayscale values ​​close to 255 (in an 8-bit grayscale image), and the average brightness may be as high as 230 or more; in a severely shaded area, many pixels have grayscale values ​​below 60, and the average brightness may be less than 50.

[0099] The third step involves adjusting the exposure gain of each sub-region based on pre-set high-reflectivity threshold Th and low-brightness threshold Tl. These two thresholds, Th and Tl, are empirical values ​​determined by the intelligent capture equipment installed at the warehouse site based on historical statistical data or on-site environmental testing. Typically, Th is set between 220 and 240, and Tl between 40 and 60. The adjustment rule is as follows: Compare the average brightness value L{i,j} of the sub-region calculated in step two with Th. If L{i,j} is greater than Th, it indicates that the sub-region has overexposed highlights, and the grayscale details of the original image (including indentation texture and light-colored text) are at risk of being buried. In this case, the exposure gain parameter of the sub-region is adjusted downward, for example, by multiplying the gain coefficient by a factor of 0.70 to 0.85 when capturing the next frame. If L{i,j} is lower than Th, it indicates that the sub-region is severely underlit, and the details on the paper are difficult to distinguish. In this case, the exposure gain of the sub-region is increased, for example, by multiplying the gain coefficient by 1.15 to 1.30. If L{i,j} is between Th and Th, the current exposure parameters are maintained without change.

[0100] Through this fine-tuned exposure adjustment by spatial partitioning, the unevenness of the reflective and shadow areas in the overall image is significantly suppressed. The original image approaches the same brightness level in all sub-regions, which is suitable for indentation detection and text recognition. At the same time, the entire process only involves adjusting the exposure gain ratio, without involving post-processing software operations such as contrast mapping, gamma curve stretching, and histogram equalization, which may destroy the slight grayscale gradient.

[0101] In one optional embodiment of this application, the irregular deformation correction of the preprocessed image includes: Detect table lines in the preprocessed image, and divide the preprocessed image into multiple cell blocks based on the table lines; Using each cell block as a correction unit, thin-plate spline interpolation transformation is performed using control points located within that cell block to obtain the correction coordinates of that cell block; Based on the corrected coordinates, the pixels of the cell block are remapped to the standard position while keeping the pixel grayscale value unchanged.

[0102] Specifically, irregular deformation correction may include three sequential sub-steps: The first step is to detect table lines in the preprocessed image and divide the image into multiple cell blocks based on these lines. Here, "table lines" refers to the horizontal and vertical lines printed on warehouse documents that define the areas of various business fields. In a digital image, table lines appear as narrow, dark bands with a grayscale value significantly lower than the surrounding paper background, continuously distributed in one direction. Detecting table lines can be achieved as follows: First, apply a low-threshold Canny edge detection operator to the preprocessed image to obtain a binarized image containing all significant edges. Then, use the Probabilistic Cumulative Hough Line Transform (PPHT) algorithm to extract all line segments that conform to straight-line characteristics from this binarized image. Since horizontal lines should be approximately horizontal and vertical lines approximately vertical, the angle search range of the Hough transform can be limited (e.g., accepting lines with angles between -5° and +5° for horizontal lines and between 85° and 95° for vertical lines) to filter out cluttered edges that do not belong to table lines. After sorting the detected horizontal and vertical lines according to their respective coordinate values ​​and merging redundant line segments that are too close together, the complete grid division of the table is obtained. Each closed area formed by the intersection of grids is a "cell block". A typical warehouse receipt may contain 8 cell blocks vertically and 5 cell blocks horizontally, for a total of 40 cell blocks.

[0103] The second step involves using each cell block as a correction unit and performing thin-plate spline interpolation transformation using the control points located within that cell block to obtain the corrected coordinates of that cell block. Thin-plate spline interpolation (TPS) is a commonly used mathematical tool for describing non-rigid, nonlinear deformations. Its basic idea is to model deformation as a surface interpolation problem constrained by minimizing the bending energy of a physical thin plate. For each cell, at least four pairs of control points are first identified within it—control points are feature points whose exact corresponding positions can be found in both the deformation image and the standard template. In warehouse documents, naturally high-quality control points include: the four corner points of the cell (i.e., the intersections of horizontal and vertical lines), because regardless of paper wrinkles, printed lines maintain an inherent intersection relationship at least locally; and the geometric center points or edge feature points of each text block (whose ideal positions on the standard document can be determined through template matching).

[0104] The third step involves remapping the pixels of the cell block to the standard positions based on the correction coordinates, while maintaining the pixel grayscale values. "Remapping" is the process of coordinate transfer—reading the pixel grayscale value at coordinate u in the deformed image and copying it to coordinate v in the corrected image. After all pixels in the entire cell have been processed, the cell will appear as a standard, wrinkle-free rectangle in the corrected image. This process is repeated for all cells to complete the correction of the entire document.

[0105] By combining a table grid adaptive partitioning strategy with TPS fine local interpolation, the problem of irregular local paper deformation that traditional four-point overall perspective correction cannot handle is overcome; and the grayscale-unmodified mapping method ensures from the lowest mathematical level that the indentation features are completely consistent with the original image after complex deformation correction.

[0106] In one optional embodiment of this application, the step of performing indentation detection on the corrected image and generating a mask representing the indentation region includes: Perform multi-scale local light and shadow difference operations on the corrected image to obtain the light and shadow difference response value; The texture gradient features of the corrected image are calculated to obtain the gradient value; The indentation pixels are marked according to the joint discrimination condition, wherein the joint discrimination condition is: the absolute value of the light and shadow difference response value is greater than the preset difference threshold, and the gradient value is between the preset lower gradient limit and the preset upper gradient limit; The mask is generated based on the marked indentation pixels, and the position information of the indentation area is extracted.

[0107] Furthermore, the neighborhood size used in the multi-scale local lighting difference operation includes 1×3 and 3×1.

[0108] Specifically, indentation detection may include the following sub-steps: The first step is to perform multi-scale local light and shadow difference operations on the corrected image to obtain the light and shadow difference response value. First, let's define and intuitively explain the algorithm of "local light and shadow difference." In a digital image, the concavity or convexity of any tiny region is manifested through differences in its reflectivity to incident light—a concave surface, due to its different orientation from the surrounding smooth surface, reflects slightly weaker light towards the camera lens, thus appearing on the image as a concave region being several gray levels darker on average than the surrounding smooth region. The purpose of local light and shadow difference operations is to quantify this difference in brightness—how much darker it is than the surrounding average—at each pixel location. Specifically, for each pixel location (x, y) in the corrected image, a small neighborhood (e.g., 3×3, 1×3, 3×1 scale) is selected centered on it. The average gray level of all surrounding pixels within this neighborhood, excluding the central pixel, is calculated. The difference between the current pixel's own gray level and the average gray level of these surrounding pixels is the light and shadow difference response value D(x, y). If the pixel is located inside an indentation, D(x,y) is negative and its absolute value is significantly greater than zero; if the pixel is on a completely flat surface, D(x,y) is approximately zero; if the pixel is a black stroke of printed text, D(x,y) may also be negative, but the text has a very sharp gradient—this will be further distinguished in the next sub-step using gradient features. The "multi-scale" approach here refers to the fact that the above difference operation does not use only one neighborhood size, but instead uses multiple neighborhood windows of different shapes and sizes to calculate the difference response map at each scale, and then summarizes the response values ​​at multiple scales according to a certain strategy (such as taking the largest absolute value among each scale) into a comprehensive response value. The advantage of multi-scale operation is that it can simultaneously enhance the sensitivity to indentations of different widths, directions, and depths.

[0109] The second step involves calculating the texture gradient features of the corrected image to obtain gradient values. The texture gradient is a second-order vector field reflecting the rate of change of local gray levels in an image. This sub-step uses the Sobel gradient operator to convolve the corrected image in both the horizontal and vertical directions to obtain the horizontal gray-level change rate Gx and the vertical gray-level change rate Gy, and then calculates the gradient magnitude for each pixel. The typical gradient magnitude distribution range in the indented area is higher than that of completely flat background paper (gradient magnitude is usually less than 5) but lower than that of sharp printed text edges (gradient magnitude is usually greater than 25), falling within a middle range. This is determined by the micro-texture of the paper surface caused by the indentation.

[0110] The third step is to mark the indentation pixels according to the joint discrimination conditions. Using this method, even if the two characters are located in the bright area (average gray level of the background 215) and dark area (average gray level of the background 130) of the document respectively, as long as the local adjustment offset is kept consistent, the strokes of the characters in the two areas can be separated in a unified and reliable manner.

[0111] The fourth step involves generating a mask based on the marked indentation pixels and extracting the location information of the indentation areas. After all indentation pixels have been marked, a connected component analysis algorithm is performed on the entire image to aggregate spatially adjacent indentation pixels into independent connected components. For each connected component, the minimum bounding rectangle is calculated, and the coordinates of its top-left corner and its width and height are recorded, forming an indentation coordinate set. Simultaneously, a binary mask image with the same size as the corrected image is created. All previously marked indentation pixels are filled with 255, and the rest are filled with 0, thus obtaining the Mask.

[0112] In one optional embodiment of this application, the step of performing character recognition processing on the region outside the mask in the corrected image, constrained by the mask, includes: Local adaptive threshold binarization is performed on the image outside the mask to obtain a binarized image; Optical character recognition is performed on the binarized image to obtain the text content and the pixel coordinates of the text content; Based on the fixed table format of the warehouse documents, the text content is semantically structured and parsed to obtain structured text data.

[0113] Furthermore, the local threshold calculation formula used in the local adaptive threshold binarization is as follows:

[0114] in, In pixels A local neighborhood window centered on the center, The total number of pixels within the local neighborhood window. Let C be the grayscale value of the corrected image within the local neighborhood window, and C be a grayscale offset constant. The local threshold is defined as follows.

[0115] Specifically, character recognition processing is performed on the regions of the corrected image located outside the mask, which is further broken down into four sequential sub-operations: The first step is to perform local adaptive threshold binarization on the image outside the mask, resulting in a binarized image. Before binarization, the prepared mask image is first used to determine the classification of each pixel in the corrected image: if the pixel value in the mask image is 255, it is skipped, and its value in the subsequent binarized image is kept the same as its original grayscale value in I_std (or directly set to the background color); if the mask value is 0, local adaptive thresholding is performed on the current pixel to determine its final black and white classification in the binarized image. The basic principle of local adaptive threshold binarization is: a local neighborhood window Ω of a pre-defined size is defined with the non-indented pixel (x,y) as the center, the arithmetic mean of the grayscale values ​​of all pixels within the window is calculated, and an offset constant C is subtracted from it. The resulting value is used as the binarization threshold T_local for that position (x,y). If I_std(x,y) is greater than T_local, it means that the pixel is brighter than its local background and is obviously not a stroke pixel, so it is marked as a white background (value 255). If I_std(x,y) is less than or equal to T_local, it means that the pixel is darker than its local background and has the grayscale characteristics of text strokes, so it is marked as a black foreground (value 0). In this way, the extraction standard for text strokes is no longer a rigid global absolute grayscale value (e.g., ≥128 for background, <128 for text), but is dynamically determined by flexibly referring to the grayscale level of the local adjacent paper surface of the stroke. This effectively solves the problem of global threshold failure caused by uneven local brightness on the document (e.g., the overall grayscale of the paper surface in the lower left shadow area is 100, and the overall grayscale of the bright area in the upper right corner is 220).

[0116] The second step involves performing Optical Character Recognition (OCR) on the binarized image to obtain the text content and its pixel coordinates. A mature commercial or open-source OCR engine can be used for OCR. The engine takes the binary image generated in the previous step as input, uses its built-in layout analysis algorithm to find the distribution areas of each line of text, performs character segmentation to cut continuous character strokes into independent character candidate boxes, and then uses a trained character classifier (such as a convolutional neural network classifier) ​​to perform character encoding recognition on each candidate box, outputting the Unicode text content corresponding to each character and its four-pixel coordinates of its bounding box. Finally, the entire printed text content on the current document and its precise pixel location information in the image are obtained.

[0117] The third step involves semantically structuring the text content based on the fixed table format of the warehouse documents to obtain structured text data. Since the types of documents used in the warehousing industry are fixed, their table format can be pre-entered as a template. The template records the coordinate range of the four corners of each cell and the expected business field labels for that cell. For example, the template specifies: "The rectangular area with coordinates (200, 350) at the top left and (500, 420) at the bottom right belongs to cell B2, and the field label is 'Product Name'." The third step involves matching the coordinates of each text block output by the OCR with the coordinate range of each cell in the template. If the center of a text block falls within the range of a certain cell, the text content of that text block is assigned to the corresponding field label of that cell, ultimately resulting in a structured text data object, Data_text, such as {"Product Name": "Hexagonal Bolts", "Quantity": "200", "Storage Location": "A-03-12"}.

[0118] In one optional embodiment of this application, establishing the correspondence between the indentation area and the text content includes: Using the table cells of the aforementioned warehouse document as the associated units, a cell number is established for each table cell; The text location information, text content, and location information of the indentation area falling within the same table cell are bound to the cell number of the table cell to form a cell-level three-dimensional mapping relationship.

[0119] Specifically, establish a correspondence between the indented area and the text content, using the table cells of the warehouse document as the basic index unit, and follow the steps below.

[0120] First, each individual cell obtained after dividing the table into a grid is assigned a unique string-type cell number. The numbering rule can directly follow the row and column identification rules printed on the document itself, such as using letters to identify columns (A, B, C...) and numbers to identify rows (1, 2, 3...). In this case, the numbering format is simply a concatenation of the column letter and the row number, such as "A3". If the document does not indicate the column letter and row number, the system will automatically assign numbers according to a preset order (A1 in the upper left, with the column letters increasing to the right and the row numbers increasing downwards).

[0121] Next, text association is performed. Each record in the structured text data object Data_text obtained earlier through character recognition and semantic parsing (each record at least contains its business field label, text content, and corresponding pixel coordinate rectangle) is spatially bound to a cell. Specifically, for the coordinate rectangle of each text block in Data_text, its center point coordinates are calculated. All the divided cells are traversed, and the cell containing this center point is found. All information of that text block is appended to the dataset corresponding to that cell number. If a cell contains two independent text blocks side by side (e.g., different information is written on different lines within the cell), these two text blocks are merged into a list belonging to the same cell.

[0122] Next, the indentation association is performed. Each indentation record in the indentation coordinate set (each record is a rectangular area) is traversed, and the coordinates of the four vertices of the rectangle are extracted. Similarly, based on the center point of the rectangle, it is assigned to a specific cell number. After binding according to the above rules, each cell number simultaneously carries its structured text data set (which may contain multiple fields or text segments), its text pixel coordinate set, and its indentation coordinate set (empty if there is no indentation), thus forming a three-dimensional mapping at the single-cell level.

[0123] In one optional embodiment of this application, the method further includes: The cell-level three-dimensional mapping relationship is verified according to the warehousing business rules; wherein, when the table cell is a verification cell and its bound indentation area is not empty, the document is determined to have been verified; when the table cell is a regular business cell and its bound indentation area is not empty, the document is determined to be abnormal and an alarm is output.

[0124] Specifically, this embodiment further adds a step of verifying the warehousing business rules for the cell-level three-dimensional mapping relationship.

[0125] After completing the 3D mapping, the system obtains detailed information about each cell and whether it contains indentations. At this point, it retrieves the warehousing business judgment rules stored in the system's rule engine and performs compliance checks on each cell.

[0126] This section provides clear definitions of "verification cells" and "ordinary business cells." "Verification cells" on warehouse documents refer to areas where, according to business procedures, the responsible person must use a hard tool to press a colorless indentation onto the document after on-site verification and confirmation. Common examples of verification cells include special areas with printed confirmation text such as "Reviewer's Signature," "Release Confirmation," "Quality Inspection Passed Endorsement," and "Inbound Physical Confirmation." The physical pressing action on these cells is a mandatory operational marker within the warehouse internal control system. "Ordinary business cells," on the other hand, refer to regular information recording areas used to record basic information about warehouse operations (such as goods name, model, quantity, location, date, etc.). Internal control rules stipulate that there should never be any intentional indentations or pressing marks on these cells.

[0127] The logic for business rule validation is as follows: Iterate through all cell numbers. For each cell, query its pre-configured category label in the document template. If the label belongs to "Verification," then determine whether the indentation coordinate set bound to the cell contains a valid record (not empty). If the indentation coordinates are not empty, it means that the cell's status meets the compliance expectation of "verified and recorded by staff," and the result is "compliant." If the verification cell lacks any indentations (the indentation set is empty), it indicates a possible omission in the process, and a reminder needs to be issued. If the label belongs to "Ordinary Business," and its bound indentation coordinates are not empty, it means that the business operator may have left non-compliant pressing marks in areas that should not be marked (e.g., accidentally pressing in the wrong position). In this case, the document status is determined to be "abnormal," an abnormal alarm record is generated, and a report is sent through the output interface.

[0128] By embedding business rules directly into the structured data of image processing results for automatic judgment, this embodiment no longer relies on traditional manual visual inspection of indentations and business matching, realizing unmanned and intelligent strict control of the warehouse internal audit process. The beneficial effect is to effectively reduce the workload and error rate of manual compliance inspection.

[0129] In one optional embodiment of this application, the method further includes: Generate an archived data package, which includes: the original voucher image retaining the indentation features, the business structured data obtained after the structured parsing of the text content, and the association index table containing the corresponding associations; In response to input text keywords, the corresponding cell number is matched from the business structured data, and the corresponding indentation area image is output according to the cell number and the association index table; and / or, in response to the selected indentation area identifier, the corresponding cell number is obtained according to the association index table, and the business structured data corresponding to the cell number is output.

[0130] Specifically, firstly, in the archive data package generation stage, three key data assets are packaged and stored uniformly. The specific operations are as follows: Images that have undergone minimal kernel noise filtering but have not been affected by any text enhancement processing and fully retain the colorless indentation features are encoded into lossless compressed image files (such as PNG format), which serve as the "original voucher archive." The structured text dataset covering all cells of the entire document (including the business field labels and extracted text values ​​for each cell) is converted into an open, human-readable, and machine-parsable text data file, such as a JSON file, which serves as the "business data archive." The cell association index table Index, which records all cell numbers and their respective bound text coordinates and indentation coordinates, along with the business verification results Check_Res, is written into the same JSON data body (or placed in a separate file) as the "association index archive." All three are organized in a uniformly named folder or compressed package (ZIP), archived using a three-level classification directory structure of collection timestamp, warehouse area number, and document type, achieving standardized storage by time and space indexing.

[0131] Secondly, regarding the two-way traceability retrieval function, this embodiment establishes two retrieval paths. Forward retrieval path: The user enters a text keyword in the system's retrieval interface, such as the specific model of a product or the name of a verification personnel. The system first performs a full-text search in the pre-built business data archive JSON file, matching the cell numbers containing the keyword. Then, it quickly finds all indentation coordinate records associated with these cell numbers through the associated index table. It then draws a highlighted rectangle at the corresponding coordinates on the archived original document image and returns a cropped image of the physical object and all business information of the document to the user. Reverse retrieval path: When auditors or managers browse the archived image, if they have doubts about an indentation mark on a piece of paper, they can directly select or click on the area near the indentation on the image. The system captures the coordinate range of the selected area, queries the associated index table to find the matched cell number, and retrieves the complete business field data (including field name, text value, and metadata of the document) corresponding to that cell from the business data archive. The system instantly displays the business context that produced the physical mark on the user interface.

[0132] The archiving and retrieval functions in this embodiment solve the problem of "images being separated from images and data being separated from data" in traditional electronic document processing, significantly improving the auditability and traceability of warehouse document archives, and meeting the requirements of enterprise internal control and external regulatory review.

[0133] Figure 3A framework diagram of a document visual scanning and processing system integrating semantic understanding and artificial intelligence is provided for embodiments of this application, such as... Figure 3 As shown, it includes: The image acquisition module 301 is used to acquire the original image of the warehouse document; The preprocessing module 302 is used to perform partitioned exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image; The deformation correction module 303 is used to perform irregular deformation correction on the preprocessed image. The irregular deformation correction corrects the paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels, thus obtaining a corrected image. Indentation detection module 304 is used to perform indentation detection on the corrected image and generate a mask representing the indentation area; The text recognition module 305 is used to perform text recognition processing on the region outside the mask in the corrected image, with the mask as a constraint, to obtain the text content and the text position information of the text content, wherein the region inside the mask is left unprocessed; The association module 306 is used to establish a corresponding association between the indentation area and the text content based at least on the location information of the indentation area and the location information of the text.

[0134] The solution provided in this application, through front-end preprocessing combining partitioned exposure control and small-size neighborhood filtering, irregular deformation correction that only performs coordinate mapping without modifying pixel grayscale values, and a dual-branch linkage architecture that uses the mask generated by the indentation detection branch as a mandatory constraint for the text processing branch, ensures that in a single image processing flow, text enhancement and recognition operations are strictly limited to non-indentation areas while the indentation area is completely isolated and protected. This fundamentally overcomes the core contradiction in the traditional single-link global processing architecture where text enhancement operations inevitably erase weak, colorless indentation marks, while retaining indentations leads to a significant decrease in text recognition accuracy. It simultaneously achieves clear and complete preservation of colorless indentations and high-precision text analysis and recognition. Furthermore, by establishing a spatial correspondence between indentation locations and text content, it realizes the automatic binding of physical traces and digital semantic information, providing a unified data foundation for bidirectional traceability retrieval and compliance auditing of warehouse documents.

[0135] In one optional embodiment of this application, the filtering process based on small-size neighborhood is a 1×N linear neighborhood median filter, where N is an integer greater than 1 and not greater than 5.

[0136] In one optional embodiment of this application, the 1×N linear neighborhood median filter is a 1×3 linear neighborhood median filter, and its processing formula is as follows:

[0137] in, For pixels in grayscale value at that location This represents the median operation. This represents the filtered pixel grayscale value.

[0138] In one optional embodiment of this application, the step of performing zoned exposure control on the original image includes: The original image is divided into M×N local sub-regions, where M and N are both integers greater than 1; Calculate the average brightness of each local sub-region. Local sub-regions with an average brightness higher than a preset highlight threshold are identified as highlight reflection areas, and the exposure gain of these areas is reduced. Local sub-regions with an average brightness lower than a preset low brightness threshold are identified as dark areas, and the exposure gain of these areas is increased. Local sub-regions with an average brightness between the preset highlight threshold and the preset low brightness threshold retain their original exposure parameters.

[0139] In one optional embodiment of this application, the irregular deformation correction of the preprocessed image includes: Detect table lines in the preprocessed image, and divide the preprocessed image into multiple cell blocks based on the table lines; Using each cell block as a correction unit, thin-plate spline interpolation transformation is performed using control points located within that cell block to obtain the correction coordinates of that cell block; Based on the corrected coordinates, the pixels of the cell block are remapped to the standard position while keeping the pixel grayscale value unchanged.

[0140] In one optional embodiment of this application, the step of performing indentation detection on the corrected image and generating a mask representing the indentation region includes: Perform multi-scale local light and shadow difference operations on the corrected image to obtain the light and shadow difference response value; The texture gradient features of the corrected image are calculated to obtain the gradient value; The indentation pixels are marked according to the joint discrimination condition, wherein the joint discrimination condition is: the absolute value of the light and shadow difference response value is greater than the preset difference threshold, and the gradient value is between the preset lower gradient limit and the preset upper gradient limit; The mask is generated based on the marked indentation pixels, and the position information of the indentation area is extracted.

[0141] In one optional embodiment of this application, the neighborhood size used in the multi-scale local lighting difference operation includes 1×3 and 3×1.

[0142] In one optional embodiment of this application, the step of performing character recognition processing on the region outside the mask in the corrected image, constrained by the mask, includes: Local adaptive threshold binarization is performed on the image outside the mask to obtain a binarized image; Optical character recognition is performed on the binarized image to obtain the text content and the pixel coordinates of the text content; Based on the fixed table format of the warehouse documents, the text content is semantically structured and parsed to obtain structured text data.

[0143] In one optional embodiment of this application, the local threshold calculation formula used for the local adaptive threshold binarization is:

[0144] in, In pixels A local neighborhood window centered on the center, The total number of pixels within the local area window. Let C be the grayscale value of the corrected image within the local neighborhood window, and C be a grayscale offset constant. The local threshold is defined as follows.

[0145] In one optional embodiment of this application, establishing the correspondence between the indentation area and the text content includes: Using the table cells of the aforementioned warehouse document as the associated units, a cell number is established for each table cell; The text location information, text content, and location information of the indentation area falling within the same table cell are bound to the cell number of the table cell to form a cell-level three-dimensional mapping relationship.

[0146] In one optional embodiment of this application, the method further includes: The cell-level three-dimensional mapping relationship is verified according to the warehousing business rules; wherein, when the table cell is a verification cell and its bound indentation area is not empty, the document is determined to have been verified; when the table cell is a regular business cell and its bound indentation area is not empty, the document is determined to be abnormal and an alarm is output.

[0147] In one optional embodiment of this application, the method further includes: Generate an archived data package, which includes: the original voucher image retaining the indentation features, the business structured data obtained after the structured parsing of the text content, and the association index table containing the corresponding associations; In response to input text keywords, the corresponding cell number is matched from the business structured data, and the corresponding indentation area image is output according to the cell number and the association index table; and / or, in response to the selected indentation area identifier, the corresponding cell number is obtained according to the association index table, and the business structured data corresponding to the cell number is output.

[0148] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a document visual scanning processing method that integrates semantic understanding and artificial intelligence. This method includes: acquiring an original image of a warehouse document; performing partitioned exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image; performing irregular deformation correction on the preprocessed image, wherein the irregular deformation correction corrects paper deformation through coordinate mapping without modifying the grayscale values ​​of the mapped pixels, to obtain a corrected image; performing indentation detection on the corrected image to generate a mask representing the indentation area; using the mask as a constraint, performing text recognition processing on the area outside the mask in the corrected image to obtain text content and text position information of the text content, wherein the area inside the mask is left unprocessed; and establishing a correspondence between the indentation area and the text content, at least based on the position information of the indentation area and the text position information.

[0149] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the document visual scanning processing method that integrates semantic understanding and artificial intelligence provided by the above methods. The method includes: acquiring an original image of a warehouse document; performing partitioned exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image; performing irregular deformation correction on the preprocessed image, wherein the irregular deformation correction corrects paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels to obtain a corrected image; performing indentation detection on the corrected image to generate a mask representing the indentation area; performing text recognition processing on the area outside the mask in the corrected image using the mask as a constraint to obtain text content and text position information of the text content, wherein the area inside the mask is left unprocessed; and establishing a correspondence between the indentation area and the text content based at least on the position information of the indentation area and the text position information.

[0151] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a document visual scanning processing method that integrates semantic understanding and artificial intelligence as described above. The method includes: acquiring an original image of a warehouse document; performing partitioned exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image; performing irregular deformation correction on the preprocessed image, wherein the irregular deformation correction corrects paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels to obtain a corrected image; performing indentation detection on the corrected image to generate a mask representing the indentation area; performing text recognition processing on the area outside the mask in the corrected image using the mask as a constraint to obtain text content and text position information of the text content, wherein the area inside the mask is left unprocessed; and establishing a correspondence between the indentation area and the text content based at least on the position information of the indentation area and the text position information.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A document visual scanning processing method integrating semantic understanding and artificial intelligence, characterized in that, include: Obtain the original image of the warehouse document; The original image is subjected to partitioned exposure control and small-neighborhood filtering to obtain a preprocessed image; The preprocessed image is subjected to irregular deformation correction. The irregular deformation correction corrects the paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels, thus obtaining a corrected image. Indentation detection is performed on the corrected image to generate a mask representing the indentation region; Using the mask as a constraint, text recognition processing is performed on the regions in the corrected image located outside the mask to obtain text content and text position information of the text content, wherein the regions located inside the mask are left unprocessed; At least based on the location information of the indentation area and the location information of the text, a corresponding association between the indentation area and the text content is established.

2. The method according to claim 1, characterized in that, The filtering process based on small-sized neighborhoods is a 1×N linear neighborhood median filter, where N is an integer greater than 1 and not greater than 5.

3. The method according to claim 1, characterized in that, The step of performing zoned exposure control on the original image includes: The original image is divided into M×N local sub-regions, where M and N are both integers greater than 1; Calculate the average brightness of each local sub-region. Local sub-regions with an average brightness higher than a preset highlight threshold are identified as highlight reflection areas, and the exposure gain of these areas is reduced. Local sub-regions with an average brightness lower than a preset low brightness threshold are identified as dark areas, and the exposure gain of these areas is increased. Local sub-regions with an average brightness between the preset highlight threshold and the preset low brightness threshold retain their original exposure parameters.

4. The method according to claim 1, characterized in that, The irregular deformation correction of the preprocessed image includes: Detect table lines in the preprocessed image, and divide the preprocessed image into multiple cell blocks based on the table lines; Using each cell block as a correction unit, thin-plate spline interpolation transformation is performed using control points located within that cell block to obtain the correction coordinates of that cell block; Based on the corrected coordinates, the pixels of the cell block are remapped to the standard position while keeping the pixel grayscale value unchanged.

5. The method according to claim 1, characterized in that, The step of performing indentation detection on the corrected image and generating a mask representing the indentation region includes: Perform multi-scale local light and shadow difference operations on the corrected image to obtain the light and shadow difference response value; The texture gradient features of the corrected image are calculated to obtain the gradient value; The indentation pixels are marked according to the joint discrimination condition, wherein the joint discrimination condition is: the absolute value of the light and shadow difference response value is greater than the preset difference threshold, and the gradient value is between the preset lower gradient limit and the preset upper gradient limit; The mask is generated based on the marked indentation pixels, and the position information of the indentation area is extracted.

6. The method according to claim 1, characterized in that, The step of performing character recognition processing on the region outside the mask in the corrected image, constrained by the mask, includes: Local adaptive threshold binarization is performed on the image outside the mask to obtain a binarized image; Optical character recognition is performed on the binarized image to obtain the text content and the pixel coordinates of the text content; Based on the fixed table format of the warehouse documents, the text content is semantically structured and parsed to obtain structured text data.

7. The method according to claim 1, characterized in that, Establishing the correspondence between the indentation area and the text content includes: Using the table cells of the aforementioned warehouse document as the associated units, a cell number is established for each table cell; The text location information, text content, and location information of the indentation area falling within the same table cell are bound to the cell number of the table cell to form a cell-level three-dimensional mapping relationship.

8. The method according to claim 7, characterized in that, The method further includes: The cell-level three-dimensional mapping relationship is verified according to the warehousing business rules; wherein, when the table cell is a verification cell and its bound indentation area is not empty, the document is determined to have been verified; when the table cell is a regular business cell and its bound indentation area is not empty, the document is determined to be abnormal and an alarm is output.

9. The method according to claim 1, characterized in that, The method further includes: Generate an archived data package, which includes: the original voucher image retaining the indentation features, the business structured data obtained after the structured parsing of the text content, and the association index table containing the corresponding associations; In response to input text keywords, the corresponding cell number is matched from the business structured data, and the corresponding indentation area image is output according to the cell number and the association index table; and / or, in response to the selected indentation area identifier, the corresponding cell number is obtained according to the association index table, and the business structured data corresponding to the cell number is output.

10. A document visual scanning and processing system integrating semantic understanding and artificial intelligence, characterized in that, include: The image acquisition module is used to acquire the original images of warehouse documents; The preprocessing module is used to perform zonal exposure control and small-neighborhood filtering on the original image to obtain a preprocessed image; The deformation correction module is used to perform irregular deformation correction on the preprocessed image. The irregular deformation correction corrects the paper deformation through coordinate mapping without modifying the grayscale value of the mapped pixels, thus obtaining a corrected image. An indentation detection module is used to perform indentation detection on the corrected image and generate a mask representing the indentation region; The text recognition module is used to perform text recognition processing on the region outside the mask in the corrected image, using the mask as a constraint, to obtain the text content and the text position information of the text content, wherein the region inside the mask is left unprocessed; The association module is used to establish a corresponding association between the indentation area and the text content, based at least on the location information of the indentation area and the location information of the text.

Images