Document data integration method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of image data integration, is applied to a document data integration uploading scene, and relates to a document data integration method, device and equipment and a storage medium thereof. Extracting data contents in all the document images; carrying out ensemble arrangement on data contents in the same document image; inputting all the data content sets into a receipt classification model; classification processing is carried out, and common and independent data contents in the receipt images are sorted; according to the integration condition and the data uploading template, obtaining a data integration result; and uploading to a target receiving end. According to the data integration method and device, the K nearest neighbor algorithm and the image feature extraction model are combined to integrate the data contained in the batch receipts, so that the data integration result is uploaded with the minimum transmission times under the condition that the number of uploaded data uploading files is the single-time maximum uploading limited quantity, and data transmission resource consumption is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of image data integration, and is applied to document data integration and uploading scenarios, and in particular to a document data integration method, device, equipment and storage medium thereof. Background Art

[0002] In financial business, there are often various documents that need to be reviewed, especially when remote business is transmitted, there are clear regulations on the number of documents, which directly leads to a large number of duplicate or similar documents in the uploaded data. In the intelligent processing process, duplicate data is redundant. Therefore, how to integrate the data of related documents is an important pre-processing process.

[0003] In current software applications, there is a lack of universal, unified systems and standards for document data integration. To achieve document data integration, a variety of integration processing components may be required depending on the document type. This not only increases the integration burden and reduces the efficiency of data integration, but also fails to guarantee the rationality of the integration results, and is very likely to waste a lot of data transmission resources. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a document data integration method, device, equipment and storage medium thereof to solve the existing problems of low data integration efficiency, inability to ensure the rationality of the integration results, and easy waste of data transmission resources during document data integration.

[0005] In order to solve the above technical problems, the present application embodiment provides a document data integration method, which adopts the following technical solutions:

[0006] A document data integration method comprises the following steps:

[0007] Step 201, obtaining batch document images to be integrated;

[0008] Step 202, inputting the batch of document images into a pre-trained image feature extraction model to obtain image features output by the image feature extraction model, wherein the image features include data content in all document images;

[0009] Step 203, the data contents in the same document image are aggregated and sorted to obtain a data content set equal to the batch document images;

[0010] Step 204, input all data content sets into a document classification model based on a K-nearest neighbor algorithm, wherein the K value in the K-nearest neighbor algorithm is an adjustable parameter value and is a positive integer;

[0011] Step 205, searching and comparing the data contents in all data content sets in pairs by using the document classification model, and classifying and processing all data content sets according to the comparison results to obtain classification processing results;

[0012] Step 206, identifying the shared data content and the unique data content in each document image according to the classification processing result;

[0013] Step 207, according to a preset integration condition, the shared data content and the unique data content are integrated into a preset data upload template to obtain a data integration result, wherein the integration condition includes: when the number of data upload files uploaded each time is the maximum single upload limit, the data integration result is uploaded with the minimum number of transmissions;

[0014] Step 208: Upload the data content in the batch document image to the target receiving end through the data upload template.

[0015] Furthermore, before executing the step of inputting the batch document images into the pre-trained image feature extraction model to obtain the image features output by the image feature extraction model, the method further includes:

[0016] Obtain unlabeled batch document images as a training sample set;

[0017] Obtain labeled batch document images as a verification sample set;

[0018] The training sample set and the verification sample set are input into the image feature extraction model to be trained, and the image feature extraction model is trained to obtain a pre-trained image feature extraction model.

[0019] Furthermore, the step of inputting the training sample set and the verification sample set into the image feature extraction model to be trained, performing learning and training on the image feature extraction model, and obtaining a pre-trained image feature extraction model specifically includes:

[0020] Step 401, inputting the training sample set into the image feature extraction model to be trained for preliminary training, obtaining the image feature extraction model after preliminary training and obtaining the model parameters of the image feature extraction model after preliminary training;

[0021] Step 402, inputting the verification sample set into the image feature extraction model after the preliminary training, and obtaining the image features corresponding to all the document images in the verification sample set output by the image feature extraction model after the preliminary training;

[0022] Step 403, calculating the loss value of the image feature extraction model after the preliminary training according to the annotation knowledge corresponding to all the document images in the verification sample set and the image features corresponding to all the document images in the verification sample set;

[0023] Step 404, determining whether the loss value satisfies a preset loss condition, wherein the loss condition includes that the loss value is less than a set value;

[0024] Step 405: if the loss value does not meet the preset loss condition, a preset optimizer is used to optimize the model parameters of the image feature extraction model after the preliminary training, and steps 401 to 404 are re-executed;

[0025] Step 406: If the loss value satisfies a preset loss condition, the image feature extraction model after the preliminary training is set as the pre-trained image feature extraction model.

[0026] Furthermore, the image feature extraction model is a Sw in Transformer-based image feature extraction model, which includes a segmentation layer, a feature extraction layer, a merging layer, a linear transformation layer, an attention layer, and a feature decoding layer. The step of inputting the batch document images into the pre-trained image feature extraction model to obtain the image features output by the image feature extraction model specifically includes:

[0027] Each document image in the batch of document images is segmented by the segmentation layer to obtain image feature extraction blocks of the same size;

[0028] The feature extraction layer is used to extract image features from the image feature extraction block to obtain an image feature extraction result corresponding to the image feature extraction block, wherein the feature extraction layer combines a self-attention mechanism and a local attention mechanism when performing image feature extraction on the image feature extraction block through the attention layer;

[0029] Performing feature fusion processing on the image feature extraction results corresponding to all image extraction feature blocks contained in the same document image through the merging layer to obtain a fused image feature representation;

[0030] Mapping the fused image feature representation to a low-dimensional feature space via the linear transformation layer to obtain a low-dimensional mapping result corresponding to each document image;

[0031] The feature decoding layer is used to perform feature decoding on the low-dimensional mapping result corresponding to each document image to obtain the image feature of each document image.

[0032] Furthermore, the step of searching and comparing data contents in all data content sets in pairs by using the document classification model, and classifying all data content sets according to the comparison results to obtain the classification processing results specifically includes:

[0033] Search and compare the data contents in all data content sets in pairs, and obtain the content difference distances between all data content sets based on the comparison results;

[0034] According to the content difference distance between all the data content sets, each document image is used as a spatial node to construct a spatial distance model including the batch document images;

[0035] Based on the spatial distance between the batch document images in the spatial distance model and the initialized K value, the classification category corresponding to each document image in the batch document images is determined, wherein the K value is a positive integer and is an initial parameter preset in the document classification model. The step of obtaining a preliminary classification processing result based on the spatial distance between the batch document images in the spatial distance model and the initialized K value specifically includes: screening out K other document images that are closest to the current document image in spatial distance, and performing statistics on the classification categories to which the K other document images belong; and according to the statistical results, setting the maximum common category corresponding to the K other document images as the classification category of the current document image;

[0036] The document images of the same classification category are sorted to obtain a document image classification set with the same number of classification categories;

[0037] Classification processing is performed on all data content sets according to the document images included in each document image classification set to obtain classification processing results of the same number as the document image classification sets.

[0038] Furthermore, the step of integrating the shared data content and the unique data content into a preset data upload template according to preset integration conditions to obtain a data integration result specifically includes:

[0039] According to the document images contained in each document image classification set, the shared data content and the unique data content contained in each document image classification set are sorted out;

[0040] Based on the maximum upload data volume set by the data upload template, the common data content and the unique data content contained in each document image classification set are added to the data upload template, and the data upload file after the data content corresponding to each document image classification set is added to the data upload template is obtained, and the number of the data upload files is counted;

[0041] Using the preset statistical algorithm:

[0042]

[0043] Count the total number of data upload files corresponding to the classification processing results, where N represents the total number of data upload files corresponding to the classification processing results, n i Indicates the number of data upload files corresponding to the document image classification set numbered i, 1≤i≤j, i and j are positive integers, and j represents the total number of document image classification sets;

[0044] The total number of data upload files corresponding to the classification processing results and all templated data upload files are taken as the data integration results.

[0045] Furthermore, after executing the step of integrating the common data content and the unique data content into a preset data upload template according to the preset integration conditions to obtain a data integration result, the method further includes:

[0046] Continue to adjust the K value and re-execute steps 205 to 207 until the number of adjustments reaches a preset number threshold, and then stop adjusting the K value;

[0047] Count the total number of data uploaded files corresponding to different K values;

[0048] Filter out the K value corresponding to the minimum total number of uploaded data files, and set it in the document classification model as the optimal K value;

[0049] When the optimal K value is obtained, all templated data upload files are used as the optimal data integration result.

[0050] In order to solve the above technical problems, the embodiment of the present application also provides a document data integration device, which adopts the following technical solutions:

[0051] A document data integration device, comprising:

[0052] A document image acquisition module is used to acquire batch document images to be integrated;

[0053] An image feature extraction module, used for inputting the batch of document images into a pre-trained image feature extraction model to obtain image features output by the image feature extraction model, wherein the image features include data content in all document images;

[0054] A data content aggregation module, used to aggregate and organize the data content in the same document image to obtain a data content aggregation equal to the batch document image;

[0055] A classification input module, used to input all data content sets into a document classification model based on a K-nearest neighbor algorithm, wherein the K value in the K-nearest neighbor algorithm is an adjustable parameter value and is a positive integer;

[0056] A classification processing module, used to search and compare the data contents in all data content sets in pairs through the document classification model, and classify and process all data content sets according to the comparison results to obtain classification processing results;

[0057] A data content type identification module, used to identify the common data content and the unique data content in each document image according to the classification processing result;

[0058] A data integration module, for integrating the common data content and the exclusive data content into a preset data upload template according to preset integration conditions, to obtain a data integration result, wherein the integration condition includes: when the number of data upload files uploaded each time is the maximum single upload limit, the data integration result is uploaded with the minimum number of transmissions;

[0059] An integrated data upload module is used to upload the data content in the batch document image to a target receiving end through the data upload template.

[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0061] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the document data integration method described above when executing the computer-readable instructions.

[0062] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0063] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the document data integration method as described above.

[0064] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0065] The document data integration method described in the embodiment of the present application is to obtain batch document images to be integrated; extract the data content in all document images by image feature extraction; collect and organize the data content in the same document image; input all data content sets into a document classification model based on the mutual K nearest neighbor algorithm; classify all data content sets to identify the shared data content and unique data content in each document image; obtain the data integration result according to the preset integration conditions and data upload template; and upload it to the target receiving end. Combining the K nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration result is uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0068] Figure 2 is a flow chart of an embodiment of a document data integration method according to the present application;

[0069] Figure 3 yes Figure 2 A flowchart of a specific embodiment of performing learning and training on the image feature extraction model before step 202 is shown;

[0070] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 303 is shown;

[0071] Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 202 is shown;

[0072] Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 205 is shown;

[0073] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 207 is shown;

[0074] Figure 8 yes Figure 2 A flowchart of a specific embodiment of performing tuning processing on the data integration result after step 207 is shown;

[0075] Fig. 9 is a schematic structural diagram of an embodiment of a document data integration device according to the present application;

[0076] Fig.10 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0077] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0078] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0079] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0080] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0081] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0082] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0083] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0084] It should be noted that the document data integration method provided in the embodiment of the present application is generally executed by a terminal device, and accordingly, the document data integration device is generally set in the terminal device.

[0085] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0086] Continue to refer Figure 2 , shows a flow chart of an embodiment of a document data integration method according to the present application. The document data integration method comprises the following steps:

[0087] Step 201, obtaining batch document images to be integrated.

[0088] In this embodiment, the documents in the batch document image are, for example, documents that need to be filled in when handling financial business, including personal information data filling forms, transaction voucher documents, financial statement documents, etc. Of course, the target document can also be a fee detail flow document in a medical business scenario, a medical expense payment document, etc.

[0089] By scanning the above documents, the batch document images are obtained, and the paper documents are converted into electronic scanned images, which is convenient for subsequent cache management of the information contained in the image and also convenient for future data processing, reuse and viewing.

[0090] Step 202: input the batch of document images into a pre-trained image feature extraction model to obtain image features output by the image feature extraction model, wherein the image features include data content in all document images.

[0091] By inputting the batch of document images into a pre-trained image feature extraction model, the data content in all the document images is obtained, which facilitates the subsequent integration of the data content in the batch of document images before uploading.

[0092] Step 203, the data contents in the same document image are aggregated and sorted to obtain a data content set equal to the batch document images.

[0093] Specifically, different document images can be numbered differently, the data contents corresponding to the different document images can be obtained, and the data contents in the same document image can be added to the same data content set, and the difference number of the document image can be set as the set number of the corresponding data content set to facilitate subsequent processing and calling.

[0094] Step 204: input all data content sets into a document classification model based on a K-nearest neighbor algorithm.

[0095] The K value in the K nearest neighbor algorithm is an adjustable parameter value and is a positive integer.

[0096] Specifically, the K nearest neighbor algorithm is used to classify documents, and the data content in the document image is used as the classification feature to classify the batch of document images. This classification method no longer pre-sets classification categories, and the number of classification categories is only related to the K value of the K nearest neighbor algorithm. As the K value changes, the number of classification categories also changes accordingly, which is more scientific.

[0097] Step 205: search and compare the data contents in all data content sets in pairs through the document classification model, and classify all data content sets according to the comparison results to obtain classification results.

[0098] Step 206: According to the classification result, the shared data content and the unique data content in each document image are respectively identified.

[0099] In this embodiment, the step of identifying the shared data content and the unique data content in each document image specifically includes: determining whether the target data content in the current document image is the data content in at least one other document image in the classification category in which the current document image is located; if so, that is, the target data content in the current document image is the data content in at least one other document image in the classification category in which the current document image is located, then the target data content in the current document image is the shared data content; if not, that is, the target data content in the current document image is not the data content in other document images in the classification category in which the current document image is located, then the target data content in the current document image is the unique data content.

[0100] By identifying the shared data content and the unique data content in each document image, it is convenient to integrate the data content in the same classification category later.

[0101] Step 207: Integrate the shared data content and the unique data content into a preset data upload template according to preset integration conditions to obtain a data integration result.

[0102] The integration condition includes: when the number of data files uploaded each time is the maximum single upload limit, the data integration result can be uploaded with the minimum number of transmission times.

[0103] Specifically, in business scenarios such as finance and foreign trade, when uploading data, the uploaded data is often complex. However, in most cases, there is a limit on the number of files uploaded at one time, and there is also a limit on the maximum data volume of each file. Therefore, under the integration conditions, the common data content and the unique data content are integrated into a preset data upload template to obtain a data integration result, which ensures that when the number of data upload files uploaded each time is the maximum single upload limit, the data integration result is uploaded with the least number of transmissions, thereby improving the upload efficiency of complex data.

[0104] In this embodiment, when the method of integrating the common data content and the unique data content into a preset data upload template to obtain a data integration result is executed, the method further includes: marking the common data content and the unique data content with the corresponding document numbers, so that the target receiving end can obtain the corresponding data content according to the document number. In order to prevent repeated addition to the data upload file, when integrating the common data content, it is necessary to first determine whether the integration has been performed. If not, the common data content is integrated.

[0105] Step 208: Upload the data content in the batch document image to the target receiving end through the data upload template.

[0106] In this embodiment, a batch of document images to be integrated is obtained; the data content in all document images is extracted by image feature extraction; the data content in the same document image is aggregated; all data content sets are input into a document classification model based on a mutual K-nearest neighbor algorithm; all data content sets are classified and processed to identify the shared data content and the unique data content in each document image; the data integration result is obtained according to the preset integration conditions and data upload template; and it is uploaded to the target receiving end. Combining the K-nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration result is uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0107] Continue to refer Figure 3 , Figure 3 yes Figure 2 The flowchart of a specific embodiment of learning and training the image feature extraction model before step 202 is shown. In some optional implementations, learning and training the image feature extraction model includes the following steps:

[0108] Step 301, obtaining unlabeled batch document images as a training sample set;

[0109] Step 302, obtaining a batch of labeled document images as a verification sample set, wherein the labeled reference refers to the data content in the document images;

[0110] Step 303: input the training sample set and the verification sample set into the image feature extraction model to be trained, perform learning training on the image feature extraction model, and obtain a pre-trained image feature extraction model.

[0111] The image feature extraction model is trained through the training sample set and the verification sample set, thereby ensuring the high availability of the image feature extraction model that is finally trained.

[0112] Continue to refer Figure 4 , Figure 4 yes Figure 3 The flowchart of a specific embodiment of step 303 is shown. In some optional implementations, step 303 includes the following steps:

[0113] Step 401, inputting the training sample set into the image feature extraction model to be trained for preliminary training, obtaining the image feature extraction model after preliminary training and obtaining the model parameters of the image feature extraction model after preliminary training;

[0114] Step 402, inputting the verification sample set into the image feature extraction model after the preliminary training, and obtaining the image features corresponding to all the document images in the verification sample set output by the image feature extraction model after the preliminary training;

[0115] Step 403, calculating the loss value of the image feature extraction model after the preliminary training according to the annotation knowledge corresponding to all the document images in the verification sample set and the image features corresponding to all the document images in the verification sample set;

[0116] Step 404, determining whether the loss value satisfies a preset loss condition, wherein the loss condition includes that the loss value is less than a set value;

[0117] Step 405: if the loss value does not meet the preset loss condition, a preset optimizer is used to optimize the model parameters of the image feature extraction model after the preliminary training, and steps 401 to 404 are re-executed;

[0118] Step 406: If the loss value satisfies a preset loss condition, the image feature extraction model after the preliminary training is set as the pre-trained image feature extraction model.

[0119] Specifically, during verification, the loss value is used to determine whether the model training is complete, and the preset optimizer is used to optimize the model parameters of the image feature extraction model, which is more scientific and ensures the availability of the model. The optimizer here can be an Adam optimizer, of course, it can also be an SGD optimizer, Momentum optimizer, AdaGrad optimizer, Nadam optimizer, etc., which can be freely selected according to the setting requirements.

[0120] In this embodiment, the image feature extraction model is a Swin Transformer-based image feature extraction model, which includes a segmentation layer, a feature extraction layer, a merging layer, a linear transformation layer, an attention layer and a feature decoding layer, wherein Swin Transformer refers to a general backbone network, which is a Transformer-based model architecture network.

[0121] Continue to refer Figure 5 , Figure 5 yes Figure 2 The flowchart of a specific embodiment of step 202 is shown. In some optional implementations, step 202 includes the following steps:

[0122] Step 501, segmenting each document image in the batch of document images through the segmentation layer to obtain image feature extraction blocks of the same size;

[0123] Step 502, using the feature extraction layer to extract image features from the image feature extraction block to obtain an image feature extraction result corresponding to the image feature extraction block, wherein the feature extraction layer combines a self-attention mechanism and a local attention mechanism when performing image feature extraction on the image feature extraction block through the attention layer;

[0124] Step 503, performing feature fusion processing on the image feature extraction results corresponding to all image extraction feature blocks contained in the same document image through the merging layer to obtain a fused image feature representation;

[0125] Step 504, mapping the fused image feature representation to a low-dimensional feature space via the linear transformation layer to obtain a low-dimensional mapping result corresponding to each document image;

[0126] Step 505: Use the feature decoding layer to perform feature decoding on the low-dimensional mapping result corresponding to each document image to obtain the image feature of each document image.

[0127] Specifically, the document image is first segmented to obtain more refined image feature extraction blocks. Then, the self-attention mechanism and local attention mechanism are introduced in combination with the Block block idea to perform feature extraction on each more refined image feature extraction block. Feature fusion, linear transformation and feature decoding are then performed to obtain the image features of each document image. The combination of the attention mechanism and more refined small block extraction ensures the accuracy of image feature extraction.

[0128] At this time, the image features of each document image obtained can be understood as a matrix-type feature decoding value, or the actual data content in the document image after decoding. The specific image feature results can be set in output format according to actual needs, and no further limitation is made here.

[0129] Continue to refer Figure 6 , Figure 6 yes Figure 2 The flowchart of a specific embodiment of step 205 is shown. In some optional implementations, step 205 includes the following steps:

[0130] Step 601, searching and comparing data contents in all data content sets in pairs, and obtaining content difference distances between all data content sets according to the comparison results;

[0131] Step 602, based on the content difference distance between all the data content sets, taking each document image as a spatial node, constructing a spatial distance model including the batch document images;

[0132] Step 603, based on the spatial distance between the batch document images in the spatial distance model and the initialized K value, determine the classification category corresponding to each document image in the batch document images, wherein the K value is a positive integer and is an initial parameter pre-set in the document classification model.

[0133] In this embodiment, the step of obtaining a preliminary classification processing result based on the spatial distance between the batch document images in the spatial distance model and the initialized K value specifically includes: screening out K other document images that are closest to the current document image in spatial distance, and performing statistics on the classification categories to which the K other document images belong; and according to the statistical results, setting the maximum common category corresponding to the K other document images as the classification category of the current document image;

[0134] For example: the K value is 10, among which 3 document images correspond to category one, another 3 document images correspond to category two, and the remaining 4 document images correspond to category three. Then the largest common category corresponding to the K other document images is category three, and the classification category of the current document image is category three. In essence, the largest common category can be understood as the category with the largest number of samples.

[0135] Step 604, sorting the document images of the same classification category to obtain a document image classification set equal in number to the classification category;

[0136] Step 605: Classify all data content sets according to the document images included in each document image classification set to obtain classification processing results of the same number as the document image classification set.

[0137] In this embodiment, the batch document images are classified by combining the K nearest neighbor algorithm and the spatial distance, and then the data content set of the document images contained in the same document image classification set is classified and processed to obtain the same number of classification processing results as the document image classification set. In fact, the classification of the data content is controlled so that the classification processing results of the data content are controlled within the corresponding document image classification set, thereby avoiding arbitrary classification processing and reducing the difficulty of subsequent recognition and integration.

[0138] Continue to refer Figure 7 , Figure 7 yes Figure 2 The flowchart of a specific embodiment of step 207 is shown. In some optional implementations, step 207 includes the following steps:

[0139] Step 701, sorting out the common data content and the unique data content contained in each document image classification set according to the document images contained in each document image classification set;

[0140] Step 702, based on the maximum upload data volume set in the data upload template, add the common data content and the unique data content contained in each document image classification set to the data upload template, obtain the data upload file after the data content corresponding to each document image classification set is added to the data upload template, and count the number of the data upload files;

[0141] Step 703, using a preset statistical algorithm:

[0142]

[0143] Count the total number of data upload files corresponding to the classification processing results, where N represents the total number of data upload files corresponding to the classification processing results, n i Indicates the number of data upload files corresponding to the document image classification set numbered i, 1≤i≤j, i and j are positive integers, and j represents the total number of document image classification sets;

[0144] Step 704: The total number of data upload files corresponding to the classification processing result and all templated data upload files are taken as the data integration result.

[0145] The data actually required to be uploaded is integrated through the data upload template and integrated into a data upload file that meets the data upload amount requirement for one time, thereby avoiding too low a data amount for a single upload and too high a data amount for a single upload. Moreover, the data actually required to be uploaded is integrated according to the data upload template. Accordingly, during the integration, the common data content and the unique data content in the document image can be structured in a preset structured form according to the template format of the data upload template, so as to facilitate subsequent extraction and identification by the target receiving end.

[0146] Continue to refer Figure 8 , Figure 8 yes Figure 2 The flowchart of a specific embodiment of performing tuning processing on the data integration result after step 207 is shown. In some optional implementations, performing tuning processing on the data integration result after step 207 includes the following steps:

[0147] Step 801, continuously adjusting the K value, and re-executing steps 205 to 207, until the number of adjustments reaches a preset number threshold, and then stopping the K value adjustment;

[0148] Specifically, the adjustment of the K value may be performed by presetting a set of K values, for example, {3, 4, 5, 7, 9, 10, 11, 13, 14, 15, 17}, and the adjustment of the K value may be performed by selecting different values ​​in the set;

[0149] Step 802, counting the total number of data upload files corresponding to different K values;

[0150] Step 803, filtering out the K value corresponding to the minimum total number of uploaded data files, and setting it in the document classification model as the optimal K value;

[0151] Step 804, when the optimal K value is obtained, all templated data upload files are used as the optimal data integration result.

[0152] Specifically, by continuously adjusting the K value, that is, the parameter value of the K nearest neighbor algorithm, the data content in the batch document images can be continuously classified and optimized, and combined with the upload restriction conditions of the target receiving end, the data content in the batch document images can be optimally integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration result is uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0153] The present application obtains batch document images to be integrated; extracts data content from all document images using image feature extraction; organizes data content from the same document image in a collection; inputs all data content collections into a document classification model based on a mutual K-nearest neighbor algorithm; classifies all data content collections to identify shared data content and unique data content in each document image; obtains data integration results based on preset integration conditions and data upload templates; and uploads to the target receiving end. Combining the K-nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration results are uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0154] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0155] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0156] In the embodiment of the present application, a batch of document images to be integrated is obtained; the data content in all document images is extracted by image feature extraction; the data content in the same document image is aggregated; all data content sets are input into a document classification model based on a mutual K-nearest neighbor algorithm; all data content sets are classified and processed to identify the shared data content and the unique data content in each document image; the data integration result is obtained according to the preset integration conditions and data upload template; and it is uploaded to the target receiving end. Combining the K-nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration result is uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0157] Further references Fig. 9 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a document data integration device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0158] like Fig. 9 As shown, the document data integration device 900 described in this embodiment includes: a document image acquisition module 901, an image feature extraction module 902, a data content aggregation module 903, a classification input module 904, a classification processing module 905, a data content type identification module 906, a data integration module 907 and an integrated data upload module 908. Among them:

[0159] The document image acquisition module 901 is used to acquire batch document images to be integrated;

[0160] An image feature extraction module 902 is used to input the batch of document images into a pre-trained image feature extraction model to obtain image features output by the image feature extraction model, wherein the image features include data content in all document images;

[0161] The data content aggregation module 903 is used to aggregate the data content in the same document image to obtain a data content aggregation equal to the batch document image;

[0162] A classification input module 904, used to input all data content sets into a document classification model based on a K-nearest neighbor algorithm, wherein the K value in the K-nearest neighbor algorithm is an adjustable parameter value and is a positive integer;

[0163] The classification processing module 905 is used to search and compare the data contents in all data content sets in pairs by using the document classification model, and classify all data content sets according to the comparison results to obtain classification processing results;

[0164] A data content type identification module 906, for identifying the common data content and the unique data content in each document image according to the classification processing result;

[0165] The data integration module 907 is used to integrate the common data content and the unique data content into a preset data upload template according to a preset integration condition to obtain a data integration result, wherein the integration condition includes: when the number of data upload files uploaded each time is the maximum single upload limit, the data integration result is uploaded in the minimum number of transmission times;

[0166] The integrated data upload module 908 is used to upload the data content in the batch document image to the target receiving end through the data upload template.

[0167] The present application obtains batch document images to be integrated; extracts data content from all document images using image feature extraction; organizes data content from the same document image in a collection; inputs all data content collections into a document classification model based on a mutual K-nearest neighbor algorithm; classifies all data content collections to identify shared data content and unique data content in each document image; obtains data integration results based on preset integration conditions and data upload templates; and uploads to the target receiving end. Combining the K-nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration results are uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0169] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0170] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0171] The computer device 10 includes a memory 10a, a processor 10b, and a network interface 10c which are interconnected and communicated via a system bus. It should be noted that: Fig.10 Only a computer device 10 having a component memory 10a, a processor 10b, and a network interface 10c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0172] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0173] The memory 10a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10a can be an internal storage unit of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 10a can also be an external storage device of the computer device 10, such as a plug-in hard disk equipped on the computer device 10, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10a can also include both the internal storage unit of the computer device 10 and its external storage device. In this embodiment, the memory 10a is generally used to store an operating system and various application software installed on the computer device 10, such as a computer-readable instruction of a document data integration method. In addition, the memory 10a can also be used to temporarily store various data that have been output or are to be output.

[0174] The processor 10b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 10b is generally used to control the overall operation of the computer device 10. In this embodiment, the processor 10b is used to run the computer-readable instructions stored in the memory 10a or process data, such as running the computer-readable instructions of the document data integration method.

[0175] The network interface 10c may include a wireless network interface or a wired network interface. The network interface 10c is generally used to establish a communication connection between the computer device 10 and other electronic devices.

[0176] The computer device proposed in this embodiment belongs to the field of image data integration technology and is applied to document data integration and uploading scenarios. This application obtains batch document images to be integrated; uses image feature extraction to extract data content from all document images; collects and organizes data content in the same document image; inputs all data content sets into a document classification model based on the mutual K nearest neighbor algorithm; classifies all data content sets to identify shared data content and unique data content in each document image; obtains data integration results according to preset integration conditions and data upload templates; and uploads to the target receiving end. Combining the K nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration results are uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0177] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the document data integration method as described above.

[0178] The computer-readable storage medium proposed in this embodiment belongs to the field of image data integration technology and is applied to document data integration and uploading scenarios. This application obtains batch document images to be integrated; extracts data content from all document images using image feature extraction; organizes data content in the same document image in a collection; inputs all data content sets into a document classification model based on a mutual K-nearest neighbor algorithm; classifies all data content sets to identify shared data content and unique data content in each document image; obtains data integration results according to preset integration conditions and data upload templates; and uploads to the target receiving end. In combination with the K-nearest neighbor algorithm and the image feature extraction model, the data contained in the batch documents is integrated to achieve the condition that the number of data upload files uploaded each time is the maximum single upload limit, and the data integration results are uploaded with the least number of transmissions, thereby avoiding the consumption of data transmission resources.

[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0180] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A document data integration method, characterized in that: The steps include: Step 201, obtaining batch document images to be integrated; Step 202, inputting the batch of document images into a pre-trained image feature extraction model to obtain image features output by the image feature extraction model, wherein the image features include data content in all document images; Step 203, the data contents in the same document image are aggregated and sorted to obtain a data content set equal to the batch document images; Step 204, input all data content sets into a document classification model based on a K-nearest neighbor algorithm, wherein the K value in the K-nearest neighbor algorithm is an adjustable parameter value and is a positive integer; Step 205, searching and comparing the data contents in all data content sets in pairs by using the document classification model, and classifying and processing all data content sets according to the comparison results to obtain classification processing results; Step 206, identifying the shared data content and the unique data content in each document image according to the classification processing result; Step 207, according to a preset integration condition, the shared data content and the unique data content are integrated into a preset data upload template to obtain a data integration result, wherein the integration condition includes: when the number of data upload files uploaded each time is the maximum single upload limit, the data integration result is uploaded with the minimum number of transmissions; Step 208: Upload the data content in the batch document image to the target receiving end through the data upload template.

2. The document data integration method according to claim 1, characterized in that: Before executing the step of inputting the batch document images into the pre-trained image feature extraction model to obtain the image features output by the image feature extraction model, the method further includes: Obtain unlabeled batch document images as a training sample set; Obtain labeled batch document images as a verification sample set; The training sample set and the verification sample set are input into the image feature extraction model to be trained, and the image feature extraction model is trained to obtain a pre-trained image feature extraction model.

3. The document data integration method according to claim 2, characterized in that: The step of inputting the training sample set and the verification sample set into the image feature extraction model to be trained, performing learning and training on the image feature extraction model, and obtaining a pre-trained image feature extraction model specifically includes: Step 401, inputting the training sample set into the image feature extraction model to be trained for preliminary training, obtaining the image feature extraction model after preliminary training and obtaining the model parameters of the image feature extraction model after preliminary training; Step 402, inputting the verification sample set into the image feature extraction model after the preliminary training, and obtaining the image features corresponding to all the document images in the verification sample set output by the image feature extraction model after the preliminary training; Step 403, calculating the loss value of the image feature extraction model after the preliminary training according to the annotation knowledge corresponding to all the document images in the verification sample set and the image features corresponding to all the document images in the verification sample set; Step 404, determining whether the loss value satisfies a preset loss condition, wherein the loss condition includes that the loss value is less than a set value; Step 405: if the loss value does not meet the preset loss condition, a preset optimizer is used to optimize the model parameters of the image feature extraction model after the preliminary training, and steps 401 to 404 are re-executed; Step 406: If the loss value satisfies a preset loss condition, the image feature extraction model after the preliminary training is set as the pre-trained image feature extraction model.

4. The document data integration method according to claim 1, characterized in that: The image feature extraction model is an image feature extraction model based on Swin Transformer, and the image feature extraction model based on Swin Transformer includes a segmentation layer, a feature extraction layer, a merging layer, a linear transformation layer, an attention layer, and a feature decoding layer. The step of inputting the batch document images into the pre-trained image feature extraction model to obtain the image features output by the image feature extraction model specifically includes: Each document image in the batch of document images is segmented by the segmentation layer to obtain image feature extraction blocks of the same size; The feature extraction layer is used to extract image features from the image feature extraction block to obtain an image feature extraction result corresponding to the image feature extraction block, wherein the feature extraction layer combines a self-attention mechanism and a local attention mechanism when performing image feature extraction on the image feature extraction block through the attention layer; Performing feature fusion processing on the image feature extraction results corresponding to all image extraction feature blocks contained in the same document image through the merging layer to obtain a fused image feature representation; Mapping the fused image feature representation to a low-dimensional feature space via the linear transformation layer to obtain a low-dimensional mapping result corresponding to each document image; The feature decoding layer is used to perform feature decoding on the low-dimensional mapping result corresponding to each document image to obtain the image feature of each document image.

5. The document data integration method according to claim 1, characterized in that: The step of searching and comparing the data contents in all data content sets in pairs by using the document classification model, and classifying and processing all data content sets according to the comparison results to obtain the classification processing results specifically includes: Search and compare the data contents in all data content sets in pairs, and obtain the content difference distances between all data content sets based on the comparison results; According to the content difference distance between all the data content sets, each document image is used as a spatial node to construct a spatial distance model including the batch document images; Based on the spatial distance between the batch document images in the spatial distance model and the initialized K value, the classification category corresponding to each document image in the batch document images is determined, wherein the K value is a positive integer and is an initial parameter preset in the document classification model. The step of obtaining a preliminary classification processing result based on the spatial distance between the batch document images in the spatial distance model and the initialized K value specifically includes: screening out K other document images that are closest to the current document image in spatial distance, and performing statistics on the classification categories to which the K other document images belong; and according to the statistical results, setting the maximum common category corresponding to the K other document images as the classification category of the current document image; The document images of the same classification category are sorted to obtain a document image classification set with the same number of classification categories; Classification processing is performed on all data content sets according to the document images included in each document image classification set to obtain classification processing results of the same number as the document image classification sets.

6. The document data integration method according to claim 5, characterized in that: The step of integrating the shared data content and the unique data content into a preset data upload template according to the preset integration conditions to obtain a data integration result specifically includes: According to the document images contained in each document image classification set, the shared data content and the unique data content contained in each document image classification set are sorted out; Based on the maximum upload data volume set by the data upload template, the common data content and the unique data content contained in each document image classification set are added to the data upload template, and the data upload file after the data content corresponding to each document image classification set is added to the data upload template is obtained, and the number of the data upload files is counted; Using the preset statistical algorithm: Count the total number of data upload files corresponding to the classification processing results, where N represents the total number of data upload files corresponding to the classification processing results, n i Indicates the number of data upload files corresponding to the document image classification set numbered i, 1≤i≤j, i and j are positive integers, and j represents the total number of document image classification sets; The total number of data upload files corresponding to the classification processing results and all templated data upload files are taken as the data integration results.

7. The document data integration method according to claim 6, characterized in that: After executing the step of integrating the common data content and the unique data content into a preset data upload template according to the preset integration conditions to obtain a data integration result, the method further includes: Continue to adjust the K value and re-execute steps 205 to 207 until the number of adjustments reaches a preset number threshold, and then stop adjusting the K value; Count the total number of data uploaded files corresponding to different K values; Filter out the K value corresponding to the minimum total number of uploaded data files, and set it in the document classification model as the optimal K value; When the optimal K value is obtained, all templated data upload files are used as the optimal data integration result.

8. A document data integration device, characterized in that: include: A document image acquisition module is used to acquire batch document images to be integrated; An image feature extraction module, used for inputting the batch of document images into a pre-trained image feature extraction model to obtain image features output by the image feature extraction model, wherein the image features include data content in all document images; A data content aggregation module, used to aggregate and organize the data content in the same document image to obtain a data content aggregation equal to the batch document image; A classification input module, used to input all data content sets into a document classification model based on a K-nearest neighbor algorithm, wherein the K value in the K-nearest neighbor algorithm is an adjustable parameter value and is a positive integer; A classification processing module, used to search and compare the data contents in all data content sets in pairs through the document classification model, and classify and process all data content sets according to the comparison results to obtain classification processing results; A data content type identification module, used to identify the common data content and the unique data content in each document image according to the classification processing result; A data integration module, for integrating the common data content and the exclusive data content into a preset data upload template according to preset integration conditions, to obtain a data integration result, wherein the integration condition includes: when the number of data upload files uploaded each time is the maximum single upload limit, the data integration result is uploaded with the minimum number of transmissions; An integrated data upload module is used to upload the data content in the batch document image to a target receiving end through the data upload template.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the document data integration method as claimed in any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the document data integration method according to any one of claims 1 to 7 are implemented.

Images