Scanned image processing method, device and storage medium

Abstract

The present disclosure provides a scanned image processing method, device, and storage medium, including performing image preprocessing on the acquired original scanned image using a pre-constructed image preprocessing model to determine a corrected scanned image; determining the target position of a target object in the corrected scanned image and extracting target features corresponding to the target object using a pre-constructed image feature extraction model based on the corrected scanned image; separately determining global information and local information corresponding to the target features, and fusing the global information and local information to determine search tag information corresponding to the original scanned image; responding to a search request for an image to be searched, determining the degree of match between the search image features and the search tag information based on the search image features corresponding to the image to be searched, and outputting search results based on the ranking results of the matching degrees. The disclosed method can improve retrieval efficiency and ensure the uniqueness of each scanned image during the retrieval process.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a scanned image processing method, device, and storage medium. Background Art

[0002] Currently, although scanning technology is becoming increasingly mature and the clarity of scanned images generally meets the requirements, there are still some paper materials that have stains on them, and the clarity of images scanned by handheld devices, such as mobile phones, and the electronic images themselves cannot meet the requirements. For example, due to the scanning angle, scanning method and scanning device properties, the clarity is insufficient, or the scanned image is incomplete or the angle is wrong.

[0003] In addition, currently, images obtained by scanning, such as certificate documents, are only stored after obtaining the scanned images, but are not further applied, such as image search, etc., which greatly limits the use scenarios of the images.

[0004] The information disclosed in the background technology section of this application is only intended to deepen the understanding of the general background technology of this application, and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to those skilled in the art. Summary of the Invention

[0005] The embodiments of the present disclosure provide a scanned image processing method, device, and storage medium, which can at least solve some of the problems in the prior art.

[0006] A first aspect of the present disclosure provides a scanned image processing method, comprising:

[0007] Based on the acquired original scan image, the original scan image is preprocessed by using a pre-built image preprocessing model to determine a corrected scan image;

[0008] Based on the corrected scanned image, determining a target position of a target object in the corrected scanned image and extracting target features corresponding to the target object using a pre-constructed image feature extraction model, wherein the target features include at least one of text features, stamp features, and anti-counterfeiting features, and the image feature extraction model includes a first network layer for extracting text features in the image, a second network layer for extracting stamp features in the image, and a third network layer for extracting anti-counterfeiting features in the image;

[0009] respectively determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image;

[0010] In response to a search request for an image to be searched, a matching degree between the search image feature and the search tag information is determined based on a search image feature corresponding to the image to be searched, and a search result is output according to a ranking result of the matching degree.

[0011] In an optional embodiment,

[0012] When the target feature includes a text feature, the method of determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image includes:

[0013] According to the corrected scanned image, performing a sliding scan on the corrected scanned image based on a preset pixel window and a preset step size to determine a text area different from a background area;

[0014] Determining a plurality of candidate text regions in the text region by using a text classifier preset in the first network layer of the image feature extraction model;

[0015] The texture features and edge features of the candidate text regions are extracted, and the texture features and edge features belonging to the same category are spatially mapped to determine the target position and target features of the target text region among the multiple candidate text regions.

[0016] In an optional embodiment,

[0017] When the target feature includes a stamp feature, the method of determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image includes:

[0018] Performing feature mapping on the corrected scanned image to convert it into a multi-dimensional grayscale binary matrix;

[0019] Determining the pixel distance of each pixel point in the multidimensional grayscale binary matrix in the corrected scan image and the weight value corresponding to each pixel point through the second network layer of the image feature extraction model;

[0020] Determine the core area and the peripheral area corresponding to each pixel point based on the feature value, pixel distance and weight value of each pixel point;

[0021] According to the positions and features corresponding to the multiple core areas, the target position corresponding to the target in the core area is determined, and the target features of the target object are extracted.

[0022] In an optional embodiment,

[0023] When the target feature includes an anti-counterfeiting feature, the method of determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image includes:

[0024] Converting all anti-counterfeiting patterns in the corrected scanned image into an anti-counterfeiting feature matrix through image morphology;

[0025] Determining, through the third network layer of the image feature extraction model and in combination with the pre-acquired anti-counterfeiting correspondence, a plaintext feature matrix corresponding to the anti-counterfeiting feature matrix;

[0026] Based on the plaintext feature matrix and the image pixel points corresponding to the anti-counterfeiting shading, the target position of the target object in the corrected scanned image is determined and the target features corresponding to the target object are extracted.

[0027] In an optional embodiment,

[0028] The method of separately determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image includes:

[0029] respectively determining first global information and first local information corresponding to the text feature, second global information and second local information corresponding to the stamp feature, and third global information and third local information corresponding to the anti-counterfeiting feature,

[0030] fusing the first global information and the first local information to obtain first fused information, fusing the second global information and the second local information to obtain second fused information, and fusing the third global information and the third local information to obtain third fused information,

[0031] Based on the attention vector mechanism, a first attention weight is assigned to the first fused information, a second attention weight is assigned to the second fused information, and a third attention weight is assigned to the third fused information;

[0032] Weighted processing is performed based on the first fusion information and the first attention weight, the second fusion information and the second attention weight, and the third fusion information and the third attention weight to determine the search tag information corresponding to the original scanned image.

[0033] In an optional embodiment,

[0034] The method of responding to a search request for an image to be searched, determining a matching degree between the search image feature and the search tag information based on a search image feature corresponding to the image to be searched, and outputting search results according to a ranking result of the matching degree includes:

[0035] Determine the feature category corresponding to the search image feature, and judge whether the feature category is the same as the target category corresponding to the target feature corresponding to the search tag information.

[0036] If they are the same, the vector distance between the feature category and the corresponding feature in the target category is determined respectively.

[0037] If the vector distances of the features corresponding to all categories meet the preset conditions, then the initial matching degree corresponding to each category is determined based on the vector distances and the matching prediction values corresponding to the vector distances;

[0038] The initial matching degrees corresponding to each category are weighted and summed to determine the search matching degree, and the search priority is determined according to the size of the search matching degree, and the search results are output according to the search priority.

[0039] In an optional embodiment,

[0040] The method of determining the initial matching degree corresponding to each category according to the vector distance and the matching prediction value corresponding to the vector distance includes:

[0041] determining a first matching degree between the search image feature and the search tag information according to a vector distance between a first feature vector corresponding to the search image feature and a second feature vector corresponding to the search tag information;

[0042] An initial matching degree corresponding to each category is determined by performing a sum calculation based on the first matching degree and a matching prediction value corresponding to the vector distance.

[0043] In an optional embodiment,

[0044] The method for determining the initial matching degree corresponding to each category is shown in the following formula:

[0045]

[0046] Among them, Mat represents the initial matching degree, N, M, and L represent the number of pixels in the text feature, stamp feature, and anti-counterfeiting feature, respectively, and W i 、P j 、F k They represent the feature vectors corresponding to the text feature, stamp feature and anti-counterfeiting feature respectively, S represents the first feature vector corresponding to the search image feature, α1, α2, α3 represent the matching prediction values corresponding to the text feature, stamp feature and anti-counterfeiting feature respectively.

[0047] In a second aspect of the embodiments of the present disclosure,

[0048] Provided is a device comprising:

[0049] processor;

[0050] a memory for storing processor-executable instructions;

[0051] The processor is configured to call the instructions stored in the memory to execute any one of the aforementioned methods.

[0052] In a third aspect of the embodiments of the present disclosure,

[0053] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, any one of the methods described above is implemented.

[0054] The present disclosure provides a scanned image processing method, comprising:

[0055] Based on the acquired original scan image, the original scan image is preprocessed by using a pre-built image preprocessing model to determine a corrected scan image;

[0056] By preprocessing the original scanned image, you can rotate, affine, crop, scale, remove black edges, adjust the image angle, etc., which reduces the amount of subsequent image processing data and helps improve processing efficiency.

[0057] Based on the corrected scanned image, determining a target position of a target object in the corrected scanned image and extracting target features corresponding to the target object using a pre-constructed image feature extraction model, wherein the target features include at least one of text features, stamp features, and anti-counterfeiting features, and the image feature extraction model includes a first network layer for extracting text features in the image, a second network layer for extracting stamp features in the image, and a third network layer for extracting anti-counterfeiting features in the image;

[0058] The target features in the present disclosure may include at least one of text features, stamp features and anti-counterfeiting features. Compared with other image feature extraction methods, the three target features in the present disclosure can ensure the accuracy of image search. In addition, compared with other certificate image searches, the present disclosure adds information on the dimension of anti-counterfeiting features, which further ensures the uniqueness of the search results.

[0059] respectively determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image;

[0060] By fusing the global information and local information of each feature in text features, stamp features and anti-counterfeiting features respectively, the dependence on the color and size information of the image itself can be reduced, the data calculation overhead can be reduced, and the overall calculation stability is strong. By fusing the global information and the local information to determine the search tag information, it is not only beneficial to improve the retrieval efficiency, but also to ensure the uniqueness of each scanned image.

[0061] In response to a search request for an image to be searched, a matching degree between the search image feature and the search tag information is determined based on a search image feature corresponding to the image to be searched, and a search result is output according to a ranking result of the matching degree.

[0062] By comparing the matching degree between the search image features and the search tag information, and outputting the search results according to the sorting results of the matching degree, the highest matching degree is given first, and other matching results are given. While meeting the search requirements, similar results can also be recommended. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram exemplarily illustrates a flow chart of a scanned image processing method according to an embodiment of the present disclosure;

[0064] Figure 2 A schematic diagram exemplarily showing a scanning certificate according to an embodiment of the present disclosure;

[0065] Figure 3 A schematic diagram exemplarily showing anti-counterfeiting information according to an embodiment of the present disclosure;

[0066] Figure 4 A flowchart showing a disclosed example of a target position of a target text region in a candidate text region and a method for obtaining target features;

[0067] Figure 5 A flowchart illustrating a second disclosed example of a method for obtaining a target position of a target text region in a candidate text region and a target feature thereof is exemplified;

[0068] Figure 6 The flowchart of the third disclosed example of the method for obtaining the target position and target features of the target text region in the candidate text region is exemplified. DETAILED DESCRIPTION

[0069] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0070] The terms "first," "second," "third," "fourth," and so forth (if any) in the description and claims of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be practiced in orders other than those illustrated or described herein.

[0071] It should be understood that in the various embodiments of the present disclosure, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0072] It should be understood that in the present disclosure, "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product or apparatus.

[0073] It should be understood that in the present disclosure, "plurality" refers to two or more. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "Contains A, B and C", "Contains A, B, C" means that A, B, and C are all included, "Contains A, B or C" means that one of A, B, and C is included, and "Contains A, B and / or C" means that any one, any two, or any three of A, B, and C are included.

[0074] It should be understood that, in this disclosure, "B corresponding to A," "B corresponding to A," "A corresponds to B," or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based solely on A; B can also be determined based on A and / or other information. A and B match when the similarity between A and B is greater than or equal to a preset threshold.

[0075] Depending on the context, "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting."

[0076] The following specific embodiments are used to describe the technical solution of the present disclosure in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0077] Figure 1 The flowchart of the scanning image processing method according to the embodiment of the present disclosure is shown as follows: Figure 1 As shown, the method includes:

[0078] S101, based on the acquired original scan image, perform image preprocessing on the original scan image by using a pre-built image preprocessing model to determine a corrected scan image;

[0079] Exemplarily, in an embodiment of the present disclosure, the scanned image may include a certificate image. In actual applications, the certificate image may include various types such as professional certificates, honorary certificates, and ordinary certificates. Among them, some professional certificates need to convert paper documents into electronic files, which need to be scanned. However, due to the scanning angle, scanning method, and scanning device properties, the clarity is insufficient, or the scanned image is incomplete or the angle is incorrect, which makes it difficult for the electronic file to meet the usage needs. In order to reduce the subsequent image processing calculation pressure, the original scanned image can be preprocessed through a pre-constructed image preprocessing model to determine the corrected scanned image.

[0080] Optionally, the image preprocessing model in the embodiment of the present disclosure can be constructed based on a neural network, and is used to perform grayscale processing, binarization processing, rotation, affine processing, cropping, black edge removal, and image angle adjustment on the image. Among them, the image preprocessing process of the embodiment of the present disclosure can be implemented in an operating environment built based on Java-OpenCV.

[0081] Among them, taking the removal of black edges in image preprocessing as an example, Figure 2 A schematic diagram showing an exemplary embodiment of the present disclosure scanning certificate is shown as follows: Figure 2As shown, there is a black edge in the picture due to scanning. The black edge information of the scan certificate can be extracted through image morphological calculation, and it is judged whether the binary mask corresponding to the image pixel point in the black edge information meets the preset pixel threshold, wherein the preset pixel threshold can be the average value of the entire image and the average value of the maximum pixel value of the corresponding pixel point in the black edge information. If it does not meet the preset pixel threshold, the edge information point of the black edge information is extracted, and the edge information point is mean-smoothed, and the pixel point after mean-smoothing is used to replace the pixel point in the black edge information, thereby achieving the technical effect of removing the black edge.

[0082] For other image preprocessing operations, reference may be made to existing image processing operations, which will not be described in detail in the present embodiment.

[0083] By preprocessing the original scanned image, you can rotate, affine, crop, scale, remove black edges, adjust the image angle, etc., which reduces the amount of subsequent image processing data and helps improve processing efficiency.

[0084] S102, determining the target position of the target object in the corrected scan image and extracting target features corresponding to the target object using a pre-built image feature extraction model based on the corrected scan image;

[0085] Exemplarily, the target feature of the embodiment of the present disclosure includes at least one of a text feature, a stamp feature, and an anti-counterfeiting feature, and the image feature extraction model includes a first network layer for extracting text features in the image, a second network layer for extracting stamp features in the image, and a third network layer for extracting anti-counterfeiting features in the image;

[0086] In actual applications, certificate images often include text information, stamp information, and anti-counterfeiting information. Among them, text information and stamp information can be obtained by scanning most certificate images, but anti-counterfeiting information is only for more professional certificates.

[0087] Figure 3 A schematic diagram exemplarily shows the anti-counterfeiting information of an embodiment of the present disclosure, as shown in FIG. Figure 3 As shown, the anti-counterfeiting information can be an image embedded in the printing process, and the background image is formed by one or more graphic arrangements and combinations of background pixels. The anti-counterfeiting information can include at least one of the certificate content, secret information, and unique coding information.

[0088] It can be understood that by extracting the target features corresponding to the target object, it is possible to ensure that the target features can comprehensively and systematically describe the scanned image while also ensuring that the target features uniquely reflect the scanned image.

[0089] Optionally, the image feature extraction model of the embodiment of the present disclosure may include a first network layer for extracting text features in the image, a second network layer for extracting stamp features in the image, and a third network layer for extracting anti-counterfeiting features in the image, wherein the first network layer, the second network layer, and the third network layer may be convolutional neural networks, or other neural networks that can realize corresponding feature extraction functions, and the embodiment of the present disclosure is not limited to this.

[0090] In an optional embodiment,

[0091] When the target feature includes a text feature, a method for determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image is as follows: Figure 4 Shown, including:

[0092] According to the corrected scanned image, performing a sliding scan on the corrected scanned image based on a preset pixel window and a preset step size to determine a text area different from a background area;

[0093] Determining a plurality of candidate text regions in the text region by using a text classifier preset in the first network layer of the image feature extraction model;

[0094] The texture features and edge features of the candidate text regions are extracted, and the texture features and edge features belonging to the same category are spatially mapped to determine the target position and target features of the target text region among the multiple candidate text regions.

[0095] For example, taking the target features including text features as an example, the corrected scanned image can be slidingly scanned through a preset pixel window according to a preset step size, wherein the size of the preset pixel window can be a pixel window of multiple sizes such as 3*3, 6*6, and 9*9, and the preset step size can be 0.5 pixels, 1 pixel, or 2 pixels. The embodiment of the present disclosure does not limit the size of the pixel window and the preset step size.

[0096] By sliding the corrected scanned image through a preset pixel window according to a preset step size, the text area in the scanned image that is different from the background area can be determined, that is, the area where the text is located is extracted from the background area for further processing.

[0097] Furthermore, a text classifier can be used to determine multiple candidate text areas in the text area, that is, in the text area distinguished from the background area, there may be multiple candidate areas of target text information, and then the text information that can truly reflect the scanned image can be selected from the candidate areas.

[0098] Specifically, the texture features and edge features of the candidate text area can be extracted. Among them, the texture features are features that can directly reflect the text, while the edge features are features that can effectively distinguish the text from the background. Combining the two and performing spatial mapping can ensure that the texture features and edge features of the same category are effectively identified and extracted, ensuring the accuracy of the extracted text features and the recognition efficiency.

[0099] In an optional embodiment,

[0100] When the target feature includes a stamp feature, a method for determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image is as follows: Figure 5 Shown, including:

[0101] Performing feature mapping on the corrected scanned image to convert it into a multi-dimensional grayscale binary matrix;

[0102] Determining the pixel distance of each pixel point in the multidimensional grayscale binary matrix in the corrected scan image and the weight value corresponding to each pixel point through the second network layer of the image feature extraction model;

[0103] Determine the core area and the peripheral area corresponding to each pixel point based on the feature value, pixel distance and weight value of each pixel point;

[0104] According to the positions and features corresponding to the multiple core areas, the target position corresponding to the target in the core area is determined, and the target features of the target object are extracted.

[0105] For example, taking the case where the target features include stamp features, in order to reduce the computational complexity of image calculations and reduce data computation overhead, the embodiment of the present disclosure may perform feature mapping on the corrected scanned image and convert it into a multi-dimensional grayscale binary matrix;

[0106] Compared to existing methods that extract and label features based on image color, the present invention effectively reduces reliance on image color by calculating pixel-by-pixel distances and combining them with the weight of each pixel, thereby increasing the application scenarios of the present invention. Some scanned images are inherently gray, white, and black, making it difficult to extract targeted color features.

[0107] According to the characteristic value, pixel distance and weight value of each pixel point, the core area and peripheral area of each pixel point can be determined, and the target features of the target object can be extracted from the positions and features corresponding to multiple core areas. This can not only ensure the effective recognition of stamps, but also improve the robustness of stamp recognition.

[0108] In an optional embodiment,

[0109] When the target feature includes an anti-counterfeiting feature, a method for determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image is as follows: Figure 6 Shown, including:

[0110] Converting all anti-counterfeiting patterns in the corrected scanned image into an anti-counterfeiting feature matrix through image morphology;

[0111] Determining, through the third network layer of the image feature extraction model and in combination with the pre-acquired anti-counterfeiting correspondence, a plaintext feature matrix corresponding to the anti-counterfeiting feature matrix;

[0112] Based on the plaintext feature matrix and the image pixel points corresponding to the anti-counterfeiting shading, the target position of the target object in the corrected scanned image is determined and the target features corresponding to the target object are extracted.

[0113] For example, taking the case where the target features include anti-counterfeiting features, all anti-counterfeiting patterns in the corrected scanned image can be converted into an anti-counterfeiting feature matrix through image morphology, wherein the anti-counterfeiting information can be retained on the floating characters. After scanning, the anti-counterfeiting information retained on the floating characters can still be identified, for example, the pixels with larger patterns can be retained.

[0114] In addition, in order to ensure the security and accuracy of anti-counterfeiting, the anti-counterfeiting feature matrix can be converted into a plaintext feature matrix, wherein the anti-counterfeiting information can be decoded by a pre-acquired key and decryption function, wherein the key and decryption function can be adaptively adjusted according to the specific anti-counterfeiting features, and the embodiments of the present disclosure do not limit this.

[0115] S103, respectively determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image;

[0116] For example, existing image retrieval methods are often based on global information. However, for the specific object of scanned images, the retrieval effect is often not ideal. Scanned images have high similarity in the dimension of global information, which makes image retrieval inefficient and the results obtained are not ideal.

[0117] By fusing the global information and local information corresponding to each target feature, the characteristics of the scanned image can be fully reflected, ensuring the accuracy of the results.

[0118] In an optional embodiment,

[0119] The method of separately determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image includes:

[0120] respectively determining first global information and first local information corresponding to the text feature, second global information and second local information corresponding to the stamp feature, and third global information and third local information corresponding to the anti-counterfeiting feature,

[0121] fusing the first global information and the first local information to obtain first fused information, fusing the second global information and the second local information to obtain second fused information, and fusing the third global information and the third local information to obtain third fused information,

[0122] Based on the attention vector mechanism, a first attention weight is assigned to the first fused information, a second attention weight is assigned to the second fused information, and a third attention weight is assigned to the third fused information;

[0123] Weighted processing is performed based on the first fusion information and the first attention weight, the second fusion information and the second attention weight, and the third fusion information and the third attention weight to determine the search tag information corresponding to the original scanned image.

[0124] Exemplarily, the global information and local information corresponding to each target feature can be fused separately, that is, the first global information and the first local information corresponding to the text feature are fused, the second global information and the second local information corresponding to the stamp feature are fused, and the third global information and the third local information corresponding to the anti-counterfeiting feature are fused.

[0125] After obtaining the fusion features, we can further assign corresponding attention weights to each fusion information, perform weighted processing on the fusion information and each attention weight information, and determine the search tag information corresponding to the original scanned image.

[0126] By fusing the global information and local information of each feature in text features, stamp features and anti-counterfeiting features respectively, the dependence on the color and size information of the image itself can be reduced, the data calculation overhead can be reduced, and the overall calculation stability is strong. By fusing the global information and the local information to determine the search tag information, it is not only beneficial to improve the retrieval efficiency, but also to ensure the uniqueness of each scanned image.

[0127] S104 , responding to a search request for an image to be searched, determining a matching degree between the search image feature and the search tag information based on a search image feature corresponding to the image to be searched, and outputting search results according to a ranking result of the matching degree.

[0128] Exemplarily, in response to a search request for an image to be searched, image features corresponding to the image to be searched, that is, at least one of text features, stamp features, and anti-counterfeiting features, can be obtained, the degree of matching between the search image features and predetermined search tag information can be determined, and the priority of the search results can be determined according to the degree of matching, and search results with a high degree of matching can be output as results with a higher priority.

[0129] By comparing the matching degree between the search image features and the search tag information, and outputting the search results according to the sorting results of the matching degree, the highest matching degree is given first, and other matching results are given. While meeting the search requirements, similar results can also be recommended.

[0130] In an optional embodiment,

[0131] The method of responding to a search request for an image to be searched, determining a matching degree between the search image feature and the search tag information based on a search image feature corresponding to the image to be searched, and outputting search results according to a ranking result of the matching degree includes:

[0132] Determine the feature category corresponding to the search image feature, and judge whether the feature category is the same as the target category corresponding to the target feature corresponding to the search tag information.

[0133] If they are the same, the vector distance between the feature category and the corresponding feature in the target category is determined respectively.

[0134] If the vector distances of the features corresponding to all categories meet the preset conditions, then the initial matching degree corresponding to each category is determined based on the vector distances and the matching prediction values corresponding to the vector distances;

[0135] The initial matching degrees corresponding to each category are weighted and summed to determine the search matching degree, and the search priority is determined according to the size of the search matching degree, and the search results are output according to the search priority.

[0136] For example, it is possible to first determine whether each feature category is the same as the target category. If the target category cannot be matched, there may be missing information. For example, the anti-counterfeiting feature may be missing, and it can be determined that the image to be searched may have certain copyright risks. If the categories are the same, the vector distance is further determined, and the initial matching degree corresponding to each category is determined based on the vector distance and the matching prediction value corresponding to the vector distance. Among them, the matching prediction value can be based on a pre-set corresponding relationship and given according to the specific value of the vector distance. That is, it can be considered that the closer the vector distance, the higher the similarity between the two, and a higher matching prediction value can be set for it. Anyway, the lower the similarity, the lower the matching prediction value set for it.

[0137] In an optional embodiment,

[0138] The method of determining the initial matching degree corresponding to each category according to the vector distance and the matching prediction value corresponding to the vector distance includes:

[0139] determining a first matching degree between the search image feature and the search tag information according to a vector distance between a first feature vector corresponding to the search image feature and a second feature vector corresponding to the search tag information;

[0140] An initial matching degree corresponding to each category is determined by performing a sum calculation based on the first matching degree and a matching prediction value corresponding to the vector distance.

[0141] In an optional embodiment,

[0142] The method for determining the initial matching degree corresponding to each category is shown in the following formula:

[0143]

[0144] Among them, Mat represents the initial matching degree, N, M, and L represent the number of pixels in the text feature, stamp feature, and anti-counterfeiting feature, respectively, and W i 、P j 、F k They represent the feature vectors corresponding to the text feature, stamp feature and anti-counterfeiting feature respectively, S represents the first feature vector corresponding to the search image feature, α1, α2, α3 represent the matching prediction values corresponding to the text feature, stamp feature and anti-counterfeiting feature respectively.

[0145] In response to a search request for an image to be searched, determining a degree of matching between the search image feature and the search tag information based on a search image feature corresponding to the image to be searched, and outputting search results according to a ranking result of the matching degree;

[0146] By comparing the matching degree between the search image features and the search tag information, and outputting the search results according to the sorting results of the matching degree, the highest matching degree is given first, and other matching results are given. While meeting the search requirements, similar results can also be recommended.

[0147] The present disclosure provides a scanned image processing method, comprising:

[0148] Based on the acquired original scan image, the original scan image is preprocessed by using a pre-built image preprocessing model to determine a corrected scan image;

[0149] By preprocessing the original scanned image, you can rotate, affine, crop, scale, remove black edges, adjust the image angle, etc., which reduces the amount of subsequent image processing data and helps improve processing efficiency.

[0150] Based on the corrected scanned image, determining a target position of a target object in the corrected scanned image and extracting target features corresponding to the target object using a pre-constructed image feature extraction model, wherein the target features include at least one of text features, stamp features, and anti-counterfeiting features, and the image feature extraction model includes a first network layer for extracting text features in the image, a second network layer for extracting stamp features in the image, and a third network layer for extracting anti-counterfeiting features in the image;

[0151] The target features in the present disclosure may include at least one of text features, stamp features and anti-counterfeiting features. Compared with other image feature extraction methods, the three target features in the present disclosure can ensure the accuracy of image search. In addition, compared with other certificate image searches, the present disclosure adds information on the dimension of anti-counterfeiting features, which further ensures the uniqueness of the search results.

[0152] respectively determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image;

[0153] By fusing the global information and local information of each feature in text features, stamp features and anti-counterfeiting features respectively, the dependence on the color and size information of the image itself can be reduced, the data calculation overhead can be reduced, and the overall calculation stability is strong. By fusing the global information and the local information to determine the search tag information, it is not only beneficial to improve the retrieval efficiency, but also to ensure the uniqueness of each scanned image.

[0154] In response to a search request for an image to be searched, a matching degree between the search image feature and the search tag information is determined based on a search image feature corresponding to the image to be searched, and a search result is output according to a ranking result of the matching degree.

[0155] By comparing the matching degree between the search image features and the search tag information, and outputting the search results according to the sorting results of the matching degree, the highest matching degree is given first, and other matching results are given. While meeting the search requirements, similar results can also be recommended.

[0156] In a second aspect of the embodiments of the present disclosure,

[0157] Provided is a device comprising:

[0158] processor;

[0159] a memory for storing processor-executable instructions;

[0160] The processor is configured to call the instructions stored in the memory to execute any one of the aforementioned methods.

[0161] In a third aspect of the embodiments of the present disclosure,

[0162] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, any one of the methods described above is implemented.

[0163] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0164] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0165] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0166] The computer program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The computer readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), is personalized by utilizing the state information of the computer readable program instructions, and the electronic circuit can execute the computer readable program instructions, thereby realizing various aspects of the present invention.

[0167] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0168] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0169] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0170] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the function or action of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0171] Note that, unless otherwise directly stated, all features disclosed in this specification (including any accompanying claims, abstracts and drawings) may be replaced by alternative features for achieving the same, equivalent or similar purposes. Therefore, unless otherwise explicitly stated, each feature disclosed is only an example of a group of equivalent or similar features. Where used, further, preferably, further and more preferably are a simple starting point for elaborating another embodiment based on the aforementioned embodiment, and the content of the further, preferably, further or more preferably followed by the above embodiment is combined with the aforementioned embodiment as a complete composition of another embodiment. Several further, preferably, further or more preferably settings following the same embodiment can be arbitrarily combined to form another embodiment.

[0172] Those skilled in the art will appreciate that the embodiments of the present invention described above and shown in the accompanying drawings are intended to be illustrative only and are not intended to limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Any variations or modifications may be made to the embodiments of the present invention without departing from the principles described.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A scanned image processing method, characterized in that: The method comprises: Based on the acquired original scan image, the original scan image is preprocessed by using a pre-built image preprocessing model to determine a corrected scan image; Based on the corrected scanned image, determining a target position of a target object in the corrected scanned image and extracting target features corresponding to the target object using a pre-constructed image feature extraction model, wherein the target features include at least one of text features, stamp features, and anti-counterfeiting features, and the image feature extraction model includes a first network layer for extracting text features in the image, a second network layer for extracting stamp features in the image, and a third network layer for extracting anti-counterfeiting features in the image; respectively determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image; In response to a search request for an image to be searched, a matching degree between the search image feature and the search tag information is determined based on a search image feature corresponding to the image to be searched, and a search result is output according to a ranking result of the matching degree.

2. The method according to claim 1, characterized in that When the target feature includes a text feature, the method of determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image includes: According to the corrected scanned image, performing a sliding scan on the corrected scanned image based on a preset pixel window and a preset step size to determine a text area different from a background area; Determining a plurality of candidate text regions in the text region by using a text classifier preset in the first network layer of the image feature extraction model; The texture features and edge features of the candidate text regions are extracted, and the texture features and edge features belonging to the same category are spatially mapped to determine the target position and target features of the target text region among the multiple candidate text regions.

3. The method according to claim 1, characterized in that When the target feature includes a stamp feature, the method of determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image includes: Performing feature mapping on the corrected scanned image to convert it into a multi-dimensional grayscale binary matrix; Determining the pixel distance of each pixel point in the multidimensional grayscale binary matrix in the corrected scan image and the weight value corresponding to each pixel point through the second network layer of the image feature extraction model; Determine the core area and the peripheral area corresponding to each pixel point based on the feature value, pixel distance and weight value of each pixel point; According to the positions and features corresponding to the multiple core areas, the target position corresponding to the target in the core area is determined, and the target features of the target object are extracted.

4. The method according to claim 1, wherein When the target feature includes an anti-counterfeiting feature, the method of determining the target position of the target object in the corrected scan image and extracting the target feature corresponding to the target object by using a pre-built image feature extraction model according to the corrected scan image includes: Converting all anti-counterfeiting patterns in the corrected scanned image into an anti-counterfeiting feature matrix through image morphology; Determining, through the third network layer of the image feature extraction model and in combination with the pre-acquired anti-counterfeiting correspondence, a plaintext feature matrix corresponding to the anti-counterfeiting feature matrix; Based on the plaintext feature matrix and the image pixel points corresponding to the anti-counterfeiting shading, the target position of the target object in the corrected scanned image is determined and the target features corresponding to the target object are extracted.

5. The method according to any one of claims 2 to 4, characterized in that The method of separately determining global information and local information corresponding to the target feature, and fusing the global information with the local information to determine search tag information corresponding to the original scanned image includes: respectively determining first global information and first local information corresponding to the text feature, second global information and second local information corresponding to the stamp feature, and third global information and third local information corresponding to the anti-counterfeiting feature, fusing the first global information and the first local information to obtain first fused information, fusing the second global information and the second local information to obtain second fused information, and fusing the third global information and the third local information to obtain third fused information, Based on the attention vector mechanism, a first attention weight is assigned to the first fused information, a second attention weight is assigned to the second fused information, and a third attention weight is assigned to the third fused information; Weighted processing is performed based on the first fusion information and the first attention weight, the second fusion information and the second attention weight, and the third fusion information and the third attention weight to determine the search tag information corresponding to the original scanned image.

6. The method according to claim 1, characterized in that The method of responding to a search request for an image to be searched, determining a matching degree between the search image feature and the search tag information based on a search image feature corresponding to the image to be searched, and outputting search results according to a ranking result of the matching degree includes: Determine the feature category corresponding to the search image feature, and judge whether the feature category is the same as the target category corresponding to the target feature corresponding to the search tag information. If they are the same, the vector distance between the feature category and the corresponding feature in the target category is determined respectively. If the vector distances of the features corresponding to all categories meet the preset conditions, then the initial matching degree corresponding to each category is determined based on the vector distances and the matching prediction values corresponding to the vector distances; The initial matching degrees corresponding to each category are weighted and summed to determine the search matching degree, and the search priority is determined according to the size of the search matching degree, and the search results are output according to the search priority.

7. The method according to claim 6, characterized in that The method of determining the initial matching degree corresponding to each category according to the vector distance and the matching prediction value corresponding to the vector distance includes: determining a first matching degree between the search image feature and the search tag information according to a vector distance between a first feature vector corresponding to the search image feature and a second feature vector corresponding to the search tag information; An initial matching degree corresponding to each category is determined by performing a sum calculation based on the first matching degree and a matching prediction value corresponding to the vector distance.

8. The method according to claim 7, characterized in that The method for determining the initial matching degree corresponding to each category is shown in the following formula: Among them, Mat represents the initial matching degree, N, M, and L represent the number of pixels in the text feature, stamp feature, and anti-counterfeiting feature, respectively, and W i 、P j 、F k They represent the feature vectors corresponding to the text feature, stamp feature and anti-counterfeiting feature respectively, S represents the first feature vector corresponding to the search image feature, α1, α2, α3 represent the matching prediction values corresponding to the text feature, stamp feature and anti-counterfeiting feature respectively.

9. A device, characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images