Work order auditing method and system

Abstract

The present application discloses a work order auditing method and system to solve the technical problem of low efficiency in work order auditing processing. Among them, a work order auditing solution filters out blurred images and duplicate images through multiple screenings of the images recording work order information, reducing the invalid data to be audited for work orders, thereby reducing the processing volume of work order auditing and improving the processing efficiency. And by identifying forged images, the incorrect data to be audited for work orders is filtered out, further reducing the processing volume of work order auditing and improving the processing efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a work order auditing method and system. Background Art

[0002] As a type of business acceptance voucher, the electronic work order form provides two-way rights protection for both customers and service providers. To ensure the data accuracy of the electronic work order, it is necessary to check the uploaded work order images and work order text information to prevent the occurrence of over-reporting, misreporting, and fraudulent claiming of work orders.

[0003] In the process of implementing the existing technology, the inventors found that:

[0004] The number of electronic work order forms is huge, and the proportion of error data or invalid data is relatively large, which results in a large processing volume and low processing efficiency for work order auditing in the existing technology.

[0005] Therefore, a new work order auditing solution is needed to solve the technical problem of low processing efficiency of work order auditing. Summary of the Invention

[0006] The embodiments of this application provide a new work order auditing solution to solve the technical problem of low processing efficiency of work order auditing.

[0007] Specifically, a work order auditing method includes the following steps:

[0008] Obtain the images recording work order information to form a first set;

[0009] Select any image from the first set and extract the first attribute value of the image; the first attribute value is used to characterize the local texture feature of the image;

[0010] Screen the images in the first set whose first attribute value is greater than or equal to the first screening threshold to form a second set;

[0011] Select any image from the second set and extract the second attribute value of the image; the second attribute value is used to characterize the pixel feature of the image;

[0012] Select any two images from the second set and calculate the difference value of the second attribute values of the two images;

[0013] When the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, retain the image with the highest first attribute value in the second set to form a third set;

[0014] Select any image from the third set as the first image;

[0015] Identify the key text information of the first image;

[0016] Obtain the descriptive text information associated with the first image;

[0017] Compare the key text information and the descriptive text information of the first image;

[0018] When the key text information of the first image is the same as the descriptive text information of the first image, store the first image and the descriptive text information of the first image.

[0019] Furthermore, select any image from the first set, and extract the first attribute value of the image, specifically including:

[0020] Select any image from the first set as the first image;

[0021] Extract the local texture features of the first image;

[0022] Cluster the local texture features of the image to obtain a centroid set;

[0023] Normalize the centroid set to obtain a normalized matrix;

[0024] Use the normalized matrix as a codebook for assignment encoding to obtain a coefficient matrix;

[0025] Pool the coefficient matrix and convert it into a local texture feature vector;

[0026] Use the local texture feature vector as the first attribute value of the first image.

[0027] Furthermore, the method further includes:

[0028] Extract at least one of the histogram features, texture features, and color features of the first image;

[0029] Input at least one of the histogram features, texture features, and color features of the first image into a forgery verification model to verify whether the first image is forged;

[0030] When the first image is not forged, identify the key text information of the first image.

[0031] Furthermore, the extraction of the texture features of the first image specifically includes:

[0032] Extract the texture features of the first image using the center symmetric local binary pattern algorithm;

[0033] Among them, the center symmetric local binary pattern algorithm is expressed as:

[0034] ,

[0035] In the formula, represents the pixel coordinates, and

[0036] ,

[0037] wherein, is the gray level of the centrosymmetric pixel, is the threshold of the centrosymmetric local binary pattern algorithm descriptor.

[0038] Furthermore, the work order auditing method is used for settling the charging list;

[0039] The key text information at least includes the work order name or the stage name.

[0040] The embodiment of the present application also provides a work order auditing system.

[0041] Specifically, a work order auditing system includes:

[0042] An acquisition module, configured to acquire an image recording work order information and form a first set;

[0043] A quality auditing module, configured to select an arbitrary image from the first set and extract a first attribute value of the image; the first attribute value is used to characterize the local texture feature of the image; and it is further configured to screen out images from the first set whose first attribute value is greater than or equal to a first screening threshold, and form a second set;

[0044] A similarity auditing module, configured to select an arbitrary image from the second set and extract a second attribute value of the image; the second attribute value is used to characterize the pixel feature of the image; and it is further configured to select any two images from the second set and calculate the difference value of the second attribute values of any two images; and it is further configured to retain the image with the highest first attribute value in the second set when the difference value of the second attribute values of any two images in the second set is less than or equal to a second screening threshold, and form a third set;

[0045] A content auditing module, configured to select an arbitrary image from the third set as a first image; and it is further configured to identify the key text information of the first image; and it is further configured to obtain the description text information associated with the first image; and it is further configured to compare the key text information and the description text information of the first image;

[0046] An archiving module, configured to store the first image and the description text information of the first image when the key text information of the first image is the same as the description text information of the first image.

[0047] Furthermore, the quality auditing module is configured to extract the first attribute value of the image in the first set, specifically:

[0048] Select an arbitrary image from the first set as a first image;

[0049] Extract the local texture feature of the first image;

[0050] Cluster the local texture features of the image to obtain a set of centroids;

[0051] Normalize the set of centroids to obtain a normalized matrix;

[0052] Use the normalized matrix as a codebook for assignment coding to obtain a coefficient matrix;

[0053] Pool the coefficient matrix and convert it into a local texture feature vector;

[0054] Use the local texture feature vector as the first attribute value of the first image.

[0055] Furthermore, the system further includes a forgery verification module, which is specifically used for:

[0056] Extract at least one of the histogram features, texture features, and color features of the first image;

[0057] Input at least one of the histogram features, texture features, and color features of the first image into a forgery verification model to verify whether the first image is forged;

[0058] When the first image is not forged, identify the key text information of the first image.

[0059] Furthermore, the forgery verification module extracts the texture features of the first image, specifically including:

[0060] Use the center symmetric local binary pattern algorithm to extract the texture features of the first work order image;

[0061] Among them, the center symmetric local binary pattern algorithm is expressed as:

[0062] ,

[0063] In the formula, represents the pixel coordinates, and

[0064] ,

[0065] In the formula, is the gray level of the center symmetric pixel, is the threshold of the center symmetric local binary pattern algorithm descriptor.

[0066] Furthermore, the work order auditing system is used to settle the billing statement;

[0067] The key text information at least includes the work order name or the stage name.

[0068] The technical solution provided by the embodiments of the present application has at least the following beneficial effects:

[0069] By screening the images containing work order information multiple times to filter out blurred images and duplicate images, the invalid data to be audited for work orders is reduced, thereby reducing the processing volume of work order audits and improving the processing efficiency. And by identifying forged images to filter out the incorrect data to be audited for work orders, the processing volume of work order audits is further reduced and the processing efficiency is improved. Brief Description of the Drawings

[0070] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0071] Figure 1 It is a flowchart of a work order auditing method provided by an embodiment of the present application.

[0072] Figure 2 It is a schematic structural diagram of a work order auditing system provided by an embodiment of the present application.

[0073] 100 Work order auditing system

[0074] 11 Acquisition module

[0075] 12 Quality auditing module

[0076] 13 Similarity auditing module

[0077] 14 Content auditing module

[0078] 15 Archiving module

[0079] 16 Forgery verification module Detailed Description of the Embodiments

[0080] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0081] Please refer to Figure 1 , to solve the technical problem of low processing efficiency of work order audits, the present application provides a work order auditing method, including the following steps:

[0082] S110: Obtain the images containing work order information to form a first set.

[0083] It should be noted that in the actual application scenario of this application, the operator will upload work order images and work order text information on the client side. In the actual application scenario, the images recording the work order information described in this application usually appear as photographed work order images or screen-captured work order images. Several work order images from the client side form the first set.

[0084] S120: Select any image from the first set and extract the first attribute value of the image.

[0085] S130: Screen the images in the first set whose first attribute value is greater than or equal to the first screening threshold to form a second set.

[0086] It can be understood that the first attribute value is used to characterize the local texture features of the image. And screening the images in the first set whose first attribute value is greater than or equal to the first screening threshold is essentially screening the image quality to filter out images with poor quality, such as blurred images and images with unclear edges. In other words, filtering out the images with poor quality in the first set is filtering out the invalid data in the data to be processed.

[0087] After filtering out the images with poor quality from the first set, the remaining several images will form the second set. It can be seen that the data to be processed in the second set is less than that in the first set, which is equivalent to reducing the processing volume of work order auditing and improving the processing efficiency.

[0088] The following introduces the specific implementation process of steps S120 - S130:

[0089] Considering that the first set can be composed of multiple images, for the sake of concise description, here we select any image from the first set as the first image; extracting the first attribute value of the first image is described, and it should not be construed as a limitation on the scope of the invention patent.

[0090] Furthermore, in a specific implementation manner provided by this application, selecting any image from the first set and extracting the first attribute value of the image specifically includes:

[0091] S1201: Select any image from the first set as the first image;

[0092] S1202: Extract the local texture features of the first image;

[0093] S1203: Cluster the local texture features of the image to obtain a centroid set;

[0094] S1204: Normalize the centroid set to obtain a normalized matrix;

[0095] S1205: Use the normalized matrix as a codebook for assignment coding to obtain a coefficient matrix;

[0096] S1206: Pool the coefficient matrix and convert it into a local texture feature vector;

[0097] S1207: Use the local texture feature vector as the first attribute value of the first image.

[0098] Specifically, assuming the first image is represented as I, extract local descriptors from a set of image patches, where .

[0099] Then, for S1202: Extract the local texture features of the first image to obtain X, the steps are as follows:

[0100] Randomly sample N different image patches uniformly from I;

[0101] Normalize each patch by subtracting the mean and dividing by the standard deviation of its elements;

[0102] Perform zero-component analysis (ZCA) whitening on the normalized patches.

[0103] After that, steps S1203 - S1204 are essentially constructing a visual codebook. The construction process is manifested as performing K-means clustering on the local texture features X extracted from unlabeled training images. The specific construction process is as follows:

[0104] Calculate its local descriptor X for each training image;

[0105] Perform K-means clustering on the local descriptors to obtain K centroids ;

[0106] For perform normalization to obtain the normalized matrix , which serves as the codebook C.

[0107] Then, step S1205 requires using the normalized codebook for soft assignment encoding. Specifically, first calculate the distance between the local descriptor and the visual codewords in using the dot product. Let be the similarity measure between the i-th local descriptor and the j-th basis . Then . The code element of the local descriptor

[0108] can be written as follows:

[0109] Then the encoding of the local descriptor can provide a coefficient matrix .

[0110] Finally, step S1206 is performed. Considering that in the image classification problem, the max-pooling method shows higher performance than the average pooling or sum pooling. This application preferably uses the max-pooling method to give a fixed-length local texture feature vector.

[0111] The image feature using max-pooling is defined as:

[0112] ,

[0113] where Ψmax is defined on each row of C, , The i-th element is given by:

[0114] ,

[0115] Then it is the local texture feature vector of the image.

[0116] Of course, this application also provides an image quality auditing model for screening images with the first attribute value greater than or equal to the first screening threshold from the first set. The image quality auditing model is a support vector machine (SVM) classifier model using a linear kernel. The training process of the image quality auditing model is as follows:

[0117] Select two types of images for training, one is a qualified image with a label of 0; the other is an unqualified image with a label of 1;

[0118] For each image, calculate the local descriptor, where B = 7 and N = 10000;

[0119] Perform K-means clustering on the local descriptors of all images, K = 10000, and calculate the codebook;

[0120] Perform soft encoding on the local descriptors of each image to obtain a coefficient matrix ;

[0121] Use the max-pooling method to obtain the local texture feature vector of the image ;

[0122] Input the local texture feature vector of each image and the label into a support vector machine (SVM) classifier with a linear kernel to train the image quality auditing model.

[0123] S140: Select an arbitrary image from the second set and extract the second attribute value of the image.

[0124] S150: Select any two images from the second set, and calculate the difference value of the second attribute values of the two images.

[0125] S160: When the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, retain the image with the highest first attribute value in the second set to form the third set.

[0126] It should be noted that the second attribute value is used to characterize the image pixel features. Retaining the image with the highest first attribute value among any two images in the second set whose difference value of the second attribute values is less than or equal to the second screening threshold is essentially filtering out similar or duplicate images and retaining the clearer images. In other words, filtering out similar or duplicate images in the second set is filtering out duplicate data in the data to be processed.

[0127] After filtering out similar or duplicate images from the second set, the remaining several images will form the third set. It can be seen that the data to be processed in the third set is further less than that in the first set or the second set, which further reduces the processing volume of work order auditing and thus improves the processing efficiency.

[0128] The following introduces the specific implementation process of steps S140 - S160:

[0129] Considering that the second set can be composed of multiple images, for the sake of concise description, here an arbitrary image is selected from the second set as the first image; the extraction of the second attribute value of the first image is described, and it should not be construed as a limitation on the scope of the invention patent.

[0130] Furthermore, in a specific implementation manner provided by the present application, selecting an arbitrary image from the second set and extracting the second attribute value of the image specifically includes:

[0131] Select an arbitrary image from the second set as the first image;

[0132] Scale the size of the first image to a preset size to generate a first preprocessed image;

[0133] Convert the first preprocessed image into a grayscale image to generate a second preprocessed image;

[0134] Subtract the pixel value of the previous row from the pixel value of the current row of the second preprocessed image to obtain the difference value between adjacent pixels;

[0135] Traverse the second preprocessed image in sequence with a preset difference matrix to construct the Hash value of the first image;

[0136] Use the Hash value of the first image as the second attribute value of the first image.

[0137] Among them, through multiple experiments by the inventor, the preset size is preferably 9 * 8 pixel points, so the first preprocessed image has 72 pixel points. The second preprocessed image is a 256 - level grayscale image. For converting an RGB three - channel image into a single - channel 256 - level grayscale image, the following algorithm can be used:

[0138] Gray =(R*76+G*151+B*28)>>8.

[0139] In this way, there will also be 8 rows of difference values, with 8 different difference values in each row, resulting in a total of 64 difference values. Finally, traverse each element of the 8×8 difference matrix row by row from left to right. If the difference value is greater than 0, then the Hash value is incremented by "1", otherwise it is incremented by "0".

[0140] Furthermore, in a specific implementation manner provided by the present application, any two images are selected from the second set, and the difference value of the second attribute values of any two images is calculated;

[0141] When the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, the image with the highest first attribute value is retained in the second set to form the third set, specifically including:

[0142] Select any image from the second set as the first image;

[0143] Select the remaining images different from the first image from the second set as the second image;

[0144] Obtain the second attribute value of the first image and the second attribute value of the second image;

[0145] Calculate the Hamming distance between the second attribute value of the first image and the second attribute value of the second image as the difference value of the second attribute values of the two images;

[0146] When the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, the image with the highest first attribute value is retained in the second set to form the third set.

[0147] Through multiple experiments by the inventor, the second screening threshold is preferably 10. In other words, when the Hamming distance between two images is less than or equal to 10, it is determined that the two images are similar or repeated, then the first attribute value of the first image and the first attribute value of the second image are obtained. Compare the first attribute values of the two images and save the image with the highest first attribute value.

[0148] When the Hamming distance between two images is greater than 10, it is determined that the two images are not similar or duplicate; continue to select the remaining images in the second set that are different from the first image and the second image as the third image; obtain the second attribute value of the third image; calculate the Hamming distance between the second attribute value of the first image and the second attribute value of the third image as the difference value of the second attribute values of the two images. Then compare the difference value of the second attribute values of the two images with the second screening threshold.

[0149] S170: Select any image from the third set as the first image.

[0150] S180: Identify the key text information of the first image.

[0151] S190: Obtain the descriptive text information associated with the first image.

[0152] S200: Compare the key text information and the descriptive text information of the first image.

[0153] S210: When the key text information of the first image is the same as the descriptive text information of the first image, store the first image and the descriptive text information of the first image.

[0154] It should be emphasized again that the work order auditing method provided in this application is used to settle the billing statement or generate an audit report. In an actual application scenario, an operator will upload a work order image and work order text information on the client side. In the actual application scenario of the image recording the work order information described in this application, it usually appears as a photographed work order image or a screenshot of a work order image. Therefore, the content of the first image records the key text information.

[0155] At the same time, there is an associated relationship between the work order image and the work order text information uploaded by the operator on the client side. In other words, the descriptive text information associated with the first image appears as the work order text information associated with the work order image in the application scenario of this application.

[0156] Generally, the key text information usually appears as a customer signature, an identity document, a work order name, or a stage name in a specific application scenario. Of course, the key text information recorded in the first image will also include a customer signature, an identity document, a work order name, or a stage name. In a specific implementation manner provided in this application, it is preferably to use the PytorchOCR algorithm to identify the key text information of the first image.

[0157] On this basis, steps S170 - S210 are essentially to verify whether the key text information of the first image is the same as the description text information of the first image. If they are the same, it means that the work order image and the work order text information uploaded by the client are consistent, meeting the archiving conditions, and the first image and the description text information of the first image can be stored for settlement of the billing statement or generation of the audit report. If the key text information of the first image is different from the description text information of the first image, it means that the work order image and the work order text information uploaded by the client are inconsistent, not meeting the archiving conditions, and the client uploading the work order image and the work order text information needs to be prompted that the uploaded data is abnormal.

[0158] After receiving the prompt of data abnormality, the client can submit appeal information or non - appeal information. Among them, the appeal information is manifested as the work order image or the work order text information re - uploaded by the client. If the client submits non - appeal information, the first image and the description text information of the first image continue to be stored for settlement of the billing statement or generation of the audit report. If the client submits appeal information, a verification process is performed on the appeal information, that is, to verify whether the key text information of the work order image re - uploaded by the client is the same as the description text information of the first image; or to verify whether the key text information of the first image is the same as the work order text information re - uploaded by the client. If they are the same, the appeal information and the unchanged information are stored for settlement of the billing statement or generation of the audit report.

[0159] Furthermore, in a specific implementation manner provided by the present application, the work order audit method further includes:

[0160] Extract at least one of the histogram feature, texture feature, and color feature of the first image;

[0161] Input at least one of the histogram feature, texture feature, and color feature of the first image into the forgery verification model to verify whether the first image is forged;

[0162] When the first image is not forged, identify the key text information of the first image.

[0163] This is because in the actual application scenario provided by the present application, there is a situation where the work order image is forged, and the forgery of the work order image will lead to the risk of false claim in the settlement of the work order cost, thus causing economic losses. To solve the technical problem of economic losses caused by the forgery of the work order image, the work order audit method provided by the present application uses the forgery verification model to verify whether the first image is forged.

[0164] Specifically, considering that forged images are usually not photos taken at the first scene, forged images may be blurred, have moiré patterns, or contain human figures. Based on the above characteristics of forged images, the inventor uses a forgery verification model to extract at least one of the histogram features, texture features, and color features of the first image, and inputs it into the forgery verification model to verify whether the first image is a re-taken image, that is, a forged image.

[0165] Taking the extraction of the histogram features of the first image as an example:

[0166] Considering that the resolution of the shooting device or printing device at the first scene may be low; or in the case of re-acquisition at the second scene, the display medium may not be within the focus range of the camera, or the screen brightness of the display medium is high, resulting in the loss of high-frequency regions. All these situations will cause the forged image to be blurred.

[0167] The inventor uses such characteristics as distinguishing features to detect whether an image is a forged image by using a differential histogram.

[0168] Specifically, assuming that the first image is represented as I, its first-order differential image is defined as , where i represents the differential images in four directions: horizontal, vertical, diagonal, and anti-diagonal. The first-order differential image can be expressed as follows:

[0169] ,

[0170] Here, the convolution kernels in the four directions of fi are defined as follows:

[0171] ,

[0172] Similarly, the second-order differential image of the first image can be defined as follows:

[0173] ,

[0174] Since , there are a total of 14 different differential images.

[0175] Next, calculate the regularized histogram Hi as follows:

[0176] ,

[0177] Here, N is the total number of pixels of the differential image Ii, and # represents the cardinality of the set. For different differential images, the following features can be obtained:

[0178] ,

[0179] In this way, there are 14(1 + k) histogram features.

[0180] Of course, in the actual application scenarios provided by this application, in addition to the histogram features of the first image, it is also possible to verify whether the first image is forged based on texture features. The texture features described here are manifested as moiré patterns or human faces, and are usually easily observed in the images re-acquired at the second scene. Considering that the human face is a complex non-rigid 3D object, and the surface characteristics of real objects, such as faces or printed materials, are different from those of the human face, these two unique characteristics may lead to characteristic specular reflections and tones. Therefore, it is very difficult to completely eliminate the texture.

[0181] The following introduces the implementation process of extracting the texture features of the first image:

[0182] The center symmetric local binary pattern algorithm is used to extract the texture features of the first image;

[0183] Among them, the center symmetric local binary pattern algorithm is manifested as:

[0184] ,

[0185] In the formula, represents the pixel coordinates, and

[0186] ,

[0187] In the formula, is the gray level of the center symmetric pixel, is the threshold of the center symmetric local binary pattern algorithm descriptor.

[0188] It should be noted that this application can use the local binary pattern algorithm (LBP) to extract the texture features of the first image, or the center symmetric local binary pattern algorithm (CS-LBP) to extract the texture features of the first image.

[0189] Generally, the basic version of the local binary pattern algorithm (LBP) only considers the eight neighboring pixels of a single pixel, resulting in a relatively long histogram (256). The center symmetric local binary pattern algorithm (CS-LBP) is commonly used for region description. Compared with the local binary pattern algorithm (LBP), the center symmetric local binary pattern algorithm (CS-LBP) greatly reduces the feature dimension and improves the calculation efficiency.

[0190] The following introduces the derivation process. The scheme functions of LBP and CS-LBP are as follows:

[0191] ,

[0192] Among them and Corresponding to the gray levels of the centrosymmetric pixel pairs and the central pixel on the circle with radius R, T is the threshold of the CS-LBP descriptor. The binary patterns of LBP and CS-LBP are calculated as

[0193] ,

[0194] where x and y represent the coordinates of the pixels. The LBP and CS-LBP operators are defined with the central pixel of the window as the threshold within a 3×3 window. Obviously, LBP generates 256 (2^8) different binary patterns, while CS-LBP only generates 16 (2^4) different binary patterns for the eight neighboring pixels, greatly reducing the feature dimension and improving the computational efficiency. At the same time, this symmetric calculation method can, to a certain extent, cope with the interference of factors such as illumination and rotation.

[0195] Of course, in the actual application scenarios provided by this application, in addition to through the histogram features or texture features of the first image, it is also possible to verify whether the first image is forged based on color features. Since the contrast and saturation of the image re-acquired at the second scene will be significantly reduced compared with the original image. Therefore, the contrast and color moments of the image can be used as discriminative features.

[0196] To extract the color moments, the first-order moments (means), second-order moments (variances), and third-order moments (skewnesses) of each channel in the HSV color space and the RGB space are used as color features. To capture these color anomalies, the inventor forms a set of 19-dimensional color features, including 1-dimensional contrast and 18-dimensional color moments.

[0197] Furthermore, after feature extraction, at least one of the histogram features, texture features, and color features of the first image is input into the forgery verification model to verify whether the first image is forged. The forgery verification model in the actual application scenario is manifested as a support vector machine (SVM) classifier with an RBF kernel, which is trained using the LSSVM tool. The training process is as follows:

[0198] Obtain training images;

[0199] Label all the training images, with 0 for the images collected at the first scene and 1 for the images re-collected at the second scene;

[0200] Extract at least one of the histogram features, texture features, and color features of the training images;

[0201] Input the features and labels of the training images into the LSSVM tool to train the SVM classifier.

[0202] After training the posterior forgery model, it can estimate the first image based on at least one of the histogram features, texture features, and color features of the first image, and output a verification result. If the verification result is 1, it means the first image is a forged image. If the verification result is 0, it means the first image is a real image. When the first image is a real image, proceed to step S180 to identify the key text information of the first image.

[0203] In summary, the work order auditing method provided by this application filters the images containing work order information multiple times to filter out blurred images and duplicate images, reducing the invalid data to be audited for work orders, thereby reducing the processing volume of work order auditing and improving the processing efficiency. And by identifying forged images to filter out the incorrect data to be audited for work orders, it further reduces the processing volume of work order auditing and improves the processing efficiency.

[0204] Please refer to Figure 2 , to support the work order auditing method, this application also provides a work order auditing system 100, including:

[0205] An acquisition module 11, configured to acquire images containing work order information and form a first set;

[0206] A quality auditing module 12, configured to select any image from the first set, extract the first attribute value of the image; the first attribute value is used to characterize the local texture feature of the image; and is also used to filter out images in the first set whose first attribute value is greater than or equal to the first filtering threshold to form a second set;

[0207] A similarity auditing module 13, configured to select any image from the second set, extract the second attribute value of the image; the second attribute value is used to characterize the pixel feature of the image; and is also used to select any two images from the second set and calculate the difference value of the second attribute values of any two images; and is also used to retain the image with the highest first attribute value in the second set when the difference value of the second attribute values of any two images in the second set is less than or equal to the second filtering threshold to form a third set;

[0208] A content auditing module 14, configured to select any image from the third set as the first image; and is also used to identify the key text information of the first image; and is also used to obtain the descriptive text information associated with the first image; and is also used to compare the key text information and the descriptive text information of the first image;

[0209] An archiving module 15, configured to store the first image and the descriptive text information of the first image when the key text information of the first image is the same as the descriptive text information of the first image.

[0210] Specifically, the acquisition module 11 acquires images containing work order information and forms a first set.

[0211] It should be noted that in the actual application scenario of this application, the operator will upload work order images and work order text information on the client side. In the actual application scenario, the image containing the work order information obtained by the acquisition module 11 usually appears as a photographed work order image or a screen-captured work order image. The acquisition module 11 forms a first set from a number of work order images from the client side.

[0212] After that, the quality audit module 12 selects an arbitrary image from the first set and extracts the first attribute value of the image; filters out the images in the first set whose first attribute value is greater than or equal to the first filtering threshold to form a second set.

[0213] It can be understood that the first attribute value is used to characterize the local texture features of the image. When the quality audit module 12 filters out the images in the first set whose first attribute value is greater than or equal to the first filtering threshold, it is actually screening the image quality to filter out images with poor quality, such as blurred images and images with unclear edges. In other words, the quality audit module 12 filtering out the images with poor quality in the first set is filtering out the invalid data in the data to be processed.

[0214] After the quality audit module 12 filters out the images with poor quality from the first set, it forms a second set from the remaining several images. It can be seen that the data to be processed in the second set is less than that in the first set, which is equivalent to reducing the processing volume of work order auditing and improving the processing efficiency.

[0215] The following introduces the specific implementation process of the quality audit module 12:

[0216] Considering that the first set can be composed of multiple images, for the sake of concise description, here an arbitrary image is selected from the first set as the first image; the description of extracting the first attribute value of the first image is used, and it should not be construed as a limitation on the scope of the invention patent.

[0217] Furthermore, in a specific implementation manner provided by this application, when the quality audit module 12 selects an arbitrary image from the first set and extracts the first attribute value of the image, it specifically includes:

[0218] Select an arbitrary image from the first set as the first image;

[0219] Extract the local texture features of the first image;

[0220] Cluster the local texture features of the image to obtain a centroid set;

[0221] Normalize the centroid set to obtain a normalized matrix;

[0222] Use the normalized matrix as a codebook for assignment coding to obtain a coefficient matrix;

[0223] Pool the coefficient matrix and convert it into a local texture feature vector;

[0224] Use the local texture feature vector as the first attribute value of the first image.

[0225] Specifically, assume that the first image is represented as I, and extract local descriptors from a set of image patches , where .

[0226] Then the quality audit module 12 extracts the local texture features of the first image to obtain X as follows:

[0227] Uniformly and randomly sample N different image patches from I;

[0228] Normalize each patch by subtracting the mean and dividing by the standard deviation of its elements;

[0229] Perform zero-component analysis (ZCA) whitening on the normalized patches.

[0230] After that, the quality audit module 12 constructs a visual codebook. The construction process is to perform K-means clustering on the local texture features X extracted from unlabeled training images. The specific construction process is as follows:

[0231] Calculate the local descriptor X for each training image;

[0232] Perform K-means clustering on the local descriptors to obtain K centroids ;

[0233] Normalize to obtain the normalized matrix , which is used as the codebook.

[0234] Next, the quality audit module 12 needs to perform soft assignment coding using the normalized codebook . Specifically, the quality audit module 12 first calculates the distance between the local descriptor and the visual codewords using the dot product. Let be the similarity measure between the i-th local descriptor and the j-th basis , then . The code element of the local descriptor can be written as follows:

[0235] ,

[0236] Then the coding of the local descriptor can provide a coefficient matrix .

[0237] The final quality auditing module 12 pools the coefficient matrix and converts it into a local texture feature vector. Considering that in image classification problems, the max pooling method exhibits higher performance than average pooling or sum pooling. This application preferably uses the max pooling method to give a local texture feature vector of a fixed length.

[0238] The image features defined by the quality auditing module 12 using max pooling are:

[0239] ,

[0240] where Ψmax is defined on each row of C, , The i-th element is given by:

[0241] ,

[0242] Then it is the local texture feature vector of the image.

[0243] Of course, this application also provides an image quality auditing model for screening images with the first attribute value greater than or equal to the first screening threshold from the first set. The image quality auditing model is manifested as a support vector machine (SVM) classifier model using a linear kernel. The training process of the image quality auditing model is as follows:

[0244] Select two types of images for training, one is a qualified image with a label of 0; the other is an unqualified image with a label of 1;

[0245] For any image, calculate the local descriptor, where B = 7 and N = 10000;

[0246] Perform K-means clustering on the local descriptors of any image, K = 10000, and calculate the codebook;

[0247] Perform soft coding on the local descriptors of any image to obtain the coefficient matrix ;

[0248] Use the max pooling method to obtain the local texture feature vector of the image ;

[0249] Input the local texture feature vector of each image and the label into a support vector machine (SVM) classifier with a linear kernel to train the image quality auditing model.

[0250] After the quality audit module 12 forms the second set, the similarity audit module 13 selects any image from the second set, extracts the second attribute value of the image; selects any two images from the second set, and calculates the difference value of the second attribute values of any two images; when the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, retain the image with the highest first attribute value in the second set to form the third set.

[0251] It should be noted that the second attribute value is used to characterize the image pixel features. The similarity audit module 13 retains the image with the highest first attribute value among any two images in the second set whose difference value of the second attribute values is less than or equal to the second screening threshold, which is essentially to filter out similar or duplicate images and retain clearer images. In other words, the similarity audit module 13 filtering out similar or duplicate images in the second set is to filter out duplicate data in the data to be processed.

[0252] After the similarity audit module 13 filters out similar or duplicate images from the second set, the remaining several images form the third set. It can be seen that the data to be processed in the third set is further less than that in the first set or the second set, which further reduces the processing volume of the work order audit and thus improves the processing efficiency.

[0253] The following introduces the specific implementation process of the similarity audit module 13:

[0254] Considering that the second set can be composed of multiple images, for the sake of concise description, here we take selecting any image from the second set as the first image; extracting the second attribute value of the first image for description, and it should not be understood as a limitation on the scope of the invention patent.

[0255] Furthermore, in a specific implementation manner provided by the present application, the similarity audit module 13 selects any image from the second set and extracts the second attribute value of the image, specifically including:

[0256] Select any image from the second set as the first image;

[0257] Scale the size of the first image to a preset size to generate a first preprocessed image;

[0258] Convert the first preprocessed image into a grayscale image to generate a second preprocessed image;

[0259] Subtract the pixel value of the current row of the second preprocessed image from the pixel value of the previous row to obtain the difference value between adjacent pixels;

[0260] Traverse the second preprocessed image in sequence with a preset difference matrix to construct the Hash value of the first image;

[0261] Use the Hash value of the first image as the second attribute value of the first image.

[0262] Among them, through multiple experiments by the inventor, the preset size is preferably 9 * 8 pixel points, so the first preprocessed image has 72 pixel points. The second preprocessed image is a 256 - level grayscale image. For an RGB three - channel image converted to a single - channel 256 - level grayscale image, the following algorithm can be used:

[0263] Gray =(R*76 + G*151 + B*28)>>8.

[0264] In this way, there will also be 8 rows of difference values, with 8 different difference values in each row, resulting in a total of 64 difference values. Finally, traverse each element of the 8×8 difference matrix row by row from left to right. If the difference value is greater than 0, then the Hash value is incremented by "1", otherwise by "0".

[0265] Furthermore, in a specific embodiment provided by the present application, the similarity auditing module 13 selects any two images from the second set, and calculates the difference value of the second attribute values of any two images; when the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, retain the image with the highest first attribute value in the second set to form the third set, specifically including:

[0266] Select any image from the second set as the first image;

[0267] Select the remaining images different from the first image in the second set as the second image;

[0268] Obtain the second attribute value of the first image and the second attribute value of the second image;

[0269] Calculate the Hamming distance between the second attribute value of the first image and the second attribute value of the second image as the difference value of the second attribute values of the two images;

[0270] When the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, retain the image with the highest first attribute value in the second set to form the third set.

[0271] Through multiple experiments by the inventor, the second screening threshold is preferably 10. In other words, when the Hamming distance between two images is less than or equal to 10, the similarity auditing module 13 determines that the two images are similar or repeated, then obtains the first attribute value of the first image and the first attribute value of the second image. The similarity auditing module 13 compares the first attribute values of the two images and saves the image with the highest first attribute value.

[0272] When the Hamming distance between two images is greater than 10, the similarity verification module 13 determines that the two images are not similar or duplicate; continue to select the remaining images different from the first image and the second image from the second set as the third image; obtain the second attribute value of the third image; calculate the Hamming distance between the second attribute value of the first image and the second attribute value of the third image as the difference value of the second attribute values of the two images. Then, the similarity verification module 13 compares the difference value of the second attribute values of the two images with the second screening threshold.

[0273] After the similarity verification module 13 forms the third set, the content verification module 14 selects any image from the third set as the first image; identifies the key text information of the first image; obtains the description text information associated with the first image; compares the key text information and the description text information of the first image. When the key text information of the first image is the same as the description text information of the first image, the archiving module 15 stores the first image and the description text information of the first image.

[0274] It should be emphasized again that the work order verification system 100 provided in this application is used to settle the billing statement or generate a verification report. In an actual application scenario, an operator uploads a work order image and work order text information on the client side. In the actual application scenario of the image recording the work order information described in this application, it usually appears as a photographed work order image or a screenshot of a work order image. Therefore, the content of the first image records the key text information.

[0275] At the same time, there is an association relationship between the work order image and the work order text information uploaded by the operator on the client side. In other words, the description text information associated with the first image appears as the work order text information associated with the work order image in the application scenario of this application.

[0276] Generally, the key text information specifically appears as a customer signature, an identity document, a work order name, or a stage name in a specific application scenario. Of course, the key text information recorded in the first image also includes a customer signature, an identity document, a work order name, or a stage name. In a specific embodiment provided in this application, the content verification module 14 preferably uses the PytorchOCR algorithm to identify the key text information of the first image.

[0277] On this basis, the content auditing module 14 essentially verifies whether the key text information of the first image is the same as the description text information of the first image. If they are the same, it means that the work order image uploaded by the client is consistent with the work order text information, meeting the filing conditions. The filing module 15 can store the first image and the description text information of the first image for settlement of the billing statement or generation of an auditing report. If the key text information of the first image is different from the description text information of the first image, it means that the work order image uploaded by the client is inconsistent with the work order text information, not meeting the filing conditions, and the filing module 15 needs to prompt the client who uploaded the work order image and work order text information that the uploaded data is abnormal.

[0278] After receiving the prompt of data abnormality, the client can submit appeal information or non - appeal information. Among them, the appeal information is manifested as the work order image or work order text information re - uploaded by the client. If the client submits non - appeal information, the filing module 15 continues to store the first image and the description text information of the first image for settlement of the billing statement or generation of an auditing report. If the client submits appeal information, the content auditing module 14 performs a verification process on the appeal information, that is, verifies whether the key text information of the work order image re - uploaded by the client is the same as the description text information of the first image; or verifies whether the key text information of the first image is the same as the work order text information re - uploaded by the client. If they are the same, the filing module 15 stores the appeal information and the unchanged information for settlement of the billing statement or generation of an auditing report.

[0279] Further, in a specific embodiment provided by the present application, the work order auditing system 100 further includes a forgery verification module 16, which is specifically used for:

[0280] extracting at least one of the histogram feature, texture feature, and color feature of the first image;

[0281] inputting at least one of the histogram feature, texture feature, and color feature of the first image into a forgery verification model to verify whether the first image is forged;

[0282] When the first image is not forged, identifying the key text information of the first image.

[0283] This is considering that in the actual application scenario provided by the present application, there is a situation where the work order image is forged, and the forgery of the work order image will lead to the risk of false claim in the settlement of the work order cost, thus causing economic losses. To solve the technical problem of economic losses caused by the forgery of the work order image, the forgery verification module 16 verifies whether the first image is forged through a forgery verification model.

[0284] Specifically, considering that forged images are usually not photos taken at the first scene, forged images may be blurred, have moiré patterns, or contain human figures. Based on the above characteristics of the forged image, the forgery verification module 16 extracts at least one of the histogram features, texture features, and color features of the first image using a forgery verification model, and inputs it into the forgery verification model to verify whether the first image is a re-taken image, that is, a forged image.

[0285] Taking the extraction of the histogram feature of the first image by the forgery verification module 16 as an example:

[0286] Considering that the resolution of the shooting device or printing device at the first scene may be low; or in the case of re-acquisition at the second scene, the display medium may not be within the focus range of the camera, or the screen brightness of the display medium is high, resulting in the loss of high-frequency regions. These situations will cause the forged image to be blurred.

[0287] The forgery verification module 16 utilizes such characteristics as distinguishing features to detect whether the image is a forged image by using the differential histogram.

[0288] Specifically, assuming that the first image is represented as I, its first-order differential image is defined as, where i represents the differential images in four directions: horizontal, vertical, diagonal, and anti-diagonal. The first-order differential image can be expressed as follows:

[0289] ,

[0290] Here, the convolution kernels in the four directions of fi are defined as follows:

[0291] ,

[0292] Similarly, the second-order differential image of the first image can be defined as follows:

[0293] ,

[0294] Since , there are a total of 14 different differential images.

[0295] Next, calculate the regularized histogram Hi as follows:

[0296] ,

[0297] Here, N is the total number of pixels of the differential image Ii, and # represents the cardinality of the set. For different differential images, the following features can be obtained:

[0298] ,

[0299] In this way, there are 14(1 + k) histogram features.

[0300] Of course, in the actual application scenarios provided by this application, in addition to the histogram features of the first image, the forgery verification module 16 can also verify whether the first image is forged based on texture features. The texture features described herein are manifested as moiré patterns or human faces, and are usually easily observed in the images re-captured at the second scene. Considering that the human face is a complex non-rigid 3D object, and the surface characteristics of real objects, such as the face or printed matter, are different from those of the human face, these two unique characteristics may lead to characteristic specular reflections and tones. Therefore, it is very difficult to completely eliminate the texture.

[0301] The following introduces the implementation process of the forgery verification module 16 for extracting the texture features of the first image:

[0302] The texture features of the first image are extracted using the center symmetric local binary pattern algorithm;

[0303] Among them, the center symmetric local binary pattern algorithm is manifested as:

[0304] ,

[0305] In the formula, represents the pixel coordinates, and

[0306] ,

[0307] In the formula, is the gray level of the center symmetric pixel, is the threshold of the center symmetric local binary pattern algorithm descriptor.

[0308] It should be noted that the forgery verification module 16 can extract the texture features of the first image using the local binary pattern algorithm (LBP), or can also extract the texture features of the first image using the center symmetric local binary pattern algorithm (CS-LBP).

[0309] Generally, the basic version of the local binary pattern algorithm (LBP) only considers the eight neighboring pixels of a single pixel, generating a relatively long histogram (256). The center symmetric local binary pattern algorithm (CS-LBP) is commonly used for region description. Compared with the local binary pattern algorithm (LBP), the center symmetric local binary pattern algorithm (CS-LBP) greatly reduces the feature dimension and improves the calculation efficiency.

[0310] The following introduces the derivation process. The scheme functions of LBP and CS-LBP are as follows:

[0311] ,

[0312] Among them and Corresponding to the gray levels of the centrosymmetric pixel pairs and the central pixel on the circle with a radius of R, T is the threshold of the CS-LBP descriptor. The binary patterns of LBP and CS-LBP are calculated as

[0313] ,

[0314] where x and y represent the coordinates of the pixel. The LBP and CS-LBP operators are defined with the central pixel of the window as the threshold within a 3*3 window. Obviously, LBP generates 256 (2^8) different binary patterns, while CS-LBP only generates 16 (2^4) different binary patterns for the eight neighboring pixels, greatly reducing the feature dimension and improving the computational efficiency. At the same time, this symmetric calculation method can, to a certain extent, cope with the interference of factors such as illumination and rotation.

[0315] Of course, in the actual application scenario provided by this application, in addition to the histogram features or texture features of the first image, the verification module 16 can also verify whether the first image is forged based on the color features. Since the contrast and saturation of the image re-acquired at the second scene will be significantly reduced compared to the original image. Therefore, the verification module 16 can use the contrast and color moments of the image as discriminative features.

[0316] To extract the color moments, the verification module 16 takes the first-order moment (mean), second-order moment (variance), and third-order moment (skewness) of each channel in the HSV color space and the RGB space as color features. To capture these color anomalies, the verification module 16 forms a set of 19-dimensional color features, including 1-dimensional contrast and 18-dimensional color moments.

[0317] Furthermore, after feature extraction, the verification module 16 inputs at least one of the histogram features, texture features, and color features of the first image into the verification model to verify whether the first image is forged. Here, the verification model is a support vector machine (SVM) classifier with an RBF kernel in the actual application scenario, which is trained using the LSSVM tool. The training process is as follows:

[0318] Obtain training images;

[0319] Label all the training images, with 0 for the images collected at the first scene and 1 for the images re-collected at the second scene;

[0320] Extract at least one of the histogram features, texture features, and color features of the training images;

[0321] Input the features and labels of the training images into the LSSVM tool to train the SVM classifier.

[0322] After training, the posterior forgery model can estimate the first image based on at least one of the histogram features, texture features, and color features of the first image, and output a verification result. If the verification result is 1, it indicates that the first image is a forged image. If the verification result is 0, it indicates that the first image is a genuine image. When the first image is a genuine image, the content auditing module 14 identifies the key text information of the first image.

[0323] In summary, the work order auditing system 100 provided by the present application filters the images recording work order information multiple times to filter out blurred images and duplicate images, reducing the invalid data to be audited for work orders, thereby reducing the processing volume of work order auditing and improving the processing efficiency. And by identifying forged images to filter out the incorrect data to be audited for work orders, the processing volume of work order auditing is further reduced and the processing efficiency is improved.

[0324] It should be noted that the term "including", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, commodity or device including the said element.

[0325] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A work order auditing method, characterized in that, It includes the following steps: Obtain images recording work order information to form a first set; Select any image from the first set and extract the first attribute value of the image; the first attribute value is used to characterize the local texture feature of the image; Filter images from the first set whose first attribute value is greater than or equal to the first screening threshold to form a second set; Select any image from the second set and extract the second attribute value of the image; the second attribute value is used to characterize the pixel feature of the image; Select any two images from the second set and calculate the difference value of the second attribute values of the two images; When the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold, retain the image with the highest first attribute value in the second set to form a third set; Select any image from the third set as the first image; Identify the key text information of the first image; Obtain the description text information associated with the first image; Compare the key text information and the description text information of the first image; When the key text information of the first image is the same as the description text information of the first image, store the first image and the description text information of the first image; select any image from the first set and extract the first attribute value of the image, specifically including: Select any image from the first set as the first image; Extract the local texture feature of the first image; Cluster the local texture features of the image to obtain a centroid set; Normalize the centroid set to obtain a normalized matrix; Use the normalized matrix as a codebook for assignment coding to obtain a coefficient matrix; Pool the coefficient matrix and convert it into a local texture feature vector; Use the local texture feature vector as the first attribute value of the first image; the method further includes: Extract at least one of the histogram feature, texture feature, and color feature of the first image; Input at least one of the histogram feature, texture feature, and color feature of the first image into a forgery verification model to verify whether the first image is forged; When the first image is not forged, identify the key text information of the first image.

2. The work order auditing method according to claim 1, wherein, The extraction of the texture feature of the first image specifically includes: Adopt the central symmetric local binary pattern algorithm to extract the texture feature of the first image.

3. The work order auditing method according to claim 1, characterized in that The work order audit method is used for settling the billing statement; The key text information includes at least the work order name or the stage name.

4. A work order auditing system, characterized in that, It includes: An acquisition module for obtaining images recording work order information to form a first set; A quality audit module for selecting any image from the first set and extracting the first attribute value of the image; the first attribute value is used to characterize the local texture feature of the image; It is also used to filter images from the first set whose first attribute value is greater than or equal to the first screening threshold to form a second set; A similarity audit module for selecting any image from the second set and extracting the second attribute value of the image; the second attribute value is used to characterize the pixel feature of the image; it is also used to select any two images from the second set and calculate the difference value of the second attribute values of the two images; it is also used to retain the image with the highest first attribute value in the second set when the difference value of the second attribute values of any two images in the second set is less than or equal to the second screening threshold to form a third set; The content auditing module is used to select any image from the third set as the first image; it is also used to identify the key text information of the first image; it is also used to obtain the descriptive text information associated with the first image; it is also used to compare the key text information and the descriptive text information of the first image; The archiving module is used to store the first image and the descriptive text information of the first image when the key text information of the first image is the same as the descriptive text information of the first image; The quality auditing module is used to extract the first attribute value of the image in the first set, specifically for: Select any image from the first set as the first image; Extract the local texture features of the first image; Cluster the local texture features of the image to obtain a centroid set; Normalize the centroid set to obtain a normalized matrix; Use the normalized matrix as a codebook for assignment encoding to obtain a coefficient matrix; Pool the coefficient matrix and convert it into a local texture feature vector; Use the local texture feature vector as the first attribute value of the first image; The system also includes a forgery verification module, specifically for: Extract at least one of the histogram features, texture features, and color features of the first image; Input at least one of the histogram features, texture features, and color features of the first image into the forgery verification model to verify whether the first image is forged; When the first image is not forged, identify the key text information of the first image.

5. The work order auditing system according to claim 4, wherein, The forgery verification module extracts the texture features of the first image, specifically including: Extract the texture features of the first work order image using the center symmetric local binary pattern algorithm.

6. The work order auditing system according to claim 5, wherein The work order auditing system is used to settle the billing list; The key text information includes at least the work order name or the stage name.

Images