Readable file readiness identification and analysis system and method based on machine learning

Abstract

The present invention discloses a readable file readiness recognition and analysis system and method based on machine learning, relates to the field of file recognition, and is used for optimizing an algorithm structure, collecting image content feature information, including historical unready image information and standard image text information; using fuzzy logic to determine the standard image evaluation time according to the average accuracy of historical unready image text and the standard image text tolerance rate, collecting historical data feature information and feature information corresponding to the standard image, performing comparison analysis, judging the rationality of the standard image, collecting new image feature information, obtaining image comprehensive information, obtaining the image comprehensive information and bringing it into the calculation formula of the SSIM index algorithm, combining parameters to calculate the SSIM index, and comparing it with the readiness threshold to determine the readiness of the new image, completing the rationality of the standard image and the time domain division of the inspection rationality, thereby improving the evaluation efficiency and accuracy of the readable file readiness.

Description

Technical Field

[0001] The present invention relates to the field of file recognition, and more specifically, to a system and method for identifying and analyzing readable file readiness based on machine learning. Background Art

[0002] Readable documents are usually expressed in the form of images. Image readiness recognition and analysis usually use image denoising, edge detection, image segmentation, etc. to prepare image data, and then use deep learning models to extract and classify the features of readable document images. In modern society, many organizations or institutions are undergoing digital transformation, converting paper documents and files into electronic formats. This transformation improves work efficiency, reduces costs, and makes information easier to access, retrieve, manage, and share.

[0003] The prior art has the following deficiencies:

[0004] At present, the image readiness recognition and analysis relies on a single standard image to evaluate the readiness of a new image. This may cause misjudgment of the recognition accuracy, affect the subsequent analysis of the system, cause errors, and waste subsequent data resources. In this regard, a readable file readiness recognition and analysis system and method based on machine learning are proposed.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a readable file readiness identification and analysis system and method based on machine learning, which solves the problems raised in the above-mentioned background technology by using different product inspection methods.

[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solution, a system and method for identifying and analyzing the readiness of readable files based on machine learning, comprising S1: collecting feature information of historical data and feature information corresponding to a standard image, performing comparison analysis, and judging the rationality of the standard image;

[0008] S2: Collect new image feature information, and obtain comprehensive image information by preprocessing the collected image, including but not limited to grayscale, noise removal, image enhancement, edge detection, etc.;

[0009] S3: Obtain comprehensive image information and bring it into the calculation formula of the SSIM indicator algorithm, calculate the SSIM indicator by combining the parameters, and compare it with the readiness threshold to determine the readiness of the new image;

[0010] S4: Collect image content feature information, including historical unready image information and standard image text information;

[0011] S5: Determine the standard image evaluation time using fuzzy logic according to the average accuracy of historical unready image texts and the standard image text tolerance rate;

[0012] S6: Perform fuzzy reasoning according to fuzzy rules to determine the standard image evaluation time.

[0013] In a preferred embodiment, the historical data feature information and the feature information corresponding to the standard image include image frequency domain values ​​and edge density;

[0014] Image frequency domain values ​​include historical image average frequency domain values And the standard image frequency domain value , edge density includes standard image edge density and the average edge density of historical images , j is the number of historical images, and j=1, 2, 3, ... N, N is the total number of historical images.

[0015] In a preferred embodiment, the standard image frequency domain value, the historical image average frequency domain value, the standard image edge density and the historical image average edge density are obtained, a data analysis model is established, and a reasonable evaluation coefficient is generated. , based on the formula:

[0016] ;

[0017] In the formula, is a reasonable assessment coefficient, and is the preset proportional coefficient of the ratio of the standard image frequency domain value to the historical image average frequency domain value, the standard image edge density to the historical image average edge density ratio, and and All are greater than 0;

[0018] The reasonable evaluation coefficient After comparison and analysis with the reasonable threshold, if the reasonable evaluation coefficient is greater than or equal to the reasonable threshold, the standard image is marked as a highly reasonable image and a reasonable signal is generated; if the reasonable evaluation coefficient is less than the reasonable threshold, the standard image is marked as an unreasonable image and a replacement signal is generated.

[0019] In a preferred embodiment, the comprehensive image information includes the average brightness information of the image, the contrast information of the image and the structural information of the image. The average brightness information includes the average brightness of the new image and the average brightness of the standard image. The contrast information of the image includes the contrast of the new image and the contrast of the standard image. The structural information of the image includes the average gradient of the new image and the average gradient of the standard image.

[0020] In a preferred embodiment, the existing SSIM indicator algorithm formula is as follows:

[0021] ;

[0022] in, is the new image readiness indicator, and are the average brightness values ​​of the new image and the standard image, respectively. and In particular, it is the contrast parameter between the new image and the standard image, that is, the standard deviation , is the structural parameter of the new image and the standard image, that is, the average gradient, and is a constant used to stabilize the calculation;

[0023] Will Compare and analyze with the ready threshold, if When the indicator value is greater than the ready threshold, the new image is marked as a ready image and a ready signal is generated; if When the indicator value is less than the ready threshold, the new image is marked as a not-ready image and an end signal is generated.

[0024] In a preferred embodiment, the image content feature information includes historical unready image information and standard image text information, and the historical unready image information includes the historical ready image text average accuracy. , standard image text information includes standard image text error tolerance .

[0025] In a preferred embodiment, the average accuracy of historical unready image texts and the error tolerance of standard image texts are defined as input variables, and they are divided into different fuzzy sets respectively;

[0026] The standard image evaluation time is defined as the output variable and divided into fuzzy sets;

[0027] Formulate fuzzy rules to describe the impact of the average accuracy of historical unready image text and the error tolerance of standard image text on the standard image evaluation time;

[0028] Fuzzy reasoning is performed based on fuzzy rules to determine the standard image evaluation time.

[0029] The machine learning-based readable file readiness recognition and analysis system includes a time domain evaluation module, a preprocessing module, a collection module, a readiness analysis module, and an output window;

[0030] The time domain evaluation module collects image content feature information, obtains the average accuracy of historical unready image text and the standard image text tolerance, uses fuzzy logic to determine the standard image evaluation time, and sends the evaluation results to the preprocessing module;

[0031] The preprocessing module receives the evaluation result, collects the feature information of the historical data and the feature information corresponding to the standard image, evaluates the rationality of the standard image, and sends the rationality information to the acquisition module;

[0032] The acquisition module receives the rationality information, collects the new image feature information, performs preprocessing to obtain the comprehensive image information, and brings it into the calculation formula of the SSIM index algorithm, combines the parameters to calculate the SSIM index and sends it to the readiness analysis module;

[0033] The readiness analysis module receives the SSIM index and compares it with the readiness threshold to determine the image readiness of the readable file, generates a readiness signal and sends it to the output window;

[0034] The output window is used to receive the ready signal and mark the readable file as displayed.

[0035] Technical effects and advantages of the present invention:

[0036] 1. The present invention is through The algorithm is optimized to judge the readiness of readable document images. First, the rationality of the standard image is judged to determine whether the standard image can be used as a comparison object with the new image. Then, the indicator operation is performed on the feature information of the new image and the standard image to obtain the readiness of the new image. Compared with the traditional algorithm, this further processes the selection of the comparison object, which improves the accuracy of the judgment of the readiness of readable document images and reduces the possibility of misjudgment.

[0037] 2. The present invention formulates a set of fuzzy rules for fuzzy reasoning based on the average accuracy of historical ready image texts and the fault tolerance rate of standard image texts to determine the standard image evaluation time. While ensuring that the readiness of new images is quickly evaluated, the standardization of standard images is improved, resources are saved, and standard images are more convincing, thereby enhancing the judgment of the accuracy of the readiness of new images. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The present invention is a flow chart of the method for identifying and analyzing the readiness of readable files based on machine learning.

[0039] Figure 2 This is a module diagram of the readable file readiness identification and analysis system based on machine learning of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] See also Figure 1 , a readable file readiness identification and analysis method based on machine learning, the specific operation process is as follows:

[0043] Readable documents are usually expressed as image files, so such documents need to be evaluated through images to get readiness;

[0044] The SSIM index algorithm, also known as the structural similarity index, is an index used to evaluate the similarity between two images. It not only considers the similarity of brightness, but also the similarity of contrast and structure. It evaluates the readiness of a new image by comparing it with a standard image.

[0045] Usually, for the selection of standard images, the SIMM algorithm assumes that the values ​​of the standard images are fixed by default. This has the advantage of being able to quickly process a large amount of image data, saving resources and playing a certain auxiliary role in the output time of subsequent data processing, which is consistent with the general actual situation. However, as the original data accumulates, the existence of a misjudgment rate will cause the accuracy of the standard image with a fixed value to be compared with the latest image, and the accuracy of judging the readiness will be greatly reduced, which is contrary to the original intention of the invention. The rapid judgment without considering the accuracy of the readiness will undoubtedly increase the risk of subsequent data processing. Therefore, there are still deficiencies in the selection of the standard algorithm. Now, the judgment on the selection of the standard algorithm is discussed;

[0046] S1: Collect feature information of historical data and feature information corresponding to the standard image, perform comparison analysis, and judge the rationality of the standard image;

[0047] The characteristic information of the historical data and the characteristic information corresponding to the standard image include the image frequency domain value and edge density;

[0048] The image frequency domain values ​​include the average frequency domain values ​​of historical images and the frequency domain values ​​of standard images. The acquisition logic is that images are usually represented and readable as two-dimensional images. For two-dimensional images, the image is grayed and zero-filled first, so that the image is single-channel and the deviation in the result after Fourier transform is reduced. It converts the image in the spatial domain into a spectrum representation in the frequency domain. The following is the mathematical expression of the two-dimensional discrete Fourier transform: Given a two-dimensional image , whose two-dimensional discrete Fourier transform Defined as:

[0049] ;

[0050] in, and are the width and height of the image, respectively. is the pixel value of the image in the spatial domain, is the representation of the image in the frequency domain, and is the coordinate in the frequency domain, and its value range is and , is the rotation factor used to transform the image from the spatial domain to the frequency domain;

[0051] It should be noted that in Fourier transform, the rotation factor is used to represent the oscillation of the signal in the time domain in the frequency domain. The rotation factor is a complex number, whose amplitude represents the amplitude of the signal and whose phase represents the phase shift of the signal. The amplitude of the rotation factor is always 1, and the phase part is and Decide when When , the phase of the rotation factor is 0, indicating the DC component of the image. and When they are not all 0 at the same time, it indicates the high-frequency components in the image. The rotation factor is often used to transform the image in the frequency domain so as to represent the frequency components of the image in the frequency domain. The standard image frequency domain value is obtained through the two-dimensional discrete Fourier transform. And the frequency domain value of each historical image , where j is the number of historical images, and j = 1, 2, 3, ... N, N is the total number of historical images, calculate the average value of the frequency domain representation of all historical images, and obtain the average frequency domain numerical representation of the historical images ;

[0052] The edge density includes the edge density of the standard image and the average edge density of the historical image. The acquisition logic is that the Canny edge detection algorithm is usually used to detect the edge of the image and generate the edge image. First, the Gaussian filter is used to reduce the noise of the original image. The mathematical expression of the Gaussian filter is:

[0053] ;

[0054] in, is the output of the Gaussian filter, indicating that at position The filter response value at is the standard deviation of the Gaussian kernel, which is usually preset based on experience or prior knowledge and controls the width of the Gaussian function. and are the horizontal and vertical offsets from the center, respectively;

[0055] Then calculate the magnitude and direction of the gradient of the smoothed image, usually using the Sobel operator, and the horizontal gradient is expressed as , the vertical gradient is expressed as , amplitude and direction The calculation formula is as follows:

[0056] , ;

[0057] Then, at each pixel position in the image, according to the gradient direction, only the maximum point of the local gradient amplitude is retained to obtain the edge of a single pixel width. For each pixel point, the two adjacent pixels in the gradient direction are checked. If the gradient amplitude of the current pixel is not the maximum value of the two adjacent pixels, the amplitude of the pixel is set to 0; the calculated gradient amplitude is used for edge detection using a double threshold, and the gradient amplitude is divided into strong edge, weak edge and non-edge. If the gradient amplitude is greater than the high threshold (strong edge threshold), it is regarded as a strong edge; if the gradient amplitude is less than the low threshold (weak edge threshold), it is excluded; if the gradient amplitude is between the high and low thresholds, it is marked as a weak edge if it is connected to a strong edge, otherwise it is marked as a non-edge; finally, the weak edge pixels connected to the strong edge are also marked as edges to obtain a complete edge image;

[0058] For each edge image, the number of edge pixels is calculated, and the number of pixels with edge values ​​in the edge image is counted; the number of edge pixels in each edge image is divided by the total number of pixels in the image to obtain the edge density of the image;

[0059] Using the above algorithm, we can get the standard image edge density and the average edge density of historical images ;

[0060] A data analysis model is established by combining the feature information of historical data with the feature information corresponding to the standard image, and a reasonable evaluation coefficient of the standard image is obtained through a weighted average algorithm;

[0061] Get the standard image frequency domain value , the average frequency domain value of historical images , Standard image edge density And the average edge density of historical images , establish a data analysis model and generate reasonable evaluation coefficients , based on the formula:

[0062] ;

[0063] In the formula, is a reasonable assessment coefficient, and is the standard image frequency domain value Average frequency domain value of historical images Ratio, standard image edge density and the average edge density of historical images The preset scaling factor of the ratio, and and All are greater than 0;

[0064] Among them, the standard image frequency domain value Average frequency domain value of historical images Ratio, standard image edge density and the average edge density of historical images Ratios are all digital manifestations that directly express the rationality of standard images;

[0065] From the formula, we can see that when the image frequency domain value The larger the value is, the greater the average frequency domain value of the historical image is. , standard image edge density Greater than the average edge density of historical images Ratio, the higher the reasonable evaluation coefficient, the higher the reasonableness of the standard image, and the standard image can be used as a comparison object for the readiness of the new image;

[0066] The reasonable evaluation coefficient Compare and analyze with reasonable thresholds to evaluate the rationality of the standard image;

[0067] The logic of obtaining reasonable thresholds is obtained through this reasonable evaluation and measurement and continuous iteration of historical image data sets;

[0068] If the reasonable evaluation coefficient is greater than or equal to the reasonable threshold, the standard image is marked as a high reasonable image and a reasonable signal is generated;

[0069] If the reasonable evaluation coefficient is less than the reasonable threshold, the standard image is marked as an unreasonable image and a replacement signal is generated;

[0070] It should be noted that the rationality calculation of the standard image this time is not to evaluate the rationality of the standard image before each readiness calculation of the new image, but according to the specific implementation situation, in the subsequent processing, the system module finds that there are more unready images in a group of data, and sends a self-check signal to start evaluating the rationality of the standard image. If there is a replacement signal, the standard image value is readjusted to ensure the prominence of the standard image; in this regard, the method of triggering the evaluation and replacement of the rationality of the standard image is not limited and will not be repeated here;

[0071] S2: Collect new image feature information, and obtain comprehensive image information by preprocessing the collected image, including but not limited to grayscale, noise removal, image enhancement, edge detection, etc.;

[0072] The comprehensive image information includes the average brightness information of the image, the contrast information of the image, and the structural information of the image;

[0073] The average brightness information includes the average brightness of the new image and the average brightness of the standard image;

[0074] The logic for obtaining the average brightness of the new image and the average brightness of the standard image is to obtain them by calculating the average brightness of all pixels in the image. The brightness of the pixel is usually calculated as follows: brightness = 0.299 × R + 0.587 × G + 0.114 × B. This formula is a common weighted average method, where R, G, and B represent the values ​​of the red, green, and blue channels, respectively, and 0.299, 0.587, and 0.114 are empirical constants that reflect the sensitivity of the human eye to different color channels. Usually, the color pixel value ranges from 0 to 225, where 0 indicates the absence of the color component and 255 indicates the maximum brightness of the color. Therefore, the mathematical expression for the average brightness is: ,in, is the average brightness value of the image, is the total number of pixels in the image, It is The brightness value of each pixel;

[0075] The contrast information of the image includes the contrast of the new image and the contrast of the standard image;

[0076] The logic of obtaining the contrast of the new image and the standard image is to estimate the contrast of the image through the standard deviation of the image. The standard deviation measures the discreteness of the pixel values ​​in the image, that is, the difference between the pixel value and the image mean. Images with higher contrast usually have larger standard deviations. First calculate the average grayscale value of the image. For images that have been grayed, simply substitute the above formula ,in, is the average gray value of the image, is the total number of pixels in the image, It is The grayscale value of each pixel indicates the brightness of the pixel. 0 indicates black and 255 indicates white. The principle is the same as that of color images and will not be repeated here. For color images, grayscale processing is first performed and then the above formula is used for calculation. The difference between each pixel and the average grayscale value of the image is calculated and expressed as an absolute value. The difference values ​​of all pixels are then averaged to obtain the average difference value of the image. The square root of the average difference value is then taken to obtain the standard deviation of the image.

[0077] The standard deviation calculation formula is: ,in, is the total number of pixels in the image, No. The gray value of a pixel, is the average gray value of the image; the standard deviation of the image is used to estimate the contrast of the image. The larger the standard deviation, the higher the contrast of the image; the smaller the standard deviation, the lower the contrast of the image;

[0078] Among them, the structural information of the image includes the average gradient of the new image and the average gradient of the standard image;

[0079] The logic of obtaining the average gradient of the new image and the average gradient of the standard image is to grayscale the color image first. Usually, for the gradient amplitude calculation, the Sobel operator is used to perform convolution operation with the image;

[0080] The Sobel operator is a commonly used edge detection operator that can be used to calculate the gradient value of each pixel in an image, including the gradient in the horizontal and vertical directions. The convolution operation is to filter the Sobel operator with the image to calculate the gradient value of each pixel. At the same time, the Sobel operator is divided into two types: horizontal and vertical. The horizontal Sobel operator;

[0081] ;

[0082] Sobel operator in vertical direction;

[0083] ;

[0084] This matrix represents the weight of the Sobel operator in the horizontal direction. In the convolution operation, the matrix is ​​convolved with the local area in the image. Specifically, the central element of this matrix is ​​0, which means that the weight of the current pixel is 0, that is, the current pixel does not participate in the calculation. The weights of other elements represent the relative importance of the pixels adjacent to the current pixel in the horizontal direction.

[0085] For example, the elements above and below the central element are -2 and 2, indicating that the adjacent pixels in the horizontal direction have a greater impact on the current pixel, while the elements in the diagonal direction are -1 and 1, indicating that the relatively far pixels have a smaller impact on the current pixel. The specific implementation depends on the specific analysis of the image pixels and will not be repeated here.

[0086] Perform convolution operation on the image with Sobel operator. For each pixel, perform dot multiplication between the Sobel operator and the corresponding area of ​​the image, and add the results to get the gradient value of the pixel; repeat this process until the entire image is convolved, and use zero padding method to process the boundary pixels; for the gradient values ​​in the horizontal and vertical directions, calculate the absolute values ​​to calculate the amplitude of the corresponding gradient;

[0087] It should be noted that zero padding is a common processing method. When performing convolution operations, if the boundary pixels are not processed, the size of the output image may be smaller than the size of the input image, which will lose some image information. The zero padding method is to fill a circle of zero-valued pixels around the boundary of the image to keep the size of the output image consistent with that of the input image. For example, the size of the input image is , and the size of the Sobel operator is , then for the boundary pixels, zero padding can be performed as follows: fill the top of the image Rows of zero-valued pixels fill the bottom of the image Rows of zero-valued pixels are filled on the left side of the image Column of zero-valued pixels. Pad on the right side of the image Column zero value pixels;

[0088] After this processing, the size of the input image will be changed from Expand to , so that the output image size after the convolution operation is consistent with the input image size. This ensures that when the convolution operation involves boundary pixels, the gradient value can still be calculated correctly without losing image information. The final result is the gradient image of the image, which can display the gradient strength and direction information of each pixel in the image;

[0089] Finally, the gradient magnitudes of all pixels in the gradient magnitude image are summed and then divided by the total number of pixels to obtain the average gradient information of the image;

[0090] This average gradient information reflects the intensity or density of the structural information in the image, and its mathematical expression is: ;in, is the total number of pixels in the image, It is The gradient magnitude of pixels;

[0091] The above embodiments are all summaries of the prior art, and the related calculations of the prior art will not be described in detail.

[0092] S3: Obtain comprehensive image information and bring it into the calculation formula of the SSIM indicator algorithm, calculate the SSIM indicator by combining the parameters, and compare it with the readiness threshold to determine the readiness of the new image;

[0093] The existing SSIM indicator algorithm formula is as follows:

[0094] ;

[0095] in, is the new image readiness indicator, and are the average brightness values ​​of the new image and the standard image, respectively. and In particular, it is the contrast parameter between the new image and the standard image, that is, the standard deviation , is the structural parameter of the new image and the standard image, that is, the average gradient, and is a constant used to stabilize the calculation;

[0096] in, and This is to ensure that the denominator and numerator will not approach zero during the numerical calculation process, thereby avoiding numerical instability. Usually, and Selection and Image Dynamic Range Related, It is the grayscale range of the image pixels, usually the maximum pixel value minus the minimum pixel value, representing the brightness range of the image; therefore, in the SSIM algorithm, it can generally be set as:

[0097] , ;

[0098] in, and is a constant, usually taken as and The value of is between 0.01 and 0.03, which is a common empirical value. Through experiments and observations, these values ​​are considered to be able to adapt to most images and application scenarios, and achieve a good effect in balancing brightness, contrast and structure. However, it needs to be adjusted according to the specific implementation. and The value of is fine-tuned, which will not be described here;

[0099] When the average brightness value, contrast standard deviation, and average gradient of the new image are greater than the average brightness value, contrast standard deviation, and average gradient of the standard image, it means The larger the indicator value, the higher the readiness of the new image;

[0100] Will The indicator value is compared with the readiness threshold to determine whether the new image is ready;

[0101] The logic for obtaining the readiness threshold is similar to the above threshold and will not be repeated here;

[0102] like When the indicator value is greater than the ready threshold, the new image is marked as a ready image and a ready signal is generated;

[0103] like When the indicator value is less than the ready threshold, the new image is marked as a not-ready image and an end signal is generated;

[0104] The present invention is through The algorithm is optimized to judge the readiness of readable document images. First, the rationality of the standard image is judged to determine whether the standard image can be used as a comparison object with the new image. Then, the indicator operation is performed on the feature information of the new image and the standard image to obtain the readiness of the new image. Compared with the traditional algorithm, this further processes the selection of the comparison object, which improves the accuracy of the judgment of the readiness of readable document images and reduces the possibility of misjudgment.

[0105] Example 2

[0106] In the first embodiment of the present invention, an example is given to illustrate that after acquiring a new image, the rationality of the standard image is evaluated first, and then the characteristic information of the new image and the standard image is collected by the SSIM algorithm and measured and evaluated to obtain the readiness of the new image, and the method for improving the accuracy of evaluating the readiness of the new image by evaluating the rationality of the standard image is used. However, in the first embodiment, the time interval of the evaluation of the rationality of the standard image is not analyzed. Obviously, this approach can improve the accuracy of evaluating the readiness of the new image, but each time the new image is evaluated, the rationality of the standard image must be evaluated, which is not only time-consuming, but also causes data redundancy and wastes resources. In view of the above problems, the second embodiment of the present invention is further refined;

[0107] S4: Collect image content feature information, including historical unready image information and standard image text information;

[0108] The historical unready image information includes the average accuracy of the historical ready image texts, and the standard image text information includes the standard image text error tolerance rate;

[0109] The logic for obtaining the average accuracy of historical unavailable image texts is to perform text recognition through OCR (optical character recognition), collect feature points through computer vision technology, detect, describe and recognize them, recognize the text content, and compare the recognized text with the original text to obtain the accuracy of historical available image texts. ; z is the number of historically available images, z=1, 2, 3, ... M, M is the total number of historically available images;

[0110] Optical character recognition is a technology that converts text content in an image into an editable text format. Computer vision technology usually uses edge detection and other methods to recognize text, obtain text areas, extract characters or words in the text area, and recognize text content through feature-based methods;

[0111] The original text refers to the text data source that needs to be recognized, extracted or analyzed. It is usually considered to be the text that meets the requirements of readiness assessment. The more similar the recognized text is to the original text, the higher the misjudgment rate of the standard image is, and the more the standard image needs to be replaced;

[0112] It should be noted that the historical time range for selecting historical unready images can be one month, and the historical ready images within two months are selected based on the number of end signals. If the number of end signals is greater, it proves that the possibility of recent misjudgment is greater, and the selected historical time should be closer to the current time;

[0113] After obtaining the historical image-text accuracy, we can obtain the historical image-text average accuracy. ;

[0114] The logic for obtaining the standard image text error tolerance rate is to use optical character recognition to obtain the text area of ​​the image text, and use computer vision technology to identify the text content, use the known text content (original text) to compare with the text obtained by OCR recognition, obtain the text where the error is recognized, count the number of texts that are recognized incorrectly, and divide it by the total number of texts in this acquisition time (the sum of the number of historical unready images and the number of historical ready images), and obtain the standard image text error tolerance rate ;

[0115] S5: Determine the standard image evaluation time using fuzzy logic according to the average accuracy of historical unready image texts and the standard image text tolerance rate;

[0116] For example, "Accuracy", "Inaccurate", "Average" are for the average accuracy of historical non-ready image text, and "High", "Low", "Medium" are for the standard image text tolerance;

[0117] Formulate a set of fuzzy rules to describe the impact of different input variables on output variables. The definition of rules can be based on professional knowledge or obtained through data analysis and experiments. For example:

[0118] The historical unready image text average accuracy is marked as X, the standard image text tolerance is marked as U, and the standard image assessment time is marked as T_Assess;

[0119] Then we can define:

[0120] Rule 1: IF (X is Accuracy) AND (U is High) THEN (T_Assess is Now)

[0121] Rule 2: IF (X is Inaccurate) AND (U is Low) THEN (C_results is Wait) ...

[0122] S6: Perform fuzzy reasoning according to fuzzy rules to determine the standard image evaluation time;

[0123] It should be noted that the division of fuzzy sets can be adjusted according to actual conditions. For example, although this embodiment takes three fuzzy sets as examples, in fact, the average accuracy of historical unready image texts and the tolerance rate of standard image texts can be divided into more than three sets to facilitate better precise adjustment according to different similarities.

[0124] Furthermore, for the judgment of the average accuracy of historical unready image texts and the high, medium and low tolerance rates of standard image texts, thresholds can be set according to actual conditions. For example, when the tolerance rate of standard image texts exceeds 5%, it is marked as "High"; when the average accuracy of historical unready image texts is higher than 85%, it is marked as "Accuracy", and so on. I will not go into details here.

[0125] The present invention formulates a set of fuzzy rules for fuzzy reasoning based on the average accuracy of historical ready image texts and the error tolerance of standard image texts to determine the standard image evaluation time. While ensuring that the readiness of new images can be quickly evaluated, the standardization of standard images is improved, resources are saved, and standard images are more convincing, which enhances the judgment of the accuracy of the readiness of new images.

[0126] Example 3

[0127] See also Figure 2 ,A readable file readiness identification and analysis system based on machine learning, including a time domain evaluation module, a preprocessing module, an acquisition module, a readiness analysis module, and an output window;

[0128] The time domain evaluation module collects image content feature information, obtains the average accuracy of historical unready image text and the standard image text tolerance, uses fuzzy logic to determine the standard image evaluation time, and sends the evaluation results to the preprocessing module;

[0129] The preprocessing module receives the evaluation result, collects the feature information of the historical data and the feature information corresponding to the standard image, evaluates the rationality of the standard image, and sends the rationality information to the acquisition module;

[0130] The acquisition module receives the rationality information, collects the new image feature information, performs preprocessing to obtain the comprehensive image information, and brings it into the calculation formula of the SSIM index algorithm, combines the parameters to calculate the SSIM index and sends it to the readiness analysis module;

[0131] The readiness analysis module receives the SSIM index and compares it with the readiness threshold to determine the image readiness of the readable file, generates a readiness signal and sends it to the output window;

[0132] The output window is used to receive the ready signal and mark the readable file as displayed.

[0133] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0134] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0135] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0136] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0137] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0138] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0142] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for identifying and analyzing readable file readiness based on machine learning, characterized in that: S1: Collect feature information of historical data and feature information corresponding to the standard image, perform comparison analysis, and judge the rationality of the standard image; S2: Collect new image feature information, and obtain comprehensive image information by preprocessing the collected image, including grayscale conversion, noise removal, image enhancement, and edge detection; S3: Obtain comprehensive image information and bring it into the calculation formula of the SSIM indicator algorithm, calculate the SSIM indicator by combining the parameters, and compare it with the readiness threshold to determine the readiness of the new image; S4: Collect image content feature information, including historical unready image information and standard image text information; S5: Determine the standard image evaluation time using fuzzy logic according to the average accuracy of historical unready image texts and the standard image text tolerance rate; S6: Perform fuzzy reasoning according to fuzzy rules to determine the standard image evaluation time; The historical data feature information and the feature information corresponding to the standard image include the image frequency domain value and edge density; Image frequency domain values ​​include historical image average frequency domain values And the standard image frequency domain value , edge density includes standard image edge density and the average edge density of historical images , j is the number of historical images, and j=1, 2, 3, ... N, N is the total number of historical images; and is the coordinate in the frequency domain, and its value range is and , M is the total number of historical ready images; Obtain the standard image frequency domain value, the historical image average frequency domain value, the standard image edge density and the historical image average edge density, establish a data analysis model, and generate a reasonable evaluation coefficient , based on the formula: ； In the formula, is a reasonable assessment coefficient, and is the preset proportional coefficient of the ratio of the standard image frequency domain value to the historical image average frequency domain value, the standard image edge density to the historical image average edge density ratio, and and All are greater than 0; The reasonable evaluation coefficient After comparison and analysis with the reasonable threshold, if the reasonable evaluation coefficient is greater than or equal to the reasonable threshold, the standard image is marked as a highly reasonable image and a reasonable signal is generated; if the reasonable evaluation coefficient is less than the reasonable threshold, the standard image is marked as an unreasonable image and a replacement signal is generated.

2. The method for identifying and analyzing readable file readiness based on machine learning according to claim 1, characterized in that: The comprehensive image information includes the average brightness information of the image, the contrast information of the image and the structural information of the image. The average brightness information includes the average brightness of the new image and the average brightness of the standard image. The contrast information of the image includes the contrast of the new image and the contrast of the standard image. The structural information of the image includes the average gradient of the new image and the average gradient of the standard image.

3. The method for identifying and analyzing readable file readiness based on machine learning according to claim 2, characterized in that: The existing SSIM indicator algorithm formula is as follows: ； in, is the new image readiness indicator, and are the average brightness values ​​of the new image and the standard image, respectively. and In particular, it is the contrast parameter between the new image and the standard image, that is, the standard deviation , is the structural parameter of the new image and the standard image, that is, the average gradient, and is a constant used to stabilize the calculation; Will Compare and analyze with the ready threshold, if When the indicator value is greater than the ready threshold, the new image is marked as a ready image and a ready signal is generated; if When the indicator value is less than the ready threshold, the new image is marked as a not-ready image and an end signal is generated.

4. The method for identifying and analyzing readable file readiness based on machine learning according to claim 1, characterized in that: Image content feature information includes historical unavailable image information and standard image text information. Historical unavailable image information includes historical available image text average accuracy. , is the accuracy of historically available image text, z is the number of historically available images, z=1, 2, 3, ... M, and the standard image text information includes the standard image text error tolerance rate .

5. The method for identifying and analyzing readable file readiness based on machine learning according to claim 4, characterized in that: The average accuracy of historical unready image text and the standard image text error tolerance are defined as input variables, and they are divided into different fuzzy sets respectively; The standard image evaluation time is defined as the output variable and divided into fuzzy sets; Formulate fuzzy rules to describe the impact of the average accuracy of historical unready image text and the error tolerance of standard image text on the standard image evaluation time; Fuzzy reasoning is performed based on fuzzy rules to determine the standard image evaluation time.

6. A system for identifying and analyzing readable file readiness based on machine learning, used to implement the method for identifying and analyzing readable file readiness based on machine learning according to any one of claims 1 to 5, characterized in that: It includes time domain evaluation module, preprocessing module, acquisition module, readiness analysis module and output window; The time domain evaluation module collects image content feature information, obtains the average accuracy of historical unready image texts and the standard image text tolerance, uses fuzzy logic to determine the standard image evaluation time, and sends the evaluation results to the preprocessing module; The preprocessing module receives the evaluation result, collects the feature information of the historical data and the feature information corresponding to the standard image, evaluates the rationality of the standard image, and sends the rationality information to the acquisition module; The acquisition module receives the rationality information, collects the new image feature information, performs preprocessing to obtain the comprehensive image information, and brings it into the calculation formula of the SSIM index algorithm, combines the parameters to calculate the SSIM index and sends it to the readiness analysis module; The readiness analysis module receives the SSIM index and compares it with the readiness threshold to determine the image readiness of the readable file, generates a readiness signal and sends it to the output window; The output window is used to receive the ready signal and mark the readable file as displayed.

Images