Text image quality detection method, device, computer equipment and storage medium
By training the quality detection model in the image database and combining noise detection, the problem of high computational volume and low accuracy of traditional image quality evaluation methods is solved, and efficient and accurate image quality detection is achieved.
Patent Information
- Application Number
- CN202210057440.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-01-19
AI Technical Summary
Traditional image quality evaluation methods have large calculation volume and low evaluation accuracy, making it difficult to effectively measure image quality.
The positive sample image is obtained in the image database, preprocessed and constructed negative sample image, trained the target quality detection model, and used image chunking processing and noise detection model to perform quality and noise detection on the detected image, and comprehensively calculate the final result.
It improves the accuracy and time efficiency of image quality detection, and ensures high accuracy of detection accuracy.
Smart Images

Figure CN114399497B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology in artificial intelligence, and in particular to a quality detection method, device, computer equipment and storage medium applied to text images. Background Art
[0002] Images are an important source of information for human perception and machine pattern recognition, and their quality plays a decisive role in the adequacy and accuracy of the information obtained. However, images are inevitably distorted to a certain extent during the process of acquisition, compression, processing, transmission, and display. How to measure the quality of images and assess whether they meet certain application requirements has become an urgent problem to be solved. To this end, it is necessary to establish an effective image quality assessment system.
[0003] Existing image quality assessment methods can be divided into subjective assessment methods and objective assessment methods. The former relies on the subjective perception of the experimenter to evaluate the quality of the object; the latter measures image quality based on the quantitative indicators given by the model and simulates the perception mechanism of the human visual system.
[0004] However, the applicant has found that traditional image quality assessment methods generally have problems such as large amount of calculation and low assessment accuracy. Summary of the invention
[0005] The purpose of the embodiments of the present application is to propose a quality detection method, device, computer equipment and storage medium for text images, so as to solve the problems commonly existing in traditional image quality assessment methods, such as large amount of calculation and low assessment accuracy.
[0006] In order to solve the above technical problems, the present application embodiment provides a quality detection method for text images, which adopts the following technical solution:
[0007] Obtain positive sample images in the image database;
[0008] Preprocessing the positive sample image to obtain a preprocessed positive sample image;
[0009] Constructing a negative sample image according to the preprocessed positive sample image, and combining the preprocessed positive sample image and the negative sample image to obtain model training data;
[0010] Inputting the model training data into the original quality inspection model for classification training until convergence, to obtain a target quality inspection model;
[0011] Receiving a quality detection request carrying an image to be detected;
[0012] Performing image block processing on the image to be detected to obtain a sub-image to be detected;
[0013] Inputting the sub-image to be detected into the target quality detection model to perform a quality detection operation, and obtaining a quality detection sub-result corresponding to the sub-image to be detected;
[0014] Performing a quality summary calculation operation on the quality detection sub-results to obtain a quality detection result;
[0015] Calling a trained noise detection model, inputting the sub-image to be detected into the trained noise detection model to perform a noise detection operation, and obtaining a noise detection sub-result corresponding to the sub-image to be detected;
[0016] Performing a noise aggregation calculation operation on the noise detection sub-results to obtain a noise detection result;
[0017] A comprehensive calculation operation is performed on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected.
[0018] In order to solve the above technical problems, the embodiment of the present application further provides a quality detection device for text images, which adopts the following technical solution:
[0019] A positive sample acquisition module is used to acquire positive sample images from an image database;
[0020] A preprocessing module, used for preprocessing the positive sample image to obtain a preprocessed positive sample image;
[0021] A first training data acquisition module, used to construct a negative sample image according to the preprocessed positive sample image, and obtain model training data by combining the preprocessed positive sample image and the negative sample image;
[0022] A classification training module, used for inputting the model training data into the original quality inspection model for classification training until convergence, so as to obtain a target quality inspection model;
[0023] A request receiving module, used for receiving a quality detection request carrying an image to be detected;
[0024] A block processing module, used for performing image block processing on the image to be detected to obtain a sub-image to be detected;
[0025] A quality detection module, used for inputting the sub-image to be detected into the target quality detection model to perform a quality detection operation, and obtaining a quality detection sub-result corresponding to the sub-image to be detected;
[0026] A quality summary module, used for performing a quality summary calculation operation on the quality detection sub-results to obtain a quality detection result;
[0027] A noise detection module is used to call a trained noise detection model, input the sub-image to be detected into the trained noise detection model to perform a noise detection operation, and obtain a noise detection sub-result corresponding to the sub-image to be detected;
[0028] A noise aggregation module, used for performing a noise aggregation calculation operation on the noise detection sub-results to obtain a noise detection result;
[0029] The comprehensive calculation module is used to perform a comprehensive calculation operation on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected.
[0030] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:
[0031] The invention comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the quality detection method applied to text images as described above when executing the computer-readable instructions.
[0032] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:
[0033] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the quality detection method applied to text images as described above are implemented.
[0034] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0035] The present application provides a quality detection method for text images, comprising: obtaining a positive sample image from an image database; preprocessing the positive sample image to obtain a preprocessed positive sample image; constructing a negative sample image based on the preprocessed positive sample image, and combining the preprocessed positive sample image and the negative sample image to obtain model training data; inputting the model training data into an original quality detection model for classification training until convergence, to obtain a target quality detection model; receiving a quality detection request carrying an image to be detected; performing image block processing on the image to be detected to obtain a sub-image to be detected; inputting the sub-image to be detected into the The target quality detection model performs a quality detection operation to obtain a quality detection sub-result corresponding to the sub-image to be detected; a quality summary calculation operation is performed on the quality detection sub-result to obtain a quality detection result; a trained noise detection model is called, and the sub-image to be detected is input into the trained noise detection model to perform a noise detection operation to obtain a noise detection sub-result corresponding to the sub-image to be detected; a noise summary calculation operation is performed on the noise detection sub-result to obtain a noise detection result; a comprehensive calculation operation is performed on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected. The present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection results and the noise detection results. The image to be detected is detected from two directions, namely, "image quality" and "image noise", thereby greatly improving the accuracy of text image detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0037] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0038] Figure 2 is a flow chart of the implementation of the quality detection method applied to text images provided in the first embodiment of the present application;
[0039] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S201;
[0040] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S202;
[0041] Figure 5 yes Figure 2 A flowchart of a specific implementation of step S203;
[0042] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S204;
[0043] Figure 7 This is a flowchart of a specific implementation method of obtaining a trained noise detection model provided in Example 1 of the present application;
[0044] Figure 8 This is a flowchart of a specific implementation method of screening sub-images to be detected provided in Example 1 of the present application;
[0045] Fig. 9 is a structural diagram of a quality detection device for text images provided in Embodiment 2 of the present application;
[0046] Fig.10 This is a structural diagram of a specific implementation of the preprocessing module provided in Example 2 of the present application;
[0047] Fig.11 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION
[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0049] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0050] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0051] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0052] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0053] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.
[0054] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .
[0055] It should be noted that the quality detection method applied to text images provided in the embodiments of the present application is generally executed by a server / terminal device, and correspondingly, the quality detection device applied to text images is generally arranged in a server / terminal device.
[0056] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0057] Continue to refer Figure 2 , shows an implementation flow chart of the quality detection method applied to text images provided in Example 1 of the present application. For the sake of convenience of explanation, only the parts related to the present application are shown.
[0058] The above-mentioned quality detection method applied to text images includes the following steps: step S201, step S202, step S203, step S204, step S205, step S206, step S207, step S208, step S209, step S210 and step S211.
[0059] Step S201: Acquire a positive sample image from an image database.
[0060] In an embodiment of the present application, visual image data for visual object recognition is stored in an image database. Specifically, the image database may be an ImageNet dataset, wherein the ImageNet project is a large visual database for visual object recognition software research. More than 14 million image URLs are manually annotated by ImageNet to indicate objects in the pictures; in at least one million images, bounding boxes are also provided. ImageNet contains more than 20,000 categories. Annotation databases of third-party image URLs can be obtained directly from ImageNet for free. It should be understood that the examples of image databases here are only for ease of understanding and are not used to limit this application.
[0061] Step S202: preprocessing the positive sample image to obtain a preprocessed positive sample image.
[0062] In the embodiment of the present application, the preprocessing may be to resize the positive sample image in a manner that keeps the aspect ratio unchanged to obtain a positive sample image of a standard size; or to segment the positive sample image of the standard size in a size of 224×224 to obtain a preprocessed positive sample image.
[0063] Step S203: constructing negative sample images according to the preprocessed positive sample images, and combining the preprocessed positive sample images and the negative sample images to obtain model training data.
[0064] In an embodiment of the present application, constructing a negative sample image can be multiplying a preprocessed positive sample image with a noise vector to obtain a noise image; deforming the noise image to obtain a deformed image; and inputting the deformed image into a generative adversarial network for upsampling to obtain a negative sample image.
[0065] Step S204: input the model training data into the original quality inspection model for classification training until convergence, to obtain the target quality inspection model.
[0066] In an embodiment of the present application, the quality detection model refers to a deep neural network constructed based on a streamlined structure using deep separable convolution to form lightweight weights. Specifically, the quality detection model consists of a clarity detection model and a noise exposure detection model, wherein the clarity detection model is mainly used to detect the clarity of the image, and the noise exposure detection model is used to detect the noise quality, overexposure / underexposure and value of the image.
[0067] In an embodiment of the present application, the quality detection model solves the problems of computational efficiency and parameter quantity of convolutional networks by adopting depthwise separable convolutions. It can show stronger performance than other network models of traditional image network classification. The quality detection model is a lightweight deep neural network based on MobileNets, wherein the original quality detection model refers to the initial parameter value model in which the model parameters have not been trained and adjusted.
[0068] In the embodiment of the present application, after obtaining the model training data, the batch gradient descent method is used for training. Examples of the batch gradient descent method include batch gradient descent method BGD, mini-batch gradient descent method MBGD, and stochastic gradient descent method SGD, etc.
[0069] In the embodiment of the present application, the training data of the clarity detection model of the quality detection model can be divided into three categories, namely notext, poor quality, and good quality. In the training data, the evaluation of notext, poor quality, and good quality can be evaluated by the subjective feelings of the experimenter, by reference images, or by a clarity algorithm.
[0070] In the embodiments of the present application, examples of clarity algorithms include Brenner gradient function, Tenengrad gradient function, Laplacian gradient function, SMD (grayscale variance) function, SMD2 (grayscale variance product) function, etc. It should be understood that the examples of clarity algorithms here are only for the convenience of understanding and are not used to limit the present application.
[0071] Step S205: receiving a quality detection request carrying an image to be detected.
[0072] Step S206: performing image segmentation processing on the image to be detected to obtain sub-images to be detected.
[0073] In the embodiment of the present application, the image block processing can be divided into three parts according to the long side and two parts according to the short side. Figure 1It is divided into 6 small pictures in total.
[0074] Step S207: input the sub-image to be detected into the target quality detection model to perform a quality detection operation, and obtain a quality detection sub-result corresponding to the sub-image to be detected.
[0075] In an embodiment of the present application, after the sub-image to be detected is input into the above-mentioned trained target quality detection model, the sub-image to be detected will be labeled with an attribute label (the attribute labels include small image, large image, long image, solid color image, black and white image, pure text image, etc.), and at the same time, the sub-image to be detected is divided into five levels (the levels include unqualified quality, low quality, medium quality, high quality and ultra-high quality) through the target quality detection model, and the clarity detection model of the quality detection model scores the clarity of the sub-image to be detected through a clarity score [0, 1]. The reference model of the target quality detection model scores the voice value, overexposure value and underexposure value of the sub-image to be detected through a noise exposure score [0, 1]. Finally, the clarity score and the voice exposure score are combined to obtain the initial detection result of the sub-image to be detected.
[0076] Step S208: Perform a quality summary calculation operation on the quality detection sub-results to obtain a quality detection result.
[0077] In the embodiment of the present application, since step S207 obtains the detection score results of each sub-image to be detected after the image to be detected is divided into blocks, when performing quality summary calculation of the quality detection score of the image to be detected, the quality detection sub-results of all sub-images to be detected can be calculated by averaging to obtain the quality detection result of the image to be detected; when calculating the score of the image to be detected, it is also possible to assign a weight value corresponding to the clarity to each sub-image to be detected, and calculate the weight score of each sub-image to be detected based on the weight value of each sub-image to be detected, and finally calculate the quality detection result of the image to be detected by averaging. It should be understood that the example of quality summary calculation here is only for the convenience of understanding and is not used to limit the present application.
[0078] Step S209: calling the trained noise detection model, inputting the sub-image to be detected into the trained noise detection model to perform noise detection operation, and obtaining a noise detection sub-result corresponding to the sub-image to be detected.
[0079] In the embodiment of the present application, the above-mentioned trained noise detection model is pre-stored in the system database for arbitrary calling.
[0080] In an embodiment of the present application, the trained noise detection model can perform noise evaluation on the input data to obtain the noise value of the input data.
[0081] Step S210: performing a noise aggregation calculation operation on the noise detection sub-results to obtain a noise detection result.
[0082] In the embodiment of the present application, since step S209 obtains the detection score results of each sub-image to be detected after the image to be detected is divided into blocks, when performing noise aggregation to calculate the noise detection score of the image to be detected, the noise detection sub-results of all sub-images to be detected can be calculated by averaging to obtain the noise detection result of the image to be detected; when calculating the score of the image to be detected, it is also possible to assign a weight value corresponding to the exposure to each sub-image to be detected, and calculate the weight score of each sub-image to be detected based on the weight value of each sub-image to be detected, and finally calculate the noise detection result of the image to be detected by averaging. It should be understood that the example of noise aggregation calculation here is only for the convenience of understanding and is not used to limit the present application.
[0083] Step S211: Perform a comprehensive calculation operation on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected.
[0084] In an embodiment of the present application, since the quality detection results and noise detection results obtained in step S208 and step S210 are the detection results of the image to be detected from the two directions of "quality" and "noise", it is necessary to perform a comprehensive calculation to obtain the final detection result of the image to be detected. When performing the comprehensive calculation, it can be calculated by an averaging method, or it can be calculated based on the assigned weights of "quality" and "noise", based on the "quality" weight score of the quality detection result and the "noise" weight score of the noise detection result, and finally the above-mentioned target detection result is obtained by averaging calculation. It should be understood that the example of comprehensive calculation here is only for the convenience of understanding and is not used to limit the present application.
[0085] In an embodiment of the present application, a quality detection method applied to text images is provided, including: obtaining a positive sample image from an image database; preprocessing the positive sample image to obtain a preprocessed positive sample image; constructing a negative sample image based on the preprocessed positive sample image, and combining the preprocessed positive sample image and the negative sample image to obtain model training data; inputting the model training data into an original quality detection model for classification training until convergence to obtain a target quality detection model; receiving a quality detection request carrying an image to be detected; performing image block processing on the image to be detected to obtain a sub-image to be detected; inputting the sub-image to be detected into a target quality detection model for quality detection operation to obtain a quality detection sub-result corresponding to the sub-image to be detected; performing a quality summary calculation operation on the quality detection sub-result to obtain a quality detection result; calling a trained noise detection model, inputting the sub-image to be detected into the trained noise detection model for noise detection operation to obtain a noise detection sub-result corresponding to the sub-image to be detected; performing a noise summary calculation operation on the noise detection sub-result to obtain a noise detection result; performing a comprehensive calculation operation on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected. The present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection results and the noise detection results. The image to be detected is detected from two directions, namely, "image quality" and "image noise", thereby greatly improving the accuracy of text image detection.
[0086] Continue reading Figure 3 , showing Figure 2 A flowchart of a specific implementation of step S201 is shown in FIG. 1 . For ease of explanation, only the part related to the present application is shown.
[0087] In some optional implementations of this embodiment, the above step S201 specifically includes: step S301 and step S302.
[0088] Step S301: classify sample images in the image database into small image categories, long image categories, pure color image categories, pure text image categories, black and white image categories, and large image categories according to the statistical model.
[0089] In an embodiment of the present application, the statistical model is preconfigured to classify image data into a small image category, a long image category, a pure color image category, a pure text image category, a black and white image category, and a large image category, wherein the statistical model is used to classify the image data, including:
[0090] When max_wh<200, the image data is judged as a small image;
[0091] When min_wh>=2000, the image data is determined to be a large image;
[0092] When max_wh / min_wh>3, the image data is judged as a long image;
[0093] When the number of channels of image data is 1, the image data is judged as a black and white image;
[0094] When v_max_1 / v_all>0.65, the image data is judged as a pure color image;
[0095] When v_max_1 / v_all<0.65 and (v_max_1+v_max_2) / v_all>0.8, the image data is determined to be a pure text image.
[0096] Among them, w, h, min_wh, max_wh represent the width, height, minimum side and maximum side of the image data respectively, and v_max_1, v_max_2 and v_all represent the maximum bin value, the second largest bin value and the sum of all bins of the histogram of the image data respectively.
[0097] In the embodiment of the present application, let hist(image) represent the histogram of the image image, where the number of columns of the histogram is 20.
[0098] In the obtained histogram of the input image, v_max_1, v_max_2 and v_all represent the maximum bin value, the second largest bin value and the sum of all bins of the histogram, respectively.
[0099] Step S302: obtaining a preset amount of image data in the small image category, the long image category, the pure color image category, the pure text image category, the black and white image category, and the large image category, respectively, and using the obtained image data as positive sample images.
[0100] In an embodiment of the present application, by classifying sample images in an image database and extracting from the classified subclass samples, the obtained positive sample images can evenly cover the above-mentioned categories, so that when the quality inspection model is subsequently trained, the training diversification of the quality inspection model is effectively guaranteed, thereby ensuring that the model can learn more realistic feature data.
[0101] Continue reading Figure 4 , showing Figure 2 A flowchart of a specific implementation of step S202 is shown in FIG. 1 . For ease of explanation, only the part related to the present application is shown.
[0102] In some optional implementations of this embodiment, the above step S202 specifically includes: step S401 and step S402.
[0103] Step S401: resizing the positive sample image in a manner that the aspect ratio remains unchanged, to obtain a positive sample image of a standard size.
[0104] In the embodiment of the present application, the resizing process refers to the process of adjusting the size of the positive sample image, and the adjustment process always keeps the aspect ratio of the image unchanged. Specifically, the image enlargement can adopt the interpolation method, that is, inserting new elements between the pixels of the original image by using a suitable interpolation algorithm. The interpolation algorithm includes, for example, an edge-based image interpolation algorithm, an area-based image interpolation algorithm, and other known or future-developed algorithms. The image reduction can be interpolated by, for example, CV_INETR_AREA.
[0105] Step S402: segmenting the standard-size positive sample image according to a preset size to obtain a preprocessed positive sample image.
[0106] In an embodiment of the present application, since the input of the quality detection model needs to meet certain size requirements, when an image that is too large is input into the quality detection model, it will affect the processing efficiency and accuracy of the model. Therefore, the positive sample image is standardized through the above-mentioned size adjustment processing and segmentation processing, so that the input data of the quality detection model meets the requirements of the model, wherein the preset size can be dynamically adjusted according to actual conditions. As an example, the preset size can be 224×224. It should be understood that the example of the preset size here is only for the convenience of understanding and is not used to limit the present application.
[0107] Continue reading Figure 5 , showing Figure 2 A flowchart of a specific implementation of step S203 is shown in FIG. 1 . For ease of explanation, only the part related to the present application is shown.
[0108] In some optional implementations of this embodiment, the above step S203 specifically includes: step S501, step S502 and step S503.
[0109] Step S501: multiply the preprocessed positive sample image by the noise vector to obtain a noise image.
[0110] In the embodiment of the present application, multiplying the preprocessed positive sample image with the noise vector refers to adding noise on the basis of the preprocessed positive sample image. Specifically, the operation of adding noise can be to use the function imnoise to pollute the preprocessed positive sample image with noise. The basic syntax of the imnoise function is g=imnoise(f,type,parameters), where f is the input image, type is the type of noise, and parameters is the parameter setting size. g=imnoise(f,'gaussian',m,var) adds Gaussian noise with a mean of m and a variance of var to the image f. The default value is noise with a mean of 0 and a variance of 0.01.
[0111] Step S502: deforming the noise image to obtain a deformed image.
[0112] In the embodiment of the present application, deformation processing refers to changing the shape of the above-mentioned noise image. Specifically, the deformation processing can be carried out by rotation, folding, mirror flipping, etc. It should be understood that the examples of deformation processing here are only for the convenience of understanding and are not used to limit the present application.
[0113] Step S503: input the deformed image into the generative adversarial network for upsampling processing to obtain a negative sample image.
[0114] In the embodiments of the present application, the Generative Adversarial Networks (GAN) refers to a network based on the concept of adversarial training, which consists of two competing neural networks, namely the generator and the discriminator. The generator or generative model attempts to capture the data distribution, and the discriminator or discriminant model estimates the probability that the sample comes from the training data instead of G.
[0115] In an embodiment of the present application, the deformed image is input into the residual block of the above-mentioned generator for upsampling, thereby obtaining the above-mentioned negative sample image, wherein the generative adversarial network skips some connections by means of residual blocks, allowing the generator to use the latent space to directly affect features of different resolutions and different levels.
[0116] Continue reading Figure 6 , showing Figure 2 A flowchart of a specific implementation of step S204 is shown in FIG. 1 . For ease of explanation, only the part related to the present application is shown.
[0117] In some optional implementations of this embodiment, the above step S204 specifically includes: step S601.
[0118] Step S601: Input the model training data into the original quality detection model for classification training until convergence, add a pooling layer, a fully connected layer and a random dropout layer to the converged original quality detection model, and train the fully connected layer according to the trained model training data until convergence to obtain the target quality detection model.
[0119] In the embodiment of the present application, considering that the fuzziness of the image belongs to the low-level semantic information of the image, the network model used preferably needs to ensure the low-level semantic information.
[0120] In the embodiment of the present application, considering that the fuzziness of the image belongs to the low-level semantic information of the image, the entire network parameters are set to be fixed (the network parameters at this time are obtained after training based on new data), and a pooling layer (Pool) and a fully connected (FC) layer and a random inactivation (dropout) layer are newly added after conv5_6, and the training data is used to train the newly added fully connected layer until convergence (the purpose is to only train the newly added FC and pool layers). In this way, the network weights of the quality assessment network model according to the embodiment of the present application are finally obtained. This network structure training method can ensure the semantic information of the middle and low layers.
[0121] In the embodiment of the present application, a pooling layer (Pool), a fully connected (FC) layer, and a random dropout layer are newly added after conv5_6, but the embodiment of the present disclosure is not limited to this. Instead, a pooling layer (Pool), a fully connected (FC) layer, and a random dropout layer can be added at other positions to train the newly added FC and pool layers.
[0122] Continue reading Figure 7 , shows a flowchart of a specific implementation method of obtaining a trained noise detection model provided in Example 1 of the present application. For the sake of ease of explanation, only the part related to the present application is shown.
[0123] In some optional implementations of this embodiment, before the above step S209, it also includes: step S701 and step S702.
[0124] Step S701: read a training database, and obtain noise training data and high-definition training data from the training database.
[0125] Step S702: Perform a model training operation on the original noise model according to the noise training data and the high-definition training data to obtain a trained noise detection model.
[0126] In the embodiment of the present application, the training data used by the noise model are noise data and high-definition data, and the output results are two categories with values of [0,1]. The larger the value, the clearer the image.
[0127] In the embodiment of the present application, noise data is defined as a picture containing noise, such as salt and pepper noise.
[0128] Continue reading Figure 8 , shows a flowchart of a specific implementation method of screening sub-images to be detected provided in Example 1 of the present application. For the sake of ease of explanation, only the parts related to the present application are shown.
[0129] In some optional implementations of this embodiment, before step S207, the following steps are further included: step S801, step S802, step S803, and step S804.
[0130] Step S801: Perform text detection processing on the sub-image to be detected to obtain a text detection result.
[0131] Step S802: Determine whether there is text content in the text detection result.
[0132] Step S803: If the text detection result shows that there is no text content, the sub-image to be detected is determined as a notext category, and the quality detection operation is stopped.
[0133] Step S804: If the text detection result contains text content, a quality detection operation is performed.
[0134] In the embodiment of the present application, the notext category refers to a sub-image to be detected that does not carry text content. Since the present application is mainly used to detect the quality of text images, after the image to be detected is divided into individual sub-images to be detected, there will be some images to be detected that do not carry text content. The result of the quality detection operation on the sub-image to be detected that does not carry text content has no reference value. Therefore, if the sub-image to be detected is confirmed to be the notext category, there is no need to perform subsequent quality detection operations.
[0135] In an embodiment of the present application, whether the image to be detected is of the notext category is confirmed by judging whether the sub-image to be detected carries text content. When the image to be detected is of the notext category, the image to be detected is directly ignored, and subsequent quality inspection operations are omitted. Only when the sub-image to be detected carries text content, the quality inspection operation is performed. By judging whether the sub-image to be detected carries text content, sub-images to be detected of the notext category are screened, thereby effectively avoiding the waste of subsequent quality inspection operations, thereby effectively improving the efficiency of quality inspection.
[0136] In summary, the present application provides a quality detection method for text images, including: obtaining a positive sample image from an image database; preprocessing the positive sample image to obtain a preprocessed positive sample image; constructing a negative sample image based on the preprocessed positive sample image, and combining the preprocessed positive sample image and the negative sample image to obtain model training data; inputting the model training data into an original quality detection model for classification training until convergence to obtain a target quality detection model; receiving a quality detection request carrying an image to be detected; performing image block processing on the image to be detected to obtain a sub-image to be detected; inputting the sub-image to be detected into a target quality detection model for quality detection operation to obtain a quality detection sub-result corresponding to the sub-image to be detected; performing a quality summary calculation operation on the quality detection sub-result to obtain a quality detection result; calling a trained noise detection model, inputting the sub-image to be detected into the trained noise detection model for noise detection operation to obtain a noise detection sub-result corresponding to the sub-image to be detected; performing a noise summary calculation operation on the noise detection sub-result to obtain a noise detection result; performing a comprehensive calculation operation on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected. The present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection results and the noise detection results. The image to be detected is detected from two directions, namely, "image quality" and "image noise", thereby greatly improving the accuracy of text image detection. At the same time, by classifying the sample images in the image database and extracting them from the classified sub-class samples, the obtained positive sample images can evenly cover the above categories, so that when the quality inspection model is subsequently trained, the training diversification of the quality inspection model is effectively guaranteed, thereby ensuring that the model can learn more realistic feature data; the positive sample image is standardized through the above-mentioned size adjustment and segmentation processing, so that the input data of the quality inspection model meets the requirements of the model; by judging whether the sub-image to be detected carries text content, the sub-image to be detected of the notext category is screened, thereby effectively avoiding the useless work of subsequent quality inspection operations, thereby effectively improving the efficiency of quality inspection.
[0137] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned images to be detected and target detection results, the above-mentioned images to be detected and target detection results can also be stored in a node of a blockchain.
[0138] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0139] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0141] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0142] Embodiment 2
[0143] Further references Fig. 9 , as a response to the above Figure 2 The present application provides an embodiment of a quality detection device for text images. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0144] like Fig. 9 As shown, the quality detection device 200 applied to text images of this embodiment includes: a positive sample acquisition module 201, a preprocessing module 202, a first training data acquisition module 203, a classification training module 204, a request receiving module 205, a block processing module 206, a quality detection module 207, a quality summary module 208, a noise detection module 209, a noise summary module 210 and a comprehensive calculation module 211. Among them:
[0145] A positive sample acquisition module 201 is used to acquire positive sample images from an image database;
[0146] A preprocessing module 202 is used to preprocess the positive sample image to obtain a preprocessed positive sample image;
[0147] A first training data acquisition module 203 is used to construct a negative sample image according to the preprocessed positive sample image, and obtain model training data by combining the preprocessed positive sample image and the negative sample image;
[0148] The classification training module 204 is used to input the model training data into the original quality inspection model for classification training until convergence, thereby obtaining a target quality inspection model;
[0149] A request receiving module 205 is used to receive a quality detection request carrying an image to be detected;
[0150] The block processing module 206 is used to perform image block processing on the image to be detected to obtain a sub-image to be detected;
[0151] The quality detection module 207 is used to input the sub-image to be detected into the target quality detection model to perform a quality detection operation, and obtain a quality detection sub-result corresponding to the sub-image to be detected;
[0152] The quality summary module 208 is used to perform a quality summary calculation operation on the quality detection sub-results to obtain a quality detection result;
[0153] The noise detection module 209 is used to call the trained noise detection model, input the sub-image to be detected into the trained noise detection model to perform noise detection operation, and obtain the noise detection sub-result corresponding to the sub-image to be detected;
[0154] A noise aggregation module 210 is used to perform a noise aggregation calculation operation on the noise detection sub-results to obtain a noise detection result;
[0155] The comprehensive calculation module 211 is used to perform comprehensive calculation operations on the quality detection results and the noise detection results to obtain the target detection results corresponding to the image to be detected.
[0156] In an embodiment of the present application, visual image data for visual object recognition is stored in an image database. Specifically, the image database may be an ImageNet dataset, wherein the ImageNet project is a large visual database for visual object recognition software research. More than 14 million image URLs are manually annotated by ImageNet to indicate objects in the pictures; in at least one million images, bounding boxes are also provided. ImageNet contains more than 20,000 categories. Annotation databases of third-party image URLs can be obtained directly from ImageNet for free. It should be understood that the examples of image databases here are only for ease of understanding and are not used to limit this application.
[0157] In the embodiment of the present application, the preprocessing may be to resize the positive sample image in a manner that keeps the aspect ratio unchanged to obtain a positive sample image of a standard size; or to segment the positive sample image of the standard size in a size of 224×224 to obtain a preprocessed positive sample image.
[0158] In an embodiment of the present application, constructing a negative sample image can be multiplying a preprocessed positive sample image with a noise vector to obtain a noise image; deforming the noise image to obtain a deformed image; and inputting the deformed image into a generative adversarial network for upsampling to obtain a negative sample image.
[0159] In an embodiment of the present application, the quality detection model refers to a deep neural network constructed based on a streamlined structure using deep separable convolution to form lightweight weights. Specifically, the quality detection model consists of a clarity detection model and a noise exposure detection model, wherein the clarity detection model is mainly used to detect the clarity of the image, and the noise exposure detection model is used to detect the noise quality, overexposure / underexposure and value of the image.
[0160] In an embodiment of the present application, the quality detection model solves the problems of computational efficiency and parameter quantity of convolutional networks by adopting depthwise separable convolutions. It can show stronger performance than other network models of traditional image network classification. The quality detection model is a lightweight deep neural network based on MobileNets, wherein the original quality detection model refers to the initial parameter value model in which the model parameters have not been trained and adjusted.
[0161] In the embodiment of the present application, after obtaining the model training data, the batch gradient descent method is used for training. Examples of the batch gradient descent method include batch gradient descent method BGD, mini-batch gradient descent method MBGD, and stochastic gradient descent method SGD, etc.
[0162] In the embodiment of the present application, the training data of the clarity detection model of the quality detection model can be divided into three categories, namely notext, poor quality, and good quality. In the training data, the evaluation of notext, poor quality, and good quality can be evaluated by the subjective feelings of the experimenter, by reference images, or by a clarity algorithm.
[0163] In the embodiments of the present application, examples of clarity algorithms include Brenner gradient function, Tenengrad gradient function, Laplacian gradient function, SMD (grayscale variance) function, SMD2 (grayscale variance product) function, etc. It should be understood that the examples of clarity algorithms here are only for the convenience of understanding and are not used to limit the present application.
[0164] In the embodiment of the present application, the image block processing can be divided into three parts according to the long side and two parts according to the short side. Figure 1 It is divided into 6 small pictures in total.
[0165] In an embodiment of the present application, after the sub-image to be detected is input into the above-mentioned trained target quality detection model, the sub-image to be detected will be labeled with an attribute label (the attribute labels include small image, large image, long image, solid color image, black and white image, pure text image, etc.), and at the same time, the sub-image to be detected is divided into five levels (the levels include unqualified quality, low quality, medium quality, high quality and ultra-high quality) through the target quality detection model, and the clarity detection model of the quality detection model scores the clarity of the sub-image to be detected through a clarity score [0, 1]. The reference model of the target quality detection model scores the voice value, overexposure value and underexposure value of the sub-image to be detected through a noise exposure score [0, 1]. Finally, the clarity score and the voice exposure score are combined to obtain the initial detection result of the sub-image to be detected.
[0166] In the embodiment of the present application, since the quality detection module 207 obtains the detection score results of each sub-image to be detected after the image to be detected is divided into blocks, when performing quality summary calculation of the quality detection score of the image to be detected, the quality detection sub-results of all sub-images to be detected can be calculated by averaging to obtain the quality detection result of the image to be detected; when calculating the score of the image to be detected, it is also possible to assign a weight value corresponding to the clarity to each sub-image to be detected, and calculate the weight score of each sub-image to be detected based on the weight value of each sub-image to be detected, and finally calculate the quality detection result of the image to be detected by averaging. It should be understood that the example of quality summary calculation here is only for the convenience of understanding and is not used to limit the present application.
[0167] In the embodiment of the present application, the above-mentioned trained noise detection model is pre-stored in the system database for arbitrary calling.
[0168] In an embodiment of the present application, the trained noise detection model can perform noise evaluation on the input data to obtain the noise value of the input data.
[0169] In the embodiment of the present application, since the noise detection module 209 obtains the detection score results of each sub-image to be detected after the image to be detected is divided into blocks, when performing noise aggregation to calculate the noise detection score of the image to be detected, the noise detection sub-results of all sub-images to be detected can be calculated by averaging to obtain the noise detection result of the image to be detected; when calculating the score of the image to be detected, a weight value corresponding to the exposure degree can be assigned to each sub-image to be detected, and the weight score of each sub-image to be detected is calculated according to the weight value of each sub-image to be detected, and finally the noise detection result of the image to be detected is calculated by averaging. It should be understood that the example of noise aggregation calculation here is only for the convenience of understanding and is not used to limit the present application.
[0170] In the embodiment of the present application, since the quality detection results and noise detection results obtained by the quality summary module 208 and the noise summary module 210 are the detection results of the image to be detected from the two directions of "quality" and "noise", it is necessary to perform a comprehensive calculation to obtain the final detection result of the image to be detected. When performing the comprehensive calculation, it can be calculated by an averaging method, or it can be calculated based on the assigned weights of "quality" and "noise", based on the "quality" weight score of the quality detection result and the "noise" weight score of the noise detection result, and finally the above-mentioned target detection result is obtained by averaging calculation. It should be understood that the example of comprehensive calculation here is only for the convenience of understanding and is not used to limit the present application.
[0171] In an embodiment of the present application, a quality detection device 200 for text images is provided, including: a positive sample acquisition module 201, used to obtain a positive sample image in an image database; a preprocessing module 202, used to preprocess the positive sample image to obtain a preprocessed positive sample image; a first training data acquisition module 203, used to construct a negative sample image according to the preprocessed positive sample image, and combine the preprocessed positive sample image and the negative sample image to obtain model training data; a classification training module 204, used to input the model training data into the original quality detection model for classification training until convergence, to obtain a target quality detection model; a request receiving module 205, used to receive a quality detection request carrying an image to be detected; a block processing module 206, used to perform image block processing on the image to be detected, to obtain a target quality detection model. sub-image; a quality detection module 207, which is used to input the sub-image to be detected into the target quality detection model for quality detection operation, and obtain the quality detection sub-result corresponding to the sub-image to be detected; a quality summary module 208, which is used to perform a quality summary calculation operation on the quality detection sub-result, and obtain the quality detection result; a noise detection module 209, which is used to call the trained noise detection model, input the sub-image to be detected into the trained noise detection model for noise detection operation, and obtain the noise detection sub-result corresponding to the sub-image to be detected; a noise summary module 210, which is used to perform a noise summary calculation operation on the noise detection sub-result, and obtain the noise detection result; a comprehensive calculation module 211, which is used to perform a comprehensive calculation operation on the quality detection result and the noise detection result, and obtain the target detection result corresponding to the image to be detected. The present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection results and the noise detection results. The image to be detected is detected from two directions, namely, "image quality" and "image noise", thereby greatly improving the accuracy of text image detection.
[0172] In some optional implementations of this embodiment, the positive sample acquisition module 201 includes: an image classification submodule and an image acquisition submodule, wherein:
[0173] An image classification submodule is used to classify sample images in an image database into small image categories, long image categories, pure color image categories, pure text image categories, black and white image categories, and large image categories according to a statistical model;
[0174] The image acquisition submodule is used to acquire a preset amount of image data in the small image category, long image category, pure color image category, pure text image category, black and white image category and large image category respectively, and use the acquired image data as positive sample images.
[0175] In an embodiment of the present application, the statistical model is preconfigured to classify image data into a small image category, a long image category, a pure color image category, a pure text image category, a black and white image category, and a large image category, wherein the statistical model is used to classify the image data, including:
[0176] When max_wh<200, the image data is judged as a small image;
[0177] When min_wh>=2000, the image data is determined to be a large image;
[0178] When max_wh / min_wh>3, the image data is judged as a long image;
[0179] When the number of channels of image data is 1, the image data is judged as a black and white image;
[0180] When v_max_1 / v_all>0.65, the image data is judged as a pure color image;
[0181] When v_max_1 / v_all<0.65 and (v_max_1+v_max_2) / v_all>0.8, the image data is determined to be a pure text image.
[0182] Among them, w, h, min_wh, max_wh represent the width, height, minimum side and maximum side of the image data respectively, and v_max_1, v_max_2 and v_all represent the maximum bin value, the second largest bin value and the sum of all bins of the histogram of the image data respectively.
[0183] In the embodiment of the present application, let hist(image) represent the histogram of the image image, where the number of columns of the histogram is 20.
[0184] In the obtained histogram of the input image, v_max_1, v_max_2 and v_all represent the maximum bin value, the second largest bin value and the sum of all bins of the histogram, respectively.
[0185] In an embodiment of the present application, by classifying sample images in an image database and extracting from the classified subclass samples, the obtained positive sample images can evenly cover the above-mentioned categories, so that when the quality inspection model is subsequently trained, the training diversification of the quality inspection model is effectively guaranteed, thereby ensuring that the model can learn more realistic feature data.
[0186] Continue reading Fig.10 , shows a structural schematic diagram of a specific implementation of the preprocessing module 202 provided in Example 2 of the present application. For the sake of ease of explanation, only the parts related to the present application are shown.
[0187] In some optional implementations of this embodiment, the pre-processing module 202 includes: a size adjustment processing sub-module 2021 and a segmentation processing sub-module 2022, wherein:
[0188] The size adjustment processing submodule 221 is used to perform size adjustment processing on the positive sample image in a manner that the aspect ratio remains unchanged, so as to obtain a positive sample image of a standard size;
[0189] The segmentation processing submodule 2022 is used to segment the standard-size positive sample image according to a preset size to obtain a pre-processed positive sample image.
[0190] In the embodiment of the present application, the resizing process refers to the process of adjusting the size of the positive sample image, and the adjustment process always keeps the aspect ratio of the image unchanged. Specifically, the image enlargement can adopt the interpolation method, that is, inserting new elements between the pixels of the original image by using a suitable interpolation algorithm. The interpolation algorithm includes, for example, an edge-based image interpolation algorithm, an area-based image interpolation algorithm, and other known or future-developed algorithms. The image reduction can be interpolated by, for example, CV_INETR_AREA.
[0191] In an embodiment of the present application, since the input of the quality detection model needs to meet certain size requirements, when an image that is too large is input into the quality detection model, it will affect the processing efficiency and accuracy of the model. Therefore, the positive sample image is standardized through the above-mentioned size adjustment processing and segmentation processing, so that the input data of the quality detection model meets the requirements of the model, wherein the preset size can be dynamically adjusted according to actual conditions. As an example, the preset size can be 224×224. It should be understood that the example of the preset size here is only for the convenience of understanding and is not used to limit the present application.
[0192] In some optional implementations of this embodiment, the first training data acquisition module 203 includes: a noise image acquisition submodule, a deformation processing submodule and an upsampling processing submodule, wherein:
[0193] A noise image acquisition submodule is used to multiply the preprocessed positive sample image with the noise vector to obtain a noise image;
[0194] A deformation processing submodule is used to perform deformation processing on the noise image to obtain a deformed image;
[0195] The upsampling processing submodule is used to input the deformed image into the generative adversarial network for upsampling processing to obtain a negative sample image.
[0196] In some optional implementations of this embodiment, the classification training module 204 includes: a classification training submodule, wherein:
[0197] The classification training submodule is used to input the model training data into the original quality inspection model for classification training until convergence, add a pooling layer, a fully connected layer and a random inactivation layer to the converged original quality inspection model, and train the fully connected layer according to the trained model training data until convergence to obtain the target quality inspection model.
[0198] In the embodiment of the present application, considering that the fuzziness of the image belongs to the low-level semantic information of the image, the network model used preferably needs to ensure the low-level semantic information.
[0199] In the embodiment of the present application, considering that the fuzziness of the image belongs to the low-level semantic information of the image, the entire network parameters are set to be fixed (the network parameters at this time are obtained after training based on new data), and a pooling layer (Pool) and a fully connected (FC) layer and a random inactivation (dropout) layer are newly added after conv5_6, and the training data is used to train the newly added fully connected layer until convergence (the purpose is to only train the newly added FC and pool layers). In this way, the network weights of the quality assessment network model according to the embodiment of the present application are finally obtained. This network structure training method can ensure the semantic information of the middle and low layers.
[0200] In the embodiment of the present application, a pooling layer (Pool), a fully connected (FC) layer, and a random dropout layer are newly added after conv5_6, but the embodiment of the present disclosure is not limited to this. Instead, a pooling layer (Pool), a fully connected (FC) layer, and a random dropout layer can be added at other positions to train the newly added FC and pool layers.
[0201] In some optional implementations of this embodiment, the quality detection device 200 applied to a text image includes:
[0202] The second training data acquisition module is used to read the training database and acquire noise training data and high-definition training data from the training database.
[0203] The model training module is used to perform model training operations on the original noise model according to the noise training data and the high-definition training data to obtain a trained noise detection model.
[0204] In the embodiment of the present application, the training data used by the noise model are noise data and high-definition data, and the output results are two categories with values of [0,1]. The larger the value, the clearer the image.
[0205] In the embodiment of the present application, noise data is defined as a picture containing noise, such as salt and pepper noise.
[0206] In some optional implementations of this embodiment, the above-mentioned quality detection device 200 applied to text images includes:
[0207] The text detection module is used to perform text detection processing on the sub-image to be detected to obtain the text detection result.
[0208] The text judgment module is used to judge whether there is text content in the text detection result.
[0209] The first result module is used to determine the sub-image to be detected as a notext category and stop performing the quality detection operation if there is no text content in the text detection result.
[0210] The second result module is used to perform a quality detection operation if the text detection result contains text content.
[0211] In the embodiment of the present application, the notext category refers to a sub-image to be detected that does not carry text content. Since the present application is mainly used to detect the quality of text images, after the image to be detected is divided into individual sub-images to be detected, there will be some images to be detected that do not carry text content. The result of the quality detection operation on the sub-image to be detected that does not carry text content has no reference value. Therefore, if the sub-image to be detected is confirmed to be the notext category, there is no need to perform subsequent quality detection operations.
[0212] In an embodiment of the present application, whether the image to be detected is of the notext category is confirmed by judging whether the sub-image to be detected carries text content. When the image to be detected is of the notext category, the image to be detected is directly ignored, and subsequent quality inspection operations are omitted. Only when the sub-image to be detected carries text content, the quality inspection operation is performed. By judging whether the sub-image to be detected carries text content, sub-images to be detected of the notext category are screened, thereby effectively avoiding the waste of subsequent quality inspection operations, thereby effectively improving the efficiency of quality inspection.
[0213] In summary, the present application provides a quality detection device 200 for text images, including: a positive sample acquisition module 201, used to obtain a positive sample image in an image database; a preprocessing module 202, used to preprocess the positive sample image to obtain a preprocessed positive sample image; a first training data acquisition module 203, used to construct a negative sample image according to the preprocessed positive sample image, and combine the preprocessed positive sample image and the negative sample image to obtain model training data; a classification training module 204, used to input the model training data into the original quality detection model for classification training until convergence, and obtain a target quality detection model; a request receiving module 205, used to receive a quality detection request carrying an image to be detected; a block processing module 206, used to perform image block processing on the image to be detected, and obtain a block to be detected. image; a quality detection module 207, used to input the sub-image to be detected into the target quality detection model for quality detection operation, and obtain the quality detection sub-result corresponding to the sub-image to be detected; a quality summary module 208, used to perform a quality summary calculation operation on the quality detection sub-result, and obtain the quality detection result; a noise detection module 209, used to call the trained noise detection model, input the sub-image to be detected into the trained noise detection model for noise detection operation, and obtain the noise detection sub-result corresponding to the sub-image to be detected; a noise summary module 210, used to perform a noise summary calculation operation on the noise detection sub-result, and obtain the noise detection result; a comprehensive calculation module 211, used to perform a comprehensive calculation operation on the quality detection result and the noise detection result, and obtain the target detection result corresponding to the image to be detected. The present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection results and the noise detection results. The image to be detected is detected from two directions, namely, "image quality" and "image noise", thereby greatly improving the accuracy of text image detection.At the same time, by classifying the sample images in the image database and extracting them from the classified sub-class samples, the obtained positive sample images can evenly cover the above categories, so that when the quality inspection model is subsequently trained, the training diversification of the quality inspection model is effectively guaranteed, thereby ensuring that the model can learn more realistic feature data; the positive sample image is standardized through the above-mentioned size adjustment and segmentation processing, so that the input data of the quality inspection model meets the requirements of the model; by judging whether the sub-image to be detected carries text content, the sub-image to be detected of the notext category is screened, thereby effectively avoiding the useless work of subsequent quality inspection operations, thereby effectively improving the efficiency of quality inspection.
[0214] To solve the above technical problems, the present application also provides a computer device. Fig.11 , Fig.11 This is a basic structural block diagram of the computer device in this embodiment.
[0215] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.
[0216] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0217] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In this embodiment, the memory 310 is generally used to store an operating system and various application software installed on the computer device 300, such as computer-readable instructions for a quality detection method for text images, etc. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.
[0218] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In this embodiment, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as running the computer-readable instructions of the quality detection method applied to the text image.
[0219] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.
[0220] The computer device provided by the present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks for image processing, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection result and the noise detection result. The image to be detected is detected from two directions of "image quality" and "image noise", thereby greatly improving the accuracy of text image detection.
[0221] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the quality detection method applied to text images as described above.
[0222] The computer-readable storage medium provided by the present application constructs training data based on high-definition images in an image database, thereby ensuring that the detection accuracy of the target quality detection model obtained through training maintains a high accuracy rate. When performing quality detection, the image to be detected is first divided into blocks for image processing, thereby greatly reducing the amount of calculation for large-image quality detection and judgment, and effectively improving the time efficiency of the process. The sub-image to be detected after block processing is then input into the quality detection model and the noise detection model for quality detection and noise detection, respectively. Finally, the final detection result of the image to be detected is calculated based on the quality detection results and the noise detection results. The image to be detected is detected from two directions of "image quality" and "image noise", thereby greatly improving the accuracy of text image detection.
[0223] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0224] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A quality detection method applied to text images, characterized in that: The steps include: Obtain positive sample images in the image database; Preprocessing the positive sample image to obtain a preprocessed positive sample image; Constructing a negative sample image according to the preprocessed positive sample image, and combining the preprocessed positive sample image and the negative sample image to obtain model training data; Inputting the model training data into the original quality inspection model for classification training until convergence, to obtain a target quality inspection model; Receiving a quality detection request carrying an image to be detected; Performing image block processing on the image to be detected to obtain a sub-image to be detected; Inputting the sub-image to be detected into the target quality detection model to perform a quality detection operation, and obtaining a quality detection sub-result corresponding to the sub-image to be detected; Performing a quality summary calculation operation on the quality detection sub-results to obtain a quality detection result; Calling a trained noise detection model, inputting the sub-image to be detected into the trained noise detection model to perform a noise detection operation, and obtaining a noise detection sub-result corresponding to the sub-image to be detected; Performing a noise aggregation calculation operation on the noise detection sub-results to obtain a noise detection result; Performing a comprehensive calculation operation on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected; The step of constructing negative sample images according to the preprocessed positive sample images to obtain model training data specifically includes the following steps: Multiplying the preprocessed positive sample image by a noise vector to obtain a noise image; Performing deformation processing on the noise image to obtain a deformed image; Inputting the deformed image into a generative adversarial network for upsampling processing to obtain the negative sample image; The step of inputting the model training data into the original quality inspection model for classification training until convergence to obtain the target quality inspection model specifically includes the following steps: The model training data is input into the original quality detection model for the classification training until convergence, a pooling layer, a fully connected layer and a random inactivation layer are added to the converged original quality detection model, and the fully connected layer is trained according to the trained model training data until convergence to obtain the target quality detection model.
2. The quality detection method for text images according to claim 1, characterized in that: The step of obtaining a positive sample image in the image database specifically includes the following steps: Classifying the sample images in the image database into a small image category, a long image category, a pure color image category, a pure text image category, a black and white image category, and a large image category according to the statistical model; A preset amount of image data is respectively acquired in the small image category, the long image category, the pure color image category, the pure text image category, the black and white image category, and the large image category, and the acquired image data is used as the positive sample image.
3. The quality detection method for text images according to claim 1, characterized in that: The step of preprocessing the positive sample image to obtain a preprocessed positive sample image specifically comprises the following steps: Resizing the positive sample image in a manner that the aspect ratio remains unchanged, to obtain a positive sample image of a standard size; The standard-size positive sample image is segmented according to a preset size to obtain the pre-processed positive sample image.
4. The quality detection method for text images according to claim 1, characterized in that: Before the step of calling the trained noise detection model, inputting the sub-image to be detected into the trained noise detection model to perform noise detection operation, and obtaining a noise detection sub-result corresponding to the sub-image to be detected, the following step is also included: Reading a training database, and obtaining noise training data and high-definition training data from the training database; A model training operation is performed on the original noise model according to the noise training data and the high-definition training data to obtain the trained noise detection model.
5. A quality detection device for text images, characterized in that: include: A positive sample acquisition module is used to acquire positive sample images from an image database; A preprocessing module, used for preprocessing the positive sample image to obtain a preprocessed positive sample image; A first training data acquisition module, used to construct a negative sample image according to the preprocessed positive sample image, and obtain model training data by combining the preprocessed positive sample image and the negative sample image; A classification training module, used for inputting the model training data into the original quality inspection model for classification training until convergence, so as to obtain a target quality inspection model; A request receiving module, used for receiving a quality detection request carrying an image to be detected; A block processing module, used for performing image block processing on the image to be detected to obtain a sub-image to be detected; A quality detection module, used for inputting the sub-image to be detected into the target quality detection model to perform a quality detection operation, and obtaining a quality detection sub-result corresponding to the sub-image to be detected; A quality summary module, used for performing a quality summary calculation operation on the quality detection sub-results to obtain a quality detection result; A noise detection module is used to call a trained noise detection model, input the sub-image to be detected into the trained noise detection model to perform a noise detection operation, and obtain a noise detection sub-result corresponding to the sub-image to be detected; A noise aggregation module, used for performing a noise aggregation calculation operation on the noise detection sub-results to obtain a noise detection result; A comprehensive calculation module, used for performing a comprehensive calculation operation on the quality detection result and the noise detection result to obtain a target detection result corresponding to the image to be detected; The first training data acquisition module includes: a noise image acquisition submodule, a deformation processing submodule and an upsampling processing submodule, wherein: The noise image acquisition submodule is used to multiply the preprocessed positive sample image with the noise vector to obtain a noise image; The deformation processing submodule is used to perform deformation processing on the noise image to obtain a deformed image; The upsampling processing submodule is used to input the deformed image into a generative adversarial network for upsampling processing to obtain the negative sample image; The classification training module includes: a classification training submodule, wherein: The classification training submodule is used to input the model training data into the original quality detection model for the classification training until convergence, add a pooling layer, a fully connected layer and a random inactivation layer to the converged original quality detection model, and train the fully connected layer according to the trained model training data until convergence to obtain the target quality detection model.
6. The quality detection device for text images according to claim 5, characterized in that: The preprocessing module comprises: A size adjustment processing submodule is used to perform size adjustment processing on the positive sample image in a manner that the aspect ratio remains unchanged, so as to obtain a positive sample image of a standard size; The segmentation processing submodule is used to segment the standard-size positive sample image according to a preset size to obtain the pre-processed positive sample image.
7. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the quality detection method applied to text images as claimed in any one of claims 1 to 4 when executing the computer-readable instructions.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the quality detection method applied to text images according to any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Image quality evaluation method and device and electronic equipment
CN110363753A
Method for generating image noise detection model and image noise detection method and device
CN110910356A
Method and device for extracting information in histogram
US20190266435A1
Dental Image Quality Prediction Platform Using Domain Specific Artificial Intelligence
US20210365736A1
