Notification compliance detection method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and provides a notification compliance detection method and device, equipment and a storage medium. The method comprises the following steps: performing image difference operation on a to-be-detected image corresponding to a to-be-detected notification and an unfilled notification template image to obtain a difference matrix corresponding to the to-be-detected image; performing edge detection on the to-be-detected image according to the difference matrix to obtain a user filling area; and calculating a pixel proportion of a user filling part in the user filling area, and comparing the pixel proportion with a pixel proportion threshold value determined based on the correctly filled compliance notification image to detect whether the notification to be detected is compliant or not. By comparing the pixel proportion with the correctly filled compliance notification image, whether the notification is compliant is determined, rapid detection of the compliance of the notification is realized, the content filled by the user does not need to be identified, the requirement on the notification image is relatively low, different templates can be set for different notifications, and the detection efficiency is improved. And manual rule making is also reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and particularly to a method, device, equipment and storage medium for detecting the compliance of a notice. Background Art

[0002] Currently, for the detection of documents or files, OCR (Optical Character Recognition) technology is usually used. For documents such as user notices, it is necessary to detect the content filled in by the user and then judge its compliance. Specifically, for documents that need to be filled in by the user, most of them need to use OCR technology to recognize the content filled in by the user, and then perform compliance detection according to the recognized content. This method has high requirements for the content filled in by the user, requiring the user's handwriting to be clear and the uploaded image to be clear enough. However, in actual applications, there are problems such as scribbled handwriting and obvious writing traces in user notices, and the results recognized by OCR technology are not credible; moreover, the detection method based on OCR technology runs slowly, resulting in too long response time, thus affecting business development, easily causing a huge system burden, and at the same time, new detection rules need to be manually formulated for different notice contents. Summary of the Invention

[0003] Embodiments of the present application provide a method, device, equipment and storage medium for detecting the compliance of a notice, so as to solve the technical problems that the existing detection methods are not credible, the detection speed is slow, resulting in too long response time, affecting business development, and new detection rules need to be manually formulated for different notice contents.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting the compliance of a notice, including:

[0005] Performing image difference operation on a to-be-detected image corresponding to a to-be-detected notice and a notice template image without filling to obtain a difference matrix corresponding to the to-be-detected image;

[0006] Performing edge detection on the to-be-detected image according to the difference matrix to obtain a user filling area of the to-be-detected notice;

[0007] Calculating a pixel ratio of a user filling part in the user filling area, and comparing the pixel ratio with a pixel ratio threshold to detect whether the to-be-detected notice is compliant; the pixel ratio threshold is determined based on an image of a compliant notice filled in correctly.

[0008] In one embodiment, the performing image difference operation on a to-be-detected image corresponding to a to-be-detected notice and a notice template image without filling to obtain a difference matrix corresponding to the to-be-detected image includes:

[0009] Perform grayscale binary conversion on the to-be-detected image corresponding to the to-be-detected notice to obtain the binary image of the to-be-detected image;

[0010] Perform difference calculation between the binary image and the template binary image of the unfilled notice template image to obtain the difference matrix corresponding to the to-be-detected image; the template binary image is obtained by performing grayscale binary conversion on the notice template image.

[0011] In one embodiment, calculating the pixel ratio of the user-filled part in the user-filled area includes:

[0012] Perform image segmentation on the user-filled area to obtain the to-be-detected area of the to-be-detected image;

[0013] According to the pixel values of each pixel point in the to-be-detected area, count the number of pixel points of the user-filled part in the to-be-detected area;

[0014] According to the counted number of pixel points of the user-filled part, calculate the pixel ratio of the user-filled part in the to-be-detected area.

[0015] In one embodiment, where there are multiple user-filled areas, comparing the pixel ratio with the pixel ratio threshold to detect whether the to-be-detected notice is compliant includes:

[0016] Obtain the pixel ratio threshold corresponding to each user-filled area;

[0017] Compare the pixel ratio of any user-filled area with the pixel ratio threshold corresponding to the user-filled area to determine the compliance detection result of the user-filled area;

[0018] Determine whether the to-be-detected notice is compliant according to the compliance detection results of each user-filled area.

[0019] In one embodiment, before performing image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image to obtain the difference matrix corresponding to the to-be-detected image, it further includes:

[0020] Based on a preset filter, use the non-maximum suppression algorithm to determine the first feature point set of the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image; the filter is constructed based on the second-order standard Gaussian function;

[0021] Obtain the sub-pixel level second feature point set by performing three-dimensional linear interpolation on the first feature point set;

[0022] Determine the main direction of each feature point in the second set of feature points using the histogram of gradients, and construct a feature description operator for each feature point in the second set of feature points; the second set of feature points includes the first feature points of the image to be detected and the second feature points of the template image of the notice; the feature description operator includes the first description operator of the first feature points and the second description operator of the second feature points;

[0023] Align the image to be detected with the template image of the notice based on the feature description operator.

[0024] In one embodiment, the aligning the image to be detected with the template image of the notice based on the feature description operator includes:

[0025] Calculate the Hamming distance between the first description operator and the second description operator in the feature description operator, and determine the matching degree between the first feature points of the image to be detected and the second feature points of the template image of the notice according to the Hamming distance;

[0026] Select the target feature points with the highest matching degree from the second set of feature points, and calculate the homography matrix between the image to be detected and the template image of the notice based on the target feature points;

[0027] Perform pixel mapping on the image to be detected according to the homography matrix to perform a rotation transformation on the image to be detected, and align the image to be detected with the template image of the notice.

[0028] In one embodiment, before performing an image difference operation on the image to be detected corresponding to the notice to be detected and the template image of the unfilled notice to obtain the difference matrix corresponding to the image to be detected, it further includes:

[0029] Perform grayscale binary conversion on the template image of the notice to obtain the template binary image of the template image of the notice and the binary matrix corresponding to the template binary image;

[0030] Perform Gaussian filtering conversion on the binary matrix to smooth the template binary image.

[0031] In a second aspect, an embodiment of the present application provides a notice compliance detection device, including:

[0032] A difference operation module, configured to perform an image difference operation on the image to be detected corresponding to the notice to be detected and the template image of the unfilled notice to obtain the difference matrix corresponding to the image to be detected;

[0033] An edge detection module, configured to perform edge detection on the image to be detected according to the difference matrix, so as to obtain the user filling area of the notice to be detected;

[0034] A compliance detection module, configured to calculate the pixel ratio of the user filled part in the user filling area, and compare the pixel ratio with a pixel ratio threshold to detect whether the notice to be detected is compliant; the pixel ratio threshold is determined based on an image of a compliant notice filled correctly.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory storing a computer program, where the processor implements the steps of the notice compliance detection method described in the first aspect when executing the program.

[0036] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored, and the computer program implements the steps of the notice compliance detection method described in the first aspect when being executed by a processor.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and the computer program implements the steps of the notice compliance detection method described in the first aspect when being executed by a processor.

[0038] The notice compliance detection method, device, equipment and storage medium provided by the embodiments of the present application perform an image difference operation on the image to be detected corresponding to the notice to be detected and the template image of the unfilled notice, perform edge detection on the image to be detected according to the difference matrix obtained by the image difference operation, and obtain the user filling area of the notice to be detected; calculate the pixel ratio of the user filled part in the user filling area, and compare it with the pixel ratio threshold to detect whether the notice to be detected is compliant. Through the image difference operation and edge detection, the user filling area of the notice is determined, and the pixel ratio of the user filled part in the user filling area is compared with the image of the correctly filled compliant notice, so as to determine whether the user filling content is substantial and whether the filling position is correct, and further determine whether the notice to be detected is compliant. It not only realizes the rapid detection of the compliance of the notice, but also, since there is no need to identify the user filling content, the clarity requirement for the notice image is relatively low. Different template images can be set corresponding to different notices, and there is no need to manually specify the detection rules. Description of the Drawings

[0039] To more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for the embodiments or the description of the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0040] Figure 1 is one of the schematic flowcharts of the notice compliance detection method provided by the embodiments of the present application;

[0041] Figure 2 is the schematic diagram of the template binary image after smoothing processing provided by the embodiments of the present application;

[0042] Figure 3 is one of the schematic structural diagrams of the notice compliance detection device provided by the embodiments of the present application;

[0043] Figure 4 is the schematic structural diagram of the electronic device provided by the embodiments of the present application. Specific Embodiments

[0044] To make the purpose, technical solutions, and advantages of the present application clearer, the following will clearly and completely describe the technical solutions in the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present application belong to the scope of protection of the present application.

[0045] The embodiments of the present application provide a notice compliance detection method. By means of image alignment, difference processing, and gray value detection, etc., the pixel ratio of the user-filled part in the user-filled area is counted, so as to determine whether the user notice is compliant.

[0046] Specifically, Figure 1 is the schematic flowchart of the notice compliance detection method provided by the embodiments of the present application. Referring to Figure 1 , the notice compliance detection method provided by the embodiments of the present application may include:

[0047] Step 100: Perform an image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the notice template image without filling to obtain the difference matrix corresponding to the to-be-detected image;

[0048] Step 200: Perform edge detection on the to-be-detected image according to the difference matrix to obtain the user-filled area of the to-be-detected notice;

[0049] Step 300, calculate the pixel ratio of the user-filled part in the user-filled area of the to-be-detected notice, and compare the pixel ratio with a pixel ratio threshold to detect whether the to-be-detected notice is compliant; the pixel ratio threshold is determined based on an image of a compliant notice filled out correctly.

[0050] When performing compliance detection on a user notice, first perform an image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the notice template image to obtain a difference matrix corresponding to the to-be-detected image. Among them, the notice template image is the image corresponding to the unfilled blank notice template, and the difference between the to-be-detected image and the notice template image is generated based on the content filled in by the user in the notice filling area, which characterizes the characteristics of the filled part in the user-filled area of the notice.

[0051] Furthermore, since the difference matrix of the to-be-detected image characterizes the characteristics of the filled part of the user in the notice filling area, therefore, based on this difference matrix, by performing edge detection on the to-be-detected image, the user-filled area of the to-be-detected notice can be obtained.

[0052] Calculate the pixel ratio of the user-filled part in the user-filled area of the to-be-detected notice, and compare the calculated pixel ratio with the pixel ratio threshold, so as to determine whether the to-be-detected notice is compliant. Among them, the pixel ratio threshold is determined based on an image of a compliant notice filled out correctly. Preferably, perform an image difference operation on the image of the compliant notice filled out correctly and the notice template image to obtain a corresponding difference matrix, perform edge detection based on this difference matrix to obtain the user-filled area of the compliant notice filled out correctly, and obtain the corresponding pixel ratio threshold by calculating the pixel ratio of the user-filled part in the user-filled area of the compliant notice.

[0053] It should be noted that different user notices correspond to different notice templates and different pixel ratio thresholds. For the same user notice, there is one or more user-filled areas. When there are multiple user-filled areas, each user-filled area corresponds to a pixel ratio threshold, or multiple user-filled areas correspond to one pixel ratio threshold.

[0054] Preferably, the pixel ratio threshold can be a definite value or a range value. When the pixel ratio threshold is a range value, generally, due to different writing habits of different users, the font sizes will vary. However, under the premise of compliance constraints, the written content generally cannot exceed the filling area. Therefore, a reasonable range value can be set as the pixel ratio threshold based on the pixel ratio of the filled part in the filling area of the correctly written compliance notice. When performing compliance detection on the notice to be detected, after counting the pixel ratio of the filled part in the filling area of the notice to be detected, if the pixel ratio is within the range corresponding to the set pixel ratio threshold, the notice to be detected is compliant; otherwise, if the counted pixel ratio is not within the range corresponding to the pixel ratio threshold, the notice to be detected is non-compliant.

[0055] Based on the notice template image, through pixel ratio calculation, compare the pixel ratio of the notice to be detected with the pixel ratio of the correctly filled compliance notice. By using the differential comparison between the notice to be detected and the compliance notice, it can be determined whether the content filled by the user is substantial and whether the filling position is correct, so as to determine whether the notice to be detected is compliant.

[0056] In this embodiment, by performing image difference operation on the detected image corresponding to the notice to be detected and the unfilled notice template image, edge detection is performed on the detected image according to the difference matrix obtained from the image difference operation to obtain the user filling area of the notice to be detected; calculate the pixel ratio of the filled part in the user filling area and compare it with the pixel ratio threshold to detect whether the notice to be detected is compliant. Through image difference operation and edge detection, the user filling area of the notice is determined, and through pixel ratio calculation, the pixel ratio of the filled part in the user filling area is compared with the image of the correctly filled compliance notice, it can be determined whether the content filled by the user is substantial and whether the filling position is correct, and then it can be determined whether the notice to be detected is compliant, which can realize the rapid detection of the compliance of the notice. And since there is no need to identify the content filled by the user, the requirement for the clarity of the notice image is relatively low. Different templates can be set for different notices, which also reduces the manual formulation of detection rules.

[0057] Preferably, before performing image difference operation on the detected image corresponding to the notice to be detected and the unfilled notice template image, on the one hand, it is necessary to preprocess the detected image to align the detected image with the notice template image, and on the other hand, it is necessary to preprocess the notice template image and the compliance notice to obtain the pixel ratio threshold.

[0058] Based on this, in step 100, before performing an image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the blank notice template image to obtain the difference matrix corresponding to the to-be-detected image, it may further include:

[0059] Step 101, based on a preset filter, using the non-maximum suppression algorithm to determine the first set of feature points of the to-be-detected image corresponding to the to-be-detected notice and the blank notice template image; the filter is constructed based on the second-order standard Gaussian function;

[0060] Step 102, obtaining a sub-pixel level second set of feature points by performing three-dimensional linear interpolation on the first set of feature points;

[0061] Step 103, using the histogram of gradients to determine the main direction of each feature point in the second set of feature points, and constructing a feature description operator for each feature point in the second set of feature points; the second set of feature points includes the first feature points of the to-be-detected image and the second feature points of the notice template image; the feature description operator includes the first description operator of the first feature points and the second description operator of the second feature points;

[0062] Step 104, aligning the to-be-detected image and the notice template image based on the feature description operator.

[0063] First, based on a preset filter, using the non-maximum suppression algorithm to determine the first set of feature points of the to-be-detected image corresponding to the to-be-detected notice and the blank notice template image, where the filter is constructed based on the second-order standard Gaussian function. By performing three-dimensional linear interpolation on the feature points in the first set of feature points, a sub-pixel level second set of feature points is obtained, and the histogram of gradients is used to determine the main direction of each feature point in the second set of feature points, and a feature description operator for each feature point in the second set of feature points is constructed. Among them, the second set of feature points includes the first feature points of the to-be-detected image and the second feature points of the notice template image. Therefore, the constructed feature description operator includes the first description operator of the first feature points of the to-be-detected image and the second description operator of the second feature points of the notice template image. Based on the constructed description operator, the to-be-detected image and the notice template image are aligned.

[0064] In one embodiment, the Hessian matrix constructs a Gaussian pyramid scale space, converting the filtering of the Gaussian second-order differential template into addition and subtraction operations on the integral image, thus greatly shortening the search time during the detection of feature points. In the constructed scale space, non-maximum suppression is used to search for the extreme points of the image as feature points, and three-dimensional linear interpolation is performed on the searched feature points to obtain sub-pixel level feature points. To ensure the rotation invariance of the feature description operator, gradient operations are performed on the image through a gradient histogram, and a main direction is assigned to each feature point, thereby constructing the feature description operator for each feature point.

[0065] Further, in step 104, based on the constructed feature description operator, aligning the image to be detected with the notice template image specifically further includes:

[0066] Step 114, calculating the Hamming distance between the first description operator and the second description operator in the feature description operator, and determining the matching degree between the first feature point of the image to be detected and the second feature point of the template notice image according to the Hamming distance;

[0067] Step 124, selecting the target feature point with the highest matching degree from the set of second feature points, and calculating the homography matrix between the image to be detected and the notice template image based on the target feature point;

[0068] Step 134, performing pixel mapping on the image to be detected according to the homography matrix to perform a rotation transformation on the image to be detected, and aligning the image to be detected with the notice template image.

[0069] When aligning the image to be detected with the notice template image, first, based on the feature description operators of each feature point, calculate the Hamming distance between the first feature description operator and the second feature description operator, determine the similarity between the first description operator and the second description operator according to the Hamming distance, and further determine the matching degree between the first feature point corresponding to the first description operator and the second feature point corresponding to the second description operator according to the similarity.

[0070] According to the calculated matching degree between the feature points in the image to be detected and the notice template image, select the target feature point with the highest matching degree from the set of second feature points, and calculate the homography matrix between the image to be detected and the notice template image based on the selected target feature point. Perform pixel mapping on the image to be detected according to the calculated homography matrix to perform a rotation transformation on the image to be detected, thereby aligning the image to be detected with the notice template image.

[0071] Preferably, in this embodiment, the technology of Random Sample Consensus is used to calculate the homography matrix N between the image to be detected and the form template image in the case of a large number of matching errors. According to the homography matrix N, pixel mapping is performed on the image to be detected to be aligned to the aligned form template image to achieve image alignment.

[0072] Preferably, in step 100, an image difference operation is performed on the image to be detected corresponding to the form to be detected and the unfilled form template image to obtain a difference matrix corresponding to the image to be detected, which specifically includes:

[0073] Step 110, performing grayscale binary conversion on the image to be detected corresponding to the form to be detected to obtain a binary image of the image to be detected;

[0074] Step 120, performing difference calculation on the binary image and the template binary image of the unfilled form template image to obtain a difference matrix corresponding to the image to be detected; the template binary image is obtained by performing grayscale binary conversion on the form template image.

[0075] The difference operation on the image to be detected is implemented based on the binary image. Specifically, first, grayscale binary conversion is performed on the image to be detected corresponding to the form to be detected to obtain a binary image of the image to be detected. Then, difference calculation is performed on the binary image and the template binary image of the unfilled form template image to obtain a difference matrix corresponding to the image to be detected. Among them, the template binary image is obtained by performing grayscale binary conversion on the form template image. The grayscale binary conversion of the binary image of the image to be detected and the grayscale binary conversion of the form template image are independent of each other, and the two can be performed simultaneously or independently. Preferably, in one embodiment, since the grayscale binary conversion of the form template image only needs to be performed once, therefore, the form template images of each form are preprocessed in advance to implement the grayscale binary conversion of the form template images to obtain the template binary images of each form template image for subsequent compliance detection.

[0076] Furthermore, for the difference matrix of the image to be detected, the elements in the matrix are the differences between the pixel values of the image to be detected and the form template image at the corresponding positions.

[0077] Preferably, for the template binary image of the form template image, gray value enhancement can also be performed in the preprocessing stage. Specifically, before performing the image difference operation on the image to be detected corresponding to the form to be detected and the unfilled form template image to obtain a difference matrix corresponding to the image to be detected in step 100, it can also include:

[0078] Step 001: Perform grayscale binary conversion on the notice template image to obtain the template binary image of the notice template image and the binary matrix corresponding to the template binary image;

[0079] Step 002: Perform Gaussian filtering conversion on the binary matrix to smooth the template binary image.

[0080] Before performing image difference operation on the image to be detected, first perform grayscale binary conversion on the notice template image to obtain the corresponding template binary image and the binary matrix of the template binary image. The elements in the binary matrix represent the pixel values at the corresponding positions of the template binary image. Then perform Gaussian filtering conversion on the binary matrix to smooth the template binary image and enhance the grayscale value of the template binary image.

[0081] In one embodiment, let the image of the compliant notice filled in correctly be P T , the blank notice template image not filled in be P N , and the image to be detected corresponding to the notice to be detected be P as an example. Before performing image difference operation, first perform grayscale binary conversion on the notice template image to obtain the corresponding binary matrix PN T , and then perform Gaussian filtering conversion on PN T to obtain the smoothed template binary image, as shown in Figure 2 . The matrix after Gaussian density conversion corresponding to the smoothed template binary image is PN-G T .

[0082] Preferably, the Gaussian probability density function used for Gaussian filtering conversion of the binary matrix is:

[0083]

[0084] In the image difference operation stage, perform difference calculation between the image to be detected P and PN-G T to obtain the difference matrix PN, and perform difference calculation between P T and PN-G T to obtain the difference matrix PN T . The difference calculation formula is shown in Formula 2 below:

[0085]

[0086] Among them, P 1 is the comparison image, that is, the image to be detected P and the compliant notice image P T , P2 is the binarization matrix PN-G of the template binarized image T , where x and y are the element subscripts of the image transformation matrix, representing the positions of the pixel points in the image.

[0087] The P-PN matrix is obtained through difference calculation, which is the binarized representation of the difference part between the image to be detected and the template image of the notice. This difference part is the user-filled part. Similarly, the compliant notice image P can also be obtained T and the difference matrix P N between the notice template image P T -PN.

[0088] Preferably, in step 300, calculate the pixel ratio of the user-filled part in the user-filled area of the notice to be detected, specifically including:

[0089] Step 310, perform image segmentation on the user-filled area to obtain the area to be detected of the image to be detected;

[0090] Step 320, count the number of pixel points of the user-filled part in the area to be detected according to the pixel values of each pixel point in the area to be detected;

[0091] Step 330, calculate the pixel ratio of the user-filled part in the area to be detected according to the counted number of pixel points of the user-filled part.

[0092] First, after determining the position of the user-filled area in the notice to be detected through edge detection, perform image segmentation on the user-filled area to obtain the area to be detected of the image to be detected. According to the pixel values of each pixel point in the area to be detected, count the number of pixel points of the user-filled part in the area to be detected. According to the counted number of pixel points of the user-filled part, calculate the pixel ratio of the user-filled part in the area to be detected.

[0093] In this embodiment, the pixel ratio of the user-filled part refers to the ratio of the number of pixel points of the user-filled part to the total number of pixel points in the user-filled area. In one embodiment, edge detection is performed according to the difference matrix P T -PN to identify one or more areas to be detected D T ={P1, P2, P3, P4}, and image segmentation is performed on the areas to be detected by means of position box marking, specifically expressed as:

[0094]

[0095] Among them, b1, b2, b3, b4 are the new fixed-point pixel coordinates that make up the detection area, and [x1, y1], [x2, y2], [x3, y3], [x4, y4] are the edge fixed points of the P T -PN matrix. By the method of fixing points with pixel coordinates, a position box is generated to enclose the area to be detected. For the difference matrix P - PN of the binary image of the image to be detected, according to the detected area D T ={P1, P2, P3, P4}, the image is segmented by region to obtain one or more areas to be detected. For each area to be detected D PN i , the ratio relationship between the proportion of pixels 1 and 0 in its area is calculated respectively to obtain the pixel proportion r of the user-filled part of the area i . Among them, the calculation method of the pixel proportion is shown in Formula 4:

[0096]

[0097] N and M are the dimensions of the binary image matrix, and p x,y is the pixel value of the binary matrix corresponding to the binary image. In this way, the pixel proportion of each area to be detected can be obtained, which can be expressed as: R S ={r1, r2,..., r n}.

[0098] Preferably, the area to be detected of the notice to be detected includes one or more. When there are multiple, in step 300, the calculated pixel proportion of the area to be detected is compared with the pixel proportion threshold to detect whether the notice to be detected is compliant. Specifically, it may further include:

[0099] Step 340, obtaining the pixel proportion threshold corresponding to each of the user-filled areas;

[0100] Step 350, comparing the pixel proportion of any one of the user-filled areas with the pixel proportion threshold corresponding to the user-filled area to determine the compliance detection result of the user-filled area;

[0101] Step 360, determining whether the notice to be detected is compliant according to the compliance detection results of each of the user-filled areas.

[0102] First, obtain the pixel ratio thresholds corresponding to each user filling area. It is known that in the user's notice, different user filling areas require different content to be filled, so the sizes of the user filling areas are different, and the corresponding pixel ratio thresholds may also be different. Compare the pixel ratio calculated for each user filling area with the pixel ratio threshold corresponding to the user filling area to determine the compliance detection result of the user filling area. Finally, based on the compliance detection results of each user filling area, determine whether the notice to be detected is compliant.

[0103] Preferably, when the compliance detection results of each user filling area are all compliant, the notice to be detected is compliant; otherwise, when the compliance detection result of any one user filling area is non-compliant, the notice to be detected is non-compliant.

[0104] In one embodiment, for the user filling part of the user notice, after multiple practices, the pixel ratio range of compliant notices is generally between 0.87 - 0.96, and the compliant notices with a ratio below 0.8 are only 0.15‰. Therefore, the pixel ratio threshold of the compliant notice can be set to 0.8 as the compliance detection threshold for the notice to be detected. That is, in one embodiment, for the same user filling area in the same notice, the pixel ratios of multiple compliant notices can be statistically analyzed, and the average value of the statistical results can be used as the pixel ratio threshold.

[0105] For the difference matrix P T -PN, calculate the pixel ratios of multiple user filling areas to obtain R T ={rt1, rt2,..., rt n}, and the set pixel ratio threshold is r i =0.8 * rt i , as the basis for compliance discrimination, compare the pixel ratios statistically obtained for the area to be detected. For any area to be detected r i <r i , then the compliance detection of the notice fails. If for any r i ∈R s there is r i ≥r i then the compliance detection of the compliant notice passes.

[0106] It should be noted that the processing process of the image corresponding to the correctly filled compliance notice is the same as the compliance detection process of the image to be detected corresponding to the notice to be detected. In one embodiment, the two can be carried out simultaneously, that is, each time the image to be detected corresponding to the notice to be detected is detected, an image of a correctly filled compliance notice of the same type is randomly selected, and based on the notice template image, the same processing is performed on the image to be detected, so that the processing result of the compliance notice is used as the control group of the notice to be detected, realizing the rapid detection of the notice to be detected. In another embodiment, in order to reduce the use of computing resources in the detection process, the processing of the correctly filled compliance notice can be performed only once, that is, in the preprocessing stage, one or more images corresponding to the correctly filled compliance notice are selected, and the selected images are subjected to the above-mentioned grayscale binary conversion, image difference operation, edge detection, pixel ratio statistics, etc., to obtain the pixel ratio of the user-filled part in the user-filled area, and the corresponding pixel ratio threshold is set according to the pixel ratio, as the discrimination basis for subsequent compliance detection.

[0107] In this embodiment, by image alignment, grayscale image binarization, image difference operation, and calculation of the pixel ratio of the filled part, and comparing with the compliance notice, it can be determined whether the content filled by the user is substantial, so as to obtain the compliance detection result of the notice and determine whether the user notice is compliant.

[0108] Furthermore, based on the notice template image, by comparing the image of the notice to be detected with the image of the correctly filled compliance notice, and using the three images for differential comparison, the comparison result of the content filled by the user and the filling position can be obtained, and the compliance of the notice to be detected can be determined. This can not only reduce the manual formulation of different notice rules and reduce the operation and maintenance workload, but also greatly save the system operation time. And compared with the OCR technology-based solution, the requirements for user handwriting and picture quality are greatly reduced, improving the detection convenience.

[0109] The compliance detection device for notices provided in the embodiments of the present application will be described below. The compliance detection device for notices described below can be mutually referred to with the compliance detection method for notices described above.

[0110] Refer to Figure 3 , the compliance detection device for notices provided in the embodiments of the present application includes:

[0111] The difference operation module 10 is used to perform an image difference operation on the image to be detected corresponding to the notice to be detected and the unfilled notice template image to obtain the difference matrix corresponding to the image to be detected;

[0112] An edge detection module 20, configured to perform edge detection on the image to be detected according to the difference matrix, so as to obtain the user filling area of the notice to be detected;

[0113] A compliance detection module 30, configured to calculate the pixel ratio of the user-filled part in the user filling area, and compare the pixel ratio with a pixel ratio threshold to detect whether the notice to be detected is compliant; the pixel ratio threshold is determined based on an image of a compliant notice filled correctly.

[0114] In one embodiment, the difference operation module 10 is further configured to:

[0115] Perform grayscale binary conversion on the image to be detected corresponding to the notice to be detected to obtain a binary image of the image to be detected;

[0116] Perform difference calculation between the binary image and the template binary image of the unfilled notice template image to obtain a difference matrix corresponding to the image to be detected; the template binary image is obtained by performing grayscale binary conversion on the notice template image.

[0117] In one embodiment, the compliance detection module 30 is further configured to:

[0118] Perform image segmentation on the user filling area to obtain a to-be-detected area of the image to be detected;

[0119] According to the pixel values of each pixel point in the to-be-detected area, count the number of pixel points of the user-filled part in the to-be-detected area;

[0120] According to the counted number of pixel points of the user-filled part, calculate the pixel ratio of the user-filled part in the to-be-detected area.

[0121] In one embodiment, there are multiple user filling areas, and the compliance detection module 30 is further configured to:

[0122] Obtain the pixel ratio threshold corresponding to each user filling area;

[0123] Compare the pixel ratio of any one of the user filling areas with the pixel ratio threshold corresponding to the user filling area to determine the compliance detection result of the user filling area;

[0124] Determine whether the notice to be detected is compliant according to the compliance detection results of each user filling area.

[0125] In one embodiment, the notice compliance detection device further includes an image alignment module, configured to:

[0126] Based on a preset filter, the non-maximum suppression algorithm is used to determine the first set of feature points of the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image; the filter is constructed based on the second-order standard Gaussian function;

[0127] By performing three-dimensional linear interpolation on the first set of feature points, a sub-pixel-level second set of feature points is obtained;

[0128] The main direction of each feature point in the second set of feature points is determined by using the histogram of gradients, and the feature description operator of each feature point in the second set of feature points is constructed; the second set of feature points includes the first feature points of the to-be-detected image and the second feature points of the notice template image; the feature description operator includes the first description operator of the first feature point and the second description operator of the second feature point;

[0129] Based on the feature description operator, the to-be-detected image is aligned with the notice template image.

[0130] In one embodiment, the image alignment module is further configured to:

[0131] Calculate the Hamming distance between the first description operator and the second description operator in the feature description operator, and determine the matching degree between the first feature points of the to-be-detected image and the second feature points of the template notice image according to the Hamming distance;

[0132] Select the target feature point with the highest matching degree from the second set of feature points, and calculate the homography matrix between the to-be-detected image and the notice template image based on the target feature point;

[0133] Perform pixel mapping on the to-be-detected image according to the homography matrix to perform a rotation transformation on the to-be-detected image and align the to-be-detected image with the notice template image.

[0134] In one embodiment, the notice compliance detection device further includes an image enhancement module, which is used to:

[0135] Perform grayscale binary conversion on the notice template image to obtain the template binary image of the notice template image and the binary matrix corresponding to the template binary image;

[0136] Perform Gaussian filtering conversion on the binary matrix to smooth the template binary image.

[0137] Figure 4 Illustrated a schematic physical structure diagram of an electronic device, such as Figure 4As shown in the figure, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 complete communication with each other through the communication bus 440. The processor 410 may call a computer program in the memory 430 to execute the steps of the notice compliance detection method, for example, including:

[0138] Perform an image difference operation on the image to be detected corresponding to the notice to be detected and the template image of the notice without filling to obtain the difference matrix corresponding to the image to be detected;

[0139] Perform edge detection on the image to be detected according to the difference matrix to obtain the user filling area of the notice to be detected;

[0140] Calculate the pixel ratio of the user-filled part in the user filling area, and compare the pixel ratio with a pixel ratio threshold to detect whether the notice to be detected is compliant; the pixel ratio threshold is determined based on the image of the compliant notice filled correctly.

[0141] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software function units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0142] On the other hand, the embodiment of the present application also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the steps of the notice compliance detection method provided in the above-mentioned various embodiments, for example, including:

[0143] Perform an image difference operation on the image to be detected corresponding to the notice to be detected and the template image of the notice without filling to obtain the difference matrix corresponding to the image to be detected;

[0144] Perform edge detection on the image to be detected according to the difference matrix to obtain the user filling area of the notice to be detected;

[0145] Calculate the pixel ratio of the user-filled part in the user filling area, and compare the pixel ratio with a pixel ratio threshold to detect whether the notice to be detected is compliant; the pixel ratio threshold is determined based on the image of a compliant notice filled out correctly.

[0146] On the other hand, an embodiment of the present application also provides a processor-readable storage medium storing a computer program for causing a processor to execute the steps of the notice compliance detection method provided in the above embodiments, for example, including:

[0147] Perform an image difference operation on the image to be detected corresponding to the notice to be detected and the template image of the unfilled notice to obtain the difference matrix corresponding to the image to be detected;

[0148] Perform edge detection on the image to be detected according to the difference matrix to obtain the user filling area of the notice to be detected;

[0149] Calculate the pixel ratio of the user-filled part in the user filling area, and compare the pixel ratio with a pixel ratio threshold to detect whether the notice to be detected is compliant; the pixel ratio threshold is determined based on the image of a compliant notice filled out correctly.

[0150] The processor-readable storage medium may be any available medium or data storage device accessible by the processor, including but not limited to magnetic memories (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memories (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROM, EPROM, EEPROM, non-volatile memories (NANDFLASH), solid state drives (SSD)).

[0151] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting the compliance of a notice, characterized in that, Including: Performing image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image to obtain the difference matrix corresponding to the to-be-detected image; Performing edge detection on the to-be-detected image according to the difference matrix to obtain the user filling area of the to-be-detected notice; Calculating the pixel ratio of the user filling part in the user filling area and comparing the pixel ratio with a pixel ratio threshold to detect whether the to-be-detected notice is compliant; The pixel ratio threshold is determined based on the image of the compliant notice filled correctly.

2. The compliance detection method for the notice as claimed in claim 1, wherein The performing image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image to obtain the difference matrix corresponding to the to-be-detected image includes: Performing gray-scale binary conversion on the to-be-detected image corresponding to the to-be-detected notice to obtain the binary image of the to-be-detected image; Performing difference calculation on the binary image and the template binary image of the unfilled notice template image to obtain the difference matrix corresponding to the to-be-detected image; the template binary image is obtained by performing gray-scale binary conversion on the notice template image.

3. The notice compliance detection method according to claim 1, wherein The calculating the pixel ratio of the user filling part in the user filling area includes: Performing image segmentation on the user filling area to obtain the to-be-detected area of the to-be-detected image; Counting the number of pixel points of the user filling part in the to-be-detected area according to the pixel values of each pixel point in the to-be-detected area; Calculating the pixel ratio of the user filling part in the to-be-detected area according to the counted number of pixel points of the user filling part.

4. The method for detecting the compliance of a notice according to claim 1, characterized in that, There are multiple user filling areas, and the comparing the pixel ratio with the pixel ratio threshold to detect whether the to-be-detected notice is compliant includes: Obtaining the pixel ratio threshold corresponding to each user filling area; Comparing the pixel ratio of any user filling area with the pixel ratio threshold corresponding to the user filling area to determine the compliance detection result of the user filling area; Determining whether the to-be-detected notice is compliant according to the compliance detection results of each user filling area.

5. The method for detecting the compliance of a notice according to claim 1, characterized in that, Before the performing image difference operation on the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image to obtain the difference matrix corresponding to the to-be-detected image, it further includes: Based on a preset filter, using the non-maximum suppression algorithm to determine the first feature point set of the to-be-detected image corresponding to the to-be-detected notice and the unfilled notice template image; the filter is constructed based on the second-order standard Gaussian function; Obtaining the sub-pixel level second feature point set by performing three-dimensional linear interpolation on the first feature point set; Using the histogram of gradients to determine the main direction of each feature point in the second feature point set and constructing the feature description operator of each feature point in the second feature point set; the second feature point set includes the first feature point of the to-be-detected image and the second feature point of the notice template image; the feature description operator includes the first description operator of the first feature point and the second description operator of the second feature point; Based on the feature description operator, align the image to be detected with the notice template image.

6. The method for detecting the compliance of the notice according to claim 5, wherein, The aligning the image to be detected with the notice template image based on the feature description operator includes: Calculate the Hamming distance between the first description operator and the second description operator in the feature description operator, and determine the matching degree between the first feature points of the image to be detected and the second feature points of the template notice image according to the Hamming distance; Select the target feature point with the highest matching degree from the set of the second feature points, and calculate the homography matrix between the image to be detected and the notice template image based on the target feature point; Perform pixel mapping on the image to be detected according to the homography matrix to perform a rotation transformation on the image to be detected, and align the image to be detected with the notice template image.

7. The compliance detection method for the notice according to claim 1, wherein Before performing the image difference operation on the image to be detected corresponding to the notice to be detected and the unfilled notice template image to obtain the difference matrix corresponding to the image to be detected, it further includes: Perform grayscale binary conversion on the notice template image to obtain the template binary image of the notice template image and the binary matrix corresponding to the template binary image; Perform Gaussian filtering conversion on the binary matrix to smooth the template binary image.

8. An advisement compliance detection device, characterized in that, It includes: A difference operation module for performing an image difference operation on the image to be detected corresponding to the notice to be detected and the unfilled notice template image to obtain the difference matrix corresponding to the image to be detected; An edge detection module for performing edge detection on the image to be detected according to the difference matrix to obtain the user filling area of the notice to be detected; A compliance detection module for calculating the pixel ratio of the user filling part in the user filling area and comparing the pixel ratio with a pixel ratio threshold to detect whether the notice to be detected is compliant; The pixel ratio threshold is determined based on the image of the compliant notice filled correctly.

9. An electronic device, comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the notice compliance detection method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the notice compliance detection method according to any one of claims 1 to 7.

Images