A picture detection method and device, electronic equipment and medium

Abstract

The application discloses a picture detection method and device, electronic equipment and medium. The method comprises the following steps: obtaining a page picture file to be detected; performing inference on a target picture in the page picture file through a pre-constructed picture detection model to determine a current coordinate of a current bounding box; performing color feature analysis on a current region framed by the current bounding box based on the current coordinate to obtain an analysis result; adjusting the current bounding box according to the analysis result to obtain a target bounding box; the bounding box is a frame used for framing the target picture; taking a target region framed by the target bounding box as the target picture; the target region does not include a non-picture region and includes all regions of the target picture. Thus, on the basis of initial picture inference of the picture detection model, the color feature analysis is used to quickly identify a picture effective region and a redundant region, and the current bounding box is adjusted to accurately frame the target picture, so that the detection accuracy of pictures in the page picture file is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical character recognition, and particularly relates to a picture detection method and device, electronic equipment and a medium. BACKGROUND

[0002] In the field of digital document processing, the image detection model in the optical character recognition (OCR) system is a core tool for extracting picture content in a document, and is widely used in important scenarios such as digital library construction, office automation and electronic archive management. With the diversification of document sources and the increasing complexity of typesetting forms, how to accurately identify and locate the effective picture area in the document image has become a key link affecting the overall system performance.

[0003] However, the current image detection model generally has the problem of insufficient detection accuracy of the bounding box in actual application, and it is difficult to meet the needs of high-reliability business scenarios. Specifically, when facing the situation that the low-contrast image is similar to the background color, the model is easy to misjudge the effective edge area of the picture as the background, thereby causing the detected bounding box to be smaller than the actual picture range. In addition, the model is difficult to effectively distinguish the picture main body from the adjacent text area or table frame, resulting in the detected bounding box being too large and covering redundant content. In addition, since the model fails to effectively distinguish the effective content boundary of the image from the blank interval in the document typesetting, the detection result may contain a large number of peripheral typesetting blank areas.

[0004] Therefore, how to improve the picture detection accuracy to meet the needs of high-reliability business scenarios is a problem to be solved by those skilled in the art. SUMMARY

[0005] Therefore, an aspect of the present application provides a picture detection method, which comprises:

[0006] obtaining a page picture file to be detected;

[0007] performing inference on a target picture in the page picture file by a pre-constructed picture detection model to determine a current coordinate of a current bounding box;

[0008] performing color feature analysis on a current region framed by the current bounding box based on the current coordinate to obtain an analysis result;

[0009] adjusting the current bounding box according to the analysis result to obtain a target bounding box; the bounding box is a frame for framing the target picture;

[0010] The target region framed by the target bounding box is taken as the target picture; the target region does not include a non-picture region and includes all regions of the target picture.

[0011] Optionally, if the current region includes the non-picture region and does not include a partial region of the target picture, the adjusting the current bounding box to obtain a target bounding box comprises:

[0012] According to the analysis result, the boundary of the current bounding box is reduced to crop the non-picture region, and a sub-bounding box is obtained;

[0013] The sub-bounding box is taken as the current bounding box, and the step of performing color feature analysis on a current region framed by the current bounding box based on the current coordinates to obtain an analysis result is executed;

[0014] According to the analysis result, the boundary of the current bounding box is expanded to frame the partial region, and the target bounding box is obtained.

[0015] Optionally, the target picture includes a first pixel point, the non-picture region includes a second pixel point, and the first pixel point and the second pixel point are different in color;

[0016] The bounding box is a rectangle; in a pre-constructed coordinate system, the current coordinates include a maximum coordinate point (X1, Y1) and a minimum coordinate point (X2, Y2), wherein X1 is greater than X2, and Y1 is greater than Y2;

[0017] The target straight lines on which the boundary of the current bounding box lies include a first straight line Y=Y1, a second straight line Y=Y2, a third straight line X=X1, and a fourth straight line X=X2.

[0018] Optionally, if the current region includes the non-picture region, the performing color feature analysis on a current region framed by the current bounding box based on the current coordinates to obtain an analysis result comprises:

[0019] According to the current coordinates, the target straight lines are determined;

[0020] Each of the target straight lines is taken as a starting scanning line, and linear pixel point scanning is gradually performed in a direction approaching a center point of the current bounding box and parallel to the target straight line, and a pixel line is obtained;

[0021] In the linear pixel point scanning process, when it is first determined that the pixel line includes the first pixel point, the linear pixel point scanning is stopped;

[0022] A current region corresponding to the pixel line including the second pixel point is taken as the non-picture region, and the analysis result is obtained.

[0023] Optionally, if the current region does not include the partial region, performing color feature analysis on the current region framed by the current bounding box based on the current coordinates to obtain an analysis result, including:

[0024] determining the target straight line according to the current coordinates;

[0025] taking each of the target straight lines as a starting scanning line, gradually performing straight line pixel point scanning along a direction away from the center point of the current bounding box and parallel to the target straight line to obtain a pixel line;

[0026] stopping the straight line pixel point scanning when it is first determined that the pixel line does not include the first pixel point in the process of the straight line pixel point scanning;

[0027] regarding a region including the pixel line of the first pixel point as the partial region to obtain the analysis result.

[0028] Optionally, adjusting the current bounding box according to the analysis result to obtain a target bounding box, including:

[0029] regarding the pixel line corresponding to the stopping of the straight line pixel point scanning as a target pixel line;

[0030] controlling each boundary of the current bounding box to perform translation with the respective target pixel line as a target to crop the non-picture region and / or expand the current bounding box to obtain the target bounding box.

[0031] Optionally, constructing the picture detection module includes the following steps:

[0032] obtaining a target page picture file and constructing an initial picture detection model;

[0033] performing data enhancement processing on the target page picture file to obtain an enhanced data set; wherein the data enhancement processing at least includes at least one of Gaussian blur processing and color change processing;

[0034] obtaining a labeled data set after performing picture labeling on the enhanced data set;

[0035] performing iterative training on the initial picture detection model through the labeled data set until an iteration condition is reached to obtain the picture detection model.

[0036] Another aspect of the present application provides a picture detection device, the device comprising:

[0037] a file obtaining module for obtaining a page picture file to be detected;

[0038] a picture inference module, configured to perform inference on a target picture in the page picture file by a pre-constructed picture detection model to determine a current coordinate of a current bounding box;

[0039] a color feature analysis module, configured to perform color feature analysis on a current region framed by the current bounding box based on the current coordinate to obtain an analysis result;

[0040] a current bounding box adjustment module, configured to adjust the current bounding box according to the analysis result to obtain a target bounding box; the bounding box is a bounding box used for framing the target picture;

[0041] a target picture determination module, configured to frame a target region framed by the target bounding box as the target picture; the target region does not include a non-picture region and includes all regions of the target picture.

[0042] Another aspect of the present application provides an electronic device, including a memory and a processor, the memory has a computer program stored thereon and executable on the processor, and the processor implements the steps of the picture detection method when executing the computer program.

[0043] Another aspect of the present application provides a computer readable storage medium, having a computer program stored thereon, and the program is executed by a processor to implement the steps of the picture detection method.

[0044] The picture detection method, device, electronic device and medium provided by the present application have the beneficial effects that: based on the initial picture inference of the picture detection model, the color feature analysis is used to quickly identify the effective region and the redundant region of the picture, and the target picture is accurately framed by adjusting the current bounding box, so as to improve the detection accuracy of the picture in the page picture file, overcome the problem of inaccurate detection of the picture bounding box by the OCR technology, and meet the demand of the high-reliability business scenario. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A flowchart of a picture detection method provided by an embodiment of the present application;

[0046] Figure 2 A picture unit diagram obtained by a picture detection model provided by an embodiment of the present application;

[0047] Figure 3 A result diagram of picture detection provided by an embodiment of the present application;

[0048] Figure 4 A result diagram of picture detection provided by another embodiment of the present application;

[0049] Figure 5A result diagram of picture detection provided by another embodiment of the present application is shown in the figure;

[0050] Figure 6 A principle diagram of a picture detection method provided by an embodiment of the present application is shown in the figure;

[0051] Figure 7 A structure diagram of a picture detection device provided by an embodiment of the present application is shown in the figure;

[0052] Figure 8 A structure diagram of an electronic device provided by an embodiment of the present application is shown in the figure.

[0053] The reference signs are as follows: 80 is a memory, 81 is a processor, 82 is a display screen, 83 is an input / output interface, 84 is a communication interface, 85 is a power supply, 86 is a communication bus, 801 is a computer program, 802 is an operating system, and 803 is data. DETAILED DESCRIPTION

[0054] The terms used in the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a," "an," and "the" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, third, etc. can be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0056] Figure 1 A flow diagram of a picture detection method provided by an embodiment of the present application is shown in the figure, Figure 1 as shown, the method comprises:

[0057] S10: obtaining a page picture file to be detected;

[0058] as Figure 1As shown, in an optional embodiment, when the page picture file is acquired, the initially acquired to-be-detected object can be a file in a non-picture format such as PDF, and at this time, the to-be-detected object needs to be converted in format, that is, converted into a page file in a picture format (that is, a page picture file), which can be directly input to a subsequent picture detection model for inference. The application does not limit the format of the initially acquired file, and does not limit the file format conversion method.

[0059] S11: determining the current coordinates of the current bounding box by performing inference on the target picture in the page picture file through the pre-constructed picture detection model;

[0060] In specific embodiments, in order to realize fast and accurate identification of pictures, a picture detection model can be pre-constructed, and inference can be performed on the target picture in the page picture file through the picture detection model, so as to obtain an output picture unit in which at least part of the target picture is framed by the current bounding box. At the same time, the current coordinates of the current bounding box can be determined according to the picture unit and the current bounding box.

[0061] The picture unit initially output by the picture detection model includes the target picture, the current bounding box, and a non-picture region. The current bounding box is used to frame the target picture, and the current bounding box can frame the target picture completely or only frame part of the target picture.

[0062] It should be noted that in specific embodiments, the current bounding box can be in different shapes such as a matrix or a circle, and the application does not limit this. In addition, it should be noted that the target picture obtained by inference of the picture detection model can include multiple target pictures, that is, the picture unit can include multiple picture units, and the current bounding box can also include multiple current bounding boxes. The number of target pictures, picture units, and current bounding boxes is the same, and they are in a one-to-one correspondence.

[0063] Figure 2 A picture unit obtained by inference of a picture detection model provided in an embodiment of the application is shown in the following figure for the convenience of understanding, which will be described in combination with Figure 2 For example, in an optional embodiment, the current bounding box is a rectangle. After inference of the page picture file by the picture detection model, the output picture unit is shown in the following figure. Figure 2

[0064] The area framed by the outermost black line is a picture area, the target picture is the area included by the blue line, that is, the picture included in the page picture file, and the current bounding box is the area framed by the green line. The picture area includes the target picture and the current bounding box, and also includes a non-picture area, that is, the area between the blue line and the black line.

[0065] ​In specific embodiments, a two-dimensional coordinate system can be established for the picture unit. After the current bounding box is obtained, the current coordinate of the current bounding box can be determined. It can be understood that the current coordinate obtained can determine the position of the current bounding box. At the same time, the relationship between the region framed by the current bounding box and the target picture can also be determined, that is, the relationship includes partial overlap, complete inclusion, and complete overlap.

[0066] In an optional embodiment, the current coordinate can include the coordinate of any point on the current bounding box, or the coordinates of the four vertices of the current bounding box as shown in the figure, and the present application is not limited thereto. Figure 2

[0067] S12: Based on the current coordinate, color feature analysis is performed on the current region framed by the current bounding box to obtain an analysis result.

[0068] S13: According to the analysis result, the current bounding box is adjusted to obtain a target bounding box; the bounding box is a bounding box for framing the target picture.

[0069] It can be understood that after inference by the picture detection model, the region framed by the current bounding box can include a non-picture region, or only include part of the target picture (i.e., does not include part of the target picture), that is, the target picture is not completely framed.

[0070] At this time, the boundary of the current bounding box needs to be adjusted to obtain a target bounding box that completely frames the target picture. The target region framed by the target bounding box does not include a non-picture region and includes all regions of the target picture.

[0071] In an optional embodiment, as shown in the figure, the pixel color of the target picture is different from the pixel color of the non-picture region. Therefore, when adjusting the current bounding box, color feature analysis can be performed on the current region framed by the current bounding box, that is, the pixel color in the current region is analyzed, the non-picture region and the part of the target picture that is not framed are identified according to the pixel color, and an analysis result is obtained. Further, according to the analysis result, the boundary of the current bounding box is adjusted to obtain a target bounding box. Figure 2 S14: The target region framed by the target bounding box is taken as the target picture; the target region does not include a non-picture region and includes all regions of the target picture.

[0072] Finally, the target region framed by the target bounding box is output as the target picture. As shown in the figure, through pixel color analysis, the green bounding box is adjusted to a black bounding box that frames the target picture, that is, the boundary of the green bounding box is adjusted to obtain a black bounding box, and the black bounding box is the target bounding box.

[0073] Figure 2 ​​​

[0074] Therefore, the image detection method provided in this application, based on the initial image inference of the image detection model, quickly identifies the effective and redundant areas of the image based on color feature analysis, and accurately selects the target image by adjusting the current bounding box, thereby improving the detection accuracy of images in page image files, overcoming the problem of inaccurate image bounding box detection by OCR technology, and thus meeting the needs of high-reliability business scenarios.

[0075] Figure 3 This is a schematic diagram of an image detection result provided in an embodiment of this application. In an optional embodiment, if the current region includes a non-image region but does not include a portion of the target image region; adjusting the current bounding box to obtain the target bounding box includes:

[0076] Based on the analysis results, the boundaries of the current bounding box are reduced to crop non-image areas, resulting in a sub-bounding box;

[0077] The sub-boundary box is used as the current bounding box, and the color feature analysis of the current region selected by the current bounding box is performed based on the current coordinates to obtain the analysis results.

[0078] Based on the analysis results, the boundaries of the current bounding box are expanded to select a portion of the region, thus obtaining the target bounding box.

[0079] In a specific embodiment, such as Figure 2 As shown, the current region selected by the current bounding box obtained from the initial inference of the image inference model may include non-image regions. These non-image regions include blank areas and text regions, such as... Figure 2 The green bounding box (current bounding box) selects the blank area and " Figure 1 The text area for "-A part". Additionally, such as Figure 2 The current area may not include a portion of the target image, namely the area between the green and blue bounding boxes.

[0080] In one optional embodiment, if the current region includes non-image regions but does not include a portion of the target image. For example... Figure 3 As shown, for image unit 1, color feature analysis of the current region determines that the current region includes non-image areas. Therefore, it is necessary to shrink the current bounding box to crop the non-image areas, resulting in a sub-bounding box, which corresponds to image unit 2.

[0081] Furthermore, if the sub-boundary is used as the current bounding box, then the area selected by the sub-boundary is considered the current region. Similarly, color feature analysis is performed on the current region. Based on the analysis results, it is determined that the current bounding box does not include a portion of the target image.

[0082] At this time, asFigure 3 As shown, it is needed to expand the sub-boundary box in the picture unit 2, that is, to expand the boundary of the current boundary box, so as to frame the target picture part region which is not currently framed, so as to obtain the target boundary box in the picture unit 3.

[0083] It should be noted that in specific embodiments, if the current region includes non-picture region and does not include the part region of the target picture, it is represented that the current boundary box needs to be reduced and expanded, that is, the non-picture region needs to be cropped and the part region which is not framed needs to be expanded. The cropping can be performed first and then the expansion can be performed, or the expansion can be performed first and then the cropping can be performed, which is not limited by the present application.

[0084] Of course, it can be understood that if the expansion is performed first, the current boundary box may also frame the extra non-picture region, for example, frame more text regions, which will cause the increase of data calculation amount when cropping, resulting in the increase of computing resource cost. Therefore, in an optional embodiment, if the current region includes non-picture region and does not include the part region of the target picture, the cropping can be performed first to remove the extra non-picture region, and then the expansion can be performed, so as to improve the resource utilization.

[0085] In addition, it should be noted that if the current region only includes non-picture region, it is represented that only the cropping of the non-picture region is needed, and the sub-boundary box obtained after the cropping can be used as the target boundary box. If the current region does not include non-picture region and does not include the part region of the target picture, it is represented that only the expansion of the current boundary box is needed, and the target boundary box can be obtained by expanding the current boundary box based on the current boundary box output by the picture detection model.

[0086] Figure 4 A picture detection result schematic diagram provided by another embodiment of the present application, in an optional embodiment, the target picture includes first pixel points, the non-picture region includes second pixel points, the colors of the first pixel points and the second pixel points are different;

[0087] The boundary box is a rectangle; in a pre-constructed coordinate system, the current coordinates include a maximum coordinate point (X1, Y1) and a minimum coordinate point (X2, Y2), wherein X1 is greater than X2, and Y1 is greater than Y2;

[0088] The target straight line where the boundary of the current boundary box is located includes a first straight line Y=Y1, a second straight line Y=Y2, a third straight line X=X1 and a fourth straight line X=X2.

[0089] It can be understood that, based on the color feature analysis of the current region, the cropping of the non-picture region and the partial region framing of the target picture, a comparison needs to be made according to the color features of the target picture and the color features of the non-picture region. Specifically, the first pixel point of the target picture is compared with the second pixel point of the non-picture region, and the colors of the first pixel point and the second pixel point are different.

[0090] In an optional embodiment, based on the first pixel point and the second pixel point, the picture unit is scanned by pixel points, so that it can be quickly determined whether the current region includes the non-picture region. Specifically, if the current region includes the second pixel point, it is determined that the non-picture region is included.

[0091] In addition, it can also be determined whether the partial region of the target picture is not included. Specifically, if the first pixel point is included in the region framed by the current non-current boundary box in any one scanning, it is determined that the current region does not include the partial region of the target picture.

[0092] In an optional embodiment, in order to facilitate calculation and improve picture detection accuracy, as shown in Figure 4 , the boundary box is a rectangle. On this basis, as an optional embodiment, a two-dimensional coordinate system is constructed in advance, for example, as shown in Figure 4 , the arrow to the right is the X-axis direction, and the arrow downward is the Y-axis direction. The application does not limit the coordinate system constructed in advance. It should be noted that the application also does not limit the origin of the coordinate system. In an optional embodiment, the center point of the picture unit can be used as the center to construct.

[0093] In this coordinate system, since the boundary box is a rectangle, the position of the current boundary box and the size of the current region framed by the current boundary box can be determined by obtaining the maximum point coordinate and the minimum point coordinate. Therefore, as shown in Figure 4 , the current coordinate of the current boundary box includes the maximum coordinate point (X1, Y1) and the minimum coordinate point (X2, Y2), wherein X1 is greater than X2, and Y1 is greater than Y2. The target straight line on which the boundary of the current boundary box is located includes the first straight line Y=Y1, the second straight line Y=Y2, the third straight line X=X1 and the fourth straight line X=X2.

[0094] Therefore, the target straight line can be determined by the current coordinate, so as to determine the position and area size of the current region. At the same time, based on the rectangular boundary box, the maximum coordinate point and the minimum coordinate point, the calculation amount of picture detection can be reduced, and the detection effect can be improved.

[0095] On the basis of the above-mentioned embodiments, as an optional embodiment, if the current region includes the non-picture region; based on the current coordinate, the color feature analysis is performed on the current region framed by the current boundary box to obtain an analysis result, including:

[0096] determining a target straight line according to the current coordinates;

[0097] taking each target straight line as a starting scanning line, gradually performing straight line pixel scanning along a direction approaching a center point of the current bounding box and parallel to the target straight line, and obtaining a pixel line;

[0098] in the straight line pixel scanning process, when the pixel line is first determined to include the first pixel point, stopping the straight line pixel scanning;

[0099] taking a current region corresponding to the pixel line including the second pixel point as a non-picture region, and obtaining an analysis result.

[0100] As shown in Figure 4 , four target straight lines can be obtained according to the current coordinates of the current bounding box. The target straight line is taken as a starting scanning line, and gradually runs in the direction of the center point of the current bounding box to perform straight line pixel scanning. That is, the boundary of the current bounding box is taken as a starting point, and scanning is performed row by row inward to the center point of the current bounding box. For example, Figure 4 , the first straight line Y=Y1 in the current bounding box in the upper left of the figure is taken as a starting scanning line, and scanning is gradually performed upward. Similarly, the other bounding boxes also perform scanning row by row inward.

[0101] In the scanning process, it can be understood that the color of the second pixel point of the non-picture region is different from the color of the first pixel point of the target picture. Therefore, if the pixel line obtained by scanning includes the second pixel point, it indicates that the current scanning has reached the non-picture region, and scanning can be continued downward.

[0102] If the pixel line first includes the first pixel point, it indicates that the target picture region will be entered, that is, the current scanning has reached the boundary of the target picture. At this time, the straight line pixel scanning can be stopped. At the same time, the current region corresponding to the pixel line including the second pixel point in the pixel line obtained by scanning before stopping the straight line pixel scanning is taken as a non-picture region. That is, in the current region of the current bounding box, the scanned region, the region including the second pixel point is taken as a non-picture region.

[0103] For example, as shown in Figure 4 , the first straight line Y=Y1 is taken as a starting scanning line, and scanning is gradually performed upward to obtain the scanning result in the lower figure. Specifically, the purple first straight line Y=Y1 is scanned and moved to the lower boundary of the target picture, and the region of the red dashed line box scanned through is a non-picture region.

[0104] On the basis of the above embodiment, that is, on the basis that the current region includes a non-picture region, as an optional embodiment, according to the analysis result, the current bounding box is adjusted to obtain a target bounding box, including:

[0105] The pixel line corresponding to the point where the straight-line pixel scanning stops is taken as the target pixel line;

[0106] Control each boundary of the current bounding box and translate it with its parallel target pixel line as the target to crop non-image areas and obtain the target bounding box.

[0107] When the scanning of linear pixels stops, it indicates that the current scan has reached the boundary of the target image, and this boundary can be used as the boundary of the adjusted bounding box. For example, as... Figure 4 In the lower part of the image, the first purple line Y=Y1 represents the boundary of the target image. Therefore, the lower green bounding box can be moved to a position that overlaps with the first purple line Y=Y1, which will serve as the new lower boundary of the current bounding box.

[0108] In other words, after the scan is completed, the pixel line corresponding to the point where the straight line pixel scanning stops is taken as the target. Each current bounding box is translated towards the target pixel line parallel to itself until they overlap, thus obtaining a new current bounding box, that is, obtaining the target bounding box.

[0109] It is understandable that when the boundary of the current bounding box is translated to the boundary of the corresponding target bounding box, the non-image area is cropped, that is, the adjustment of the current bounding box is completed.

[0110] For example, in Figure 4 In the example shown, the first pixel is a non-black and white color, and the second pixel is both black and white. Therefore, the non-image area includes blank areas (where the second pixel is white) and text areas (where the second pixel is black). In a specific embodiment, when scanning with straight pixel lines, the current area corresponding to a pixel line consisting entirely of white pixels or including both black and white pixels can be considered the non-image area.

[0111] It should be noted that, as Figure 4 As shown, the purple first straight line Y=Y1 gradually scans upwards, passing through blank areas, text areas, and secondary blank areas. Therefore, in one optional embodiment, the non-image area includes blank areas and text areas. During scanning and cropping, the entire non-image area can be scanned before cropping. Alternatively, the blank areas can be scanned and cropped first, then the text areas can be cropped, and finally, a secondary cropping of the blank areas can be performed. This application does not limit this approach.

[0112] Figure 5 This application also provides a schematic diagram of an image detection result according to an embodiment. Based on the above embodiment, as an optional embodiment, if the current region does not include a certain region; based on the current coordinates, color feature analysis is performed on the current region selected by the current bounding box to obtain the analysis result, including:

[0113] determining a target straight line according to the current coordinate;

[0114] taking each target straight line as a starting scanning line, gradually performing straight line pixel scanning along a direction away from a center point of the current bounding box and parallel to the target straight line, and obtaining a pixel line;

[0115] in the straight line pixel scanning process, when it is first determined that the pixel line does not include the first pixel point, stopping the straight line pixel scanning;

[0116] taking a region of the pixel line including the first pixel point as a partial region, and obtaining an analysis result.

[0117] In a specific embodiment, if the current region does not include the partial region of the target picture, it indicates that the current bounding box is too small and does not completely frame the entire target picture. At this time, in an optional embodiment, the starting scanning line is also taken as the target straight line, and the starting scanning line is translated and scanned along a direction away from the center point of the current bounding box. That is, taking the boundary of the current bounding box as a starting point, the scanning is performed row by row outward away from the center point of the current bounding box.

[0118] For example, as shown in the upper left figure of FIG. 1, the green current bounding box does not completely frame the entire target picture. At this time, the left boundary of the current bounding box needs to be moved to the left, the right boundary of the current bounding box needs to be moved to the right, and the upper boundary of the current bounding box needs to be moved upward, so as to obtain the lower figure of FIG. 1, that is, the target bounding box completely frames the entire target picture. Figure 5 Figure 5

[0119] Specifically, when expanding the current bounding box, as shown in the upper left figure of FIG. 1, the third straight line X=X1 is taken as the starting scanning line, the starting scanning line is parallel to the third straight line X=X1, and the starting scanning line is translated and scanned gradually in a direction toward the center point of the original boundary. Figure 5

[0120] Further, based on the color difference between the first pixel point and the second pixel point, if the first pixel point is included in the pixel line obtained by scanning, it indicates that the scanning passes through the picture region not framed by the current bounding box, and at this time, the scanning can be continued.

[0121] If the first pixel point is not included in the pixel line in the scanning process, that is, the pixel line is composed of the second pixel point, it indicates that the scanning reaches the boundary of the target picture, and the straight line pixel scanning is stopped. At this time, a region of the pixel line including the first pixel point in the scanned pixel line is taken as a partial region, and an analysis result is obtained.

[0122] On the basis of the above embodiment, that is, on the basis that the current region does not include the partial region, as an optional embodiment, according to the analysis result, the current bounding box is adjusted to obtain a target bounding box, including: ​​​

[0123] the pixel line corresponding to the stopping of the straight line pixel point scanning is taken as a target pixel line;

[0124] The boundaries of the current bounding box are controlled to be translated along the target pixel line parallel to each boundary, so as to expand the current bounding box and obtain a target bounding box.

[0125] When the straight line pixel point scanning is stopped, it is indicated that the current scanning has reached the boundary of the target picture, that is, the boundary can be taken as the boundary of the adjusted bounding box. For example, as shown in the lower part of the figure, the purple third straight line X=X1 is the boundary of the target picture, so the green right bounding box can be moved to the position overlapping the purple third straight line X=X1 as the right boundary of the new current bounding box. Figure 5

[0126] That is, after the scanning is completed, the pixel line corresponding to the stopping of the straight line pixel point scanning is taken as a target, and each current bounding box is translated along the target pixel line parallel to itself until overlapping, so as to obtain a new current bounding box, that is, a target bounding box. It can be understood that the boundary of the current bounding box is translated to the boundary of the corresponding target bounding box, so that the boundary of the current bounding box is expanded.

[0127] For example, in the example shown in Figure 5 , the color of the first pixel point is a non-black-and-white color, and the second pixel point is black and white. Thus, the current bounding box is the first pixel point in the color-enclosed partial region, and in specific embodiments, by straight line pixel point scanning, the pixel line region including the color pixel point can be taken as a partial region of the target picture, so as to adjust the boundary of the current bounding box to frame the corresponding region, so as to obtain a target bounding box for framing the entire target picture.

[0128] The non-picture region includes a blank region (the second pixel point is white) and a text region (the second pixel point is black). In specific embodiments, by straight line pixel point scanning, the current region corresponding to the pixel line which is all white pixel points or includes black pixel points and white pixel points can be taken as a non-picture region.

[0129] In an optional embodiment, the picture detection module is constructed by the following steps:

[0130] obtaining a target page picture file and constructing an initial picture detection model;

[0131] performing data enhancement processing on the target page picture file to obtain an enhanced data set; wherein the data enhancement processing at least includes at least one of Gaussian blur processing and color change processing;

[0132] obtaining a labeled data set after labeling the enhanced data set; ​

[0133] The initial picture detection model is iteratively trained through the labeled data set until an iteration condition is reached, and a picture detection model is obtained.

[0134] It can be understood that if the detection accuracy of the picture detection model is improved, the subsequent adjustment efficiency of the current bounding box can be improved, and the calculation resources can be saved. Therefore, in an optional embodiment, an initial picture detection model is pre-constructed, and a target page picture file is obtained. The target page picture file is used to train the initial picture detection model, and the target page picture file can be part of the page picture file to be detected or data selected by a user for model training. The present application does not limit this.

[0135] In order to further improve the model accuracy, the quality of the training data can be improved, specifically, in an optional embodiment, the target page picture file is subjected to data enhancement processing such as Gaussian blur processing and color change processing, thereby obtaining an enhanced data set.

[0136] Further, the pictures in the enhanced data set can be labeled by a human, thereby obtaining a labeled data set. Thus, the initial picture detection model can be iteratively trained through the labeled data set, and a picture detection model that can be directly used for picture detection is obtained.

[0137] In an optional embodiment, the labeled data set can be divided into a training data set, a validation data set and a test data set according to a preset ratio. The training data set is used to input into the initial picture detection model to iteratively train the model to update the parameters of the initial picture model.

[0138] At the same time, in the training process, the performance of the model is evaluated through the validation data set, and the hyperparameters are adjusted according to the evaluation result or until the iteration condition is met. The iteration condition can be a preset iteration number or model convergence, and the present application does not limit this. After the model training is completed, the performance of the trained model is evaluated through the test data set to determine the generalization ability of the model, thereby obtaining a final picture detection model.

[0139] Figure 6 A schematic diagram of the principle of a picture detection method provided by the embodiments of the present application. In order to make those skilled in the art more clearly understand the technical solutions of the present application, the following will be further described in combination with Figure 6 The specific implementation of the present application will be further described in combination with the accompanying drawings.

[0140] As Figure 6As shown, in specific embodiments, an image file to be detected is acquired, which can be in a file format such as PDF and cannot be directly input into the image detection model for training. Therefore, it is necessary to first determine whether the currently acquired image file to be detected is in a page image format. If so, the image detection model can be directly input for inference. If not, format conversion is needed to obtain a page image file that can be directly input into the image detection model.

[0141] Further, after inference by the image detection model, the image unit, the current bounding box for framing the target image, and the current coordinates of the current bounding box can be obtained. According to the current coordinates, color feature analysis is performed on the initial region framed by the current bounding box, so as to determine whether the current bounding box needs to be adjusted.

[0142] It can be understood that if the current region framed by the current bounding box includes a non-image region or does not include a partial region of the target image, it indicates that the current bounding box does not completely frame the target image, and the boundary of the bounding box needs to be adjusted. The determination of whether the current region includes a non-image region or does not include a partial region of the target image can be achieved by pixel scanning as in the above embodiments.

[0143] Further, as shown, Figure 6 if the current region includes a non-image region or does not include a partial region of the target image, the current bounding box needs to be adjusted. Specifically, the adjustment operation includes at least one of cropping the non-image region and framing the partial region. After adjustment, a target bounding box is obtained, and the region framed by the target bounding box is output as the target image.

[0144] In the above embodiments, the image detection method is described in detail, and the present application also provides an embodiment of an image detection device.

[0145] Figure 7 The structure diagram of an image detection device provided by the embodiments of the present application is shown in FIG. 7. The device includes: Figure 7

[0146] A file acquisition module 70 is configured to acquire a page image file to be detected.

[0147] An image inference module 71 is configured to perform inference on a target image in the page image file by a pre-constructed image detection model to determine the current coordinates of the current bounding box.

[0148] A color feature analysis module 72 is configured to perform color feature analysis on a current region framed by the current bounding box based on the current coordinates to obtain an analysis result.

[0149] ​The current bounding box adjusting module 73 is configured to adjust the current bounding box according to the analysis result to obtain a target bounding box; the bounding box is a bounding box used for bounding the target picture;

[0150] The target picture determining module 74 is configured to take a target region bounded by the target bounding box as the target picture; the target region does not include a non-picture region and includes all regions of the target picture.

[0151] In addition, the picture detection apparatus provided in the embodiment of the present application further includes:

[0152] The cropping module is configured to reduce the boundary of the current bounding box to crop the non-picture region and obtain a sub-bounding box according to the analysis result;

[0153] The expanding module is configured to take the sub-bounding box as the current bounding box, perform color feature analysis on a current region bounded by the current bounding box based on the current coordinates to obtain the analysis result, and expand the boundary of the current bounding box to bound a partial region to obtain the target bounding box according to the analysis result.

[0154] The target straight line determining module is configured to determine the target straight line according to the current coordinates;

[0155] The first pixel scanning module is configured to take each target straight line as a starting scanning line, gradually perform straight line pixel point scanning along a direction close to a center point of the current bounding box and parallel to the target straight line, and obtain a pixel line;

[0156] The first control module is configured to stop the straight line pixel point scanning when it is first determined that the pixel line includes the first pixel point in the process of the straight line pixel point scanning;

[0157] The first determining module is configured to take a current region corresponding to the pixel line including the second pixel point as the non-picture region to obtain the analysis result.

[0158] The second pixel scanning module is configured to take each target straight line as a starting scanning line, gradually perform straight line pixel point scanning along a direction away from a center point of the current bounding box and parallel to the target straight line, and obtain a pixel line;

[0159] The second control module is configured to stop the straight line pixel point scanning when it is first determined that the pixel line includes the first pixel point in the process of the straight line pixel point scanning;

[0160] The second determining module is configured to take a region of the pixel line including the first pixel point as the partial region to obtain the analysis result.

[0161] The target pixel line determining module is configured to take the pixel line corresponding to the stopping of the straight line pixel point scanning as the target pixel line;

[0162] The boundary adjusting module is configured to control each boundary of the current boundary box to be translated along a target pixel line parallel to the each boundary, so as to crop a non-picture area and / or expand the current boundary box, and obtain a target boundary box.

[0163] The target acquisition module is configured to acquire a target page picture file and construct an initial picture detection model.

[0164] The data enhancement module is configured to perform data enhancement processing on the target page picture file to obtain an enhanced data set, wherein the data enhancement processing at least includes at least one of Gaussian blur processing and color change processing.

[0165] The labeled data set acquisition module is configured to acquire a labeled data set after picture labeling on the enhanced data set.

[0166] The iterative training module is configured to perform iterative training on the initial picture detection model through the labeled data set until an iteration condition is reached, and obtain a picture detection model.

[0167] Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1. Figure 8 As shown in FIG. 1, the electronic device includes a memory 80 configured to store a computer program.

[0168] A processor 81 is configured to implement steps of the picture detection method mentioned in the above embodiments when executing the computer program.

[0169] The electronic device provided by the embodiment can include but is not limited to a notebook computer or a desktop computer, etc.

[0170] The processor 81 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 81 can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), etc. The processor 81 can also include a main processor and a co-processor. The main processor is a processor for processing data in a wake-up state, also referred to as a central processing unit (CPU). The co-processor is a low-power processor for processing data in a standby state. In some embodiments, the processor 81 can be integrated with a graphics processor (GPU) for rendering and drawing content to be displayed by the display screen. In some embodiments, the processor 81 can further include an artificial intelligence (AI) processor for processing machine learning-related computing operations.

[0171] The memory 80 can include one or more computer-readable storage media that can be non-transitory. The memory 80 can further include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In this embodiment, the memory 80 is at least used to store the following computer program 801, wherein the computer program is loaded and executed by the processor 81, and can implement the related steps of the picture detection method disclosed in any of the preceding embodiments. In addition, the resources stored in the memory 80 can further include an operating system 802 and data 803, etc., and the storage mode can be temporary storage or permanent storage. The operating system 802 can include Windows, Unix, Linux, etc. The data 803 can include, but is not limited to, related data involved in the picture detection method, etc.

[0172] In some embodiments, the electronic device can further include a display screen 82, an input / output interface 83, a communication interface 84, a power supply 85, and a communication bus 86.

[0173] Those skilled in the art can understand that, Figure 8 The structure shown in the figure does not constitute a limitation on the electronic device, and can include more or fewer components than those shown.

[0174] The electronic device provided by the embodiment of the present application comprises a memory and a processor. The processor can realize the picture detection method in the above embodiment when executing the program stored in the memory.

[0175] It should be noted that, although the operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring the operations to be performed in the particular order shown or sequentially, or requiring all of the illustrated operations to be performed to achieve a desired result. In some cases, multi-tasking and parallel processing can be advantageous. In addition, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A picture detection method, characterized by, The method comprises: acquiring a page picture file to be detected; performing inference on a target picture in the page picture file by a pre-constructed picture detection model to determine a current coordinate of a current bounding box; performing color feature analysis on a current region framed by the current bounding box based on the current coordinate to obtain an analysis result; adjusting the current bounding box according to the analysis result to obtain a target bounding box; the bounding box is a frame for framing the target picture; framing a target region framed by the target bounding box as the target picture; the target region does not include a non-picture region and includes all regions of the target picture; the target picture includes a first pixel point, the non-picture region includes a second pixel point, and the first pixel point and the second pixel point are different in color; the bounding box is a rectangle; in a pre-constructed coordinate system, the current coordinate includes a maximum coordinate point (X1, Y1) and a minimum coordinate point (X2, Y2), wherein X1 is greater than X2, and Y1 is greater than Y2; a target straight line on which a boundary of the current bounding box is located includes a first straight line Y=Y1, a second straight line Y=Y2, a third straight line X=X1, and a fourth straight line X=X2; if the current region includes the non-picture region; the color feature analysis on the current region framed by the current bounding box based on the current coordinate to obtain an analysis result comprises: determining the target straight line according to the current coordinate; taking each target straight line as a starting scanning line, gradually performing straight-line pixel point scanning in a direction approaching a center point of the current bounding box and parallel to the target straight line to obtain a pixel line; stopping the straight-line pixel point scanning when it is first determined that the pixel line includes the first pixel point in the straight-line pixel point scanning process; taking a current region corresponding to a pixel line including the second pixel point as the non-picture region to obtain the analysis result; if the current region does not include a part of the target picture; the color feature analysis on the current region framed by the current bounding box based on the current coordinate to obtain an analysis result comprises: determining the target straight line according to the current coordinate; taking each target straight line as a starting scanning line, gradually performing straight-line pixel point scanning in a direction away from a center point of the current bounding box and parallel to the target straight line to obtain a pixel line; stopping the straight-line pixel point scanning when it is first determined that the pixel line does not include the first pixel point in the straight-line pixel point scanning process; taking a region of a pixel line including the first pixel point as the part of the target picture to obtain the analysis result.

2. The picture detection method of claim 1, wherein, if the current region includes the non-picture region and does not include the part of the target picture; the adjusting the current bounding box to obtain a target bounding box comprises: reducing the boundary of the current bounding box to crop the non-picture region according to the analysis result to obtain a sub-bounding box; taking the sub-bounding box as the current bounding box and performing the color feature analysis on the current region framed by the current bounding box based on the current coordinate to obtain an analysis result. According to the analysis result, the boundary of the current bounding box is extended to frame the partial region, and a target bounding box is obtained.

3. The picture detection method of claim 1, wherein, The adjusting the current bounding box according to the analysis result to obtain a target bounding box comprises: The pixel line corresponding to the stop of the straight line pixel point scanning is taken as a target pixel line; The boundaries of the current bounding box are controlled to translate along the target pixel lines parallel to each other, so as to crop the non-picture region and / or extend the current bounding box, and obtain the target bounding box.

4. The picture detection method of claim 1, wherein, The construction of the picture detection module comprises the following steps: An initial picture detection model is constructed by the following steps: An initial picture detection model is constructed by the following steps: The data enhancement processing at least comprises at least one of a Gaussian blur processing and a color change processing. An initial picture detection model is constructed by the following steps:

5. An image detecting apparatus characterized by comprising: The device comprises: A file acquisition module is configured to acquire a page picture file to be detected. A picture inference module is configured to infer a target picture in the page picture file by using a pre-constructed picture detection model, so as to determine a current coordinate of a current bounding box. A color feature analysis module is configured to perform color feature analysis on a current region framed by the current bounding box based on the current coordinate, and obtain an analysis result. A current bounding box adjustment module is configured to adjust the current bounding box according to the analysis result, and obtain a target bounding box. A target picture determination module is configured to take a target region framed by the target bounding box as the target picture. The target picture comprises a first pixel point, and the non-picture region comprises a second pixel point. The color of the first pixel point is different from that of the second pixel point. The current coordinate comprises a maximum coordinate point (X1, Y1) and a minimum coordinate point (X2, Y2) in a pre-constructed coordinate system, wherein X1 is greater than X2, and Y1 is greater than Y2. The boundaries of the current bounding box are on a target straight line, which comprises a first straight line Y=Y1, a second straight line Y=Y2, a third straight line X=X1, and a fourth straight line X=X2. If the current region comprises the non-picture region, a target straight line determination module, a first pixel scanning module, a first control module, and a first determination module are sequentially called. The target straight line determination module is configured to determine the target straight line according to the current coordinate. The first pixel scanning module is configured to take each target straight line as a starting scanning line, and gradually perform straight line pixel point scanning along a direction close to a center point of the current bounding box and parallel to the target straight line, so as to obtain a pixel line. The first control module is configured to stop the straight line pixel point scanning when the pixel line first comprises the first pixel point during the straight line pixel point scanning. The first determining module is configured to take a current region corresponding to a pixel line including the second pixel point as the non-picture region, and obtain the analysis result. If the current region does not include a partial region of the target picture, the target straight line determining module, the second pixel scanning module, the second control module and the second determining module are sequentially called. The second pixel scanning module is configured to take each target straight line as a starting scanning line, and gradually perform straight line pixel point scanning along a direction away from a center point of the current bounding box and parallel to the target straight line, to obtain a pixel line. The second control module is configured to stop the straight line pixel point scanning when it is first determined that the pixel line does not include the first pixel point during the straight line pixel point scanning. The second determining module is configured to take a region of a pixel line including the first pixel point as the partial region, and obtain the analysis result.

6. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program operable to run on said processor, characterized in that, The processor executes the computer program to implement the steps of the picture detection method in any one of claims 1 to 4.

7. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the picture detection method in any one of claims 1 to 4.

Images