Region identification method and device for image, medium and equipment

Abstract

The invention provides a region recognition method for an image, a region recognition device for the image, a computer readable storage medium and electronic equipment, and relates to the technical field of image processing. The false filling area in the at least one reference filling area is filtered based on the contour detection result of the to-be-detected image, it is avoided that the false filling area is determined as the to-be-filled area, the problem of false detection of the to-be-filled area can be solved, and the detection precision of the to-be-filled area is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for identifying regions of an image, a device for identifying regions of an image, a computer-readable storage medium, and an electronic device. Background Art

[0002] In order to facilitate online business processing, most applications provide users with online form filling interfaces. For example, e-commerce platforms provide users with express delivery form filling interfaces, banking systems provide users with card opening information filling interfaces, medical procedures provide users with registration form filling interfaces, etc. After entering these interfaces, users can fill in various information using a virtual keyboard.

[0003] In order to save the user's time in filling out forms, the related art provides an automatic filling function, which is specifically implemented as follows: identifying the area to be filled based on the text information in the interface, and automatically filling the area according to the pre-stored data.

[0004] However, in actual application scenarios, the interfaces to be filled in usually vary greatly, and the areas to be filled in identified based on text information may not be accurate, which can easily lead to false detection problems.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute relevant technology known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present application is to provide a method for region recognition for an image, a device for region recognition for an image, a computer-readable storage medium and an electronic device, which can filter out false fill areas (such as areas that look like areas to be filled but are actually option areas) in at least one reference fill area in the image to be detected based on the contour detection result of the image to be detected after identifying at least one reference fill area in the image to be detected, so as to avoid determining false fill areas as areas to be filled, solve the problem of false detection of areas to be filled, and improve the detection accuracy of areas to be filled.

[0007] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0008] According to one aspect of the present application, a method for region recognition of an image is provided, the method comprising:

[0009] identifying at least one reference fill region in the image to be detected;

[0010] Perform contour detection on the image to be detected to obtain a contour detection result;

[0011] Based on the contour detection result, the false filling area in the at least one reference filling area is filtered to obtain at least one area to be filled.

[0012] According to one aspect of the present application, a region recognition device for an image is provided, the device comprising:

[0013] A region identification unit, used to identify at least one reference filling region in the image to be detected;

[0014] A contour detection unit, used for performing contour detection on the image to be detected to obtain a contour detection result;

[0015] The area filtering unit is used to filter the false filling area in at least one reference filling area based on the contour detection result to obtain at least one area to be filled.

[0016] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0017] According to one aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above methods is implemented.

[0018] According to one aspect of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above methods by executing the executable instructions.

[0019] The exemplary embodiments of the present application may have some or all of the following beneficial effects:

[0020] In a region recognition method for an image provided in an example embodiment of the present application, after identifying at least one reference fill region in the image to be detected, based on the contour detection result of the image to be detected, a pseudo-fill region (e.g., a region that looks like a region to be filled but is actually an option region) in at least one reference fill region can be filtered to avoid determining the pseudo-fill region as the region to be filled, thereby solving the problem of false detection of the region to be filled and improving the detection accuracy of the region to be filled. In addition, since the present application can improve the detection accuracy of the region to be filled, it can be beneficial to ensure the automatic filling accuracy of the region to be filled.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a method for region recognition of an image according to an embodiment of the present application is schematically shown;

[0024] Figure 2 A schematic diagram of an image / interface according to an embodiment of the present application is schematically shown;

[0025] Figure 3 A schematic diagram of a grayscale image according to an embodiment of the present application is schematically shown;

[0026] Figure 4 A first image schematic diagram according to an embodiment of the present application is schematically shown;

[0027] Figure 5 A schematic diagram of an image to be detected according to an embodiment of the present application is schematically shown;

[0028] Figure 6 A schematic diagram of the structure of an interface analysis model according to an embodiment of the present application is shown;

[0029] Figure 7 A schematic diagram of a contour image according to an embodiment of the present application is schematically shown;

[0030] Figure 8 A schematic diagram of a filtered contour image according to an embodiment of the present application is schematically shown;

[0031] Fig. 9 A schematic diagram of a reference filling area according to an embodiment of the present application is schematically shown;

[0032] Fig.10 A schematic diagram of a region to be filled according to an embodiment of the present application is shown schematically;

[0033] Fig.11 A flowchart of a method for region recognition of an image according to another embodiment of the present application is schematically shown;

[0034] Fig.12A schematic diagram of the structure of a region recognition device for an image according to an embodiment of the present application is shown;

[0035] Fig.13 The structure diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present application will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical solutions of the present application may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present application.

[0037] See also Figure 1 , Figure 1 The flowchart of the method for region recognition of an image according to an embodiment of the present application is schematically shown. Figure 1 As shown, the method includes the following steps.

[0038] Step S110: identifying at least one reference filling area in the image to be detected.

[0039] Step S120: performing contour detection on the image to be detected to obtain a contour detection result.

[0040] Step S130: based on the contour detection result, filtering out the pseudo-filled regions in at least one reference filled region to obtain at least one region to be filled.

[0041] Implementation Figure 1 The method shown can, after identifying at least one reference filling area in the image to be detected, filter out the false filling areas (such as areas that are shaped like areas to be filled but are actually option areas) in at least one reference filling area based on the contour detection result of the image to be detected, so as to avoid determining the false filling areas as areas to be filled, solve the problem of false detection of areas to be filled, and improve the detection accuracy of areas to be filled. In addition, since the present application can improve the detection accuracy of areas to be filled, it can be beneficial to ensure the automatic filling accuracy of areas to be filled.

[0042] Next, the above steps of this exemplary embodiment are described in more detail.

[0043] In step S110 , at least one reference filling region in the image to be detected is identified.

[0044] When performing automatic interface filling, the related technology usually recognizes the text in the image / interface (such as name, certificate type, ID number, mobile phone number, etc.), and if the text belongs to the area that needs to be filled (such as the area where the "name" is located), it will be automatically filled. However, the method of determining the area that needs to be filled based on text has the problem of low accuracy. The reason is that: the area to which the text in the image / interface belongs does not necessarily belong to the area to be filled, but may belong to the area to be selected; among which, the area to be selected (such as certificate type) can respond to the user trigger operation and display multiple options (such as ID card, passport, driver's license) for the user to choose, and the area to be filled (such as name) can respond to the automatic filling operation and display the information automatically filled in by the related technology (such as Zhang**). For details, please refer to Figure 2 , Figure 2 The schematic diagram of an image / interface according to an embodiment of the present application is schematically shown. According to the related art definition of identifying the area to be filled according to the text, it is easy to identify the area where the "document type" is located as the area to be filled. The incorrect identification of the area to be filled can easily affect the subsequent automatic filling process, resulting in filling errors or failure to fill.

[0045] In order to solve the problems existing in the related art, the present application proposes to screen at least one initially identified reference filling area, thereby improving the recognition accuracy of the area to be filled.

[0046] Specifically, it is possible to first identify whether there is at least one reference filling area in the image to be detected based on the interface analysis model. If not, the process is terminated; if so, at least one reference filling area in the identified image to be detected is obtained, and the reference filling area is a preliminary recognition result for the image to be detected; wherein, the areas between the reference filling areas may be different or the same, and the number of reference filling areas may also depend on actual conditions, which is not limited in the embodiments of the present application; the interface analysis model may divide the image to be detected into regions to determine the reference filling areas and the feature vectors of the reference filling areas; wherein the feature vectors may be expressed as [x, y, w, h, c], where x represents the horizontal coordinate of the upper left corner of the reference filling area, y represents the vertical coordinate of the upper left corner of the reference filling area, w represents the area width of the reference filling area, h represents the area height of the reference filling area, and c represents the confidence of the reference filling area. Optionally, the confidence value range may be (0 to 1).

[0047] In addition, optionally, before identifying at least one reference fill area in the image to be detected, it is also possible to: determine whether there is a specified image to be detected in response to an automatic fill request; if so, perform the above-mentioned identification of at least one reference fill area in the image to be detected; if not, take a snapshot of the current user interface to obtain the image to be detected, and the current user interface can be a software interface, a mini-program interface, etc.; wherein the automatic fill request can be generated based on touch screen operations, click operations, voice control operations, gesture control operations, and other operations.

[0048] Further, optionally, before taking a snapshot of the current user interface to obtain the image to be detected, it is also possible to: when it is detected that the current user interface corresponds to a split-screen scene, extract keywords from the current user interface to obtain keywords corresponding to each sub-interface (e.g., the keyword corresponding to sub-interface A is "video", and the keyword corresponding to sub-interface B is "table"); determine at least one sub-interface to be filled according to the keywords of each sub-interface; take snapshots of at least one sub-interface to be filled to obtain at least one image to be detected; and perform steps S210 to S230 for each image to be detected. Among them, the number of keywords for each sub-interface may be consistent or inconsistent, for example, the number of keywords corresponding to sub-interface A is 2, and the number of keywords corresponding to sub-interface B is 5.

[0049] In addition, optionally, in some non-split-screen scenarios, the user interface can respond to the interface sliding operation to display some previously undisplayed areas to be filled that are limited by the size of the display screen. That is to say, in this scenario, the area to be filled cannot be displayed at one time through the display screen. In order to save user operations and improve the efficiency of automatic filling, the above method can also include: responding to the automatic filling request, detecting the size of the current user interface. If the size is larger than the display screen size and there is a fillable area in the currently undisplayed interface content, a snapshot of the displayed interface content and the undisplayed interface content is taken. This can facilitate the one-time filling of a long interface, without having to complete the filling of a part of the interface, and then respond to the interface sliding operation to fill the remaining previously undisplayed interface.

[0050] In step S120, contour detection is performed on the image to be detected to obtain a contour detection result.

[0051] Specifically, contour detection can be implemented based on edge enhancement technology. Contour detection can highlight the text, icons, and symbols in the image to be detected. Filtering the reference fill area based on the contour detection result can improve the recognition accuracy of the area to be filled. One or more contours are recorded in the contour detection result.

[0052] In step S130, based on the contour detection result, the pseudo-filled regions in at least one reference filled region are filtered to obtain at least one region to be filled.

[0053] Specifically, the number of pseudo-filled areas in at least one reference filling area may be one or more, and a pseudo-filled area refers to an area that cannot be filled essentially. Based on the contour detection result, the pseudo-filled areas in at least one reference filling area are filtered to obtain at least one area to be filled, including: if the reference filling area contains one or more contours recorded by the contour detection result, the reference filling area is determined to be a pseudo-filled area, and each reference filling area is traversed by the above determination method until a pseudo-filled area is determined and filtered to obtain at least one area to be filled.

[0054] In addition, optionally, the functions defined in steps S110 to S130 of the present application can be encapsulated as independent modules in the system or software, eliminating the need for developers to develop separate filling functions for different interfaces, thereby saving development costs.

[0055] As an optional embodiment, identifying at least one reference filling area in the image to be detected includes: processing the image to be detected based on at least one of a normalization process of a graphic size and a normalization process of a pixel value; and identifying at least one reference filling area in the processed image to be detected. In this way, the image to be detected can be standardized through normalization processing, thereby reducing the difficulty of subsequent processing of the image to be detected and improving processing efficiency.

[0056] Specifically, in addition to the normalization of the image size and the normalization of the pixel value, the image to be detected can optionally be processed based on other normalizations, which are not limited in the embodiments of the present application. In addition, the normalization of the image size includes at least one of reduction processing, enlargement processing, and convolution processing; the normalization of the pixel value includes at least one of maximum value normalization processing and minimum value normalization processing, so that the pixel value range of the image to be detected is reduced to -1 to 1.

[0057] As an optional embodiment, the method further includes: converting the original image into a grayscale image; binarizing the grayscale image based on the first edge enhancement parameter to obtain a first image; and fusing the first image and the original image to obtain an image to be detected. In this way, the edge segmentation line in the image to be detected can be enhanced by grayscale and binarization, which is conducive to improving the detection accuracy of the area to be filled in the image to be detected.

[0058] Specifically, based on the edge enhancement module in the interface analysis model, the original image (e.g., RGB image) can be converted into Figure 3 The grayscale image shown in FIG. 1 is then binarized based on the first edge enhancement parameter (eg, 230) to obtain the grayscale image as shown in FIG. Figure 4 The first image shown is a fusion image of the original image to obtain the image to be detected.

[0059] As an optional embodiment, fusing the first image and the original image to obtain the image to be detected includes: stacking the color channels corresponding to the same pixel in the first image and the original image to obtain the image to be detected. This can further enhance the edges in the image to be detected to improve the detection accuracy of the area to be filled.

[0060] Specifically, for the specific fusion method of the first image and the original image, the color channels corresponding to the same pixel can be stacked based on the channel stacking module in the interface analysis model to achieve the fusion of the first image and the original image, thereby obtaining the following: Figure 5 The image to be detected is shown.

[0061] The structure of the interface analysis model can be found in Figure 6 , Figure 6 The structure diagram of the interface analysis model according to an embodiment of the present application is schematically shown. Figure 6 As shown, the interface analysis model can receive an input original image, and convert the original image into a grayscale image through an edge enhancement module 620, and binarize the grayscale image based on a first edge enhancement parameter to obtain a first image. Furthermore, the color channels corresponding to the same pixel in the first image and the original image can be stacked through the channel stacking module 610 to obtain an image to be detected. Furthermore, the image to be detected can be input into a feature extraction layer 641, a feature extraction layer 642, ..., a feature extraction layer 643; wherein the number of feature extraction layers is not limited. Feature extraction layer 641, feature extraction layer 642, ..., feature extraction layer 643 are used to extract image features and input them into a detector 630. The detector 630 can output at least one reference filling area in the image to be detected based on the image features, and feature vectors corresponding to at least one reference filling area, respectively.

[0062] As an optional embodiment, contour detection is performed on the image to be detected to obtain a contour detection result, including: binarizing the grayscale image based on the second edge enhancement parameter to obtain a second image; contour marking the second image to obtain a contour image; determining a preset range of pixel numbers within the contour; wherein the range of pixel numbers within the contour includes a lower limit and an upper limit; and filtering out contours in the contour image that are greater than the upper limit and less than the lower limit to obtain a contour detection result. This can reduce the false detection rate of the area to be filled, which is conducive to improving the accuracy of subsequent automatic filling.

[0063] Specifically, after obtaining the grayscale image, the grayscale image can be binarized based on the second edge enhancement parameter (e.g., 200) to obtain a second image, so as to eliminate the influence of the segmentation line used to segment the display area in the image; wherein the second image can be in the form of Figure 4 Since the first edge enhancement parameter and the second edge enhancement parameter are different, there is a difference in the binarization effects of the first image and the second image.

[0064] Then, the second image can be contour marked to obtain Figure 7 The contour image shown, wherein the contour image includes multiple contours, such as text contours, icon contours, etc. Further, based on a preset range of the number of pixels in the contour (e.g., an upper limit of 1000 pixels to a lower limit of 10000 pixels), the number of pixels of each independent contour in the contour image can be detected, and contours with a number greater than the upper limit in the contour image can be filtered out, and contours with a number less than the lower limit can be filtered out, so as to obtain a contour detection result.

[0065] Among them, the contour detection result may include multiple contours whose pixel numbers fall within the pixel number range within the contour, such as Figure 8 As shown, through Figure 7 The contours of the image are filtered to obtain contour detection results including illegal contours 810, 820, 830, 840 and 850. It can be understood that the illegal contours 840 and 850 both indicate that the project is an area to be selected rather than an area to be filled.

[0066] Further optionally, considering that icons usually appear in edge areas, it also includes: determining the image edge area, the size of the image edge area can be set according to actual conditions; filtering out contours that are not in the image edge area in the contour detection results to further avoid misdetection of areas to be filled.

[0067] As an optional embodiment, based on the contour detection result, filtering the pseudo-filled area in at least one reference fill area to obtain at least one area to be filled includes: filtering out the reference fill area containing the contour detection result in at least one reference fill area as the pseudo-filled area to obtain at least one area to be filled. In this way, the pseudo-filled area in the reference fill area can be filtered out through the inclusion relationship between the reference fill area and the contour detection result, which can improve the detection accuracy of the area to be filled.

[0068] Specifically, in at least one reference filling area, the reference filling area containing the contour detection result is filtered out as a false filling area, and the specific implementation method of obtaining at least one area to be filled can be: since the image to which the reference filling area belongs and the image to which the contour detection result belongs are essentially the same image, the reference filling area and the contour detection result can be mapped to the same image, and based on the position and area of ​​the reference filling area, the inclusion relationship between the reference filling area and the contour detection result is detected. If the reference filling area contains the contour detection result, it means that the contour detection result should be the area to be selected, rather than the area to be filled. Furthermore, the contour detection result can be filtered out as a false filling area. After completing the traversal of each reference filling area, the reference filling area that has not been filtered out can be determined as the area to be filled.

[0069] For reference fill areas, see Fig. 9 Since the image to be detected is used for model recognition, it is not convenient to display the reference filling area. Therefore, the reference filling area is displayed based on the grayscale image. Fig. 9 In the embodiment, the identified reference filling area may include: reference filling area 910, reference filling area 920, reference filling area 930, reference filling area 940, and reference filling area 950; wherein the reference filling area 930 and the reference filling area 940 are essentially areas to be selected. Figure 8 The contour 850 in is mapped to Fig. 9 After that, the reference fill region 930 includes the outline 850; Figure 8 The contour 840 in is mapped to Fig. 9 After that, the reference filling area 940 contains the contour 840. Therefore, the reference filling area 930 and the reference filling area 940 should be determined as false filling areas and filtered out.

[0070] The result after filtering out the reference filling area 930 and the reference filling area 940 can be found in Fig.10 , based on Fig. 9 For similar reasons, the area to be filled is displayed here based on the grayscale image. Fig.10 , includes a to-be-filled area 1010, a to-be-filled area 1020, and a to-be-filled area 1030, but does not include a pseudo-filled area.

[0071] As an optional embodiment, the method further includes: detecting overlapping region pairs in at least one region to be filled; determining a type of intersection-and-union ratio of the region pairs; and if the type of intersection-and-union ratio is greater than a preset intersection-and-union ratio, deleting the region to be filled with the lowest confidence in the region pair. In this way, the false detection rate of the region to be filled can be further reduced.

[0072] Specifically, if there is a partially overlapping / completely overlapping area to be filled in at least one area to be filled, a type of intersection over union (IoU) of the overlapping area pairs can also be calculated. If a type of IoU is greater than a preset IoU, it means that there is a falsely detected area to be filled in the area pair, and then based on the confidence in the feature vector of the area to be filled, the area to be filled with the lowest confidence can be deleted, and the area to be filled with the highest confidence can be retained. Among them, the intersection over union (IoU) is used as a measurement indicator to describe the overlap between two areas.

[0073] As an optional embodiment, detecting overlapping region pairs in at least one region to be filled includes: obtaining a feature vector of at least one region to be filled identified by the interface analysis model; wherein the feature vector includes at least one of the upper left corner horizontal coordinate, the upper left corner vertical coordinate, the region width, the region height, and the confidence level; and detecting overlapping region pairs in at least one region to be filled based on the feature vector. In this way, the region pairs can be detected based on the feature vector that accurately describes the region to be filled, and the detection accuracy of the region pairs is higher.

[0074] Specifically, overlapping area pairs in at least one area to be filled can be detected based on at least one of the upper left corner horizontal coordinate, the upper left corner vertical coordinate, the area width, the area height, and the confidence level. If there are no overlapping area pairs, the process can be terminated.

[0075] As an optional embodiment, the method further includes: identifying at least one text box in the image to be detected; filtering at least one area to be filled according to the at least one text box to obtain at least one target filling area. In this way, filtering of the area to be filled can be further implemented, and the accuracy of subsequent automatic filling can be further improved.

[0076] Specifically, at least one text box in the image to be detected can be identified based on an optical character recognition model (OCR), and the identification result can also include text information contained in at least one text box; wherein the amount of text information contained in each text box can be the same or different. Furthermore, after filtering at least one area to be filled according to at least one text box, the number of target filling areas obtained can be less than or equal to the number of areas to be filled.

[0077] In addition, optionally, the optical character recognition model may use other algorithms besides OCR, which is not limited here.

[0078] As an optional embodiment, at least one area to be filled is filtered according to at least one text box to obtain at least one target filling area, including: calculating the two-class intersection-and-union ratio of at least one text box and each area to be filled; for the two-class effective intersection-and-union ratio greater than a first preset threshold, recording the inclusion relationship between the text box and the area to be filled corresponding to the two-class effective intersection-and-union ratio; filtering out the area to be filled without a corresponding inclusion relationship in at least one area to be filled, to obtain at least one target filling area. In this way, the false detection rate of the area to be filled can be further reduced.

[0079] Specifically, for each text box, it is necessary to calculate the two-class intersection and union ratio with each area to be filled. If the two-class intersection and union ratio is greater than the first preset threshold, it is considered to be a two-class valid intersection and union ratio. The pair of text boxes corresponding to the two-class valid intersection and union ratio and the area to be filled are related and have an inclusion relationship. If the two-class intersection and union ratio is less than or equal to the first preset threshold, it is considered to be a two-class invalid intersection and union ratio. There is no effective overlap between the pair of text boxes corresponding to the two-class invalid intersection and union ratio and the area to be filled. Therefore, this pair of text boxes is irrelevant to the area to be filled.

[0080] In all the inclusion relationships, the area to be filled may be irrelevant to text box A but relevant to text box B. If an area to be filled is irrelevant to all text boxes, that is, there is no corresponding area to be filled with inclusion relationships. Then, such areas to be filled can be filtered out to obtain at least one target filling area. In addition, if a text box is irrelevant to all areas to be filled, such text boxes can also be filtered out.

[0081] As an optional embodiment, if at least two effective intersection-and-union ratios of the second category are detected to correspond to the same specific text box, recording the inclusion relationship between the text box corresponding to the effective intersection-and-union ratio of the second category and the area to be filled includes: selecting a specific area to be filled from the areas to be filled corresponding to the at least two effective intersection-and-union ratios of the second category; and recording the correspondence between the specific area to be filled and the specific text box. In this way, a one-to-one correspondence can be maintained between the text box and the area to be filled, so that the content to be filled in the area to be filled can be accurately determined later.

[0082] Specifically, in some cases, a text box may be contained in multiple areas to be filled, that is, at least two areas to be filled corresponding to the two-category effective intersection and union ratio. At this time, one area to be filled can be selected as the specific area to be filled, so that there is a one-to-one correspondence between the text box and the area to be filled.

[0083] As an optional embodiment, selecting a specific area to be filled from the areas to be filled corresponding to at least two effective intersection-and-union ratios of the two categories includes: determining the three-category intersection-and-union ratios between the areas to be filled corresponding to the at least two effective intersection-and-union ratios of the two categories and the specific text box; and determining the area to be filled corresponding to the maximum three-category intersection-and-union ratio as the specific area to be filled. In this way, a one-to-one correspondence between the text box and the area to be filled can be guaranteed, which is conducive to improving the subsequent automatic filling accuracy.

[0084] Specifically, the three-category intersection-and-union ratio is used to characterize the overlap between the to-be-filled area corresponding to each two-category effective intersection-and-union ratio and the specific text box. The higher the overlap, the higher the correlation between the to-be-filled area and the specific text box. Therefore, the to-be-filled area corresponding to the maximum three-category intersection-and-union ratio can be determined as the specific to-be-filled area.

[0085] As an optional embodiment, it also includes: filling at least one target filling area based on the text box contained in the at least one target filling area; or filling the program interface corresponding to the image to be detected based on the text box contained in the at least one target filling area. In this way, automatic interface filling can be achieved, saving user filling time and improving interface filling efficiency.

[0086] Specifically, the first-class coordinates of at least one target filling area and the second-class coordinates of the text boxes contained in each target filling area can be obtained first, and the first-class coordinates and the second-class coordinates can be returned to the automatic filling framework. The automatic filling framework can directly fill at least one target filling area based on the first-class coordinates, the second-class coordinates and the inclusion relationship between the target filling area and the text box, and can also fill the original image corresponding to the image to be detected, or can fill the program interface corresponding to the image to be detected. The automatic filling framework can be personalized based on the filling efficiency requirements. Therefore, the automatic filling frameworks developed by different developers may be different, but they can all realize the automatic filling function.

[0087] As an optional embodiment, filling at least one target filling area based on a text box contained in at least one target filling area includes: identifying a field to be filled corresponding to at least one target filling area based on the text box contained in the at least one target filling area; and filling the corresponding target filling area with information according to the field to be filled. In this way, the field to be filled can be determined based on the text box contained in the target filling area, which is conducive to achieving accurate filling.

[0088] Specifically, after the to-be-filled field corresponding to the target filling area is identified, information corresponding to the to-be-filled field can be determined from pre-stored information and information filling can be performed.

[0089] As an optional embodiment, based on the text boxes contained in the at least one target filling area, identifying the to-be-filled field corresponding to the at least one target filling area includes: for each target filling area, splicing the text information of the text boxes in the area to obtain the target text corresponding to the at least one target filling area (e.g., see Fig.10 The area to be filled 1020 in the image is identified as the target area to be filled after the inclusion relationship is determined. The text information in the area to be filled 1020 is spliced ​​to obtain the target text "telephone-extension number"); the field to be filled corresponding to the target text (such as account number) is identified based on the Named Entity Recognition (NER) algorithm. In this way, accurate and automatic interface filling can be achieved.

[0090] Specifically, the method of splicing the text information of the text boxes in the area can be: sorting the text information of the text boxes according to the information position in order from top to bottom and from left to right, and splicing the sorted text information; or, sorting the text information of the text boxes according to the information position in order from bottom to top and from right to left, and splicing the sorted text information.

[0091] In addition, optionally, it also includes: if the target text contains specific keywords (such as selection), the target filling area corresponding to the target text is filtered out to further improve the automatic filling accuracy.

[0092] In addition, optionally, in addition to NER, other algorithms may be used as the entity recognition algorithm, which is not limited here.

[0093] See also Fig.11 , Fig.11 The following schematically shows a flow chart of a method for region recognition of an image according to another embodiment of the present application. Fig.11 As shown, the region recognition method for an image includes: steps S1110 to S1134.

[0094] Step S1110: converting the original image into a grayscale image, and binarizing the grayscale image based on a first edge enhancement parameter to obtain a first image.

[0095] Step S1112: stacking the color channels corresponding to the same pixel in the first image and the original image to obtain an image to be detected.

[0096] Step S1114: Based on at least one of the graphic size normalization processing and the pixel value normalization processing, the image to be detected is processed, and then at least one reference filling area in the processed image to be detected is identified.

[0097] Step S1116: Binarize the grayscale image based on the second edge enhancement parameter to obtain a second image, and perform contour marking on the second image to obtain a contour image, and determine a preset range of the number of pixels within the contour; wherein the range of the number of pixels within the contour includes a lower limit and an upper limit, and then, contours in the contour image that are greater than the upper limit and less than the lower limit can be filtered out to obtain contour detection results.

[0098] Step S1118: Filter out the reference filling area including the contour detection result in at least one reference filling area as a false filling area to obtain at least one area to be filled.

[0099] Step S1120: Obtain a feature vector of at least one area to be filled identified by the interface analysis model; wherein the feature vector includes at least one of the upper left corner horizontal coordinate, the upper left corner vertical coordinate, the area width, the area height, and the confidence. Then, based on the feature vector, overlapping area pairs in at least one area to be filled can be detected. If an intersection-and-union ratio of one type is greater than a preset intersection-and-union ratio, the area to be filled with the lowest confidence in the area pair will be deleted.

[0100] Step S1122: Identify at least one text box in the image to be detected.

[0101] Step S1124: Calculate the two-class intersection and union ratio of at least one text box and each area to be filled. If at least two two-class valid intersection and union ratios are not detected to correspond to the same specific text box, execute step S1126; if at least two two-class valid intersection and union ratios are detected to correspond to the same specific text box, execute step S1128.

[0102] Step S1126: for two-category effective intersection-and-union ratios greater than a first preset threshold, record the inclusion relationship between the text box and the area to be filled corresponding to the two-category effective intersection-and-union ratios.

[0103] Step S1128: For the two-category effective intersection-and-union ratio that is greater than the first preset threshold, determine the three-category intersection-and-union ratios between the to-be-filled areas corresponding to at least two two-category effective intersection-and-union ratios and the specific text boxes, and then determine the to-be-filled area corresponding to the maximum three-category intersection-and-union ratio as the specific area to be filled, and record the correspondence between the specific area to be filled and the specific text box.

[0104] Step S1130: Filter out the to-be-filled areas without corresponding inclusion relationships in at least one to-be-filled area to obtain at least one target filling area.

[0105] Step S1132: For each target filling area, the text information of the text boxes in the area is spliced ​​to obtain the target text corresponding to at least one target filling area, and then, based on the entity recognition algorithm, the to-be-filled field corresponding to the target text is identified.

[0106] Step S1134: fill the corresponding target filling area with information according to the field to be filled.

[0107] It should be noted that steps S1110 to S1134 are Figure 1 The steps and their embodiments shown correspond to each other. For the specific implementation of steps S1110 to S1134, please refer to Figure 1 The steps and embodiments shown are not described in detail here.

[0108] It can be seen that the implementation Fig.11 The method shown can, after identifying at least one reference filling area in the image to be detected, filter out the false filling areas (such as areas that are shaped like areas to be filled but are actually option areas) in at least one reference filling area based on the contour detection result of the image to be detected, so as to avoid determining the false filling areas as areas to be filled, solve the problem of false detection of areas to be filled, and improve the detection accuracy of areas to be filled. In addition, since the present application can improve the detection accuracy of areas to be filled, it can be beneficial to ensure the automatic filling accuracy of areas to be filled.

[0109] See also Fig.12 , Fig.12 The structure block diagram of the region recognition device for an image according to an embodiment of the present application is schematically shown. Fig.12 As shown, the region recognition device 1200 for an image may include the following units.

[0110] The region identification unit 1201 is used to identify at least one reference filling region in the image to be detected;

[0111] A contour detection unit 1202 is used to perform contour detection on the image to be detected and obtain a contour detection result;

[0112] The region filtering unit 1203 is configured to filter the pseudo-filled regions in at least one reference filled region based on the contour detection result to obtain at least one region to be filled.

[0113] It can be seen that the implementation Fig.12 The device shown can, after identifying at least one reference filling area in the image to be detected, filter out the false filling areas (such as areas that are shaped like areas to be filled but are actually option areas) in at least one reference filling area based on the contour detection result of the image to be detected, so as to avoid determining the false filling areas as areas to be filled, solve the problem of false detection of areas to be filled, and improve the detection accuracy of areas to be filled. In addition, since the present application can improve the detection accuracy of areas to be filled, it can be beneficial to ensure the automatic filling accuracy of areas to be filled.

[0114] As an optional embodiment, the region identifying unit 1201 identifies at least one reference filling region in the image to be detected, including:

[0115] Processing the image to be detected based on at least one of a graphic size normalization process and a pixel value normalization process;

[0116] At least one reference filling region in the image to be detected after identification processing.

[0117] It can be seen that by implementing this optional embodiment, the image to be detected can be standardized through normalization processing, thereby reducing the difficulty of subsequent processing of the image to be detected and improving processing efficiency.

[0118] As an optional embodiment, it also includes:

[0119] An image conversion unit, used for converting an original image into a grayscale image;

[0120] The image conversion unit is further used to binarize the grayscale image based on the first edge enhancement parameter to obtain a first image;

[0121] The image fusion unit is used to fuse the first image and the original image to obtain an image to be detected.

[0122] It can be seen that by implementing this optional embodiment, the edge segmentation line in the image to be detected can be enhanced through grayscale and binarization, which is beneficial to improving the detection accuracy of the area to be filled in the image to be detected.

[0123] As an optional embodiment, the image fusion unit fuses the first image and the original image to obtain the image to be detected, including:

[0124] The color channels corresponding to the same pixel in the first image and the original image are stacked to obtain an image to be detected.

[0125] It can be seen that the implementation of this optional embodiment can further enhance the edges in the image to be detected, so as to improve the detection accuracy of the area to be filled.

[0126] As an optional embodiment, the contour detection unit 1202 performs contour detection on the image to be detected to obtain a contour detection result, including:

[0127] Binarize the grayscale image based on the second edge enhancement parameter to obtain a second image;

[0128] Performing contour marking on the second image to obtain a contour image;

[0129] Determine a preset range of the number of pixels within the contour; wherein the range of the number of pixels within the contour includes a lower limit and an upper limit;

[0130] The contours in the contour image that are greater than the upper limit and less than the lower limit are filtered out to obtain the contour detection result.

[0131] It can be seen that implementing this optional embodiment can reduce the false detection rate of the area to be filled, which is beneficial to improving the accuracy of subsequent automatic filling.

[0132] As an optional embodiment, the region filtering unit 1203 filters the pseudo-filled region in the at least one reference filled region based on the contour detection result to obtain at least one region to be filled, including:

[0133] In at least one reference filling region, the reference filling region containing the contour detection result is filtered out as a false filling region to obtain at least one region to be filled.

[0134] It can be seen that by implementing this optional embodiment, the false filling area in the reference filling area can be filtered out through the inclusion relationship between the reference filling area and the contour detection result, which can improve the detection accuracy of the area to be filled.

[0135] As an optional embodiment, it also includes:

[0136] A region pair detection unit, used to detect overlapping region pairs in at least one region to be filled;

[0137] An intersection-and-union ratio calculation unit, used to determine a type of intersection-and-union ratio of a region pair;

[0138] The region filtering unit 1203 is further configured to delete the to-be-filled region with the lowest confidence in the region pair if the intersection-over-union ratio of one type is greater than a preset intersection-over-union ratio.

[0139] It can be seen that implementing this optional embodiment can further reduce the false detection rate of the area to be filled.

[0140] As an optional embodiment, the region pair detection unit detects overlapping region pairs in at least one region to be filled, including:

[0141] Acquire a feature vector of at least one area to be filled identified by the interface analysis model; wherein the feature vector includes at least one of the upper left corner horizontal coordinate, the upper left corner vertical coordinate, the area width, the area height, and the confidence level;

[0142] Based on the feature vector, a pair of overlapping regions in at least one region to be filled is detected.

[0143] It can be seen that by implementing this optional embodiment, the detection of region pairs can be achieved based on the feature vector that accurately describes the region to be filled, and the detection accuracy of region pairs is higher.

[0144] As an optional embodiment, it also includes:

[0145] A text box recognition unit, used to recognize at least one text box in the image to be detected;

[0146] The area filtering unit 1203 is further configured to filter at least one area to be filled according to at least one text box to obtain at least one target filling area.

[0147] It can be seen that the implementation of this optional embodiment can further achieve filtering of the area to be filled, and further improve the accuracy of subsequent automatic filling.

[0148] As an optional embodiment, the area filtering unit 1203 filters at least one area to be filled according to at least one text box to obtain at least one target filling area, including:

[0149] Calculate the two-class intersection-and-union ratio of at least one text box and each area to be filled;

[0150] For two-category effective intersection-and-union ratios greater than a first preset threshold, record the inclusion relationship between the text box corresponding to the two-category effective intersection-and-union ratio and the area to be filled;

[0151] In at least one area to be filled, areas to be filled that have no corresponding inclusion relationship are filtered out to obtain at least one target filling area.

[0152] It can be seen that implementing this optional embodiment can further reduce the false detection rate of the area to be filled.

[0153] As an optional embodiment, if at least two two-class effective intersection-and-union ratios are detected to correspond to the same specific text box, the region filtering unit 1203 records the inclusion relationship between the text box corresponding to the two-class effective intersection-and-union ratios and the to-be-filled region, including:

[0154] Selecting a specific area to be filled from at least two areas to be filled corresponding to the effective intersection and union ratio of the second category;

[0155] Record the corresponding relationship between the specific area to be filled and the specific text box.

[0156] It can be seen that the implementation of this optional embodiment can maintain a one-to-one correspondence between the text box and the area to be filled, so as to accurately determine the content that needs to be filled in the area to be filled later.

[0157] As an optional embodiment, the region filtering unit 1203 selects a specific region to be filled from at least two regions to be filled corresponding to the effective intersection-and-union ratios of the second category, including:

[0158] Determine three types of intersection-and-union ratios between the to-be-filled areas corresponding to at least two two types of valid intersection-and-union ratios and the specific text box respectively;

[0159] The to-be-filled area corresponding to the maximum three-category intersection-and-union ratio is determined as the specific to-be-filled area.

[0160] It can be seen that implementing this optional embodiment can ensure a one-to-one correspondence between the text box and the area to be filled, which is conducive to improving the subsequent automatic filling accuracy.

[0161] As an optional embodiment, it also includes:

[0162] A filling unit, configured to fill at least one target filling area based on a text box contained in at least one target filling area;

[0163] Alternatively, the filling unit is used to fill the program interface corresponding to the image to be detected based on a text box contained in at least one target filling area.

[0164] It can be seen that the implementation of this optional embodiment can realize automatic interface filling, save user filling time, and improve interface filling efficiency.

[0165] As an optional embodiment, the filling unit fills at least one target filling area based on a text box contained in the at least one target filling area, including:

[0166] Based on a text box contained in at least one target filling area, identifying a to-be-filled field corresponding to at least one target filling area;

[0167] Fill the corresponding target filling area with information according to the fields to be filled.

[0168] It can be seen that by implementing this optional embodiment, the fields to be filled can be determined based on the text boxes contained in the target filling area, which is conducive to achieving accurate filling.

[0169] As an optional embodiment, the filling unit identifies a to-be-filled field corresponding to at least one target filling area based on a text box contained in at least one target filling area, including:

[0170] For each target filling area, the text information of the text boxes in the area are spliced ​​to obtain the target text corresponding to at least one target filling area;

[0171] Identify the fields to be filled corresponding to the target text based on the entity recognition algorithm.

[0172] It can be seen that the implementation of this optional embodiment can achieve accurate and automatic interface filling.

[0173] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0174] Since the various functional modules of the region recognition device for images in the example embodiment of the present application correspond to the steps of the example embodiment of the region recognition device for images described above, for details not disclosed in the device embodiment of the present application, please refer to the above-mentioned embodiment of the region recognition device for images in the present application.

[0175] See also Fig.13 , Fig.13 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0176] It should be noted that Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0177] like Fig.13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 into a random access memory (RAM) 1303. Various programs and data required for system operation are also stored in the RAM 1303. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0178] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0179] In particular, according to an embodiment of the present application, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, various functions defined in the method and apparatus of the present application are executed.

[0180] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method in the above embodiment.

[0181] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0182] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0184] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

Claims

1. A method for region recognition of an image, characterized in that: include: identifying at least one reference fill region in the image to be detected; Performing contour detection on the image to be detected to obtain a contour detection result; Based on the contour detection result, the pseudo-filled regions in the at least one reference filled region are filtered to obtain at least one region to be filled.

2. The method according to claim 1, characterized in that: Identifying at least one reference fill region in the image to be detected, including: Processing the image to be detected based on at least one of a graphic size normalization process and a pixel value normalization process; At least one reference filling region in the image to be detected after identification processing.

3. The method according to claim 1, characterized in that Also includes: Convert the original image to a grayscale image; Binarize the grayscale image based on a first edge enhancement parameter to obtain a first image; The first image and the original image are fused to obtain the image to be detected.

4. The method according to claim 3, characterized in that The first image and the original image are fused to obtain the image to be detected, including: The color channels corresponding to the same pixel in the first image and the original image are stacked to obtain the image to be detected.

5. The method according to claim 3, characterized in that: Performing contour detection on the image to be detected to obtain a contour detection result includes: Binarize the grayscale image based on a second edge enhancement parameter to obtain a second image; Performing contour marking on the second image to obtain a contour image; Determine a preset range of the number of pixels within the outline; wherein the range of the number of pixels within the outline includes a lower limit and an upper limit; Contours in the contour image that are greater than the upper limit of quantity and less than the lower limit of quantity are filtered out to obtain a contour detection result.

6. The method according to claim 5, characterized in that Based on the contour detection result, filtering the pseudo-filled area in the at least one reference filled area to obtain at least one area to be filled includes: Among the at least one reference filling region, the reference filling region containing the contour detection result is filtered out as a false filling region to obtain at least one region to be filled.

7. The method according to claim 1, characterized in that Also includes: Detecting a pair of overlapping regions in the at least one region to be filled; determining a class intersection-over-union ratio of the region pairs; If the intersection-and-union ratio of one type is greater than a preset intersection-and-union ratio, the to-be-filled region with the lowest confidence in the region pair is deleted.

8. The method according to claim 7, characterized in that Detecting an overlapping region pair in the at least one region to be filled, comprising: Acquire a feature vector of the at least one area to be filled identified by the interface analysis model; wherein the feature vector includes at least one of the upper left corner horizontal coordinate, the upper left corner vertical coordinate, the area width, the area height, and the confidence level; Based on the feature vector, a pair of overlapping regions in the at least one region to be filled is detected.

9. The method according to claim 1, characterized in that: Also includes: Identifying at least one text box in the image to be detected; According to the at least one text box, the at least one area to be filled is filtered to obtain at least one target filling area.

10. The method according to claim 9, characterized in that According to the at least one text box, filtering the at least one area to be filled to obtain at least one target filling area includes: Calculating a two-category intersection-and-union ratio of the at least one text box and each of the areas to be filled; For two types of effective intersection-and-union ratios greater than a first preset threshold, recording the inclusion relationship between the text box corresponding to the two types of effective intersection-and-union ratios and the area to be filled; Among the at least one area to be filled, areas to be filled that have no corresponding inclusion relationship are filtered out to obtain at least one target filling area.

11. The method according to claim 10, characterized in that If it is detected that at least two of the two types of effective intersection-and-union ratios correspond to the same specific text box, the inclusion relationship between the text box corresponding to the two types of effective intersection-and-union ratios and the area to be filled is recorded, including: Selecting a specific area to be filled from at least two areas to be filled corresponding to the two types of effective intersection-and-union ratios; The corresponding relationship between the specific area to be filled and the specific text box is recorded.

12. The method according to claim 11, characterized in that Selecting a specific area to be filled from at least two areas to be filled corresponding to the two types of effective intersection-and-union ratios comprises: Determine three types of intersection-and-union ratios between the to-be-filled areas corresponding to at least two of the two types of effective intersection-and-union ratios and the specific text box respectively; The to-be-filled area corresponding to the maximum three-category intersection-and-union ratio is determined as the specific to-be-filled area.

13. The method according to claim 9, characterized in that Also includes: Filling the at least one target filling area based on a text box contained in the at least one target filling area; Alternatively, based on the text box contained in the at least one target filling area, the program interface corresponding to the image to be detected is filled.

14. The method according to claim 13, characterized in that Filling the at least one target filling area based on a text box contained in the at least one target filling area includes: Based on a text box contained in the at least one target filling area, identifying a to-be-filled field corresponding to the at least one target filling area; The corresponding target filling area is filled with information according to the field to be filled.

15. The method according to claim 14, characterized in that Based on a text box contained in the at least one target filling area, identifying a to-be-filled field corresponding to the at least one target filling area includes: For each target filling area, the text information of the text boxes in the area are spliced ​​to obtain the target text corresponding to the at least one target filling area; Identify the to-be-filled fields corresponding to the target text based on an entity recognition algorithm.

16. A region recognition device for an image, characterized in that: include: A region identification unit, used to identify at least one reference filling region in the image to be detected; A contour detection unit, used to perform contour detection on the image to be detected to obtain a contour detection result; The area filtering unit is used to filter the pseudo-filled area in the at least one reference filling area based on the contour detection result to obtain at least one area to be filled.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

18. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1-15 by executing the executable instructions.

Images