Image recognition method, image review method and electronic equipment

Abstract

The present invention relates to the field of computer technology, and specifically provides an image recognition method, an image review method, and an electronic device, which are intended to solve the problem that the existing method extracts information from different types of images to be recognized separately based on the same image recognition model, resulting in low image recognition efficiency and poor robustness. To this end, the image recognition method of the present invention includes: obtaining the image type of each image to be recognized in the image set to be recognized and calling the image recognition model corresponding to the image to be recognized according to the image type of each image to be recognized, so as to obtain the recognition result of the corresponding image to be recognized, thereby effectively improving the recognition efficiency. When the image recognition model is used for recognition, the capture area is determined according to the image type, the text information of the first text box, the coordinates of the two diagonal fixed points, and the deflection angle, and the second text box whose text information matches the target recognition content is screened out according to the capture area. This method is suitable for images to be recognized in tilted conditions, and effectively improves the robustness.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and specifically provides an image recognition method, an image review method, and an electronic device. Background Art

[0002] In many current scenarios, it is necessary to review multiple user documents. Relying solely on manual review is inefficient and prone to omissions or false detections. In addition, existing technologies obtain document images and extract information from document images based on OCR (Optical Character Recognition) technology for the purpose of reviewing relevant information.

[0003] However, when existing OCR technology extracts information from document images, it is based on the same OCR model and indiscriminately extracts information from document images of different types of documents, which has the problem of low efficiency. In addition, when the image tilt problem occurs, it can only be solved through tilt correction preprocessing, and the model has poor robustness. Summary of the Invention

[0004] The present invention aims to solve the above technical problems, that is, to solve the problem that the existing image recognition model is based on the same image recognition model to extract information from different types of images to be recognized separately, resulting in low image recognition efficiency and poor robustness.

[0005] In a first aspect, the present invention provides an image recognition method, comprising:

[0006] For each of the images to be recognized, determining an image recognition model corresponding to the image to be recognized according to the image type of the image to be recognized, and inputting the image to be recognized into the image recognition model corresponding to the image to be recognized, so as to obtain a recognition result of the image to be recognized using the image recognition model;

[0007] Wherein, obtaining the recognition result of the image to be recognized by using the image recognition model includes:

[0008] Obtaining a text box of the image to be recognized and obtaining the coordinates of two diagonal vertices, a deflection angle, and text information of the text box; wherein the two diagonal vertex coordinates include the coordinates of the upper left vertex and the coordinates of the lower right vertex;

[0009] According to the target recognition content corresponding to the image type, a first text box whose text information is the target recognition content is selected from the acquired text boxes;

[0010] determining a capture area according to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle;

[0011] For the remaining text boxes except the first text box among the text boxes corresponding to the image to be recognized, a second text box located in the capture area is selected from the remaining text boxes, and the text information corresponding to the second text box matches the target recognition content;

[0012] A recognition result of the image to be recognized is obtained at least according to the text information in the first text box and the text information in the second text box.

[0013] In some embodiments, determining the capture area according to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle includes:

[0014] Determining the preset empirical values ​​corresponding to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle, wherein the upper left vertex coordinates include the first direction coordinates and the second direction coordinates of the upper left vertex, the lower right vertex coordinates include the first direction coordinates and the second direction coordinates of the lower right vertex, and the preset empirical values ​​include a first empirical value, a second empirical value, a third empirical value, and a fourth empirical value;

[0015] Determining the first direction coordinate of a first vertex of the capture area according to the first direction coordinate of the lower right corner vertex of the first text box and the first empirical value, where the first vertex is the vertex of the capture area that is closest to the upper left corner vertex of the first text box;

[0016] Determine the first direction coordinate of a second vertex of the capture area according to the first direction coordinate of the lower right corner vertex and the second empirical value; the second vertex is a diagonal vertex of the first vertex;

[0017] Determine the second direction coordinate of the first vertex according to the first direction coordinate of the first vertex, the first direction coordinate of the upper left corner vertex, the second direction coordinate of the upper left corner vertex, the deflection angle, and the third empirical value;

[0018] Determine the second direction coordinate of the second vertex according to the first direction coordinate of the second vertex, the first direction coordinate of the lower right corner vertex, the second direction coordinate of the lower right corner vertex, the deflection angle, and the fourth empirical value;

[0019] The capture area is determined according to the first direction coordinate and the second direction coordinate of the first vertex and the first direction coordinate and the second direction coordinate of the second vertex.

[0020] In some embodiments, the capture area is determined by the following expression:

[0021] Ma1 =A a2 -T1

[0022] M b1 =A b1 -(M a1 -A a1 )·tanθ-T3

[0023] M a2 =A a2 +T2

[0024] M b2 =A b2 -(M a2 -A a2 )·tanθ+T4

[0025] Among them, M a1 and M b1 Respectively represent the first direction coordinate and the second direction coordinate of the first vertex, M a2 and M b2 Represent the first direction coordinate and the second direction coordinate of the second vertex respectively, A a1 and A b1 Respectively represent the first direction coordinate and the second direction coordinate of the upper left corner vertex, A a2 and A b2 represent the first direction coordinate and the second direction coordinate of the lower right corner vertex respectively, θ represents the deflection angle, T1 represents the first experience value, T2 represents the second experience value, T3 represents the third experience value, and T4 represents the fourth experience value.

[0026] In some embodiments, before determining the capture area according to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle, the method further includes:

[0027] For each of the image types, based on the first text box and the second text box at different deflection angles, respectively setting an actual capture area covering the second text box;

[0028] Obtaining the coordinates of the two diagonal vertices of the first text box and the coordinates of the third vertex and the fourth vertex of the actual capture area, wherein the third vertex is the vertex of the actual capture area closest to the upper left corner vertex of the first text box, and the fourth vertex is the diagonal vertex of the third vertex;

[0029] Based on the deflection angle, the coordinates of the two diagonal vertices of the first text box, and the coordinates of the third vertex and the fourth vertex, the preset experience value is determined and stored in correspondence with the image type, the text information of the first text box, the deflection angle and the coordinates of the two diagonal vertices.

[0030] In some embodiments, determining the preset empirical value based on the deflection angle, the coordinates of the two diagonal vertices of the first text box, and the coordinates of the third vertex and the fourth vertex includes:

[0031] Determining a first intersection point according to an intersection point of a first auxiliary line and a first extension line, wherein the first auxiliary line passes through the third vertex of the actual capture area and is parallel to the second direction, and the first extension line is an extension line passing through the upper left corner vertex and the upper right corner vertex of the first text box;

[0032] Determining a second intersection point based on an intersection point of a second auxiliary line and a second extension line, wherein the second auxiliary line passes through the fourth vertex of the actual capture area and is parallel to the second direction, and the second extension line passes through the lower right corner vertex of the first text box and an inclination angle of the second extension line is equal to the deflection angle;

[0033] Constructing a positional relationship model between the actual capture area and the first text box according to the first intersection point, the second intersection point, the preset empirical value, and the deflection angle;

[0034] Based on the positional relationship model, determining the preset empirical value according to the coordinates of the two diagonal vertices of the first text box and the coordinates of the third vertex and the fourth vertex of the actual capture area;

[0035] The preset experience values ​​include a first experience value, a second experience value, a third experience value, and a fourth experience value, and the position relationship model is expressed as:

[0036] M a01 =A a2 -T1

[0037] M b01 =E b -T3

[0038] E b =A b1 -(M a01 -A a1 )·tanθ

[0039] M a02 =A a2 +T2

[0040] M b02 =Fb +T4

[0041] F b =A b2 -(M a02 -A a2 )·tanθ

[0042] Among them, M a01 and M b01 Respectively represent the first direction coordinate and the second direction coordinate of the third vertex of the actual capture area, M a02 and M b02 They represent the first direction coordinate and the second direction coordinate of the fourth vertex of the actual capture area, E b represents the second direction coordinate of the first intersection point, F b represents the second direction coordinate of the second intersection point, A a1 and A b1 Respectively represent the first direction coordinate and the second direction coordinate of the upper left corner vertex, A a2 and A b2 represent the first direction coordinate and the second direction coordinate of the lower right corner vertex respectively, θ represents the deflection angle, T1 represents the first experience value, T2 represents the second experience value, T3 represents the third experience value, and T4 represents the fourth experience value.

[0043] In some embodiments, the image recognition model includes a text detection sub-model and a character recognition sub-model, and obtaining a text box in an image to be recognized and obtaining the coordinates of two diagonal vertices, a deflection angle, and character information of the text box includes:

[0044] Inputting the image to be recognized into a text detection sub-model, obtaining a text box area map of the image to be recognized and the coordinates of the two diagonal vertices and the deflection angle of each text box in the text box area map;

[0045] The text box area map is input into a text recognition model to obtain text information corresponding to the text box.

[0046] In a second aspect, the present invention provides an image review method comprising:

[0047] Apply any one of the above-mentioned image recognition methods to perform image recognition on each image to be reviewed in the image set to be reviewed, and obtain a recognition result for each image to be reviewed;

[0048] For each of the images to be reviewed, the image to be reviewed is reviewed according to the recognition result of the image to be reviewed and the review mode corresponding to the image type of the image to be reviewed.

[0049] In some embodiments, reviewing the image to be reviewed based on the recognition result of the image to be reviewed and the review mode corresponding to the image type of the image to be reviewed includes:

[0050] When the audit mode is information matching, obtaining pre-stored information corresponding to the image type of the image to be audited;

[0051] The recognition result of the image to be reviewed is matched with the pre-stored information, and the image to be reviewed is reviewed according to the matching result.

[0052] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any one of the above-mentioned image recognition methods or image review methods.

[0053] In a fourth aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements any one of the above-mentioned image recognition methods or image review methods.

[0054] When the above technical solution is adopted, the present invention can obtain the image type of each image to be identified in the set of images to be identified and call the image recognition model corresponding to the image to be identified based on the image type of each image to be identified, so as to obtain the recognition result of the corresponding image to be identified using the image recognition model. Among them, when using the image recognition model to identify the image to be identified, it is possible to determine a first text box whose text information is the target recognition content from the text boxes of the image to be identified, where the target recognition content corresponds to the image type. The capture area is determined based on the image type, the text information of the first text box, the coordinates of two diagonal fixed points, and the deflection angle. The second text box whose text information matches the target recognition content is filtered based on the capture area, so that the recognition result can be obtained based on at least the text information of the first text box and the text information of the second text box. This method can simultaneously identify multiple images of different image types to be identified and use image recognition models corresponding to the image type for different image types, effectively improving recognition efficiency. In addition, the capture area can be determined based on the image type, deflection angle, etc., and the second text box that matches the first text box is determined based on the capture area. This method is suitable for images to be identified in tilted conditions, effectively improving the robustness of image recognition.

[0055] In addition, the image recognition method provided by the present invention can also be applied to image review methods, thereby effectively improving review efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The preferred embodiments of the present invention are described below with reference to the accompanying drawings, in which:

[0057] Figure 1 1 is a flow chart of an image recognition method provided by an embodiment of the present invention;

[0058] Figure 2 This is a flow chart of a method for obtaining a recognition result of an image to be recognized based on an image recognition model provided by an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of an image to be identified provided by a specific example of the present invention;

[0060] Figure 4 1 is a flow chart of a method for determining a preset experience value provided by an embodiment of the present invention;

[0061] Figure 5 is a schematic diagram of a sample image provided by an embodiment of the present invention;

[0062] Figure 6 is another sample image schematic diagram provided by an embodiment of the present invention;

[0063] Figure 7 It is a flowchart of an image review method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0065] When existing OCR technology extracts information from document images, it is based on the same OCR model and indiscriminately extracts information from document images of different types of documents, resulting in low efficiency. In addition, when the image tilt problem occurs, it can only be solved through tilt correction preprocessing, and the model has poor robustness.

[0066] In view of this, the present invention provides an image recognition method. The method obtains the image type of each image to be recognized in a set of images to be recognized and, based on the image type of each image to be recognized, calls an image recognition model corresponding to the image to be recognized, thereby obtaining a recognition result for the corresponding image to be recognized using the image recognition model. When the image recognition model is used to recognize the image to be recognized, a first text box containing text information as target recognition content can be determined from text boxes in the image to be recognized, where the target recognition content corresponds to the image type. A capture area is determined based on the image type, the text information in the first text box, the coordinates of two diagonal fixed points, and the deflection angle. A second text box containing text information matching the target recognition content is then screened based on the capture area. Thus, a recognition result can be obtained based on at least the text information in the first text box and the text information in the second text box. This method can simultaneously recognize images of multiple different image types and utilize image recognition models corresponding to the image type for each image type, effectively improving recognition efficiency. Furthermore, the capture area can be determined based on the image type, deflection angle, and other factors, and the second text box matching the first text box is determined based on the capture area. This method is suitable for images to be recognized under tilted conditions, effectively improving the robustness of image recognition.

[0067] See also Figure 1 As shown, Figure 1 FIG. 1 is a flow chart of an image recognition method provided by an embodiment of the present invention, which may include:

[0068] Step S11: obtaining the image type of each image to be identified in the image set to be identified;

[0069] Step S12: For each image to be recognized, determine the image recognition model corresponding to the image to be recognized according to the image type of the image to be recognized, and input the image to be recognized into the image recognition model corresponding to the image to be recognized to obtain the recognition result of the image to be recognized using the image recognition model.

[0070] In some embodiments, step S11 may specifically be to use a pre-trained classifier to classify each image to be identified in the image set to be identified and obtain the image type of each image to be identified.

[0071] As an example, the image type may include at least one of an identity card, a bank card, a business license, a confirmation letter, and a contract.

[0072] Different image types correspond to different text templates. Based on this, in some embodiments, different image types can be associated with different image recognition models. This allows for simultaneous recognition of multiple different image types, and uses the corresponding image recognition model for each image type, effectively improving recognition efficiency.

[0073] Among them, see Figure 2 As shown, Figure 2 : is a flow chart of a method for obtaining a recognition result of an image to be recognized based on an image recognition model provided by an embodiment of the present invention, which may include:

[0074] Step S21: obtaining a text box of the image to be recognized and obtaining the coordinates of two diagonal vertices of the text box, a deflection angle, and text information; wherein the two diagonal vertex coordinates include the coordinates of the upper left vertex and the coordinates of the lower right vertex;

[0075] Step S22: selecting a first text box whose text information is the target recognition content from the acquired text boxes according to the target recognition content corresponding to the image type;

[0076] Step S23: determining a capture area according to the image type, the text information in the first text box, the coordinates of two diagonal vertices, and the deflection angle;

[0077] Step S24: for the text boxes corresponding to the image to be recognized, excluding the first text box, selecting a second text box located in the capture area from the remaining text boxes, wherein the text information corresponding to the second text box matches the target recognition content;

[0078] Step S25: obtaining a recognition result of the image to be recognized based on at least the text information in the first text box and the text information in the second text box.

[0079] See also Figure 3 As shown, Figure 3 The schematic diagram of the image to be recognized provided by the specific example of the present invention can establish a 0-xy two-dimensional coordinate system, and obtain the coordinates of the two diagonal vertices of the text box A and the text box B in the image to be recognized in the two-dimensional coordinate system. The coordinates of the two diagonal vertices of the text box A can be expressed as follows: the coordinates of the upper left corner vertex are (A x1 ,A y1 ), the coordinates of the lower right corner vertex are (A x2 ,A y2 ), deflection angle θ1; the coordinates of the two diagonal vertices of text box B can be expressed as: the coordinates of the upper left vertex are (B x1 ,B y1 ), the coordinates of the lower right corner vertex are (B x2 ,B y2 ), deflection angle θ2.

[0080] In some embodiments, step S21 may specifically be to directly obtain the text box of the image to be recognized and the coordinates of the two diagonal vertices of the text box, the deflection angle and the text information based on pre-stored data.

[0081] In some other embodiments, the image recognition model may include a text detection sub-model and a text recognition sub-model. Step S21 may be specifically as follows:

[0082] Input the image to be recognized into the text detection sub-model to obtain the text box area map of the image to be recognized, as well as the coordinates of the two diagonal vertices and the deflection angle of each text box in the text box area map.

[0083] Input the text box area map into the text recognition sub-model to obtain the text information corresponding to the text box.

[0084] As an example, the text detection sub-model may adopt the DBNet (Differentiable Binarization Net) model; the text recognition model sub may adopt the DenseNet model.

[0085] Among them, the coordinates of the two diagonal vertices, the deflection angle and the text information of the text box can be stored in the form of a list. Taking Figure 3 as an example, the obtained list can be expressed as {[(A x1 , A y1 ), (A x2 , A y2 ), angle_θ1, text_“Legal Representative”], [(B x1 , B y1 ), (B x2 , B y2 ), angle_θ2, text_“Li Ming”]}.

[0086] In some embodiments, step S22 may be specifically as follows: According to the target recognition content corresponding to the image type, use the traversal method to screen out the text box whose text information is the target recognition content from the multiple text boxes of the image to be recognized, and use this text box as the first text box.

[0087] As an example, when the image type is an ID card, the target recognition content corresponding to it is at least one of the ID card number, name and gender; when the image type is a business license, the target recognition content corresponding to it is at least one of the domicile, legal representative, registration number and enterprise type. Taking Figure 3 as an example, when it is necessary to recognize the legal representative in the business license, “Legal Representative” can be used as the target recognition content, the text box A corresponding to “Legal Representative” can be used as the first text box, and “Li Ming” can be used as another target recognition content that matches “Legal Representative”.

[0088] In some embodiments, determining the capture area according to the image type, the text information of the first text box, the coordinates of the two diagonal vertices and the deflection angle in step S23 includes:

[0089] Determine preset experience values ​​corresponding to the image type, text information of the first text box, coordinates of two diagonal vertices, and a deflection angle, wherein the upper left vertex coordinates include first and second direction coordinates of the upper left vertex, the lower right vertex coordinates include first and second direction coordinates of the lower right vertex, and the preset experience values ​​include a first experience value, a second experience value, a third experience value, and a fourth experience value;

[0090] Determine the first direction coordinate of the first vertex of the capture area according to the first direction coordinate of the lower right corner vertex of the first text box and the first empirical value, where the first vertex is the vertex of the capture area closest to the upper left corner vertex of the first text box;

[0091] Determine the first direction coordinate of the second vertex of the capture area according to the first direction coordinate of the lower right corner vertex and the second empirical value; the second vertex is the diagonal vertex of the first vertex;

[0092] Determine the second direction coordinate of the first vertex according to the first direction coordinate of the first vertex, the first direction coordinate of the upper left corner vertex, the second direction coordinate of the upper left corner vertex, the deflection angle, and the third empirical value;

[0093] Determine the second direction coordinate of the second vertex according to the first direction coordinate of the second vertex, the first direction coordinate of the lower right corner vertex, the second direction coordinate of the lower right corner vertex, the deflection angle, and the fourth empirical value;

[0094] The capture area is determined according to the first direction coordinate and the second direction coordinate of the first vertex and the first direction coordinate and the second direction coordinate of the second vertex.

[0095] A two-dimensional coordinate system can be constructed in advance based on a first direction and a second direction perpendicular to each other. Figure 3 For example, the first direction may be the x direction, and the second direction may be the y direction, thereby obtaining the coordinates of two diagonal vertices of the text box in the corresponding coordinate system.

[0096] In some embodiments, the capture area may be determined by the following expression:

[0097] M a1 =A a2 -T1

[0098] M b1 =A b1 -(M a1 -A a1 )·tanθ-T3

[0099] M a2 =A a2 +T2

[0100] M b2 =Ab2 -(M a2 -A a2 )·tanθ+T4

[0101] Among them, M a1 and M b1 Represent the first direction coordinate and the second direction coordinate of the first vertex respectively, M a2 and M b2 Represent the first direction coordinate and the second direction coordinate of the second vertex respectively, A a1 and A b1 Represents the first direction coordinate and the second direction coordinate of the upper left corner vertex, A a2 and A b2 They represent the first direction coordinate and the second direction coordinate of the lower right corner vertex respectively, θ represents the deflection angle, T1 represents the first experience value, T2 represents the second experience value, T3 represents the third experience value, and T4 represents the fourth experience value.

[0102] In the embodiment of the present invention, see Figure 4 As shown, before step S23, the preset experience value can be determined by the following steps:

[0103] Step S41: for each image type, based on the first text box and the second text box at different deflection angles, respectively set an actual capture area covering the second text box;

[0104] Step S42: Obtain the coordinates of two diagonal vertices of the first text box and the coordinates of the third and fourth vertices of the actual capture area, where the third vertex is the vertex of the actual capture area closest to the upper left corner of the first text box, and the fourth vertex is the diagonal vertex of the third vertex;

[0105] Step S43: Determine a preset experience value based on the deflection angle, the coordinates of the two diagonal vertices of the first text box, and the coordinates of the third vertex and the fourth vertex, and store the preset experience value in correspondence with the image type, the text information of the first text box, the deflection angle, and the coordinates of the two diagonal vertices.

[0106] See also Figure 5 As shown, Figure 5 A schematic diagram of a sample image provided by an embodiment of the present invention is shown, based on which a preset experience value can be determined.

[0107] A two-dimensional coordinate system is established with a first direction a and a second direction b perpendicular to each other. The sample image includes a first text box C and a second text box D. The actual capture area is a dotted rectangular box. The coordinates of the two diagonal vertices of the first text box C can be obtained as (A a1 ,A b1 ) and (A a2 ,A b2), the coordinates of the two diagonal vertices of the second text box D are (B a1 ,B b1 ) and (B a2 ,B b2 ), the deflection angles of the first text box C and the second text box D are both θ, and the coordinates of the third vertex of the actual capture area are (M a01 ,M b01 ) and (M a02 ,M b02 ).

[0108] In some embodiments, step S41 may specifically involve adjusting the size of the actual capture area as needed, while ensuring that the actual capture area completely covers the second text box and does not overlap other text boxes. The actual capture area may be rectangular and have the same deflection angle as the first text box. In other embodiments, the capture area may also have other shapes.

[0109] In some embodiments, determining a preset empirical value in step S43 based on the deflection angle, the coordinates of the two diagonal vertices of the first text box, and the coordinates of the third vertex and the fourth vertex includes:

[0110] Determine the first intersection point based on the intersection of the first auxiliary line and the first extension line, where the first auxiliary line passes through the third vertex of the actual capture area and is parallel to the second direction, and the first extension line is an extension line passing through the upper left corner vertex and the upper right corner vertex of the first text box;

[0111] Determine a second intersection point based on the intersection of the second auxiliary line and the second extension line, the second auxiliary line passes through the fourth vertex of the actual capture area and is parallel to the second direction, the second extension line passes through the lower right corner vertex of the first text box, and the inclination angle of the second extension line is equal to the deflection angle;

[0112] Constructing a positional relationship model between the actual capture area and the first text box according to the first intersection point, the second intersection point, the preset experience value, and the deflection angle;

[0113] Based on the position relationship model, a preset experience value is determined according to the coordinates of the two diagonal vertices of the first text box and the coordinates of the third vertex and the fourth vertex of the actual capture area;

[0114] The preset experience values ​​include a first experience value, a second experience value, a third experience value, and a fourth experience value. The position relationship model is expressed as:

[0115] M a01 =A a2 -T1

[0116] M b01 =E b -T3

[0117] Eb =A b1 -(M a01 -A a1 )·tanθ

[0118] M a02 =A a2 +T2

[0119] M b02 =F b +T4

[0120] F b =A b2 -(M a02 -A a2 )·tanθ

[0121] Among them, M a01 and M b01 They represent the first direction coordinate and the second direction coordinate of the third vertex of the actual capture area, M a02 and M b02 They represent the first direction coordinate and the second direction coordinate of the fourth vertex of the actual capture area, E b Indicates the second direction coordinate of the first intersection point, F b Indicates the second direction coordinate of the second intersection point, A a1 and A b1 Represents the first direction coordinate and the second direction coordinate of the upper left corner vertex, A a2 and A b2 They represent the first direction coordinate and the second direction coordinate of the lower right corner vertex respectively, θ represents the deflection angle, T1 represents the first experience value, T2 represents the second experience value, T3 represents the third experience value, and T4 represents the fourth experience value.

[0122] See also Figure 6 As shown, a perpendicular line to the first auxiliary line can be drawn through the upper left corner vertex of the first text box C. The first intersection point E, the upper left corner vertex of the first text box C, and the intersection of the perpendicular line and the first auxiliary line can form a right triangle, wherein the angle between the first extension line and the perpendicular line is equal to the deflection angle θ. Based on this, the relationship in the position relationship model can be determined: E b =A b1 -(M a01 -A a1 )·tanθ.

[0123] Similarly, a perpendicular line to the second auxiliary line can be drawn through the lower right corner vertex of the first text box C. The second intersection point F, the lower right corner vertex of the first text box C, and the intersection of the perpendicular line and the second auxiliary line can form a right triangle, where the angle between the second extension line and the perpendicular line is equal to the deflection angle θ. Based on this, the relationship F in the position relationship model can be determined.b =A b2 -(M a02 -A a2 )·tanθ.

[0124] In some embodiments, step S24 may specifically include:

[0125] According to the first direction coordinate M of the first vertex of the capture area a1 and the first direction coordinate M of the second vertex a2 , determine that the first direction coordinate of the upper left corner vertex of the text box except the first text box is less than M a1 And the first direction coordinate of the lower right vertex is greater than M a2 and use this text box as the second text box.

[0126] As an example, when the text information of the first text box is "legal representative", the corresponding text box whose text information is the legal representative's specific name "Li Ming" is used as the second text box; when the text information of the first text box is "ID number", the corresponding text box whose text information is the specific ID number is used as the second text box.

[0127] In some embodiments, step S25 may specifically include: when a second text box is obtained, the text information of the first text box and the text information of the second text box are combined into a dictionary to obtain a recognition result of the image to be recognized.

[0128] As an example, the text information in the first text box is "legal representative", and the text information in the second text box is "Li Ming", so the dictionary can be obtained as {"legal representative": "Li Ming"}.

[0129] In other embodiments, step S25 may specifically include: when a plurality of second text boxes are obtained, sorting the plurality of second text boxes from closest to farthest according to the distance between the upper left corner vertex or the lower right corner vertex of each second text box and the upper left corner vertex or the lower right corner vertex of the first text box;

[0130] Based on the sorted order of the plurality of second text boxes, the text information of the first text box and the text information of the plurality of second text boxes are spliced ​​into a dictionary to obtain a recognition result of the image to be recognized.

[0131] As an example, multiple second text boxes are sorted from near to far according to the distance between their upper left corner vertices and the upper left corner vertices of the first text box, and the following is obtained: second text box B1 < second text box B2 < second text box B3. After the text information of the first text box and the text information of the multiple second text boxes are spliced ​​into a dictionary, {text_A: text_B1 + text_B2 + text_B3} can be obtained.

[0132] In other embodiments, step S25 may specifically include: when multiple second text boxes are obtained, sorting the multiple second text boxes according to the first direction coordinate size of the upper left corner vertices or the lower right corner vertices; or sorting the multiple second text boxes according to the second direction coordinate size of the upper left corner vertices or the lower right corner vertices;

[0133] The text information in the first text box and the sorted text information in the plurality of second text boxes are combined into a dictionary to obtain a recognition result of the image to be recognized.

[0134] The order of the multiple second text boxes can be determined based on the relative positions of the first text box and the second text box. For example, when the first text box is to the left of the second text box, and the first direction coordinate of the upper left corner vertex of the first text box is smaller than the first direction coordinates of the upper left corner vertices of all the second text boxes, the multiple second text boxes can be sorted from small to large based on the first direction coordinates of the upper left corner vertices of the second text boxes.

[0135] This method can determine the capture area based on image type, deflection angle, etc., and determine the second text box that matches the first text box based on the capture area. It is suitable for images to be recognized in tilted conditions and effectively improves the robustness of image recognition.

[0136] See also Figure 7 As shown, Figure 7 FIG. 1 is a flow chart of an image review method provided by an embodiment of the present invention, which may include:

[0137] Step S71: applying the image recognition method described in any of the above embodiments to perform image recognition on each image to be reviewed in the image set to be reviewed, and obtaining a recognition result for each image to be reviewed;

[0138] Step S72: for each image to be reviewed, review the image to be reviewed according to the recognition result of the image to be reviewed and the review mode corresponding to the image type of the image to be reviewed.

[0139] In some embodiments, step S72 may specifically include:

[0140] When the audit mode is information matching, the pre-stored information corresponding to the image type of the image to be audited is obtained; the recognition result of the image to be audited is matched with the pre-stored information, and the image to be audited is audited according to the matching result.

[0141] When the matching result is a successful match, the image to be reviewed passes the review; when the matching result is a failed match, the image to be reviewed fails the review.

[0142] In some other embodiments, step S72 may further be specifically as follows:

[0143] For each image type to be reviewed, the review mode corresponding to the text information is determined according to the text information corresponding to each second text box in the image to be reviewed, wherein the review mode may include information matching and / or condition matching.

[0144] When the audit mode is information matching, the pre-stored text information corresponding to the second text box is obtained from the pre-stored information corresponding to the image type, and it is determined whether the text information of the identified second text box is consistent with the pre-stored text information. If they are consistent, the match is successful and the audit is passed; if they are inconsistent, the match fails and the audit fails.

[0145] When the review mode is conditional matching, determine whether the text information corresponding to the second text box meets the preset conditions. As an example, when the text information corresponding to the second text box is a date, it can be determined whether the date is within the validity period, and whether the review is passed is determined based on the judgment result.

[0146] The above is an image review method provided by an embodiment of the present invention, which can realize the simultaneous review of multiple different image types of images to be reviewed, and call the image recognition model and review mode corresponding to each image type. It can effectively improve the review accuracy while improving the review speed. In addition, this method identifies the images to be reviewed based on the image recognition model described in the above embodiment, which can effectively improve the robustness of image recognition and further improve the review accuracy.

[0147] Another aspect of the present invention provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the image recognition method or image review method described in any of the above embodiments. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, in embodiments of the present invention, the computer-readable storage medium is non-transitory.

[0148] In another aspect of the present invention, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the image recognition method or image review method described in any of the above embodiments is implemented.

[0149] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. An image recognition method, characterized in that: include: Obtain the image type of each image to be identified in the image set to be identified; For each of the images to be recognized, determining an image recognition model corresponding to the image to be recognized according to the image type of the image to be recognized, and inputting the image to be recognized into the image recognition model corresponding to the image to be recognized, so as to obtain a recognition result of the image to be recognized using the image recognition model; Wherein, obtaining the recognition result of the image to be recognized by using the image recognition model includes: Obtaining a text box of the image to be recognized and obtaining the coordinates of two diagonal vertices, a deflection angle, and text information of the text box; wherein the two diagonal vertex coordinates include the coordinates of the upper left vertex and the coordinates of the lower right vertex; According to the target recognition content corresponding to the image type, a first text box whose text information is the target recognition content is selected from the acquired text boxes; determining a capture area according to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle; For the remaining text boxes except the first text box among the text boxes corresponding to the image to be recognized, a second text box located in the capture area is selected from the remaining text boxes, and the text information corresponding to the second text box matches the target recognition content; Obtaining a recognition result of the image to be recognized based at least on the text information in the first text box and the text information in the second text box; wherein determining a capture area based on the image type, the text information in the first text box, the coordinates of the two diagonal vertices, and the deflection angle includes: Determining preset empirical values ​​corresponding to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle, wherein the upper left vertex coordinates include first and second direction coordinates of the upper left vertex, the lower right vertex coordinates include first and second direction coordinates of the lower right vertex, and the preset empirical values ​​include a first empirical value, a second empirical value, a third empirical value, and a fourth empirical value; Determining the first direction coordinate of a first vertex of the capture area according to the first direction coordinate of the lower right corner vertex of the first text box and the first empirical value, where the first vertex is the vertex of the capture area that is closest to the upper left corner vertex of the first text box; Determine the first direction coordinate of a second vertex of the capture area according to the first direction coordinate of the lower right corner vertex and the second empirical value; the second vertex is a diagonal vertex of the first vertex; Determine the second direction coordinate of the first vertex according to the first direction coordinate of the first vertex, the first direction coordinate of the upper left corner vertex, the second direction coordinate of the upper left corner vertex, the deflection angle, and the third empirical value; Determine the second direction coordinate of the second vertex according to the first direction coordinate of the second vertex, the first direction coordinate of the lower right corner vertex, the second direction coordinate of the lower right corner vertex, the deflection angle, and the fourth empirical value; The capture area is determined according to the first direction coordinate and the second direction coordinate of the first vertex and the first direction coordinate and the second direction coordinate of the second vertex.

2. The method according to claim 1, characterized in that The capture area is determined by the following expression: , , , ,in, and represent the first direction coordinate and the second direction coordinate of the first vertex respectively, and represent the first direction coordinate and the second direction coordinate of the second vertex respectively, and Respectively represent the first direction coordinate and the second direction coordinate of the upper left corner vertex, and denote the first direction coordinate and the second direction coordinate of the lower right corner vertex respectively, θ denotes the deflection angle, represents the first experience value, represents the second experience value, represents the third experience value, represents the fourth experience value.

3. The method according to claim 1 or 2, characterized in that Before determining the capture area according to the image type, the text information of the first text box, the coordinates of the two diagonal vertices, and the deflection angle, the method further includes: For each of the image types, based on the first text box and the second text box at different deflection angles, respectively setting an actual capture area covering the second text box; Obtaining the coordinates of the two diagonal vertices of the first text box and the coordinates of the third vertex and the fourth vertex of the actual capture area, wherein the third vertex is the vertex of the actual capture area closest to the upper left corner vertex of the first text box, and the fourth vertex is the diagonal vertex of the third vertex; Based on the deflection angle, the coordinates of the two diagonal vertices of the first text box, and the coordinates of the third vertex and the fourth vertex, the preset experience value is determined and stored in correspondence with the image type, the text information of the first text box, the deflection angle and the coordinates of the two diagonal vertices.

4. The method according to claim 3, characterized in that The determining the preset empirical value according to the deflection angle, the coordinates of the two diagonal vertices of the first text box, and the coordinates of the third vertex and the fourth vertex includes: Determining a first intersection point according to an intersection point of a first auxiliary line and a first extension line, wherein the first auxiliary line passes through the third vertex of the actual capture area and is parallel to the second direction, and the first extension line is an extension line passing through the upper left corner vertex and the upper right corner vertex of the first text box; Determining a second intersection point based on an intersection point of a second auxiliary line and a second extension line, wherein the second auxiliary line passes through the fourth vertex of the actual capture area and is parallel to the second direction, and the second extension line passes through the lower right corner vertex of the first text box and an inclination angle of the second extension line is equal to the deflection angle; Constructing a positional relationship model between the actual capture area and the first text box according to the first intersection point, the second intersection point, the preset empirical value, and the deflection angle; Based on the positional relationship model, determining the preset empirical value according to the coordinates of the two diagonal vertices of the first text box and the coordinates of the third vertex and the fourth vertex of the actual capture area; The preset experience values ​​include a first experience value, a second experience value, a third experience value, and a fourth experience value, and the position relationship model is expressed as: , , , , , ,in, and respectively represent the first direction coordinate and the second direction coordinate of the third vertex of the actual capture area, and respectively represent the first direction coordinate and the second direction coordinate of the fourth vertex of the actual capture area, represents the second direction coordinate of the first intersection point, represents the second direction coordinate of the second intersection point, and Respectively represent the first direction coordinate and the second direction coordinate of the upper left corner vertex, and denote the first direction coordinate and the second direction coordinate of the lower right corner vertex respectively, θ denotes the deflection angle, represents the first experience value, represents the second experience value, represents the third experience value, represents the fourth experience value.

5. The method according to claim 1, wherein The image recognition model includes a text detection sub-model and a character recognition sub-model. The step of obtaining a text box of an image to be recognized and obtaining the coordinates of two diagonal vertices, a deflection angle, and character information of the text box includes: Inputting the image to be recognized into a text detection sub-model, obtaining a text box area map of the image to be recognized and the coordinates of the two diagonal vertices and the deflection angle of each text box in the text box area map; The text box area map is input into a text recognition model to obtain text information corresponding to the text box.

6. An image review method, characterized in that: include: Applying the image recognition method according to any one of claims 1 to 5 to perform image recognition on each image to be reviewed in the image set to be reviewed, and obtaining a recognition result for each image to be reviewed; For each of the images to be reviewed, the image to be reviewed is reviewed according to the recognition result of the image to be reviewed and the review mode corresponding to the image type of the image to be reviewed.

7. The method according to claim 6, characterized in that The reviewing of the image to be reviewed according to the recognition result of the image to be reviewed and the review mode corresponding to the image type of the image to be reviewed includes: When the audit mode is information matching, obtaining pre-stored information corresponding to the image type of the image to be audited; The recognition result of the image to be reviewed is matched with the pre-stored information, and the image to be reviewed is reviewed according to the matching result.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the image recognition method described in any one of claims 1 to 5 or the image review method described in claim 6 or 7.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the image recognition method according to any one of claims 1 to 5 or the image review method according to claim 6 or 7 is implemented.

Images