Image recognition method and device, electronic equipment and computer readable medium
By preprocessing the images and identifying the handwritten text and seal areas of interest, the problem of identifying large-scale image content in the prior art is solved, and accurate content recognition and storage is achieved.
Patent Information
- Application Number
- CN202411515387.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-10-29
AI Technical Summary
The prior art is difficult to effectively and accurately identify large quantities of image content containing handwritten text, non-handwritten text and seals.
By obtaining the image to be identified, image preprocessing is performed to generate preprocessed images, identify handwritten text and areas of interest of seals, non-handwritten text, handwritten text and seal recognition, content verification and extraction are performed, and data records are generated for storage.
It realizes effective and accurate recognition of content in the image, improves image quality, automatically extracts non-handwritten text and locates handwritten text and seal areas, verifys the validity of the content, and generates data records for data storage.
Smart Images

Figure CN119049072A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an image recognition method, device, electronic device, and computer-readable medium. Background Art
[0002] With the development of computer-related technologies, data electronicization has become a mainstream trend. However, there are a large number of text files (for example, power of attorney) that are retained in the form of images. How to effectively and accurately identify the content of large quantities of images has become an urgent problem to be solved.
[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention
[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0005] Some embodiments of the present disclosure propose image recognition methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.
[0006] In a first aspect, some embodiments of the present disclosure provide an image recognition method, the method comprising: acquiring an image to be recognized, wherein the image to be recognized is an image containing handwritten text, non-handwritten text and a seal image; performing image preprocessing on the image to be recognized to generate a preprocessed image; performing non-handwritten text recognition on the preprocessed image to generate a first text information; determining a handwritten text interest region and a seal interest region in the image to be recognized based on the first text information; performing handwritten text recognition on the handwritten text interest region in the image to be recognized to generate a second text information; performing seal recognition on the seal interest region in the image to be recognized to generate seal information; performing content verification on the first text information, the second text information and the seal information; in response to passing the content verification, performing content extraction on the first text information, the second text information and the seal information to generate a data record for the image to be recognized; and performing data storage on the data record and the image to be recognized.
[0007] Optionally, the above-mentioned image preprocessing of the image to be identified to generate a preprocessed image includes: performing distortion correction on the image to be identified to generate a distortion-corrected image; performing image resizing on the distortion-corrected image to obtain an image after image resizing; performing image enhancement on the image after image resizing to generate an image enhanced image; performing character region of interest recognition on the image enhanced image to determine a set of character regions of interest; and performing character shadow removal on the image enhanced image based on the set of character regions of interest to obtain the preprocessed image.
[0008] Optionally, the above-mentioned content verification of the above-mentioned first text information, the above-mentioned second text information and the above-mentioned seal information includes: performing tampering verification based on the above-mentioned first text information to generate a first verification result; performing format verification based on the above-mentioned second text information to generate a second verification result; performing seal anti-counterfeiting verification based on the above-mentioned seal information to generate a third verification result; generating a content verification result based on the above-mentioned first verification result, the above-mentioned second verification result and the above-mentioned third verification result.
[0009] Optionally, the first text information includes: a first text description information set, the first text description information in the first text description information set includes: a character description information group and a text semantic feature; and the non-handwritten text recognition on the preprocessed image to generate the first text information includes: according to the area position corresponding to the character interest area, filtering out the character interest area that meets the first filtering condition from the character interest area set as the first character interest area, and obtaining the first character interest area set; according to the area position corresponding to the character interest area, filtering out the character interest area that meets the second filtering condition from the character interest area set as the second character interest area, and obtaining the second character interest area set; according to the above According to the first character area of interest set, a first semantic association is determined, wherein the first semantic association represents the character semantic association of the character set corresponding to the first character area of interest set; according to the second character area of interest set, a second semantic association is determined, wherein the second semantic association represents the character semantic association of the character set corresponding to the second character area of interest set; in response to the first semantic association being greater than a preset semantic association threshold and the first semantic association being greater than the second semantic association, the horizontal recognition direction is determined as the text recognition direction; in response to the second semantic association being greater than the preset semantic association threshold and the second semantic association being greater than the first semantic association, the vertical recognition direction is determined as the text recognition direction.
[0010] Optionally, the above-mentioned non-handwritten text recognition is performed on the above-mentioned preprocessed image to generate the first text information, and further includes: according to the above-mentioned text recognition direction, dividing the above-mentioned character interest area set into a character interest area group set, wherein the character interest areas in the character interest area group are located in the same text line; for each character interest area group in the above-mentioned character interest area group set, performing the following processing steps: determining the character features corresponding to the characters in each character interest area in the above-mentioned character interest area group through a pre-trained character feature extraction model to obtain a character feature group, wherein the character features in the above-mentioned character feature group include: font shape features, font size features, font color features and font meaning features; determining the first text description information corresponding to the above-mentioned character interest area group in the above-mentioned first text description information set through the character recognition model and the text semantic extraction model according to the above-mentioned character feature group.
[0011] In a second aspect, some embodiments of the present disclosure provide an image recognition device, the device comprising: an acquisition unit, configured to acquire an image to be recognized, wherein the image to be recognized is an image containing handwritten text, non-handwritten text and a seal image; an image preprocessing unit, configured to perform image preprocessing on the image to be recognized to generate a preprocessed image; a non-handwritten text recognition unit, configured to perform non-handwritten text recognition on the preprocessed image to generate first text information; a determination unit, configured to determine a handwritten text interest region and a seal interest region in the image to be recognized based on the first text information; the handwritten text recognition unit, configured to perform non-handwritten text recognition on the preprocessed image to generate first text information; The handwritten text is recognized in the handwritten text interest area in the to-be-recognized image to generate the second text information; the seal recognition unit is configured to perform seal recognition on the seal interest area in the to-be-recognized image to generate the seal information; the content verification unit is configured to perform content verification on the first text information, the second text information and the seal information; the content extraction unit is configured to extract content from the first text information, the second text information and the seal information in response to passing the content verification to generate a data record for the to-be-recognized image; the data storage unit is configured to store the data record and the to-be-recognized image.
[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.
[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.
[0014] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the image processing method of some embodiments of the present disclosure, effective and accurate content recognition for the content contained in the image is achieved. Specifically, first, an image to be recognized is obtained, wherein the above-mentioned image to be recognized is an image containing handwritten text, non-handwritten text and a seal image. That is, the image to be recognized contains a variety of contents to be recognized, such as handwritten text, non-handwritten text and a seal image. Then, the above-mentioned image to be recognized is image preprocessed to generate a preprocessed image. The image quality is improved by image preprocessing. Further, the above-mentioned preprocessed image is subjected to non-handwritten text recognition to generate a first text information. In this way, the non-handwritten text contained in the preprocessed image is automatically extracted. In addition, according to the above-mentioned first text information, the handwritten text interest area and the seal interest area in the above-mentioned image to be recognized are determined. In this way, according to the first text information, the area where the handwritten text and the seal image may exist is located. In addition, handwritten text recognition is performed on the above-mentioned handwritten text interest area in the above-mentioned image to be recognized to generate a second text information. Next, seal recognition is performed on the seal area of interest in the image to be recognized to generate seal information. In this way, automatic recognition of handwritten text and seals is achieved. Next, content verification is performed on the first text information, the second text information and the seal information. In this way, the validity of the content is verified. Further, in response to the content verification being passed, content extraction is performed on the first text information, the second text information and the seal information to generate a data record for the image to be recognized. In this way, attribute extraction is performed on the first text information, the second text information and the seal information to obtain corresponding data records for data storage. Finally, data is stored on the data record and the image to be recognized. In this way, effective and accurate content recognition of the content contained in the image is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0016] Figure 1 is a flow chart of some embodiments of the image recognition method according to the present disclosure; Figure 2 is a schematic diagram of an image to be recognized in some embodiments of the image recognition method disclosed herein; Figure 3 is another schematic diagram of an image to be recognized in some embodiments of the image recognition method of the present disclosure; Figure 4 is a schematic diagram of the positions of the first character region of interest set and the second character region of interest set in some embodiments of the image recognition method disclosed herein; Figure 5 is a schematic diagram of a process of generating first text information in some embodiments of the image recognition method disclosed herein; Figure 6 is a schematic diagram of the position of a text fence in a pre-processed image in some embodiments of the image recognition method disclosed herein; Figure 7 A schematic diagram of the structure of some embodiments of the image recognition device according to the present disclosure; Figure 8 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0023] refer to Figure 1 , shows a process 100 of some embodiments of the image recognition method according to the present disclosure. The image recognition method comprises the following steps: Step 101: Obtain an image to be recognized.
[0024] In some embodiments, the execution subject (e.g., a computing device) of the image recognition method can obtain the image to be recognized through a wired connection or a wireless connection. The image to be recognized is an image containing handwritten text, non-handwritten text, and a seal image. In practice, the execution subject can obtain the image to be recognized uploaded by the user from the user end in real time.
[0025] As an example, see Figure 2 A schematic diagram of an image to be recognized is shown, and Figure 3 Another schematic diagram of the image to be recognized is shown in FIG. Figure 2 and Figure 3 The text in the solid line frame is handwritten text, the text in the dotted line frame is the seal image, and the text outside the dotted line frame and the solid line frame is non-handwritten text. Figure 2 and Figure 3 The text and seal can have corresponding colors according to actual needs, such as the seal can be red.
[0026] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.
[0027] It should be noted that the above-mentioned computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here. It should be understood that the number of computing devices can be any number according to the implementation needs.
[0028] Step 102: perform image preprocessing on the image to be recognized to generate a preprocessed image.
[0029] In some embodiments, the execution subject may perform image preprocessing on the image to be identified to generate a preprocessed image. In practice, the execution subject may perform image cropping on the image to be identified to obtain the preprocessed image.
[0030] In some optional implementations of some embodiments, the execution subject performs image preprocessing on the image to be recognized to generate a preprocessed image, which may include the following steps: The first step is to perform distortion correction on the image to be identified to generate a distortion-corrected image.
[0031] In practice, first, the execution subject may collect feature points of the image to be identified to obtain a feature point set. Then, the difference between the feature points in the feature point set and the feature points in the real world is compared to determine the distortion parameters. Then, according to the distortion parameters, the image to be identified is corrected to obtain the above-mentioned distortion-corrected image.
[0032] In the second step, the image size is adjusted for the above-mentioned distortion-corrected image to obtain an image after image size adjustment.
[0033] In practice, the execution subject may adjust the image size of the distortion-corrected image to a preset size through operations such as stretching and cropping to obtain the above-mentioned image after image size adjustment. Specifically, when the image size of the distortion-corrected image is adjusted through cropping, the distortion-corrected image is cropped with the boundary points of the blank areas around the image as the boundaries to obtain the above-mentioned image after image size adjustment.
[0034] The third step is to perform image enhancement on the above-mentioned image after image resizing to generate an image enhanced image.
[0035] In practice, the execution subject may perform image enhancement on the image after the image resizing by using an image enhancement method based on a Laplace operator to generate an image after image enhancement.
[0036] The fourth step is to perform character region of interest recognition on the enhanced image to determine a set of character region of interest.
[0037] In practice, the execution subject may perform character region of interest recognition on the image after image enhancement through a character region of interest recognition model to determine a set of character regions of interest. The character region of interest recognition model includes: a first downsampling network, a second downsampling network, a feature fusion layer and a detection head. The first downsampling network includes: a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer and a fifth convolution layer. The output of the first convolution layer is the input of the second convolution layer. The output of the second convolution layer is the input of the third convolution layer. The output of the third convolution layer is the input of the fourth convolution layer. The output of the fourth convolution layer is the input of the fifth convolution layer. The output size of the first convolution layer is 1 / 2 of the image size of the image after image enhancement. The output size of the second convolution layer is 1 / 4 of the image size of the image after image enhancement. The output size of the third convolution layer is 1 / 4 of the image size of the image after image enhancement. The output size of the third convolution layer is 1 / 8 of the image size of the image after image enhancement. The output size of the fourth convolution layer is 1 / 16 of the image size of the image after image enhancement. The output size of the fifth convolutional layer is 1 / 32 of the image size of the image after image enhancement. The second downsampling network includes: the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer. Among them, the input of the sixth convolutional layer is the superposition of the output of the second convolutional layer and the output of the seventh convolutional layer. The input of the seventh convolutional layer is the superposition of the output of the third convolutional layer and the output of the eighth convolutional layer. The input of the eighth convolutional layer is the superposition of the output of the fourth convolutional layer and the output of the fifth convolutional layer. The feature fusion layer is used to superimpose the output of the fifth convolutional layer, the output of the sixth convolutional layer, the input of the seventh convolutional layer and the output of the eighth convolutional layer. Specifically, before the feature fusion layer is superimposed, the output of the fifth convolutional layer will be upsampled 8 times, the output of the seventh convolutional layer will be upsampled 2 times, and the output of the eighth convolutional layer will be upsampled 4 times.
[0038] The fifth step is to remove character shadows from the image after image enhancement according to the character region of interest set to obtain the preprocessed image.
[0039] In practice, the above-mentioned execution subject can perform binarization processing on the pixels in the character's region of interest by setting a pixel threshold, so as to eliminate the shadow part existing in the image acquisition process due to jitter.
[0040] Step 103: Perform non-handwritten text recognition on the preprocessed image to generate first text information.
[0041] In some embodiments, the above-mentioned execution entity may perform non-handwritten text recognition on the preprocessed image to generate first text information. For example, the above-mentioned execution entity may perform non-handwritten text recognition on the preprocessed image through general OCR (Optical Character Recognition) to generate first text information.
[0042] Optionally, the first text information includes: a first set of text description information, and the first text description information in the first set of text description information includes: a character description information group and a text semantic feature. The first text description information represents a text description at the sentence granularity. The character description information represents a description of a character. For example, the character description information may include: characters. The text semantic feature represents the sentence semantics of the sentence corresponding to the first text description information.
[0043] In some optional implementation manners of some embodiments, the above-mentioned execution entity performing non-handwritten text recognition on the preprocessed image to generate first text information may include the following steps: First step, according to the regional position corresponding to the character region of interest, screen out the character regions of interest that meet the first screening condition from the above-mentioned set of character regions of interest as the first character regions of interest, and obtain a first set of character regions of interest.
[0044] Among them, the first character region of interest in the first set of character regions of interest is the character region of interest whose corresponding regional position is in the row where the middle regional position is located.
[0045] Second step, according to the regional position corresponding to the character region of interest, screen out the character regions of interest that meet the second screening condition from the above-mentioned set of character regions of interest as the second character regions of interest, and obtain a second set of character regions of interest.
[0046] Among them, the second character region of interest in the second set of character regions of interest is the character region of interest whose corresponding regional position is in the column where the middle regional position is located.
[0047] As an example, Figure 4 As shown in the position schematic diagrams of the first set of character regions of interest and the second set of character regions of interest, where the first set of character regions of interest 401 includes 8 second character regions of interest for "show" and 8 first character regions of interest for "example". The second set of character regions of interest 402 includes 3 second character regions of interest for "example".
[0048] Third step, determine a first semantic association degree according to the above-mentioned first set of character regions of interest.
[0049] Among them, the above-mentioned first semantic association degree represents the character semantic association degree of the character set corresponding to the first character region of interest set.
[0050] In practice, the above-mentioned execution entity determines the first semantic association degree according to the above-mentioned first character region of interest set through a semantic association degree prediction model. Among them, the semantic association degree prediction model includes: a recurrent neural network model and a predictor. The predictor is used to output the semantic association degree of the character set corresponding to the first character region of interest set.
[0051] Determine the second semantic association degree according to the above-mentioned second character region of interest set.
[0052] In practice, the above-mentioned execution entity determines the second semantic association degree according to the above-mentioned second character region of interest set through a semantic association degree prediction model. Among them, the above-mentioned second semantic association degree represents the character semantic association degree of the character set corresponding to the second character region of interest set.
[0053] As an example, further refer to Figure 4 , the first character region of interest set 401 includes 8 second character regions of interest for "示" and 8 first character regions of interest for "例". There is a certain semantic relationship between them. Therefore, the corresponding first semantic association degree is greater than 0. The 3 second character regions of interest for "例" included in the second character region of interest set 402 do not have a semantic relationship with the corresponding character "例". Therefore, the corresponding second semantic association degree can be 0. Corresponding to Figure 2 and Figure 3 the text arrangement method, when the text is arranged horizontally, the first semantic association degree is greater than the second semantic association degree. When the text is arranged vertically, the second semantic association degree is greater than the first semantic association degree.
[0054] Fourth step, in response to the above-mentioned first semantic association degree being greater than the preset semantic association degree threshold and the above-mentioned first semantic association degree being greater than the above-mentioned second semantic association degree, determine the horizontal recognition direction as the text recognition direction.
[0055] Fifth step, in response to the above-mentioned second semantic association degree being greater than the above-mentioned preset semantic association degree threshold and the above-mentioned second semantic association degree being greater than the above-mentioned first semantic association degree, determine the vertical recognition direction as the text recognition direction.
[0056] Sixth step, according to the above-mentioned text recognition direction, divide the above-mentioned character region of interest set into a character region of interest group set.
[0057] Among them, the character regions of interest in the character region of interest group are located in the same text line.
[0058] Seventh step, for each character region of interest group in the above-mentioned character region of interest group set, perform the following processing steps: In the first sub-step, a character feature extraction model trained in advance is used to determine the character feature corresponding to each character in the character ROI group to obtain a character feature group.
[0059] The character features in the above character feature group include: font shape features, font size features, font color features and font meaning features.
[0060] In practice, see Figure 5 The schematic diagram of the generation process of the first text information shown in the figure, the character feature extraction model 502 includes: a shallow character feature extraction model 5021, a font feature extraction model 5022, a font size feature extraction model 5023, a font color feature extraction model 5024 and a character meaning feature extraction model 5025. Among them, the shallow character feature extraction model 5021 adopts a residual neural network model as the backbone network. The model structures of the font feature extraction model 5022, the font size feature extraction model 5023, the font color feature extraction model 5024 and the character meaning feature extraction model 5025 are consistent, and all adopt a convolutional neural network model with 5 layers of convolutional layers connected in series.
[0061] The second sub-step is to determine the first text description information corresponding to the above-mentioned character area of interest group in the above-mentioned first text description information set through a character recognition model and a text semantic extraction model according to the above-mentioned character feature group.
[0062] In practice, further reference Figure 5, the character region of interest group 501 may include: the character region of interest corresponding to the character "执", the character region of interest corresponding to the character "行", the character region of interest corresponding to the character "事", the character region of interest corresponding to the character "务", the character region of interest corresponding to the character "合", the character region of interest corresponding to the character "伙", and the character region of interest corresponding to the character "人". Specifically, for each character region of interest, the character feature extraction model 502 performs corresponding character feature extraction and inputs the extracted character features into the character recognition model 503. Specifically, the character recognition model includes a character recognizer for determining the character and confidence corresponding to the character features, and takes the recognized character, character type (handwritten or non-handwritten character), and character position (representing the position of the character in the image to be recognized) as character description information 505. In addition, the character feature extraction model 502 inputs the character features corresponding to the character region of interest corresponding to the character "执", the character region of interest corresponding to the character "行", the character region of interest corresponding to the character "事", the character region of interest corresponding to the character "务", the character region of interest corresponding to the character "合", the character region of interest corresponding to the character "伙", and the character region of interest corresponding to the character "人" into the text semantic extraction model 504 respectively to obtain corresponding text semantic features 506. In practice, the text semantic extraction model uses a recurrent neural network model.
[0063] Step 104, determine the handwritten text region of interest and the seal region of interest in the image to be recognized according to the first text information.
[0064] In some embodiments, the above execution entity may determine the handwritten text region of interest and the seal region of interest in the image to be recognized according to the first text information. Among them, the handwritten text region of interest is the region that may contain handwritten text, and the seal region of interest is the region that may contain a seal.
[0065] In some optional implementation manners of some embodiments, the above execution entity determines the handwritten text region of interest and the seal region of interest in the image to be recognized according to the first text information, which may include the following steps: The first step is to determine the text fence according to the above first text information.
[0066] In practice, the above execution entity may use the region in the preprocessed image where the positions of the characters included in the first text information are located as the text fence.
[0067] As an example, as Figure 6 shown in the schematic diagram of the position of the text fence in the preprocessed image, where the text fence 601 is the region where the positions of the characters included in the first text information are located.
[0068] The second step is to determine the difference between the text fence and the preprocessed image to obtain a difference image.
[0069] In practice, difference image = preprocessed image - preprocessed image containing the text fence.
[0070] The third step is to divide the above difference image into a set of image blocks.
[0071] Among them, the image blocks in the above-mentioned image block set have the same size.
[0072] In the fourth step, for each image block in the image block set, the handwritten text confidence and seal confidence corresponding to the image block are determined according to the image block and the first text information.
[0073] Among them, the handwritten text confidence represents the confidence that the image block contains handwritten text, and the seal confidence represents the confidence that the image block contains a seal. In practice, the above-mentioned execution entity can determine the handwritten text confidence and seal confidence corresponding to the image block through binary classifier A and binary classifier B, respectively. Specifically, binary classifier A is used to classify whether the image block contains handwritten text and the corresponding confidence. Binary classifier B is used to classify whether the image block contains the confidence corresponding to the seal box.
[0074] In the fifth step, according to the handwritten text confidence and seal confidence corresponding to the image block, the above-mentioned image block set is subjected to image block filtering to obtain a first image block set and a second image block set.
[0075] In practice, the execution subject can set the handwritten text confidence threshold and the seal confidence threshold respectively, and filter the image block set according to the handwritten text confidence and seal confidence corresponding to the image block to obtain the first image block set and the second image block set. The first image block in the first image block set is an image block containing handwritten text. The second image block in the second image block set is an image block containing a seal.
[0076] Step 6: stitch adjacent image blocks of the first image block set to obtain the handwritten text region of interest.
[0077] In the seventh step, adjacent image blocks of the second image block set are spliced to obtain the seal region of interest.
[0078] The above-mentioned content of "in some optional implementations of some embodiments" is used as an inventive point. Specifically, by determining the text fence, that is, blocking the area corresponding to the non-handwritten text in the preprocessed image, which is specifically expressed as setting the pixel value to 0, the non-handwritten text area is avoided from participating in the subsequent feature calculation, which greatly reduces the feature calculation amount. In addition, by dividing the image blocks, that is, constraining the recognition granularity, the problem of large data calculation amount due to too small granularity and inaccurate recognition due to too large granularity is avoided. Thereby, the handwritten text area of interest and the seal area of interest are effectively and accurately identified.
[0079] Step 105 , performing handwritten text recognition on the handwritten text interest region in the image to be recognized to generate second text information.
[0080] In some embodiments, the execution subject may perform handwritten text recognition on the handwritten text interest area in the image to be recognized to generate the second text information. The second text description information set includes: a character description information group and text semantic features. In practice, the execution subject may generate the second text information through a shallow character feature extraction model 5021, a glyph feature extraction model 5022, a character recognition model 503, and a text semantic feature extraction model 504. Specifically, since there are differences between handwritten characters and non-handwritten characters, before performing handwritten text recognition, the shallow character feature extraction model 5021, the glyph feature extraction model 5022, the character recognition model 503, and the text semantic feature extraction model 504 may be trained with training samples corresponding to non-handwritten texts. After the training, the model may be trained with a small amount of training samples corresponding to handwritten texts by means of transfer learning to ensure the accuracy of the model in non-handwritten text recognition.
[0081] Step 106 , performing seal recognition on the seal interest region in the image to be recognized to generate seal information.
[0082] In some embodiments, the execution entity may perform seal recognition on the seal interest region in the image to be recognized to generate seal information.
[0083] In some optional implementations of some embodiments, the execution subject performs seal recognition on the seal interest area in the image to be recognized to generate seal information, which may include the following steps: The first step is to determine the seal boundary, seal text direction and seal type based on the above seal area of interest.
[0084] The seal type represents the boundary type corresponding to the seal boundary.
[0085] In practice, first, the above-mentioned execution entity can determine the seal boundary according to the above-mentioned seal area of interest through the Hough transform algorithm. Then, the seal type is determined according to the graphic type of the seal boundary. For example, if the seal boundary is circular or elliptical, the corresponding seal type can be an official seal type. For another example, if the seal boundary is rectangular or square, the corresponding seal type can be a private seal type. In addition, the above-mentioned execution entity can extract the characters within the seal boundary through the shallow character feature extraction model 5021, and according to the position of the extracted characters in the seal area of interest, fit the direction line corresponding to the direction of the seal text.
[0086] In the second step, according to the seal type and the seal text direction, the seal boundary is segmented axially symmetrically to obtain a first segmented seal boundary and a second segmented seal boundary.
[0087] The first segmented seal boundary and the second segmented seal boundary are axisymmetric closed boundaries.
[0088] The third step is to segment the seal region of interest according to the seal boundary after the first segmentation to obtain the seal region of interest after the first segmentation.
[0089] The seal region of interest after the first segmentation is a region surrounded by the seal boundary after the first segmentation.
[0090] The fourth step is to segment the seal region of interest according to the seal boundary after the second segmentation to obtain the seal region of interest after the second segmentation.
[0091] The second segmented seal region of interest is a region surrounded by the second segmented seal boundary.
[0092] The fifth step is to perform seal text recognition on the seal region of interest after the first segmentation to obtain the first seal text information.
[0093] The first seal text information includes: seal text content and seal text position. Specifically, the execution subject can perform character recognition on the seal interest region after the first segmentation through a character recognition model to obtain the seal text content, and obtain the seal text position according to the position of the characters contained in the seal text content in the seal interest region after the first segmentation.
[0094] The sixth step is to perform seal text recognition on the corresponding position in the second segmented seal region of interest according to the seal text position included in the first seal text information to obtain the second seal text information.
[0095] Wherein, the second seal text information includes: seal text content and seal text position. Specifically, the execution subject can perform character recognition on the second segmented seal region of interest through a character recognition model to obtain the seal text content, and according to the seal text content, the position of the characters contained in the second segmented seal region of interest is obtained to obtain the seal text position. Specifically, when identifying the second segmented region of interest, axial symmetry can be relied on to directly extract features from the corresponding positions in the second segmented region of interest that contain text in the first segmented seal region of interest, thereby reducing the amount of feature processing.
[0096] The seventh step is to generate the above-mentioned seal information according to the above-mentioned first seal text information and the above-mentioned second seal text information.
[0097] In practice, the first seal text information and the second seal text information are spliced together to obtain the seal information.
[0098] The above-mentioned content of "in some optional implementations of some embodiments" is an inventive point of the present disclosure. Specifically, since the appearance of a seal often has axial symmetry, the present disclosure realizes the symmetrical division of the seal's region of interest by determining the seal boundary. At the same time, considering that a seal may correspond to multiple axes of symmetry, in order to perform axial symmetrical division of the content contained in the seal (such as text, etc.), further axial symmetrical division is performed on the basis of the seal boundary and in combination with the direction of the seal text. Then, the region of interest after the first segmentation is identified to identify part of the content contained in the seal. At the same time, due to the axial symmetry, the content can be directly identified at the symmetrical position in the region of interest after the second segmentation, which reduces the amount of feature processing and greatly improves the recognition efficiency.
[0099] Step 107: Verify the contents of the first text information, the second text information, and the seal information.
[0100] In some embodiments, the above-mentioned execution entity can verify the content of the first text information, the second text information and the seal information in various ways.
[0101] In some optional implementations of some embodiments, the execution subject performs content verification on the first text information, the second text information, and the seal information, which may include the following steps: The first step is to perform tampering verification based on the first text information to generate a first verification result.
[0102] In practice, the execution subject can determine whether the characters corresponding to the first text information are erased or tampered with based on the glyph features of the characters corresponding to the first text information and the text erase classifier. The text erase classifier is a binary classifier that takes the output of the character feature extraction model as input. The first verification result indicates whether the first text information contains erased or tampered characters.
[0103] The second step is to perform format verification based on the second text information to generate a second verification result.
[0104] In practice, the execution subject can perform format verification on the second text information through regular expressions related to multiple pre-built filling rules to generate a second verification result. The second verification result indicates whether the format verification is passed. For example, the second text information may include "Securities Account Opening" and "Securities Account Cancellation", which indicates that the user applies for account opening and account cancellation at the same time, so it does not meet the actual business process, and the regular expression can be used to filter out the format errors.
[0105] The third step is to perform seal anti-counterfeiting verification based on the seal information to generate a third verification result.
[0106] In practice, when the seal is an official seal, it is possible to search the seal database to see whether there is a seal record matching the seal information, so as to generate a third verification result indicating whether the seal is abnormal.
[0107] The fourth step is to generate a content verification result based on the first verification result, the second verification result and the third verification result.
[0108] In practice, the execution subject may generate a content verification result based on the first verification result, the second verification result, and the third verification result. Specifically, when any of the first verification result, the second verification result, and the third verification result indicates that the verification has not been passed, a content verification result indicating that the content has not been verified is generated. When the first verification result, the second verification result, and the third verification result all indicate that the verification has been passed, a content verification result indicating that the content has been verified is generated.
[0109] Optionally, the execution entity may also generate a fourth verification result according to the positional relationship between the seal information, the first text information and the second text information, and generate a content verification result according to the first verification result, the second verification result, the third verification result and the fourth verification result.
[0110] In practice, first, when the positional relationship between the seal corresponding to the seal information and the handwritten text corresponding to the second text information is an overlapping relationship, a fourth verification result representing the verification is passed is generated. Specifically, when the seal and the handwritten text are in an overlapping relationship, the representation has been manually reviewed and verified to be correct. Secondly, when the positional relationship between the seal corresponding to the seal information and the handwritten text corresponding to the second text information is a non-overlapping relationship, the text attributes of the non-handwritten text corresponding to the adjacent first text information and the handwritten text corresponding to the second text information are extracted. For example, the non-handwritten text is "handling" and the handwritten text is "business A, business B". At this time, the text attributes of "handling" and "business A, business B" are different, and the corresponding fourth verification result representing the verification can be generated. For another example, when the non-handwritten text is "business A" and the handwritten text is "business B, business C". At this time, the text attributes of "business A" and "business A, business B" are both business tags of noun attributes. At this time, it is considered that the verification has not passed, so the fourth verification result representing the verification has not passed is generated. In addition, when any of the first verification result, the second verification result, the third verification result, and the fourth verification result indicates that the verification is not passed, a content verification result indicating that the content is not verified is generated. When the first verification result, the second verification result, the third verification result, and the fourth verification result all indicate that the verification is passed, a content verification result indicating that the content is verified is generated.
[0111] Step 108 , in response to the content verification being passed, extracting the content of the first text information, the second text information and the seal information to generate a data record for the image to be identified.
[0112] In some embodiments, in response to the content verification being passed, the execution subject may extract the first text information, the second text information, and the seal information through the NER (Named Entity Recognition) technology to generate a data record for the image to be identified. For example, through the BERT model, the first text information, the second text information, and the seal information are extracted to generate a data record for the image to be identified.
[0113] Step 109, data storage of the data record and the image to be recognized.
[0114] In some embodiments, the execution subject may store data records and images to be identified. Specifically, the data records and images to be identified may be stored in different databases and associated through foreign keys, or the data records and images to be identified may be stored in the same database. The specific storage method is not limited here.
[0115] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the image processing method of some embodiments of the present disclosure, effective and accurate content recognition for the content contained in the image is achieved. Specifically, first, an image to be recognized is obtained, wherein the above-mentioned image to be recognized is an image containing handwritten text, non-handwritten text and a seal image. That is, the image to be recognized contains a variety of contents to be recognized, such as handwritten text, non-handwritten text and a seal image. Then, the above-mentioned image to be recognized is image preprocessed to generate a preprocessed image. The image quality is improved by image preprocessing. Further, the above-mentioned preprocessed image is subjected to non-handwritten text recognition to generate a first text information. In this way, the non-handwritten text contained in the preprocessed image is automatically extracted. In addition, according to the above-mentioned first text information, the handwritten text interest area and the seal interest area in the above-mentioned image to be recognized are determined. In this way, according to the first text information, the area where the handwritten text and the seal image may exist is located. In addition, handwritten text recognition is performed on the above-mentioned handwritten text interest area in the above-mentioned image to be recognized to generate a second text information. Next, seal recognition is performed on the seal area of interest in the image to be recognized to generate seal information. In this way, automatic recognition of handwritten text and seals is achieved. Next, content verification is performed on the first text information, the second text information and the seal information. In this way, the validity of the content is verified. Further, in response to the content verification being passed, content extraction is performed on the first text information, the second text information and the seal information to generate a data record for the image to be recognized. In this way, attribute extraction is performed on the first text information, the second text information and the seal information to obtain corresponding data records for data storage. Finally, data is stored on the data record and the image to be recognized. In this way, effective and accurate content recognition of the content contained in the image is achieved.
[0116] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an image recognition device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the image recognition device can be specifically applied to various electronic devices.
[0117] like Figure 7As shown, the image recognition device 700 of some embodiments includes: an acquisition unit 701, an image preprocessing unit 702, a non-handwritten text recognition unit 703, a determination unit 704, a handwritten text recognition unit 705, a seal recognition unit 706, a content verification unit 707, a content extraction unit 708 and a data storage unit 709. The acquisition unit 701 is configured to acquire an image to be recognized, wherein the image to be recognized is an image containing handwritten text, non-handwritten text and a seal image; the image preprocessing unit 702 is configured to perform image preprocessing on the image to be recognized to generate a preprocessed image; the non-handwritten text recognition unit 703 is configured to perform non-handwritten text recognition on the preprocessed image to generate a first text information; the determination unit 704 is configured to determine a handwritten text interest region and a seal interest region in the image to be recognized according to the first text information; the handwritten text recognition unit 705 is configured to perform non-handwritten text recognition on the preprocessed image to generate a first text information. The handwritten text is recognized in the interest area to generate the second text information; the seal recognition unit 706 is configured to perform seal recognition on the seal interest area in the image to be recognized to generate seal information; the content verification unit 707 is configured to perform content verification on the first text information, the second text information and the seal information; the content extraction unit 708 is configured to extract content from the first text information, the second text information and the seal information in response to passing the content verification to generate a data record for the image to be recognized; the data storage unit 709 is configured to store the data record and the image to be recognized.
[0118] It is understandable that the units described in the image recognition device 700 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the image recognition device 700 and the units included therein, and will not be described in detail here.
[0119] Reference below Figure 8 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 800 suitable for implementing some embodiments of the present disclosure. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0120] like Figure 8As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory 802 or a program loaded from a storage device 808 to a random access memory 803. Various programs and data required for the operation of the electronic device 800 are also stored in the random access memory 803. The processing device 801, the read-only memory 802, and the random access memory 803 are connected to each other via a bus 804. An input / output interface 805 is also connected to the bus 804.
[0121] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 8 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0122] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a read-only memory 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0123] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0124] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains an image to be identified, wherein the image to be identified is an image containing handwritten text, non-handwritten text and a seal image; performs image preprocessing on the image to be identified to generate a preprocessed image; performs non-handwritten text recognition on the preprocessed image to generate a first text information; determines a handwritten text interest region and a seal interest region in the image to be identified based on the first text information; performs handwritten text recognition on the handwritten text interest region in the image to be identified to generate a second text information; performs seal recognition on the seal interest region in the image to be identified to generate seal information; performs content verification on the first text information, the second text information and the seal information; in response to passing the content verification, performs content extraction on the first text information, the second text information and the seal information to generate a data record for the image to be identified; and stores data on the data record and the image to be identified.
[0126] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0127] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0128] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, an image preprocessing unit, a non-handwritten text recognition unit, a determination unit, a handwritten text recognition unit, a seal recognition unit, a content verification unit, a content extraction unit and a data storage unit. Among them, the names of these units do not constitute limitations on the units themselves in certain circumstances. For example, the content extraction unit may also be described as "a unit that extracts content from the above-mentioned first text information, the above-mentioned second text information and the above-mentioned seal information in response to passing the content verification, so as to generate a data record for the above-mentioned image to be recognized".
[0129] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0130] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.
Claims
1. An image recognition method, comprising: Acquire an image to be recognized, wherein the image to be recognized is an image containing handwritten text, non-handwritten text, and a seal image; Performing image preprocessing on the image to be identified to generate a preprocessed image; Performing non-handwritten text recognition on the preprocessed image to generate first text information; Determine a handwritten text region of interest and a seal region of interest in the image to be recognized according to the first text information; Performing handwritten text recognition on the handwritten text region of interest in the image to be recognized to generate second text information; Performing seal recognition on the seal interest region in the image to be recognized to generate seal information; Performing content verification on the first text information, the second text information and the seal information; In response to the content verification being passed, extracting the content of the first text information, the second text information and the seal information to generate a data record for the image to be identified; The data record and the image to be recognized are stored as data.
2. The method according to claim 1, wherein: The performing image preprocessing on the image to be identified to generate a preprocessed image includes: Performing distortion correction on the image to be identified to generate a distortion-corrected image; Performing image resizing on the distortion-corrected image to obtain an image after image resizing; Performing image enhancement on the image after image resizing to generate an image after image enhancement; Performing character region of interest recognition on the image after image enhancement to determine a set of character regions of interest; According to the character region of interest set, character shadows are removed from the image enhanced image to obtain the preprocessed image.
3. The method according to claim 2, wherein: The performing content verification on the first text information, the second text information and the seal information includes: Performing tampering verification according to the first text information to generate a first verification result; Performing format verification according to the second text information to generate a second verification result; Performing seal anti-counterfeiting verification according to the seal information to generate a third verification result; A content verification result is generated according to the first verification result, the second verification result and the third verification result.
4. The method according to claim 3, wherein: The first text information includes: a first text description information set, wherein the first text description information in the first text description information set includes: a character description information group and a text semantic feature; and The performing non-handwritten text recognition on the preprocessed image to generate first text information includes: According to the area positions corresponding to the character ROIs, character ROIs that meet the first screening condition are screened out from the character ROI set as first character ROIs, thereby obtaining a first character ROI set; According to the area positions corresponding to the character ROIs, character ROIs that meet the second screening condition are screened out from the character ROI set as second character ROIs, thereby obtaining a second character ROI set; Determining a first semantic association according to the first character interest region set, wherein the first semantic association represents a character semantic association between the first character interest region set and the character set; Determining a second semantic association according to the second character region of interest set, wherein the second semantic association represents a character semantic association between the second character region of interest set and the character set; In response to the first semantic association being greater than a preset semantic association threshold and the first semantic association being greater than the second semantic association, determining a horizontal recognition direction as a text recognition direction; In response to the second semantic association being greater than the preset semantic association threshold and the second semantic association being greater than the first semantic association, the vertical recognition direction is determined as the text recognition direction.
5. The method according to claim 4, wherein: The performing non-handwritten text recognition on the preprocessed image to generate first text information further includes: According to the text recognition direction, the character region of interest set is divided into a character region of interest group set, wherein the character regions of interest in the character region of interest group are located in the same text line; For each character ROI group in the character ROI group set, the following processing steps are performed: Determine the character features corresponding to the characters in each character region of interest in the character region of interest group by using a pre-trained character feature extraction model to obtain a character feature group, wherein the character features in the character feature group include: font features, font size features, font color features and font meaning features; The first text description information corresponding to the character region of interest group in the first text description information set is determined according to the character feature group through a character recognition model and a text semantic extraction model.
6. An image recognition device, comprising: An acquisition unit is configured to acquire an image to be recognized, wherein the image to be recognized is an image containing handwritten text, non-handwritten text, and a seal image; An image preprocessing unit, configured to perform image preprocessing on the image to be recognized to generate a preprocessed image; a non-handwritten text recognition unit, configured to perform non-handwritten text recognition on the preprocessed image to generate first text information; A determination unit, configured to determine a handwritten text interest region and a seal interest region in the image to be recognized according to the first text information; a handwritten text recognition unit, configured to perform handwritten text recognition on the handwritten text region of interest in the image to be recognized, so as to generate second text information; A seal recognition unit is configured to perform seal recognition on the seal interest area in the image to be recognized to generate seal information; a content verification unit, configured to perform content verification on the first text information, the second text information and the seal information; a content extraction unit configured to extract content from the first text information, the second text information and the seal information in response to content verification being passed, so as to generate a data record for the image to be identified; The data storage unit is configured to store the data record and the image to be recognized.
7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.
8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Business execution method and device based on handwriting verification
CN110874579A
Authenticity identification method of circular seal
CN112016563A
Text detection method, model training method and related equipment
CN112464931A
Intelligent seal identification method
CN114782953A
Document image information extraction method and device, computer equipment and storage medium
CN116311292A
Cited By
Stamp character recognition method
CN122531031A
A seal character recognition method
CN122531031B
