A manuscript image recognition system and method based on fusion detection

Abstract

The present invention discloses a manuscript image recognition system and method based on fusion detection, belonging to the technical field of optical character recognition. The method specifically comprises: acquiring a manuscript image, and preprocessing the acquired manuscript image, performing angle recognition on the preprocessed manuscript image, and performing rotation correction processing, recognizing the text position in the manuscript image after the rotation correction, and recognizing the text content in the manuscript image based on the recognized text position in the manuscript image. By fusing a plurality of detection methods, the advantages of each method can be combined, the defects of a single method can be reduced, and the recognition accuracy of the manuscript image can be improved. The method is applicable to various complex manuscript image recognition scenarios, including but not limited to manuscript images in the financial field, etc.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical character recognition, and in particular to a system and method for manuscript image recognition based on fusion detection. Background Art

[0002] Optical character recognition generally includes image preprocessing, text detection and text recognition processes. Among the text detection methods, the more popular ones are pixel segmentation-based methods and regression-based methods. Draft images contain various text images in financial scenarios, such as bank statements, invoices, vouchers and various ID photos, and the printing quality varies, including photography, scanning and PDF printing. Therefore, the text line size, shape and page layout in the draft image vary, and the fonts are also different. There are problems such as small characters, transparent characters, and text adhesion. Due to the complexity of the scene, it is difficult to achieve good text detection results using the existing single text detection model, resulting in poor overall recognition quality.

[0003] Text detection methods based on pixel segmentation usually draw on the ideas of object semantic segmentation and instance segmentation, and use a fully convolutional neural network to classify each pixel in the image into text and non-text, thereby obtaining a text area mask map, and then aggregate the text pixels belonging to the same text through some pixel aggregation post-processing methods to obtain the final text instance bounding box. Typical algorithms in this type of method include PSENet, PAN, MSR, and DBNet. Text detection methods based on regression are mainly based on target detection technology or instance segmentation technology based on deep learning. It regards text as a general target and then directly detects the entire text instance. This type of method usually directly regresses horizontal rectangles or multi-directional arbitrary shape polygons to solve the problem of text detection. Typical algorithms in this type of method include Text-Boxes, EAST, and Yolo. The main advantage of the pixel segmentation-based method is that it can accurately predict the text boundary, can adapt to texts of various shapes, and has strong anti-noise interference ability; at the same time, it also has some defects, such as the inability to detect overlapping text and insensitivity to small targets. The main advantages of the regression-based method are that it can accurately detect overlapping text and has good detection effects on small objects; at the same time, its disadvantage is that it is not suitable for scenarios where the angles and shapes of text lines are diverse.

[0004] For example, the Chinese patent with the publication number CN116935405A discloses a method for extracting service terms from customer document scans based on OCR and NLP, and the steps are as follows: 1) the server obtains the scan uploaded by the user; 2) the scan is converted into a JPG format image; 3) the DBnet network is used to identify the lines containing text in the image; 4) the identified lines containing text are converted into independent images; 5) the independent images containing text are converted into standardized images; 6) the standardized images are handed over to the text recognition network to identify the text; 7) the titles and paragraphs in the identified text are switched to the main text; 8) the tables in the document are identified and switched to the main text; 9) the service content and requirements for engineers in the document are extracted and switched to the main text; 10) keyword search is performed and the extracted content is standardized. The invention reduces the communication cost and time cost of technical personnel and business personnel through system automation, and quickly locates the service content required by customers.

[0005] For example, a Chinese patent with publication number CN118314563A discloses a text detection method based on a rotating frame, which belongs to the field of text detection. The method includes: S1, obtaining a picture data set and preprocessing it; S2, constructing a text detection model based on an improved YOLOv8s-obb algorithm; S3, inputting the picture data set preprocessed in step S1 into the text detection model of step S2 for iteration to obtain the optimal text detection model; S4, using the optimal text detection model to perform text detection on the picture data set preprocessed in step S1. This invention generates a rotatable text detection frame through an improved YOLOv8s-obb algorithm, which solves the problem that the detection frame cannot fully cover targets such as traffic signs due to image angle problems, and effectively improves detection accuracy.

[0006] The defects of the above patent are: 1) It cannot process overlapping texts, which will lead to errors in subsequent recognition results; 2) The detection effect for long texts is not good enough and cannot be applied to manuscript text recognition scenarios. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention proposes a manuscript image recognition system and method based on fusion detection, which mainly include rotation correction, a detection module and image text recognition, wherein the detection module uses a segmentation-based text detection method and a regression-based text detection method, and fuses the detection results of the two, and finally outputs the positions of all text lines in the manuscript image.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A manuscript image recognition method based on fusion detection includes the following specific steps:

[0010] Acquire a draft image, and pre-process the acquired draft image;

[0011] Perform angle recognition on the pre-processed manuscript image and perform rotation correction processing;

[0012] Identify the text position in the rotation-corrected manuscript image;

[0013] Based on the identified text position in the draft image, the text content in the draft image is recognized.

[0014] Specifically, the angle recognition of the preprocessed draft image includes:

[0015] Use the draft image dataset with pre-annotated angle values ​​to train the MobileNet-v3 model. After the model training is completed, the draft orientation recognition model is obtained.

[0016] The obtained draft image is set as I(x, y), and the draft image I(x, y) is input into the draft orientation recognition model to obtain the orientation angle of the draft image I(x, y);

[0017] Convert the manuscript image I(x,y) from the spatial domain to the frequency domain to generate a spectrum. The specific formula for generating the spectrum is:

[0018] ,

[0019] Among them, F(u,v) represents the frequency spectrum value of frequency (u,v) in the frequency domain, u represents the horizontal frequency component in the frequency domain, v represents the vertical frequency component in the frequency domain, I(i,j) represents the gray value of the coordinate position (i,j) in the draft image, M represents the width of the draft image I(x,y), and N represents the height of the draft image I(x,y). represents an imaginary unit and satisfies =-1, e represents exponential function;

[0020] Perform adaptive radial projection in the frequency domain of the draft image I(x,y) to extract the tilt angle of the draft image I(x,y) .

[0021] Specifically, the tilt angle of the extracted draft image I (x, y) is ,include:

[0022] Calculate the amplitude value of the frequency (u,v) in the frequency domain. The calculation formula is:

[0023] ,

[0024] Among them, A(u,v) represents the amplitude value of the frequency (u,v) in the frequency domain, Re(F(u,v)) represents the real part of the frequency (u,v) in the frequency domain, and Im(F(u,v)) represents the imaginary part of the frequency (u,v) in the frequency domain;

[0025] The amplitude of the spectrum in the frequency domain is radially projected, converted into polar coordinates, and the projections are summed at different angles to obtain the projection values. The specific formula for the projection summation is:

[0026] ,

[0027] in, Represents the amplitude of the spectrum in the frequency domain in degrees The projection value on represents the projection angle, r represents the radial distance, and R represents the maximum radius of the spectrum in the frequency domain;

[0028] The tilt angle of the draft image I(x,y) is detected, and the specific formula is: , where argmax Indicates that the projection value The angle at which the maximum value is reached, Indicates the tilt angle of the draft image I(x,y).

[0029] Specifically, the rotation correction process includes:

[0030] According to the tilt angle of the extracted draft image I(x,y) , perform rotation correction on the draft image I(x,y), the specific formula is:

[0031] ,

[0032] in, Represents the draft image after rotation correction.

[0033] Specifically, the text position in the rotation-corrected manuscript image in the financial field is identified, including:

[0034] Obtain a public dataset of manuscript images, and manually annotate text line position information in the public dataset of manuscript images;

[0035] Using a public dataset of manuscript images containing text line position annotation information to train a first text detection model and a second text detection model, obtaining a trained first text detection model and a trained second text detection model;

[0036] Inputting the rotation-corrected manuscript image in the financial field into the first text detection model and the second text detection model respectively, to obtain the prediction result of the first text line position and the prediction result of the second text line position;

[0037] Set a detection box in the prediction result of the first text line position as A, and a detection box in the prediction result of the second text line position as B, and calculate the Ious value between the detection boxes A and B. The calculation formula is:

[0038] ,

[0039] Among them, Ious represents the Ious value between detection boxes A and B, areaA represents the area of ​​detection box A, areaB represents the area of ​​detection box B, and areaC represents the area of ​​detection box C;

[0040] Calculate the Ious between the detection box in the prediction result of the first text line position and the detection box in the prediction result of the second text line position, and obtain the Ious set Ious_map, where Ious_map[p, q] represents the Ious between the p-th detection box in the prediction result of the first text line position and the q-th detection box in the prediction result of the second text line position;

[0041] The fusion strategy is used to screen and judge the detection boxes in Ious_map.

[0042] Specifically, the fusion strategy includes:

[0043] The detection box in the prediction result of the second text line position detects a single-character text, if none of the detection boxes in the prediction result of the first text line position intersects with the single-character text detection box, and the aspect ratio of the single-character text detection box is less than 1.5, and the prediction result of the first text line position does not include the single-character text detection box, then the single-character text detection box is added to the prediction result of the first text line position;

[0044] If the detection box in the prediction result of the first text line position corresponds to several detection boxes in the prediction result of the second text line position, then delete the detection box in the prediction result of the first text line position, and add several detection boxes in the prediction result of the second text line position to the prediction result of the first text line position.

[0045] A manuscript image recognition system based on fusion detection, which is used to implement the manuscript image recognition method based on fusion detection, comprising: an image acquisition module, a rotation correction module, a position recognition module and a text recognition module;

[0046] The image acquisition module is used to acquire the draft image and pre-process the acquired draft image;

[0047] The rotation correction module is used to identify the angle of the pre-processed manuscript image and perform rotation correction processing;

[0048] The position recognition module is used to recognize the text position in the rotation-corrected manuscript image;

[0049] The text recognition module is used to identify the text position in the manuscript image and recognize the text content in the manuscript image.

[0050] Specifically, the rotation correction module includes an angle recognition unit and a rotation correction unit.

[0051] The angle recognition unit is used to perform angle recognition on the preprocessed draft image;

[0052] The rotation correction unit is used to perform rotation correction on the draft image according to the identified orientation angle of the draft image.

[0053] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a manuscript image recognition method based on fusion detection when executing the computer program.

[0054] A computer-readable storage medium stores computer instructions, which, when executed, execute the steps of a manuscript image recognition method based on fusion detection.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. The present invention proposes a manuscript image recognition method based on fusion detection. By fusing multiple detection methods, the advantages of each method can be combined, the defects of a single method can be reduced, and the recognition accuracy of the manuscript image can be improved.

[0057] 2. The present invention proposes a manuscript image recognition method based on fusion detection, which is applicable to various complex manuscript image recognition scenarios, including but not limited to manuscript images in the financial field, etc.

[0058] 3. The present invention proposes a manuscript image recognition method based on fusion detection, which can effectively reduce the probability of misidentification and improve the accuracy of the overall system by comprehensively analyzing multiple detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flow chart of a manuscript image recognition method based on fusion detection provided by the present invention;

[0060] Figure 2 A flow chart of text position detection provided by the present invention;

[0061] Figure 3 A text detection block diagram provided by the present invention;

[0062] Figure 4 An architecture diagram of a manuscript image recognition system based on fusion detection provided by the present invention;

[0063] Figure 5 A schematic diagram of an electronic device provided by the present invention;

[0064] Figure 6 A schematic diagram of the computer-readable storage medium structure provided by the present invention. DETAILED DESCRIPTION

[0065] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, in the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "No. 1", "No. 2" and "No. 3" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The present invention is further explained below in conjunction with specific embodiments.

[0066] Example 1

[0067] See also Figure 1-3 The present invention provides an embodiment: a manuscript image recognition method based on fusion detection, comprising the following specific steps:

[0068] Step S1: obtaining a draft image in the financial field, and preprocessing the obtained draft image in the financial field, including: denoising and graying;

[0069] Step S2: performing angle recognition on the pre-processed manuscript image in the financial field and performing rotation correction processing;

[0070] The specific steps of step S2 are:

[0071] Step S201: Use a draft image dataset with pre-annotated angle values ​​to train a MobileNet-v3 model. After the model training is completed, a draft orientation recognition model is obtained;

[0072] Step S202: setting the acquired draft image of the financial field as I(x, y), inputting the draft image I(x, y) of the financial field into the draft orientation recognition model, and obtaining the orientation angle of the draft image I(x, y) of the financial field;

[0073] The output result of the orientation angle here may be one of '0 degrees, 90 degrees, 180 degrees, 270 degrees', and the specific orientation angle cannot be directly obtained;

[0074] Step S203: Convert the manuscript image I(x, y) in the financial field from the spatial domain to the frequency domain to generate a spectrum. The specific formula for generating the spectrum is:

[0075] ,

[0076] Among them, F(u,v) represents the frequency spectrum value of frequency (u,v) in the frequency domain, u represents the horizontal frequency component in the frequency domain, v represents the vertical frequency component in the frequency domain, I(i,j) represents the gray value of the coordinate position (i,j) in the draft image of the financial field, M represents the width (number of pixels) of the draft image I(x,y) in the financial field, and N represents the height (number of pixels) of the draft image I(x,y) in the financial field. represents an imaginary unit and satisfies =-1, e represents exponential function;

[0077] Step S204: Perform adaptive radial projection in the frequency domain of the financial field draft image I(x,y) to extract the tilt angle of the financial field draft image I(x,y) ;

[0078] The specific steps of step S204 are:

[0079] Step S2041: Calculate the amplitude value of the frequency (u, v) in the frequency domain, and the calculation formula is:

[0080] ,

[0081] Among them, A(u,v) represents the amplitude value of the frequency (u,v) in the frequency domain, Re(F(u,v)) represents the real part of the frequency (u,v) in the frequency domain, and Im(F(u,v)) represents the imaginary part of the frequency (u,v) in the frequency domain;

[0082] Step S2042: radially project the amplitude of the spectrum in the frequency domain, convert it into polar coordinates, and sum the projections at different angles to obtain projection values. The specific formula for the projection sum is:

[0083] ,

[0084] in, Represents the amplitude of the spectrum in the frequency domain in degrees The projection value on The cumulative sum of the spectral amplitudes on represents the projection angle, that is, the angle component in the polar coordinate system, r represents the radial distance, that is, the distance from the center of the spectrum to the edge of the spectrum, and R represents the maximum radius of the spectrum in the frequency domain, that is, the maximum distance from the center of the spectrum to the edge;

[0085] Step S2043: Detect the tilt angle of the manuscript image I (x, y) in the financial field. The specific formula is: , where argmax Indicates that the projection value The angle at which the maximum value is reached, Indicates the tilt angle of the manuscript image I(x,y) in the financial field.

[0086] Step S205: Based on the tilt angle of the extracted financial field manuscript image I(x,y) , the manuscript image I(x,y) in the financial field is rotated and corrected. The specific formula is:

[0087] ,

[0088] in, Represents a manuscript image in the financial field after rotation correction.

[0089] Step S3: identifying the text position in the rotation-corrected manuscript image in the financial field;

[0090] The specific steps of step S3 are:

[0091] Step S301: obtaining a public dataset of draft images, and manually marking text line position information in the public dataset of draft images;

[0092] The information that needs to be annotated for each text line is the coordinates of the four corner points of the text line [(x1,y1), (x2, y1), (x2, y2), (x1, y2)], which respectively represent the coordinate values ​​of the top left, top right, bottom right and bottom left vertices of the text line;

[0093] Step S302: using a public dataset of manuscript images containing text line position annotation information to train a first text detection model and a second text detection model to obtain trained first text detection models and second text detection models;

[0094] The first text detection model and the second text detection model are: DBNet model and Yolo-v7 model respectively;

[0095] Step S303: inputting the rotation-corrected manuscript image in the financial field into the first text detection model and the second text detection model respectively, to obtain the prediction result of the first text line position and the prediction result of the second text line position;

[0096] This method mainly uses the detection results of DBNet and supplemented by the detection results of Yolo-v7. The Yolo-v7 algorithm can detect small targets in the image more accurately (mainly single-character text in the draft image), and can also detect multiple overlapping (or contiguous) text lines. This method optimizes the single-character text that is missed in the DBNet detection results and the overlapping and contiguous text lines that are wrongly detected through the result merging process.

[0097] Step S304: Set a detection box in the prediction result of the first text line position as A, and a detection box in the prediction result of the second text line position as B, and calculate the Ious value between the detection boxes A and B. The calculation formula is:

[0098] ,

[0099] Among them, Ious represents the Ious value between detection boxes A and B, areaA represents the area of ​​detection box A, areaB represents the area of ​​detection box B, and areaC represents the area of ​​detection box C;

[0100] Step S305: Calculate the Ious between the detection box in the prediction result of the first text line position and the detection box in the prediction result of the second text line position, and obtain an Ious set Ious_map, where Ious_map[p, q] represents the Ious between the p-th detection box in the prediction result of the first text line position and the q-th detection box in the prediction result of the second text line position;

[0101] ious_map can represent the intersection between all detection boxes of DBNet and all detection boxes of Yolo-v7;

[0102] Step S306: using the fusion strategy to screen and judge the detection boxes in the lous_map.

[0103] The specific steps of the fusion strategy in step S306 are:

[0104] Step S3061: if the detection box in the prediction result of the second text line position detects the single-character text, if none of the detection boxes in the prediction result of the first text line position intersects with the single-character text detection box, and the aspect ratio of the single-character text detection box is less than 1.5, and the prediction result of the first text line position does not include the single-character text detection box, then the single-character text detection box is added to the prediction result of the first text line position;

[0105] Explanation of step S3061: According to the intersection relationship between the two detection results, the single-character text missed by DBNet but detected by Yolo-v7 is determined. For a Yolo-v7 detection box, if all DBNet detection boxes do not intersect with it, and the aspect ratio of this detection box is less than 1.5, it is determined to be a single-character detection box, and this character box is missed by the DBNet method, and it is added to the detection result of DBNet;

[0106] Step S3062: If the detection box in the prediction result of the first text line position corresponds to several detection boxes in the prediction result of the second text line position, delete the detection box in the prediction result of the first text line position, and add several detection boxes in the prediction result of the second text line position to the prediction result of the first text line position.

[0107] Explanation of step S3062: The situation of falsely detected overlapping frames in the DBNet detection result is determined based on the intersection relationship between the two detection results. For a detection frame in the DBNet result, if its position corresponds to multiple Yolo frames, it can be determined that the DBNet frame is a false detection. It is necessary to delete this detection frame from the DBNet detection result and add the corresponding multiple Yolo-v7 detection frames to the DBNet detection result.

[0108] Step S4: Based on the identified text position in the manuscript image in the financial field, identify the text content in the manuscript image in the financial field.

[0109] The CRNN model is trained using a large number of manually annotated text line images. The trained CRNN model can recognize the characters in the text lines.

[0110] Example 2

[0111] See also Figure 4 ,Another embodiment provided by the present invention: A manuscript image recognition system based on fusion detection, comprising: an image acquisition module, a rotation correction module, a position recognition module and a text recognition module;

[0112] The image acquisition module is used to acquire the draft image and pre-process the acquired draft image;

[0113] The rotation correction module is used to identify the angle of the pre-processed manuscript image and perform rotation correction processing;

[0114] The position recognition module is used to recognize the text position in the rotation-corrected manuscript image;

[0115] The text recognition module is used to identify the text position in the manuscript image and recognize the text content in the manuscript image.

[0116] The rotation correction module includes an angle recognition unit and a rotation correction unit.

[0117] The angle recognition unit is used to perform angle recognition on the preprocessed draft image;

[0118] The rotation correction unit is used to perform rotation correction on the draft image according to the identified orientation angle of the draft image.

[0119] Example 3

[0120] Figure 5 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5 As shown, according to another aspect of the present application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are run by one or more processors, a method for recognizing a manuscript image based on fusion detection can be executed.

[0121] The method or system according to the embodiment of the present application can also be used by Figure 5 The electronic device architecture shown in FIG. Figure 5 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a ROM 503, a RAM 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, may store a method for manuscript image recognition based on fusion detection provided in the present application. A method for manuscript image recognition based on fusion detection may, for example, include: acquiring a manuscript image, and preprocessing the acquired manuscript image, performing angle recognition on the preprocessed manuscript image, and performing rotation correction processing, recognizing the text position in the manuscript image after rotation correction, and identifying the text position in the manuscript image, and recognizing the text content in the manuscript image. Further, the electronic device 500 may also include a user interface 508. Of course, Figure 5 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 5 One or more components of an electronic device are shown.

[0122] Example 4

[0123] Figure 6 Schematic diagram of a computer-readable storage medium structure provided by an embodiment of the present application. Figure 6As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, a method for recognizing manuscript images based on fusion detection according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0124] It should be understood that the methods, apparatuses, and devices of the present application can be implemented in many ways. For example, the methods, apparatuses, and devices of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0125] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0126] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A manuscript image recognition method based on fusion detection, characterized in that: include: Acquire a draft image, and pre-process the acquired draft image; Perform angle recognition on the pre-processed manuscript image and perform rotation correction processing; Identify the text position in the rotation-corrected manuscript image; Based on the identified text position in the draft image, the text content in the draft image is identified; The identifying of the text position in the rotation-corrected manuscript image in the financial field includes: Obtain a public dataset of draft images, and manually annotate text line position information in the public dataset of draft images; Using a public dataset of manuscript images containing text line position annotation information to train a first text detection model and a second text detection model, obtaining a trained first text detection model and a trained second text detection model; Inputting the rotation-corrected manuscript image in the financial field into the first text detection model and the second text detection model respectively, to obtain the prediction result of the first text line position and the prediction result of the second text line position; Set a detection box in the prediction result of the first text line position as A, and a detection box in the prediction result of the second text line position as B, and calculate the Ious value between the detection boxes A and B. The calculation formula is: ， Among them, Ious represents the Ious value between detection boxes A and B, areaA represents the area of ​​detection box A, areaB represents the area of ​​detection box B, and areaC represents the area of ​​detection box C; Calculate the Ious between the detection box in the prediction result of the first text line position and the detection box in the prediction result of the second text line position, and obtain the Ious set Ious_map, where Ious_map[p, q] represents the Ious between the p-th detection box in the prediction result of the first text line position and the q-th detection box in the prediction result of the second text line position; Use the fusion strategy to screen and judge the detection boxes in Ious_map; The fusion strategy includes: The detection box in the prediction result of the second text line position detects a single-character text, if none of the detection boxes in the prediction result of the first text line position intersects with the single-character text detection box, and the aspect ratio of the single-character text detection box is less than 1.5, and the prediction result of the first text line position does not include the single-character text detection box, then the single-character text detection box is added to the prediction result of the first text line position; If the detection box in the prediction result of the first text line position corresponds to several detection boxes in the prediction result of the second text line position, then delete the detection box in the prediction result of the first text line position, and add several detection boxes in the prediction result of the second text line position to the prediction result of the first text line position.

2. A manuscript image recognition method based on fusion detection as claimed in claim 1, characterized in that: The step of performing angle recognition on the preprocessed draft image includes: Use the draft image dataset with pre-annotated angle values ​​to train the MobileNet-v3 model. After the model training is completed, the draft orientation recognition model is obtained. The obtained draft image is set as I(x, y), and the draft image I(x, y) is input into the draft orientation recognition model to obtain the orientation angle of the draft image I(x, y); Convert the manuscript image I(x,y) from the spatial domain to the frequency domain to generate a spectrum. The specific formula for generating the spectrum is: ， Among them, F(u,v) represents the frequency spectrum value of frequency (u,v) in the frequency domain, u represents the horizontal frequency component in the frequency domain, v represents the vertical frequency component in the frequency domain, I(i,j) represents the gray value of the coordinate position (i,j) in the draft image, M represents the width of the draft image I(x,y), and N represents the height of the draft image I(x,y). represents an imaginary unit and satisfies =-1, e represents exponential function; Perform adaptive radial projection in the frequency domain of the draft image I(x,y) to extract the tilt angle of the draft image I(x,y) .

3. A manuscript image recognition method based on fusion detection as claimed in claim 2, characterized in that: The tilt angle of the extracted draft image I (x, y) ,include: Calculate the amplitude value of the frequency (u,v) in the frequency domain. The calculation formula is: ， Among them, A(u,v) represents the amplitude value of the frequency (u,v) in the frequency domain, Re(F(u,v)) represents the real part of the frequency (u,v) in the frequency domain, and Im(F(u,v)) represents the imaginary part of the frequency (u,v) in the frequency domain; The amplitude of the spectrum in the frequency domain is radially projected, converted into polar coordinates, and the projections are summed at different angles to obtain the projection values. The specific formula for the projection summation is: ， in, Represents the amplitude of the spectrum in the frequency domain in degrees The projection value on represents the projection angle, r represents the radial distance, and R represents the maximum radius of the spectrum in the frequency domain; The tilt angle of the draft image I(x,y) is detected, and the specific formula is: , where argmax Indicates that the projection value The angle at which the maximum value is reached, Indicates the tilt angle of the draft image I(x,y).

4. A manuscript image recognition method based on fusion detection as claimed in claim 3, characterized in that: The rotation correction process comprises: According to the tilt angle of the extracted draft image I(x,y) , perform rotation correction on the draft image I(x,y), the specific formula is: ， in, Represents the draft image after rotation correction.

5. A manuscript image recognition system based on fusion detection, which is used to implement a manuscript image recognition method based on fusion detection as described in any one of claims 1 to 4, characterized in that: It includes: image acquisition module, rotation correction module, position recognition module and text recognition module; The image acquisition module is used to acquire the draft image and pre-process the acquired draft image; The rotation correction module is used to identify the angle of the pre-processed draft image and perform rotation correction processing; The position recognition module is used to recognize the text position in the rotation-corrected manuscript image; The text recognition module is used to recognize the text position in the manuscript image and recognize the text content in the manuscript image.

6. A manuscript image recognition system based on fusion detection as claimed in claim 5, characterized in that: The rotation correction module includes an angle recognition unit and a rotation correction unit. The angle recognition unit is used to perform angle recognition on the preprocessed draft image; The rotation correction unit is used to perform rotation correction on the draft image according to the identified orientation angle of the draft image.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a manuscript image recognition method based on fusion detection described in any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the steps of a manuscript image recognition method based on fusion detection described in any one of claims 1-4 are executed.

Images