A Multi-Classification-Based Text Detection Method and System

Abstract

This application proposes a multi-classification-based text detection method and system, comprising: acquiring and organizing text box samples; segmenting the text box samples into text units; labeling the coordinates and flags of the text units; generating training and validation sets proportionally from the text box samples; assigning class labels to the flags of the text units; constructing a text unit detection network model; training the training set using the text unit detection network model until the loss converges; obtaining the output of the trained model, which is set B; constructing a text unit merging algorithm; dividing set B into multiple subsets based on the class labels of the flags; calculating the bounding rectangle of the point set cnt based on the current element and the elements in the subset to which the current element belongs; and inputting the validation set into the trained model to finally obtain all the text boxes. This method effectively solves various problems such as inaccurate text localization, missed detections, and false detections in natural scenes, and can also handle multi-angle text detection problems.

Description

Technical Field

[0001] This application relates to the field of text detection technology, specifically to a text detection method and system based on multi-classification. Background Technology

[0002] Optical Character Recognition (OCR) technology refers to the detection and recognition of text in video images; it is a technique for mining textual information from video images. Text detection, as a subtask, locates the specific region in an image that needs to be recognized. With the continuous updates in technology and requirements, traditional recognition techniques used for printed documents in the past are no longer suitable for natural scenes. Deep learning-based text detection methods have become the mainstream technology, which can be broadly divided into pixel-segmentation-based text detection and candidate box-based text detection.

[0003] Pixel-based text detection utilizes general semantic segmentation techniques to obtain text localization by regressing or aggregating text pixels, such as DBNet. Candidate box-based text detection, building upon general object detection, decomposes the regression target to address multi-deformation issues, and then selectively merges or removes duplicates to obtain text boxes, such as FCENet. Because both techniques have their advantages and disadvantages, attempts to fuse the two techniques have emerged, such as Pixel-Anchor.

[0004] Pixel-segmentation-based text detection is highly sensitive to text scale, often performing poorly with dense or excessively sparse text. Furthermore, because the algorithm requires detailed pixel-level analysis, it generally involves significant computational cost, long processing time, and higher annotation costs. Candidate box-based text detection algorithms that directly use text boxes as regression targets suffer from poor convergence. Additionally, the model struggles to adapt to text viewed from multiple angles. Some have proposed segmenting text boxes, detecting them as individual text units, and then merging them. However, algorithms that merge text units into lines currently rely heavily on complex programming and strong, manually designed priors, resulting in poor performance even with multi-angle text merging and issues such as missing characters and over-merging. Fusion schemes, on the other hand, introduce even greater computational costs without significantly improving detection performance.

[0005] To address the aforementioned issues, this paper proposes a multi-classification-based text detection method. This method takes into account the challenges of text connection and introduces a multi-classification judgment mechanism for the connection direction of text units into the single-stage text unit detection module. This allows the module to output the connection direction and connection start and end indicators while outputting the text unit coordinates. Summary of the Invention

[0006] To address the existing problems of difficult text connection, inaccurate text localization in natural scenes, missed detections, and false detections, this application provides a multi-classification-based text detection method and system to solve the aforementioned technical deficiencies.

[0007] According to one aspect of the present invention, a multi-classification-based text detection method is proposed, the method comprising the following steps:

[0008] S1. Obtain and organize the text box sample, divide the text box sample into text units, and label the coordinates and flags of the text units;

[0009] S2. Generate training and validation sets from the text box samples proportionally, and assign class labels to the flag bits of the text units;

[0010] S3. Construct a text unit detection network model, train the training set using the text unit detection network model until the loss converges, and obtain the output of the trained model, which is set B.

[0011] S4. Construct a text unit merging algorithm. Divide the set B into multiple subsets based on the class label of the flag bit. Calculate the bounding rectangle of the point set cnt based on the current element and the elements within its subset. The bounding rectangle represents the merged text box.

[0012] S5. Input the validation set into the model trained in step S3 to finally obtain all the text boxes on the validation set.

[0013] The aforementioned technical solution, by leveraging the directional and connectivity characteristics of text itself, introduces a multi-classification judgment mechanism for each text unit and employs a rasterized segmentation and merging detection scheme. This effectively solves various problems such as inaccurate text localization, missed detections, and false detections in natural scenes, while also handling multi-angle text detection. This method not only ensures the algorithm's generalization ability across multiple scenarios but also avoids introducing excessive parameters, guaranteeing model inference performance and enabling the model to be deployed and used on various devices.

[0014] In a specific embodiment, obtaining and organizing the text box sample includes the following sub-steps:

[0015] S11. Obtain the text detection dataset and segment the text boxes in the text detection dataset using a rasterization method to form text units;

[0016] S12. Label each text unit and calculate the coordinates of the segmented text units. The label format is (x,y,w,h), where (x,y) are the coordinates of the center point of the text unit and (w,h) are the width and height of the text unit.

[0017] S13. Mark the flags of the first and last text cells of the text box, and mark the flags between every two text cells.

[0018] The above technical solution collects relevant data for text detection, and completes the segmentation and labeling of text boxes for subsequent operations.

[0019] In a specific embodiment, step S13 involves marking the flag bit between every two text units, including the following sub-steps, where (x pre ,y pre (x) represents the center coordinates of the current text cell. next ,y next () represents the center coordinates of the next text unit.

[0020] S131. Calculate the distance between the center point of the current text cell and the center point of the next text cell. If y next -y pre =0, then the end flag of the current text box and the start flag of the next text box are marked horizontally;

[0021] S132, if x next -x pre =0, then the end flag of the current text box and the start flag of the next text box are marked as vertical;

[0022] S133, if y next -y pre If the value is less than 0, then the end flag of the current text box and the start flag of the next text box are both set to "up".

[0023] S134, if y next -y pre If the value is greater than 0, then the end flag of the current text box and the start flag of the next text box are both marked as "down".

[0024] Repeat step S13 for all text cells within the text box to complete marking the flag between every two text cells.

[0025] In a specific embodiment, in step S4, a text unit merging algorithm is constructed, which divides set B according to the class label of the flag bit, including dividing the elements in set B into multiple subsets according to the different class labels of the start flag bit: B first (start=0), B start1 (start=1), B start_2 (start=2), B start_3 (start=3), B start_4 (start=4).

[0026] In a specific embodiment, in step S4, the bounding rectangle of the point set cnt is calculated and obtained based on the current element and the elements in the subset to which the current element belongs. The bounding rectangle is the merged text box, and includes the following sub-steps:

[0027] S41, Based on subset B first The current element b in pre Calculate the coordinates of the text cell box and add them to the point set cnt, and then calculate the coordinates based on the current element b. pre End flag value b pre (end) The set B with the same starting flag (start) index and class label. next ;

[0028] S42. Based on the current element b pre and set B next Inner element b nexti Calculate the following values ​​respectively:

[0029] x dis =x c_pre -x c_next

[0030] y dis =y c_pre -y c_next

[0031]

[0032]

[0033]

[0034] t = w pre +h pre

[0035] Among them, (x c_pre ,y c_pre ) represents the current element b pre The center coordinates of (w) pre ,h pre ) represents the current element b pre The width and length; (x c_next ,y c_next ) is element b nexti The center coordinate point;

[0036] S43, if b pre If (end) = 0, then directly return the point set cnt; otherwise...

[0037]

[0038] where ω is an adjustable threshold. If D < t, then the corresponding b is taken min_dis and b is updated pre = b min_dis ;

[0039] S44. Repeat steps S41 - S43 until b pre (end) = 0, calculate the bounding rectangle of the return point set cnt, and the obtained bounding rectangle is the text box after merging completion.

[0040] In a specific embodiment, in step S3, obtaining the output of the trained model further includes obtaining the confidence value conf output through the sigmoid function, filtering the confidence value conf, and only retaining the bbox, start flag bit, and end flag bit corresponding to conf ≥ θ, and outputting the set B(x, y, w, h, start, end), where θ is an adjustable threshold.

[0041] In a specific embodiment, in step S3, the training set is trained by the text unit detection network model until the loss converges. The expression of the total loss function of the text unit detection network model is:

[0042] L = αL obj + βL GIoU + γ(L start + L end ),

[0043] where α, β, and γ are weight parameters, L obj is the binary cross - entropy loss function, L GIoU is the GIoU loss function, L start , L end are the multi - classification cross - entropy loss functions.

[0044] In a specific embodiment, it further includes that the determination of the text area adopts the binary cross - entropy loss function. The expression of the binary cross - entropy loss function is:

[0045]

[0046] In the formula, y is the true label of the sample, is the network prediction label;

[0047] The text box regression adopts the GIoU loss function. The expression of the GIoU loss function is:

[0048]

[0049]

[0050] In the formula, area(A) represents the real text region of the text unit, area(B) represents the text region predicted by the model, and area(C) represents the minimum convex closed box region containing A and B.

[0051] The category determination of the start and end flags uses the multi-class cross-entropy loss function, the expression of which is:

[0052]

[0053]

[0054] In the formula, y i Let y' be the true class of the sample. i This refers to the network prediction category.

[0055] Secondly, this application provides a multi-classification-based text detection system, which includes:

[0056] The sample acquisition module is used to acquire and organize text box samples, divide the text box samples into text units, and label the coordinates and flags of the text units;

[0057] The generation module is used to generate training and validation sets from text box samples proportionally, and to assign class labels to the flag bits of the text units;

[0058] The model training module is used to build a text unit detection network model. The training set is trained through the text unit detection network model until the loss converges, and the output of the trained model is obtained, which is set B.

[0059] The algorithm merging module is used to construct a text unit merging algorithm. It divides the set B into multiple subsets based on the class label of the flag bit. It calculates and obtains the bounding rectangle of the point set cnt based on the current element and the elements within its subset. This bounding rectangle represents the merged text box.

[0060] The validation module is used to input the validation set into the model trained in the model training module, and finally obtain all the text boxes on the validation set.

[0061] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0062] Compared with the prior art, the beneficial results of the present invention are as follows:

[0063] This invention addresses the challenges of text concatenation by introducing a multi-classification mechanism for text unit connection direction into a single-stage text unit detection module. This mechanism outputs the connection direction and start / end markers along with the text unit coordinates. This solves the problem of multi-angle text detection and significantly improves the accuracy of text line merging results. While maintaining model generalization, the model's inference speed is not significantly affected, enabling better industrial deployment. Attached Figure Description

[0064] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments, taken with reference to the accompanying drawings:

[0065] Figure 1 This is a flowchart of the multi-classification-based text detection method according to this application;

[0066] Figure 2 This is a structural diagram of the text unit detection network model according to this application;

[0067] Figure 3 This is a structural diagram of the multi-classification-based text detection system according to this application;

[0068] Figure 4 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0069] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0070] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0071] Figure 1 A flowchart of the multi-classification-based text detection method of this application is shown below. Please refer to it. Figure 1 The method includes the following steps:

[0072] S1. Obtain and organize the text box sample, divide the text box sample into text units, and label the coordinates and flags of the text units.

[0073] In this embodiment, the text units within each text box are labeled as follows:

[0074] a) Calculate the coordinates of the segmented text cells and save the annotations in the format (x, y, w, h). Here, (x, y) represents the coordinates of the center point of the text cell, and (w, h) represents the width and height of the text cell.

[0075] b) The first text cell in the text box is marked as "first";

[0076] c) Calculate the distance between the center point of the current text cell and the center point of the next text cell. If y next -y pre =0, then the end flag of the current text box and the start flag of the next text box are marked horizontally; if x next -x pre =0, then the end flag of the current text box and the start flag of the next text box are marked as vertical; if y next -y pre If y < 0, then the end flag of the current text box and the start flag of the next text box are marked as up; if y next -y pre If the value is greater than 0, then the end flag of the current text box and the start flag of the next text box are both set to "down".

[0077] d) Repeat step c for all text cells in the current text box;

[0078] e) The last text cell in the text box is marked as "last".

[0079] S2. Generate training and validation sets from the text box samples according to the proportion, and assign class labels to the flag bits of the text units.

[0080] In this embodiment, 0, 1, 2, 3, and 4 are used to represent the first (last) class, horizontal class, vertical class, up class, and down class, respectively. The samples obtained in step S1 are used to construct a training set and a validation set according to a certain ratio. This application uses a ratio of 10:1 by default, but not limited to.

[0081] S3. Construct a text unit detection network model, train the training set using the text unit detection network model until the loss converges, and obtain the output of the trained model, which is set B.

[0082] In this embodiment, a single-stage detection network is used. Figure 2 The structural diagram of the text unit detection network model of this application is shown. Figure 2 The neural network model is trained on the training set generated in step S2 until the loss converges.

[0083] The determination of text regions uses the binary cross-entropy loss function, the expression of which is:

[0084]

[0085] In the formula, y represents the true label of the sample. For network prediction labels.

[0086] Text box regression uses the GIoU loss function, which is expressed as follows:

[0087]

[0088]

[0089] In the formula, area(A) represents the actual text region of the text unit, area(B) represents the text region predicted by the model, and area(C) represents the minimum convex closed box region containing A and B.

[0090] The category determination of the start and end flags uses the multi-class cross-entropy loss function, the expression of which is:

[0091]

[0092]

[0093] In the formula, y i Let y'i be the true class of the sample, and y'i be the class predicted by the network.

[0094] The expression for the total loss function of the text unit detection network model is:

[0095] L=αL obj +βL GIoU +γ(L start +L end )

[0096] In the formula, α, β, and γ are weight parameters with values ​​ranging from [1, 10], which can be adjusted according to the needs of the network.

[0097] In this embodiment, the output of the trained model is obtained by using the sigmoid function to get the confidence value (conf).

[0098]

[0099] Filter the conf and keep only the bounding boxes, start flags, and end flags that satisfy conf≥θ, and output B(x,y,w,h,start,end), where θ is an adjustable threshold.

[0100] S4. Construct a text unit merging algorithm. Divide the set B according to the class label of the flag bit to form multiple subsets. Calculate and obtain the bounding rectangle of the point set cnt based on the current element and the elements in the subset to which the current element belongs. The bounding rectangle is the merged text box.

[0101] In this embodiment, the elements in set B can be divided into multiple subsets based on different categories of the start flag: B first (start=0), B start1 (start=1), B start_2 (start=2), B start_3 (start=3), B start_4 (start=4).

[0102] Pair of subset B first All elements b in the set are calculated as follows:

[0103] a) Based on the current element b pre Calculate the coordinates of the text cell (x1, y1, x2, y2, x3, y3, x4, y4) (top left, top right, bottom left, bottom right corners). Add these coordinates to the point set cnt. Based on b... pre (end) The start set B with the same class index as the value index. next ;

[0104] b) According to b pre (bbox) and B next Inner element b nexti (bbox), calculate the following values ​​respectively:

[0105] x dis =x c_pre -x c_next

[0106] y dis =y c_pre -y c_next

[0107]

[0108]

[0109]

[0110] t = w pre +h pre

[0111] Among them, (x c_pre ,y c_pre ) for b pre The center coordinates of (w) pre ,hpre ) is b pre The width and length (x c_next , y c_next ) is b nexti The central coordinate point of

[0112] If b pre (end) = 0, then directly return the point set cnt, otherwise

[0113]

[0114] Among them, ω is an adjustable threshold, and this application uses 0.17. If D < t, then take the corresponding b min_dis And update b pre = b min_dis .

[0115] d) Repeat steps a - c until b pre (end) = 0. Calculate and return the bounding rectangle of the point set cnt, and the obtained bounding rectangle is the text box after merging;

[0116] S5. Input the validation set into the model trained in step S3. After obtaining the model output, finally obtain all the text boxes on the validation set through the text unit merging algorithm.

[0117] Text in natural scenes is often difficult to capture accurate text information due to various influences such as text angle and illumination angle. And accurate text information is beneficial for the computer to further analyze the content of video images and promote the further development of various related applications. The present invention provides a text detection method based on multi - classification. By utilizing the direction characteristics and connection characteristics of the text itself, introducing a multi - classification judgment mechanism for each text unit, and using a detection scheme of rasterization segmentation and merging, it can effectively solve various problems such as inaccurate text positioning, missed detection, and false detection in natural scenes, and at the same time can handle text detection problems at multiple angles. This method can not only ensure the generalization ability of the algorithm in various scenes, but also does not introduce too many parameters, guaranteeing the inference performance of the model and enabling the model to be used on various devices.

[0118] Further referring to Figure 3 , as an implementation of the above method, this application provides an embodiment of a text detection system based on multi - classification. This system embodiment corresponds to Figure 1 the method embodiment shown, and this system can be specifically applied to various electronic devices. This system 300 includes the following modules:

[0119] A sample acquisition module 310, configured to obtain and organize text box samples, cut the text box samples into text units, and label the coordinates and flag bits of the text units;

[0120] The generation module 320 is used to generate training and validation sets from text box samples proportionally, and to assign class labels to the flag bits of text units;

[0121] The model training module 330 is used to build a text unit detection network model. The training set is trained through the text unit detection network model until the loss converges, and the output of the trained model is obtained, which is set B.

[0122] Algorithm merging module 340 is used to construct a text unit merging algorithm. It divides set B into multiple subsets based on the class label of the flag bit. It calculates and obtains the bounding rectangle of the point set cnt based on the current element and the elements within its subset. This bounding rectangle represents the merged text box.

[0123] The validation module 350 is used to input the validation set into the model trained in the model training module 330, and finally obtain all the text boxes on the validation set.

[0124] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs any of the methods described above.

[0125] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer system 400 suitable for implementing terminal devices or servers in the embodiments of this application. Figure 4 The terminal device or server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0126] like Figure 4 As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 402 or programs loaded from storage section 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the system 400. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0127] The following components are connected to I / O interface 405: input section 406 including keyboard, mouse, etc.; output section 407 including liquid crystal display (LCD) and speakers, etc.; storage section 408 including hard disk, etc.; and communication section 409 including network interface card such as LAN card, modem, etc. Communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from them can be installed into storage section 408 as needed.

[0128] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable medium or any combination thereof. The computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0129] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The modules described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a receiving module, an acquiring module, a determining module, a calculating module, and a generating module. The names of these units do not necessarily limit the specific unit itself; for example, the receiving unit can also be described as "a module that, in response to determining that the verification request information includes a username, request time, user signature encoding, and client application encoding, acquires preset configuration information of the target user."

[0132] In another aspect, this application also provides a computer-readable medium, which may be included in the server described in the above embodiments; or it may exist independently and not assembled into the server. The computer-readable medium carries one or more programs, which, when executed by the server, cause the server to: receive verification request information sent by a client of a target user; in response to determining that the verification request information includes a username, request time, user signature code, and client application code, obtain preset configuration information of the target user, wherein the configuration information includes a preset user password corresponding to the username; determine whether the verification request information is valid based on the request time; in response to determining that it is valid, determine whether a preset storage area includes a user signature code; in response to determining that it does not include a user signature code, store the user signature code in the preset storage area, and calculate the server application code based on the user password, request time, and user signature code; in response to determining that the server application code and the client application code match, generate verification success information to characterize the verification request as a legitimate request.

[0133] Furthermore, the aforementioned computer-readable medium may be included in the terminal device described in the above embodiments; or it may exist independently and not assembled into the terminal device. The aforementioned computer-readable medium carries one or more programs that, when executed by the terminal device, cause the terminal device to: acquire user information input by the target user, wherein the user information includes a username and a password; generate a user signature code representing the target user based on the user information; determine the request time; calculate a client application code based on the user password, the request time, and the user signature code; generate verification request information including the username, request time, user signature code, and client application code; and send the verification request information to the server.

[0134] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method of text detection based on multi-classification, characterized in that, The method comprises the following steps: S1, obtaining and sorting text box samples, cutting the text box samples into text units, and labeling the coordinates and flag bits of the text units; S2, generating a training set and a verification set according to the proportion of the text box samples, and assigning class labels to the flag bits of the text units; S3, constructing a text unit detection network model, training the training set through the text unit detection network model until the loss converges, and obtaining the model output after the training is completed, wherein the output is a set B; S4, according to the different classes of the start flag, the elements in the set B are divided into multiple subsets: According to the current element and the elements in the subset to which the current element belongs, a point set cnt is calculated, and the circumscribed rectangle of the point set cnt is a merged text box. According to the current element and the elements in the subset to which the current element belongs, a point set cnt is obtained, and the outer rectangle of the point set cnt is a merged text box, comprising the following sub-steps: S41、according to the subset current element b pre Calculate the text unit frame coordinates and add them to the point set cnt, and according to the end flag value b pre of the current element b pre (start) index same start flag start class tag set ; S42、according to the current element b pre and the set the following values are calculated for each element within ； in,( , ) is the current element b pre The center coordinates of ( , ) is the current element b pre Width and length; , ) is the element b nexti The center coordinate point; S43, if b pre (end) = 0, then directly return the point set cnt, otherwise ； where ω is an adjustable threshold, and if D < t, take the corresponding b min_dis and update b pre = b min_dis ; S44, repeat steps S41-S43 until b pre (end) = 0, calculate the circumscribed rectangle of the point set cnt, and the obtained circumscribed rectangle is the completed text box; and S5, input the verification set into the model trained in step S3, and finally obtain all the text boxes on the verification set.

2. The multi-class based text detection method of claim 1, wherein, In step S1, the text box samples are obtained and sorted, comprising the following sub-steps: S11, obtaining a text detection data set, cutting the text boxes of the text detection data set into text units by rasterization; S12, labeling each text unit, calculating the coordinates of the cut text unit, and labeling the format as (x, y, w, h), wherein (x, y) is the center point coordinates of the text unit, and (w, h) is the width and height of the text unit; S13, marking the flag bits of the first and last text units of the text box, and marking the flag bits between every two text units.

3. The multi-class based text detection method of claim 2, wherein, In step S13, a flag is marked between each two text units, including the following sub-steps, wherein, (x pre ,y pre ) is the center coordinate point of the current text unit, (x next ,y next ) is the center coordinate point of the next text unit: S131、calculate the distance between the center point of the current text unit and the next text unit, if y next- y pre = 0, the current text box end flag and the next text box start flag are marked as horizontal; S132, if x next -x pre = 0, then the current textbox end flag and the next textbox start flag are marked as vertical; S133、if y next -y pre <0, the current textbox end flag and the next textbox start flag are marked as up; S134, if y next -y pre > 0, then the current textbox end flag and the next textbox start flag are marked as down.

4. The multi-class based text detection method of claim 2, wherein, In step S3, the model output after the training is completed further comprises obtaining a confidence value conf output through a sigmoid function, filtering the confidence value conf, and only retaining the corresponding bbox, start flag bit and end flag bit that satisfy conf >= theta, and outputting a set B(x, y, w, h, start, end), wherein theta is an adjustable threshold.

5. The multi-class based text detection method of claim 1, wherein, In step S3, the training set is trained through the text unit detection network model until the loss converges, and the total loss function expression of the text unit detection network model is: ； wherein, , , is a weight parameter, is a binary cross-entropy loss function, is a GIoU loss function, , is a multi-class cross-entropy loss function.

6. The multi-class based text detection method of claim 5, wherein, Further comprising that the determination of the character region adopts the binary cross-entropy loss function, and the expression of the binary cross-entropy loss function is: ； In the formula, y is a true label of a sample, is a network predicted label; The character box regression adopts the GIoU loss function, and the expression of the GIoU loss function is: ； In the formula, area(A) represents the real character region of the text unit, area(B) represents the model predicted character region, and area(C) represents the minimum convex closed frame region containing A and B; The category determination of the start and end flag bits adopts the multi-classification cross-entropy loss function, and the expression of the multi-classification cross-entropy loss function is: ； In the formula, is the true class of the sample, is the network predicted class.

7. A multi-class based text detection system, characterized in that, The system comprises: A sample collection module for obtaining and sorting text box samples, cutting the text box samples into text units, and labeling the coordinates and flag bits of the text units; A generation module for generating a training set and a verification set according to the proportion of the text box samples, and assigning class labels to the flag bits of the text units; A model training module for constructing a text unit detection network model, training the training set through the text unit detection network model until the loss converges, and obtaining the model output after the training is completed, wherein the output is a set B; An algorithm merging module is configured to divide the elements in the set B into multiple subsets according to different classes of the start flag start: A point set cnt is calculated according to the current element and the elements in the subset to which the current element belongs, and the circumscribed rectangle of the point set cnt is a completed text box. Calculate and obtain the bounding rectangle of the point set cnt based on the current element and the elements in the subset to which the current element belongs. The bounding rectangle is the merged text box. This includes the following sub-steps: Calculate the bounding rectangle based on the subset... The current element b in pre Calculate the coordinates of the text cell box and add them to the point set cnt, and according to the current element b pre End flag value b pre (end) A collection of indexes with the same starting flag (start) of the index class. According to the current element b pre and the set The inner elements are calculated as follows: ； wherein, , ) is the center coordinate point of the current element b pre , ( , ) is the width and length of the current element b pre ; ( , ) is the center coordinate point of the element b nexti ; if b pre (end) = 0, directly return the point set cnt, otherwise ； where ω is an adjustable threshold, and if D < t, take the corresponding b min_dis and update b pre = b min_dis ; Repeat the above sub-steps until b pre If (end) = 0, calculate the circumscribed rectangle of the point set cnt, and the obtained circumscribed rectangle is the completed text box.

8. A computer readable storage medium having stored therein a computer program which, when executed by a processor, carries out the method of any one of claims 1-6.

Images