Image text detection method and system, terminal and storage medium
By combining the YOLO and VL models with OCR technology, the problem of accurate text localization and recognition is solved, achieving efficient structured information output, which is suitable for text processing in complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-04-03
AI Technical Summary
Existing technologies struggle to simultaneously achieve accurate text localization, efficient recognition, and structured information output, resulting in poor performance when processing text in complex scenarios.
The YOLO model is used for text region detection, combined with the VL model and OCR recognition. Image cropping and recognition tasks are executed concurrently through a thread pool, and the results are integrated by deduplication decision to output structured data.
It achieves high-precision and high-speed text recognition, can handle complex text structures, understand text context relationships, and improve system response speed and processing efficiency.
Smart Images
Figure CN121789236A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text detection technology, and in particular to an image text detection method, system, terminal, and computer-readable storage medium. Background Technology
[0002] Object detection technology, as a core task of computer vision, has continuously evolved under the impetus of deep learning. From the R-CNN series to single-stage detectors represented by YOLO, it has significantly improved both detection accuracy and speed, and has been widely applied in practical scenarios, including text localization. Meanwhile, optical character recognition technology has shifted from traditional methods to end-to-end systems based on deep learning. While it is relatively mature in printed text recognition, it still faces challenges in recognizing complex scenes, handwritten text, and texts with special formats.
[0003] In recent years, visual language models have made breakthroughs in cross-modal understanding and reasoning, demonstrating powerful image content understanding and natural language interaction capabilities. However, existing models still have shortcomings in achieving accurate text localization and structured information output, making it difficult to directly replace specialized detection and recognition technologies.
[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention
[0005] The main objective of this invention is to provide an image text detection method, system, terminal, and computer-readable storage medium, aiming to solve the problem that existing technologies are unable to simultaneously achieve accurate text positioning, efficient recognition, and structured information output, resulting in poor performance when processing text in complex scenarios.
[0006] To achieve the above objectives, the present invention provides an image text detection method, which includes the following steps: The system receives image files uploaded by users, calls the core processing function to perform image processing on the image files, and obtains preprocessed image files. Load a pre-trained YOLO model, use the YOLO model to detect text regions in the preprocessed image file, generate bounding boxes, filter and remove the bounding boxes to obtain the optimal detection result; Based on the optimal detection results, image cropping tasks are distributed, image cropping operations are executed concurrently using a thread pool, cropped images are cached, VL model tasks and OCR processing tasks are distributed, VL model recognition and OCR accurate recognition are executed concurrently, and VL recognition results and OCR recognition results are obtained. The VL recognition result and the OCR recognition result are analyzed to obtain the text content. The similarity of the text content is calculated. Based on the deduplication decision, overlapping areas are removed. The deduplicated VL recognition result and the OCR recognition result are integrated to output structured data.
[0007] Optionally, the image text detection method, wherein receiving the image file uploaded by the user and calling the core processing function to perform image processing on the image file to obtain a preprocessed image file, specifically includes: Provides a RESTful API interface to receive image files uploaded by users; Create a temporary storage space and store the image file in the storage space; The core processing function is invoked to perform image processing on the image file in the storage space to obtain a preprocessed image file. Returns a structured JSON response to the user and automatically cleans up temporary files.
[0008] Optionally, the image text detection method, wherein loading a pre-trained YOLO model, performing text region detection on the preprocessed image file using the YOLO model, generating bounding boxes, and filtering and removing the bounding boxes to obtain the optimal detection result, specifically includes: Load the pre-trained YOLO model and adjust the image size of the preprocessed image file to fit the model input requirements; The preprocessed image file with adjusted image size is input into the YOLO model, and the YOLO model performs text region detection on the preprocessed image file to generate multiple bounding boxes. Filter and remove nested bounding boxes, retain the best detection results, and extract and output coordinate information.
[0009] Optionally, the image text detection method, wherein based on the optimal detection result, distributes image cropping tasks, performs image cropping operations concurrently using a thread pool, caches the cropped image, distributes VL model tasks and OCR processing tasks, and concurrently performs VL model recognition and OCR accurate recognition to obtain VL recognition results and OCR recognition results, specifically includes: Based on the text region in the optimal detection result, the image cropping task is distributed to multiple threads; Use a thread pool to concurrently perform image cropping operations on each text region, create a unique cache directory using UUID, and cache the cropped images. Distribute VL modeling tasks and OCR processing tasks, concurrently execute VL model recognition and OCR accurate recognition, apply flow limiting control, and obtain VL recognition results and OCR recognition results.
[0010] Optionally, in the image text detection method, the VL model task represents processing the cropped image using a VL model; the OCR processing task represents processing the cropped image using an OCR model.
[0011] Optionally, the image text detection method, wherein parsing the VL recognition result and the OCR recognition result to obtain text content, calculating the similarity of the text content, removing overlapping regions based on deduplication decisions, integrating the deduplicated VL recognition result and the OCR recognition result, and outputting structured data, specifically includes: The VL recognition result and the OCR recognition result are parsed to obtain the text content. The text content is then analyzed to determine whether secondary processing is required. If secondary processing is required, an improved deduplication logic is applied, which checks the inclusion relationship and overlapping areas of bounding boxes based on text content and coordinate relationships. Calculate the similarity of text content, where the similarity represents the degree of overlap between text contents; Based on the similarity, a deduplication decision is made to remove overlapping regions, and the deduplicated VL recognition result and the OCR recognition result are integrated to output structured data.
[0012] Optionally, the image text detection method further includes, in the steps of parsing the VL recognition result and the OCR recognition result to obtain text content, calculating the similarity of the text content, removing overlapping regions based on deduplication decisions, integrating the deduplicated VL recognition result and the OCR recognition result to output structured data, and then further including: Returns a structured JSON response, cleans up temporary files and cache directories, and releases system resources.
[0013] Furthermore, to achieve the above objectives, the present invention also provides an image text detection system, wherein the image text detection system comprises: The image processing module is used to receive image files uploaded by users, call the core processing function to perform image processing on the image files, and obtain preprocessed image files. The text region detection module is used to load a pre-trained YOLO model, perform text region detection on the preprocessed image file using the YOLO model, generate bounding boxes, filter and remove the bounding boxes, and obtain the optimal detection result. The multi-model parallel processing module is used to distribute image cropping tasks based on the optimal detection results, execute image cropping operations concurrently using a thread pool, cache the cropped images, distribute VL model tasks and OCR processing tasks, and concurrently execute VL model recognition and OCR accurate recognition to obtain VL recognition results and OCR recognition results. The result fusion and deduplication module is used to parse the VL recognition result and the OCR recognition result to obtain the text content, calculate the similarity of the text content, remove overlapping areas based on the deduplication decision, integrate the deduplicated VL recognition result and the OCR recognition result, and output structured data.
[0014] In addition, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an image text detection program stored in the memory and executable on the processor, wherein when the image text detection program is executed by the processor, it implements the steps of the image text detection method as described above.
[0015] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image text detection program, which, when executed by a processor, implements the steps of the image text detection method as described above.
[0016] In this invention, an image file uploaded by a user is received, and a core processing function is invoked to process the image file, resulting in a preprocessed image file. A pre-trained YOLO model is loaded, and text region detection is performed on the preprocessed image file using the YOLO model to generate bounding boxes. The bounding boxes are then filtered and removed to obtain the optimal detection result. Based on the optimal detection result, an image cropping task is distributed, and the image cropping operation is executed concurrently using a thread pool. The cropped image is cached, and VL model tasks and OCR processing tasks are distributed and executed concurrently for VL model recognition and accurate OCR recognition, resulting in VL recognition results and OCR recognition results. The VL recognition results and OCR recognition results are parsed to obtain the text content. The similarity of the text content is calculated, and overlapping regions are removed based on deduplication decisions. The deduplicated VL recognition results and OCR recognition results are integrated to output structured data. This invention can handle complex text structures, understand text context relationships, achieve high-precision and high-efficiency text recognition, and improve system response speed and processing efficiency. Attached Figure Description
[0017] Figure 1 This is a flowchart of a preferred embodiment of the image text detection method of the present invention; Figure 2This is a schematic diagram of the entire execution process in a preferred embodiment of the image text detection method of the present invention; Figure 3 This is a schematic diagram of the image receiving and preprocessing process in a preferred embodiment of the image text detection method of the present invention; Figure 4 This is a schematic diagram of the text region detection process in a preferred embodiment of the image text detection method of the present invention; Figure 5 This is a schematic diagram of the multi-model parallel processing process in a preferred embodiment of the image text detection method of the present invention; Figure 6 This is a schematic diagram of the result fusion and deduplication process in a preferred embodiment of the image text detection method of the present invention; Figure 7 This is a structural diagram of a preferred embodiment of the image text detection system of the present invention; Figure 8 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0019] Object detection, as one of the core tasks of computer vision, has made significant progress in recent years with the development of deep learning. From traditional R-CNN, Fast R-CNN, and Faster R-CNN to single-stage detectors such as the YOLO series and SSD, object detection technology has greatly improved in both accuracy and speed. The YOLO (You Only Look Once) series of models has been widely used in industry due to its efficient real-time detection capabilities, especially in the field of text detection, where YOLO models can quickly locate text regions in images.
[0020] Optical Character Recognition (OCR) technology has evolved over decades from early template matching and feature engineering methods to end-to-end recognition systems based on deep learning. Modern OCR systems such as Tesseract, Baidu OCR, and Google Cloud Vision have achieved high accuracy in printed text recognition. However, these systems still face challenges when processing text in complex scenarios, handwritten text, multilingual text, and text with special formats (such as tables and formulas).
[0021] Visual-language models (VLMs) represent a major breakthrough in the field of artificial intelligence in recent years. Models such as GPT-4V, Qwen-VL, and Doubao-VL have demonstrated powerful visual understanding and language reasoning capabilities. These models can understand image content and perform natural language descriptions, but they still have limitations in accurate text localization and structured output.
[0022] (1) The disadvantages of the traditional object detection + OCR combination scheme are as follows: Inaccurate localization: Traditional object detection models are not accurate enough in locating the bounding boxes of text regions, resulting in excessive background noise or truncated text during subsequent OCR recognition.
[0023] Fragmented processing flow: Object detection and OCR are processed as independent modules, lacking end-to-end optimization, which limits overall performance.
[0024] Weak ability to process complex text: Insufficient ability to process complex structures such as tables, formulas, and multi-directional text.
[0025] Lack of contextual understanding: The text lacks semantic understanding of its content, making it impossible to perform intelligent text classification and structuring.
[0026] (2) Disadvantages of the pure OCR solution: Reliance on pre-localized text regions: Most OCR systems require pre-provided text regions and cannot automatically handle full-image text detection and recognition.
[0027] Sensitive to complex layouts: Performance drops significantly in scenarios with dense text, overlapping text, and multi-directional text.
[0028] Lack of semantic understanding: Focusing only on character recognition, it fails to understand the semantic relationships and structural information of the text.
[0029] Duplicate recognition problem: Duplicate recognition results may be generated for the same text region, and there is a lack of effective deduplication mechanism.
[0030] (3) Disadvantages of the pure visual language model scheme: Insufficient localization accuracy: Although VLM can understand image content, it is not as good as dedicated object detection models in terms of accurate text bounding box localization.
[0031] Limitations of structured output: It is difficult to output standardized structured data, such as precise coordinate information and classification labels.
[0032] High computational resource consumption: Large-scale VLM models have high inference costs and are not suitable for large-scale real-time applications.
[0033] Stability issues: It has strict requirements on the format and size of the input image, resulting in insufficient robustness.
[0034] The image text detection method described in the preferred embodiment of the present invention, such as... Figure 1 and Figure 2 As shown, the image text detection method includes the following steps: Step S10: Receive the image file uploaded by the user, call the core processing function to perform image processing on the image file, and obtain the preprocessed image file.
[0035] Specifically, such as Figure 3 As shown, a RESTful API interface is provided (a RESTful API is an application programming interface designed based on the REST architectural style, which utilizes the characteristics of the HTTP protocol to conduct network communication centered on resources). The API receives image files uploaded by users; creates temporary storage space and stores the image files in the storage space; calls the core processing function to perform image processing on the image files in the storage space to obtain preprocessed image files; returns a structured JSON response to the user, and automatically cleans up temporary files.
[0036] Step S20: Load the pre-trained YOLO model, use the YOLO model to detect text regions in the preprocessed image file, generate bounding boxes, filter and remove the bounding boxes to obtain the optimal detection result.
[0037] Specifically, such as Figure 4 As shown, a pre-trained YOLO model (You Only Look Once, a revolutionary real-time object detection algorithm model whose core idea is to reconstruct the object detection task into a regression problem of a single neural network forward propagation, achieving a breakthrough balance between speed and accuracy) is loaded. The image size of the pre-processed image file is adjusted to meet the model input requirements. The pre-processed image file with adjusted image size is input into the YOLO model, and the YOLO model performs text region detection on the pre-processed image file, generating multiple bounding boxes. Nested bounding boxes are filtered and removed, the best detection results are retained, and coordinate information is extracted and output (the coordinate information is used to determine the boundaries of the text region).
[0038] The YOLO model is used to achieve precise localization of text regions, overcoming the problems of insufficient localization accuracy of VLM and OCR's reliance on pre-positioning.
[0039] Step S30: Based on the optimal detection result, distribute image cropping tasks, use a thread pool to concurrently execute image cropping operations, cache the cropped images, distribute VL model tasks and OCR processing tasks, concurrently execute VL model recognition and OCR accurate recognition, and obtain VL recognition results and OCR recognition results.
[0040] Specifically, such as Figure 5 As shown, based on the text region in the optimal detection result, image cropping tasks are distributed to multiple threads for multi-threaded concurrent operation (using multi-threaded concurrency to improve speed); a thread pool is used to concurrently perform image cropping operations on each text region, a unique cache directory is created using UUID, and the cropped image is cached; VL model tasks and OCR processing tasks are distributed, and VL model recognition and OCR accurate recognition are performed concurrently, with rate limiting applied to avoid API call overruns, to obtain VL recognition results and OCR recognition results.
[0041] The VL model task refers to using the VL model (Visual Language) to process the cropped image; the OCR processing task refers to using the OCR model (Optical Character Recognition) to process the cropped image.
[0042] Thread pools are used to implement concurrent processing of image cropping and model inference, token bucket algorithm is applied to implement API rate limiting, and caching mechanism is used to improve system response speed.
[0043] Step S40: Analyze the VL recognition result and the OCR recognition result to obtain the text content, calculate the similarity of the text content, remove overlapping areas based on deduplication decision, integrate the deduplicated VL recognition result and the OCR recognition result, and output structured data.
[0044] Specifically, such as Figure 6As shown, the VL recognition result and the OCR recognition result are parsed to obtain the text content. The text content is analyzed to determine whether secondary processing is needed (e.g., if multiple paragraphs are found, secondary processing is required). If secondary processing is needed, an improved deduplication logic is applied. Based on the text content and coordinate relationship, the inclusion relationship and overlapping areas of the bounding boxes are checked. The similarity of the text content is calculated, where the similarity represents the degree of overlap between text content. If two texts have a high degree of overlap and similar coordinates, deduplication should be performed. Based on the similarity, a deduplication decision is adopted to remove overlapping areas. The deduplicated VL recognition result and the OCR recognition result are then integrated to output structured data, providing a unified structured output format to meet the needs of different application scenarios. Finally, a structured JSON response is returned, temporary files and cache directories are cleaned up, and system resources are released.
[0045] Combining YOLO object detection, VL model recognition, and OCR for accurate recognition, this system employs the most suitable recognition method for different types of text content, and improves recognition accuracy through a secondary processing mechanism. Based on bounding box inclusion relationships and overlapping areas, it incorporates a deduplication strategy that considers text content similarity, supporting intelligent deduplication even in cases of partial overlap.
[0046] The technical effects that this invention can bring are as follows: (1) High-precision detection and recognition: Through multi-model fusion, the accuracy of text detection and recognition is significantly improved; it has better processing capabilities for complex scenarios (such as multi-line text and cross-line text); combining the precise positioning capability of the YOLO model and the semantic understanding of the visual language model, it achieves precise detection and recognition of text regions.
[0047] (2) Strong complex text processing capability: It can process scattered, overlapping or complex text regions; it supports the recognition of various text formats and layouts; it can process complex text structures and understand text context relationships by utilizing the powerful semantic understanding capability of visual language models.
[0048] (3) High efficiency in deduplication: effectively removes duplicate detection results and retains the most accurate information; based on multi-dimensional (coordinates, content) deduplication strategy, it improves the accuracy of deduplication; innovative deduplication algorithm effectively avoids duplicate recognition and improves the quality of results.
[0049] (4) System stability: Through rate limiting control and exception handling, the system is guaranteed to operate stably in high-concurrency scenarios; a sound resource management mechanism is in place to avoid resource leakage.
[0050] (5) Improved processing efficiency: The image processing speed is significantly improved through concurrent processing and caching mechanisms; the optimized algorithms and data structures reduce computational overhead and improve system response speed and processing efficiency.
[0051] (6) Multi-format compatibility: It can handle a variety of text formats, including continuous text, phrases, individual characters, table content, etc.
[0052] (7) Standardized output: Provides a unified structured output format (structured JSON output, including text content, precise coordinates, classification information, etc.) to facilitate subsequent processing and application; supports multiple output formats to meet the needs of different application scenarios.
[0053] (8) Flexible and scalable: Modular design facilitates functional expansion and model upgrades, adapting to different application scenarios.
[0054] (9) Strong robustness: It has strong adaptability to factors such as image quality, size, and orientation, which improves the practicality and reliability of the system.
[0055] This invention solves key technical problems in text detection and recognition in images through innovative multimodal fusion technology and intelligent deduplication algorithm, realizing a high-precision and high-efficiency text recognition system. It has strong practical value and broad application prospects, achieving a breakthrough in text detection and recognition technology and providing strong technical support for fields such as digital education, document processing, and intelligent content analysis.
[0056] Furthermore, such as Figure 7 As shown, based on the above image text detection method, the present invention also provides an image text detection system, wherein the image text detection system includes: Image processing module 51 is used to receive image files uploaded by users, call core processing functions to perform image processing on the image files, and obtain preprocessed image files; The text region detection module 52 is used to load a pre-trained YOLO model, perform text region detection on the preprocessed image file using the YOLO model, generate bounding boxes, filter and remove the bounding boxes, and obtain the optimal detection result. The multi-model parallel processing module 53 is used to distribute image cropping tasks based on the optimal detection results, perform image cropping operations concurrently using a thread pool, cache the cropped images, distribute VL model tasks and OCR processing tasks, and perform VL model recognition and OCR accurate recognition concurrently to obtain VL recognition results and OCR recognition results. The result fusion and deduplication module 54 is used to parse the VL recognition result and the OCR recognition result to obtain text content, calculate the similarity of the text content, remove overlapping areas based on deduplication decision, integrate the deduplicated VL recognition result and the OCR recognition result, and output structured data.
[0057] Furthermore, such as Figure 8 As shown, based on the above image text detection method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 8 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.
[0058] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores an image text detection program 40, which can be executed by the processor 10 to implement the image text detection method of this application.
[0059] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the image text detection method.
[0060] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The terminal's processor 10, memory 20, and display 30 communicate with each other via a system bus.
[0061] In one embodiment, when the processor 10 executes the image text detection program 40 in the memory 20, the following steps are performed: The system receives image files uploaded by users, calls the core processing function to perform image processing on the image files, and obtains preprocessed image files. Load a pre-trained YOLO model, use the YOLO model to detect text regions in the preprocessed image file, generate bounding boxes, filter and remove the bounding boxes to obtain the optimal detection result; Based on the optimal detection results, image cropping tasks are distributed, image cropping operations are executed concurrently using a thread pool, cropped images are cached, VL model tasks and OCR processing tasks are distributed, VL model recognition and OCR accurate recognition are executed concurrently, and VL recognition results and OCR recognition results are obtained. The VL recognition result and the OCR recognition result are analyzed to obtain the text content. The similarity of the text content is calculated. Based on the deduplication decision, overlapping areas are removed. The deduplicated VL recognition result and the OCR recognition result are integrated to output structured data.
[0062] Specifically, receiving the image file uploaded by the user and calling the core processing function to perform image processing on the image file to obtain a preprocessed image file includes: Provides a RESTful API interface to receive image files uploaded by users; Create a temporary storage space and store the image file in the storage space; The core processing function is invoked to perform image processing on the image file in the storage space to obtain a preprocessed image file. Returns a structured JSON response to the user and automatically cleans up temporary files.
[0063] The loading of the pre-trained YOLO model, the detection of text regions in the preprocessed image file using the YOLO model, the generation of bounding boxes, and the filtering and removal of the bounding boxes to obtain the optimal detection result specifically include: Load the pre-trained YOLO model and adjust the image size of the preprocessed image file to fit the model input requirements; The preprocessed image file with adjusted image size is input into the YOLO model, and the YOLO model performs text region detection on the preprocessed image file to generate multiple bounding boxes. Filter and remove nested bounding boxes, retain the best detection results, and extract and output coordinate information.
[0064] Specifically, based on the optimal detection result, image cropping tasks are distributed, image cropping operations are executed concurrently using a thread pool, cropped images are cached, VL model tasks and OCR processing tasks are distributed, and VL model recognition and accurate OCR recognition are executed concurrently to obtain VL recognition results and OCR recognition results. Based on the text region in the optimal detection result, the image cropping task is distributed to multiple threads; Use a thread pool to concurrently perform image cropping operations on each text region, create a unique cache directory using UUID, and cache the cropped images. Distribute VL modeling tasks and OCR processing tasks, concurrently execute VL model recognition and OCR accurate recognition, apply flow limiting control, and obtain VL recognition results and OCR recognition results.
[0065] The VL model task refers to using a VL model to process the cropped image; the OCR processing task refers to using an OCR model to process the cropped image.
[0066] Specifically, the process of parsing the VL recognition result and the OCR recognition result to obtain text content, calculating the similarity of the text content, removing overlapping regions based on deduplication decisions, integrating the deduplicated VL recognition result and OCR recognition result, and outputting structured data includes: The VL recognition result and the OCR recognition result are parsed to obtain the text content. The text content is then analyzed to determine whether secondary processing is required. If secondary processing is required, an improved deduplication logic is applied, which checks the inclusion relationship and overlapping areas of bounding boxes based on text content and coordinate relationships. Calculate the similarity of text content, where the similarity represents the degree of overlap between text contents; Based on the similarity, a deduplication decision is made to remove overlapping regions, and the deduplicated VL recognition result and the OCR recognition result are integrated to output structured data.
[0067] The process includes parsing the VL recognition result and the OCR recognition result to obtain text content, calculating the similarity of the text content, removing overlapping regions based on deduplication decisions, integrating the deduplicated VL recognition result and OCR recognition result, and outputting structured data. The process further includes: Returns a structured JSON response, cleans up temporary files and cache directories, and releases system resources.
[0068] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image text detection program, which, when executed by a processor, implements the steps of the image text detection method described above.
[0069] In summary, this invention provides an image text detection method, system, terminal, and computer-readable storage medium. The method includes: receiving an image file uploaded by a user; calling a core processing function to process the image file to obtain a preprocessed image file; loading a pre-trained YOLO model; performing text region detection on the preprocessed image file using the YOLO model to generate bounding boxes; filtering and removing the bounding boxes to obtain the optimal detection result; based on the optimal detection result, distributing image cropping tasks; concurrently executing image cropping operations using a thread pool; caching the cropped image; distributing VL model tasks and OCR processing tasks; concurrently executing VL model recognition and OCR accurate recognition to obtain VL recognition results and OCR recognition results; parsing the VL recognition results and the OCR recognition results to obtain text content; calculating the similarity of the text content; removing overlapping regions based on deduplication decisions; integrating the deduplicated VL recognition results and the OCR recognition results to output structured data. This invention can handle complex text structures, understand text context relationships, achieve high-precision and high-efficiency text recognition, and improve system response speed and processing efficiency.
[0070] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.
[0071] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.
[0072] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. An image text detection method, characterized in that, The image text detection method includes: The system receives image files uploaded by users, calls the core processing function to perform image processing on the image files, and obtains preprocessed image files. Load a pre-trained YOLO model, use the YOLO model to detect text regions in the preprocessed image file, generate bounding boxes, filter and remove the bounding boxes to obtain the optimal detection result; Based on the optimal detection results, image cropping tasks are distributed, image cropping operations are executed concurrently using a thread pool, cropped images are cached, VL model tasks and OCR processing tasks are distributed, VL model recognition and OCR accurate recognition are executed concurrently, and VL recognition results and OCR recognition results are obtained. The VL recognition result and the OCR recognition result are analyzed to obtain the text content. The similarity of the text content is calculated. Based on the deduplication decision, overlapping areas are removed. The deduplicated VL recognition result and the OCR recognition result are integrated to output structured data.
2. The image text detection method according to claim 1, characterized in that, The process of receiving an image file uploaded by the user and calling a core processing function to perform image processing on the image file to obtain a preprocessed image file includes: Provides a RESTful API interface to receive image files uploaded by users; Create a temporary storage space and store the image file in the storage space; The core processing function is invoked to perform image processing on the image file in the storage space to obtain a preprocessed image file. Returns a structured JSON response to the user and automatically cleans up temporary files.
3. The image text detection method according to claim 1, characterized in that, The loading of the pre-trained YOLO model, the use of the YOLO model to detect text regions in the preprocessed image file, the generation of bounding boxes, and the filtering and removal of the bounding boxes to obtain the optimal detection result, specifically includes: Load the pre-trained YOLO model and adjust the image size of the pre-processed image file to fit the model input requirements; The preprocessed image file with adjusted image size is input into the YOLO model, and the YOLO model performs text region detection on the preprocessed image file to generate multiple bounding boxes. Filter and remove nested bounding boxes, retain the best detection results, and extract and output coordinate information.
4. The image text detection method according to claim 1, characterized in that, Based on the optimal detection result, image cropping tasks are distributed, image cropping operations are executed concurrently using a thread pool, cropped images are cached, VL model tasks and OCR processing tasks are distributed, VL model recognition and OCR accurate recognition are executed concurrently, and VL recognition results and OCR recognition results are obtained, specifically including: Based on the text region in the optimal detection result, the image cropping task is distributed to multiple threads; Use a thread pool to concurrently perform image cropping operations on each text region, create a unique cache directory using UUID, and cache the cropped images. Distribute VL modeling tasks and OCR processing tasks, concurrently execute VL model recognition and OCR accurate recognition, apply flow limiting control, and obtain VL recognition results and OCR recognition results.
5. The image text detection method according to claim 4, characterized in that, The VL model task refers to using a VL model to process the cropped image; the OCR processing task refers to using an OCR model to process the cropped image.
6. The image text detection method according to claim 1, characterized in that, The process involves parsing the VL recognition result and the OCR recognition result to obtain text content, calculating the similarity of the text content, removing overlapping regions based on deduplication decisions, integrating the deduplicated VL recognition result and OCR recognition result, and outputting structured data. Specifically, this includes: The VL recognition result and the OCR recognition result are parsed to obtain the text content. The text content is then analyzed to determine whether secondary processing is required. If secondary processing is required, an improved deduplication logic is applied, which checks the inclusion relationship and overlapping areas of bounding boxes based on text content and coordinate relationships. Calculate the similarity of text content, where the similarity represents the degree of overlap between text contents; Based on the similarity, a deduplication decision is made to remove overlapping regions, and the deduplicated VL recognition result and the OCR recognition result are integrated to output structured data.
7. The image text detection method according to claim 1, characterized in that, The process involves parsing the VL recognition result and the OCR recognition result to obtain text content, calculating the similarity of the text content, removing overlapping regions based on deduplication decisions, integrating the deduplicated VL recognition result and OCR recognition result, and outputting structured data. The process further includes: Returns a structured JSON response, cleans up temporary files and cache directories, and releases system resources.
8. An image text detection system, characterized in that, The image text detection system includes: The image processing module is used to receive image files uploaded by users, call the core processing function to perform image processing on the image files, and obtain preprocessed image files. The text region detection module is used to load a pre-trained YOLO model, perform text region detection on the preprocessed image file using the YOLO model, generate bounding boxes, filter and remove the bounding boxes, and obtain the optimal detection result. The multi-model parallel processing module is used to distribute image cropping tasks based on the optimal detection results, execute image cropping operations concurrently using a thread pool, cache the cropped images, distribute VL model tasks and OCR processing tasks, and concurrently execute VL model recognition and OCR accurate recognition to obtain VL recognition results and OCR recognition results. The result fusion and deduplication module is used to parse the VL recognition result and the OCR recognition result to obtain the text content, calculate the similarity of the text content, remove overlapping areas based on the deduplication decision, integrate the deduplicated VL recognition result and the OCR recognition result, and output structured data.
9. A terminal, characterized in that, The terminal includes: a memory, a processor, and an image text detection program stored in the memory and executable on the processor. When the image text detection program is executed by the processor, it implements the steps of the image text detection method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image text detection program, which, when executed by a processor, implements the steps of the image text detection method as described in any one of claims 1-7.