Geological exploration handwritten revealing board OCR (optical character recognition) method and system

Abstract

The invention discloses an OCR (Optical Character Recognition) method and an OCR system for a geological exploration handwritten revealing board, and the method comprises the steps: obtaining a handwritten revealing board image, and carrying out the image preprocessing, thereby obtaining a preprocessed handwritten revealing board image; on the basis of an OCR recognition model, character detection and character recognition are carried out on the preprocessed handwritten revealing board image, and a character recognition result is obtained; and based on the large language model, performing semantic analysis on the character recognition result, and outputting structured data. According to the method, the PP-OCRv4 model and the Qwen-7B large model are fused, the PP-OCRv4 model focuses on detection and recognition of handwritten characters and can effectively deal with various interference factors in a complex field environment, and the Qwen-7B large language model makes up for the defects of traditional OCR in the aspect of structured information extraction through semantic comprehension and context association ability, so that the method has the advantages of being high in practicability and high in practicability. And a brand-new technical path is provided for digital acquisition of the geological exploration handwritten revealing board.

Description

Technical Field

[0001] This invention relates to the field of geological exploration technology, specifically to an OCR recognition method and system for handwritten geological exploration signs. Background Technology

[0002] In field operations such as geological exploration and mineral resource exploration, on-site personnel typically need to record key information such as exploration point coordinates, rock strata characteristics, and sampling depth by hand on metal or plastic signboards. Accurate digitization of this handwritten information, as a crucial carrier of raw exploration data, presents numerous challenges.

[0003] 1) Difficulty in recognition under complex environmental interference. Display boards are often exposed to harsh environments such as rain, snow, and dust storms, leading to reflections, dirt, and corrosion on the board surface. Traditional OCR technology has poor tolerance to low contrast (such as the low color difference between faded white paint and the metal base) and localized defects (such as broken strokes caused by rust at the corners of the board). Changes in ambient lighting can cause unstable image quality, especially since the specular reflection on the metal board surface can create highlight areas, causing broken or deformed character strokes.

[0004] 2) Bottlenecks in recognizing non-standardized handwriting. Differences in writing habits among exploration personnel lead to variations in character forms, primarily manifesting in three categories: numerical distortion (e.g., the connected strokes of "2" and "Z"), ambiguity in professional symbols (e.g., the visual similarity between the geological symbol "∠" and the mathematical symbol "<"), and multilingual mixing (the alternating arrangement of Chinese annotations and English abbreviations). For example, the number "7" written by different members of the same exploration team may have horizontal strokes at different angles; this individual variation renders traditional template matching methods ineffective.

[0005] 3) Semantic Faults in Structured Information Extraction. While the information on the display plaques possesses a specific semantic structure, handwritten character recognition faces three major obstacles: positional encoding failure (irregular character spacing leads to coordinate alignment deviations), lack of context (isolated recognition cannot infer numerical attributes), and multimodal interference (mixed use of numbers and symbols of different units). For example, "15°" might represent the dip angle of a rock stratum or the azimuth angle of a borehole, requiring semantic association with other fields on the plaque. Summary of the Invention

[0006] This invention provides an OCR recognition method and system for handwritten geological exploration signs. By integrating the PP-OCRv4 model and the Qwen-7B large language model, a lightweight end-to-end recognition system is constructed to solve the problem of difficulty in digitizing handwritten sign information in existing geological exploration.

[0007] According to a first aspect, one embodiment provides an OCR recognition method for handwritten geological exploration signboards, the method comprising:

[0008] Acquire the image of the handwritten notice board and perform image preprocessing to obtain the preprocessed image of the handwritten notice board;

[0009] Based on the OCR recognition model, text detection and text recognition are performed on the preprocessed handwritten sign image to obtain the text recognition result.

[0010] Based on a large language model, semantic parsing is performed on the text recognition results, and structured data is output.

[0011] Further, the handwritten sign image is acquired and preprocessed to obtain a preprocessed handwritten sign image, specifically including:

[0012] The image is converted to grayscale by performing grayscale processing. Then, Gaussian filtering or median filtering is used to remove noise from the image. Histogram equalization or adaptive contrast enhancement algorithms are used to improve the contrast of the image. Finally, the grayscale image is converted to a binary image for subsequent text detection.

[0013] Furthermore, based on the OCR recognition model, text detection and recognition are performed on the preprocessed handwritten sign image to obtain the text recognition results, specifically including:

[0014] Use the text detection module of PP-OCRv4 to locate the text region and extract the coordinates of the text region;

[0015] Then, the text recognition module of PP-OCRv4 is used to convert the text in the text area into a text string.

[0016] Furthermore, the text detection module of PP-OCRv4 is used to locate the text region and extract the coordinates of the text region, specifically including:

[0017] PP-OCRv4 employs a text detection algorithm based on the DBNet network, which achieves accurate segmentation of text regions through a differentiable binarization network.

[0018] Furthermore, the text detection module of PP-OCRv4 is used to locate the text region and extract the coordinates of the text region, specifically including:

[0019] DBNet's loss function is defined as:

[0020] LDB=α·Lbce(P,G)+β·Ldice(P,G)

[0021] Where P represents the predicted probability map, G represents the true label, Lbce represents the binary cross-entropy loss, Ldice represents the Dice coefficient loss, and α and β are balance parameters. The binary cross-entropy loss is used to measure the difference between the predicted value and the true value, while the Dice coefficient loss is used to measure the degree of overlap between the predicted region and the true region. By combining the two loss functions, DBNet can accurately segment text regions.

[0022] Furthermore, the text recognition module of PP-OCRv4 is used to convert the text in the text region into a text string, specifically including:

[0023] PP-OCRv4 uses a text recognition algorithm based on CRNN network.

[0024] Furthermore, the text recognition module of PP-OCRv4 is used to convert the text in the text region into a text string, specifically including:

[0025] The structure of the CRNN network includes convolutional layers, recurrent layers, and fully connected layers. Convolutional layers are used to extract image features, recurrent layers are used to encode feature sequences, and fully connected layers are used to map the encoded features to character categories. The loss function of CRNN is CTC, which is used to handle sequence recognition problems.

[0026] Furthermore, based on a large language model, semantic parsing is performed on the text recognition results, and structured data is output, specifically including:

[0027] The prompt words are constructed, and the Qwen-7B large model based on the Transformer architecture is used for semantic parsing. The recognized text strings are input into the Qwen-7B model, and structured data is generated by combining the prompt words.

[0028] According to a second aspect, one embodiment provides an OCR recognition system for handwritten geological exploration signs, the system comprising:

[0029] The image preprocessing module is used to acquire the handwritten sign image and perform image preprocessing to obtain the preprocessed handwritten sign image;

[0030] The OCR recognition module is used to perform text detection and text recognition on the pre-processed handwritten sign image based on the OCR recognition model, and obtain the text recognition result.

[0031] The large language model processing module is used to perform semantic parsing on the text recognition results based on the large language model and output structured data.

[0032] According to a third aspect, one embodiment provides an electronic device, the device comprising: a processor and a memory;

[0033] The memory is used to store one or more program instructions;

[0034] The processor is configured to run one or more program instructions to perform the steps of an OCR recognition method for handwritten geological exploration signs as described in any of the preceding claims.

[0035] According to a fourth aspect, one embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of an OCR recognition method for handwritten geological exploration signs as described in any of the preceding claims.

[0036] This invention provides an OCR recognition method and system for handwritten geological exploration signs. The method involves acquiring an image of the handwritten sign and performing image preprocessing to obtain a preprocessed image. Based on an OCR recognition model, text detection and recognition are performed on the preprocessed handwritten sign image to obtain text recognition results. Finally, based on a large language model, semantic parsing is performed on the text recognition results, and structured data is output. Specifically, this method offers the following advantages:

[0037] 1) This invention achieves a significantly higher recognition accuracy than existing technologies by integrating the PP-OCRv4 model and the Qwen-7B large language model. In complex outdoor environments, facing challenges such as handwritten character deformation, glare, and dirt, this invention can stably and efficiently perform text recognition, effectively overcoming the limitations of traditional OCR technology in complex scenarios. Furthermore, the end-to-end recognition system constructed by this invention is lightweight, does not rely on high-performance GPUs / NPUs, and can operate efficiently in edge computing scenarios, significantly reducing deployment costs and improving the system's practicality and scalability.

[0038] 2) Furthermore, this invention possesses high flexibility and adaptability, quickly adapting to new sign styles and the writing habits of different exploration personnel, ensuring stable application in diverse scenarios. For specialized terminology and special symbols in the field of geological exploration, this invention utilizes the semantic parsing capabilities of a large language model to further optimize recognition accuracy, eliminating the need for extensive fine-tuning with labeled data, significantly reducing implementation costs and technical barriers. Simultaneously, this invention achieves full automation from image acquisition to structured data output, reducing manual intervention, improving work efficiency, and providing an efficient and reliable solution for the digital acquisition of handwritten geological exploration signs. Attached Figure Description

[0039] Figure 1 A flowchart illustrating an OCR recognition method for handwritten geological exploration signs, provided as an embodiment of the present invention;

[0040] Figure 2This is a flowchart illustrating the specific implementation of an OCR recognition method for handwritten geological exploration signs, as provided in one embodiment of the present invention.

[0041] Figure 3 The large language model prompt word 1 is provided in an OCR recognition method for handwritten geological exploration signs according to an embodiment of the present invention;

[0042] Figure 4 In an embodiment of the present invention, a method for OCR recognition of handwritten geological exploration signs provides a large language model prompt word 2;

[0043] Figure 5 This invention provides an example of a handwritten sign in an OCR recognition method for geological exploration handwritten signage, as provided in one embodiment of the present invention.

[0044] Figure 6 The results of recognizing handwritten notice boards using the recognition model (Tencent Cloud Handwriting OCR);

[0045] Figure 7 The results of handwritten sign recognition using document image analysis technology (NetEase YiDun) in the field of pattern recognition;

[0046] Figure 8 The results of handwritten sign recognition using DeepSeek (a multimodal large model);

[0047] Figure 9 This is a handwritten sign recognition result provided by an OCR recognition method for geological exploration handwritten signage according to an embodiment of the present invention;

[0048] Figure 10 This invention provides an embodiment of a geological exploration handwritten sign OCR recognition method, which includes an intelligent drilling system management page with handwritten license plate OCR recognition functionality. Detailed Implementation

[0049] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0050] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0051] The first embodiment of this invention provides an OCR recognition method for handwritten geological exploration signs. It combines the PP-OCRv4 model based on the PaddleOCR framework with the Qwen-7B large language model based on the Transformer architecture to achieve OCR recognition of handwritten geological exploration signs. This method can improve recognition accuracy in complex scenarios and achieve full-process automation. The following describes the method in conjunction with... Figure 1 and Figure 2 Please provide a detailed explanation.

[0052] like Figure 1 As shown, in step S100, the handwritten sign image is acquired and preprocessed to obtain the preprocessed handwritten sign image.

[0053] Specifically, image preprocessing is a crucial step in the OCR recognition process. Its purpose is to optimize and transform the acquired image to improve the accuracy of subsequent text detection and recognition. In geological exploration scenarios, metal or plastic signs may be affected by factors such as reflection, dirt, and corrosion, making image preprocessing particularly important. First, grayscale processing is performed, converting the color image to a grayscale image to reduce computational load and preserve text information. Next, Gaussian filtering or median filtering is used to remove noise from the image, and histogram equalization or adaptive contrast enhancement algorithms are used to improve image contrast. Finally, the grayscale image is converted to a binary image for subsequent text detection. The grayscale conversion formula is:

[0054] I gray =0.299R + 0.587G + 0.114B

[0055] Among them, I gray R represents the pixel value of the grayscale image, while R, G, and B represent the pixel values ​​of the red, green, and blue channels of the original color image, respectively.

[0056] like Figure 1 As shown, in step S200, based on the OCR recognition model, text detection and text recognition are performed on the preprocessed handwritten sign image to obtain the text recognition result.

[0057] Text detection:

[0058] PP-OCRv4 employs a text detection algorithm based on DBNet (Differentiable Binarization Network). Its core idea is to achieve accurate segmentation of text regions through a differentiable binarization network. The loss function of DBNet is defined as:

[0059] LDB=α·Lbce(P,G)+β·Ldice(P,G)

[0060] Where P represents the predicted probability map, G represents the ground truth label, Lbce represents the binary cross-entropy loss, Ldice represents the Dice coefficient loss, and α and β are balancing parameters. The binary cross-entropy loss measures the difference between the predicted and ground truth values, while the Dice coefficient loss measures the degree of overlap between the predicted and ground truth regions. By combining these two loss functions, DBNet can accurately segment text regions.

[0061] Text recognition:

[0062] PP-OCRv4 uses a character recognition algorithm based on CRNN (Convolutional Recurrent Neural Network). The CRNN structure includes convolutional layers, recurrent layers, and fully connected layers. Convolutional layers extract image features, recurrent layers encode the feature sequences, and fully connected layers map the encoded features to character categories. The CRNN uses CTC (Connectionist Temporal Classification) as its loss function to handle sequence recognition problems. In this way, PP-OCRv4 can accurately convert text within a text region into a text string.

[0063] like Figure 1 As shown, in step S300, based on the large language model, the text recognition results are semantically parsed and structured data is output.

[0064] This embodiment uses the Qwen-7B large language model based on the Transformer architecture for semantic parsing. The core of the Transformer architecture is the self-attention mechanism, which can capture the dependencies between different positions in a sequence. The calculation formula for the self-attention mechanism is as follows:

[0065]

[0066] Where Q represents the query vector, K represents the key vector, V represents the value vector, and d kThis represents the dimension of the key vector. The encoder part of the Transformer consists of multiple layers, each composed of a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism can capture various dependencies between different positions in the sequence, while the feedforward neural network is used to further process the features.

[0067] Prompt word engineering:

[0068] Cue word engineering is a key technology for using large language models to perform specific tasks. By designing appropriate cue words, the model can be guided to generate output that meets the task requirements. In the semantic parsing task of handwritten geological exploration signs, this embodiment designs cue words such as... Figure 3 and Figure 4 The prompts shown enable the Qwen-7B model to accurately understand task requirements and generate structured data that conforms to the required format and domain knowledge.

[0069] Integration Solution:

[0070] The fusion process of PP-OCRv4 and Qwen-7B is as follows: First, there is image input, followed by image preprocessing, which involves grayscale conversion, noise reduction, and contrast enhancement to improve image quality. Next, text detection is performed, using the PP-OCRv4 text detection module to locate text regions and extract their coordinates. After text detection, the text recognition stage begins, where the PP-OCRv4 text recognition module converts the text in the regions into text strings. Following this is semantic parsing, where the recognized text strings are input into the Qwen-7B model, and structured data is generated by combining prompt words. The final step is output, where the obtained structured data is output in JSON or other formats for subsequent system use.

[0071] Comparison results:

[0072] The specific comparison results are as follows: Figure 5-9 As shown, where Figure 5 Original image of the handwritten notice board. Figure 6 To use only the results of the recognition model, Figure 7 The results of document image analysis techniques using the field of pattern recognition, Figure 8 The results are from identification using only a multimodal large model. Figure 9 The results demonstrate the effectiveness of using the technical solution described in this embodiment. Comparative results show that the present invention, through the fusion of PP-OCRv4 and Qwen-7B, significantly improves the recognition accuracy of handwritten signboards while maintaining a high processing speed. The structured data output meets the practical needs of the geological exploration field, providing a reliable foundation for subsequent data processing and analysis.

[0073] System Integration:

[0074] After implementing the technical solution of this embodiment, this embodiment embeds it into the management of the intelligent drilling system. Figure 10 This is a management page for intelligent drilling systems that includes OCR recognition functionality for handwritten license plates.

[0075] This invention integrates the PP-OCRv4 model from the PaddleOCR framework with the Qwen-7B large language model based on the Transformer architecture. This combination overcomes the limitations of traditional OCR technology, organically combining advanced computer vision technology and powerful natural language processing capabilities. The PP-OCRv4 model focuses on the detection and recognition of handwritten text, effectively handling various interference factors in complex field environments; while the Qwen-7B large language model, through semantic understanding and contextual association capabilities, compensates for the shortcomings of traditional OCR in extracting structured information. The integration of the two not only improves recognition accuracy but also achieves full automation from image acquisition to structured data output, providing a new technical path for the digital acquisition of handwritten geological exploration signs.

[0076] In addition to the fusion scheme of the PP-OCRv4 model and the Qwen-7B large language model used in this embodiment, the following alternatives can be considered: using other large language models (such as the GPT series, Wenxin Yiyan, etc.) instead of Qwen-7B. These large language models have powerful capabilities in natural language processing, but require further optimization and adaptation for professional terms and special symbols in the field of geological exploration. Furthermore, a combination scheme using lightweight OCR models (such as EasyOCR) with other lightweight language models (such as ChatGLM) can be explored to further reduce hardware dependence and deployment costs. However, these alternatives may not be as good as the fusion scheme of this invention in terms of recognition accuracy and professional domain optimization, especially when dealing with complex scenes and non-standardized handwritten text, where the technical advantages of this invention will be more obvious.

[0077] Corresponding to the OCR recognition method for handwritten geological exploration signs disclosed above, this invention also discloses an OCR recognition system for handwritten geological exploration signs, which specifically includes:

[0078] The image preprocessing module is used to acquire the handwritten sign image and perform image preprocessing to obtain the preprocessed handwritten sign image;

[0079] The OCR recognition module is used to perform text detection and text recognition on the pre-processed handwritten sign image based on the OCR recognition model, and obtain the text recognition result.

[0080] The large language model processing module is used to perform semantic parsing on the text recognition results based on the large language model and output structured data.

[0081] It should be noted that for a detailed description of the OCR recognition system for handwritten geological exploration signs provided in the embodiments of the present invention, please refer to the relevant description of the OCR recognition method for handwritten geological exploration signs provided in the embodiments of the present invention, which will not be repeated here.

[0082] In addition, embodiments of the present invention also provide an electronic device, the device comprising: a processor and a memory; the memory being used to store one or more program instructions; the processor being used to execute one or more program instructions to perform the steps of the OCR recognition method for a handwritten geological exploration sign as described in any of the preceding embodiments.

[0083] It should be noted that for a detailed description of the electronic device provided in the embodiments of the present invention, please refer to the relevant description of the OCR recognition method for a handwritten geological exploration sign provided in the embodiments of the present invention, which will not be repeated here.

[0084] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the OCR recognition method for handwritten geological exploration signs as described in any of the preceding embodiments.

[0085] It should be noted that for a detailed description of a computer-readable storage medium provided in the embodiments of the present invention, please refer to the relevant description of an OCR recognition method for a handwritten geological exploration sign provided in the embodiments of the present invention, which will not be repeated here.

[0086] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0087] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A method for OCR recognition of handwritten geological exploration signs, characterized in that, The method includes: Acquire the image of the handwritten notice board and perform image preprocessing to obtain the preprocessed image of the handwritten notice board; Based on the OCR recognition model, text detection and text recognition are performed on the preprocessed handwritten sign image to obtain the text recognition result. Based on a large language model, semantic parsing is performed on the text recognition results, and structured data is output.

2. The OCR recognition method for handwritten geological exploration signs as described in claim 1, characterized in that, Acquire the image of the handwritten notice board and perform image preprocessing to obtain the preprocessed handwritten notice board image, specifically including: The image is converted to grayscale by performing grayscale processing. Then, Gaussian filtering or median filtering is used to remove noise from the image. Histogram equalization or adaptive contrast enhancement algorithms are used to improve the contrast of the image. Finally, the grayscale image is converted to a binary image for subsequent text detection.

3. The OCR recognition method for handwritten geological exploration signs as described in claim 1, characterized in that, Based on the OCR recognition model, text detection and recognition are performed on the preprocessed handwritten sign image to obtain the text recognition results, specifically including: Use the text detection module of PP-OCRv4 to locate the text region and extract the coordinates of the text region; Then, the text recognition module of PP-OCRv4 is used to convert the text in the text area into a text string.

4. The OCR recognition method for handwritten geological exploration signs as described in claim 3, characterized in that, The text detection module of PP-OCRv4 is used to locate text regions and extract the coordinates of the text regions, specifically including: PP-OCRv4 employs a text detection algorithm based on the DBNet network, which achieves accurate segmentation of text regions through a differentiable binarization network.

5. The OCR recognition method for handwritten geological exploration signs as described in claim 4, characterized in that, The text detection module of PP-OCRv4 is used to locate text regions and extract the coordinates of the text regions, specifically including: DBNet's loss function is defined as: LDB=α·Lbce(P,G)+β·Ldice(P,G) Where P represents the predicted probability map, G represents the true label, Lbce represents the binary cross-entropy loss, Ldice represents the Dice coefficient loss, and α and β are balance parameters. The binary cross-entropy loss is used to measure the difference between the predicted value and the true value, while the Dice coefficient loss is used to measure the degree of overlap between the predicted region and the true region. By combining the two loss functions, DBNet can accurately segment text regions.

6. The OCR recognition method for handwritten geological exploration signs as described in claim 3, characterized in that, The text recognition module of PP-OCRv4 is used to convert the text in the text region into a text string, specifically including: PP-OCRv4 uses a text recognition algorithm based on CRNN network.

7. The OCR recognition method for handwritten geological exploration signs as described in claim 6, characterized in that, The text recognition module of PP-OCRv4 is used to convert the text in the text region into a text string, specifically including: The structure of the CRNN network includes convolutional layers, recurrent layers, and fully connected layers. Convolutional layers are used to extract image features, recurrent layers are used to encode feature sequences, and fully connected layers are used to map the encoded features to character categories. The loss function of CRNN is CTC, which is used to handle sequence recognition problems.

8. The OCR recognition method for handwritten geological exploration signs as described in claim 1, characterized in that, Based on a large language model, semantic parsing is performed on the text recognition results, and structured data is output, specifically including: The prompt words are constructed, and the Qwen-7B large model based on the Transformer architecture is used for semantic parsing. The recognized text strings are input into the Qwen-7B model, and structured data is generated by combining the prompt words.

9. A geological exploration handwritten sign OCR recognition system, characterized in that, The system includes: The image preprocessing module is used to acquire the handwritten sign image and perform image preprocessing to obtain the preprocessed handwritten sign image; The OCR recognition module is used to perform text detection and text recognition on the pre-processed handwritten sign image based on the OCR recognition model, and obtain the text recognition result. The large language model processing module is used to perform semantic parsing on the text recognition results based on the large language model and output structured data.

10. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is configured to run one or more program instructions to perform the steps of the OCR recognition method for handwritten geological exploration signs as described in any one of claims 1 to 8.

Images