Multi-source heterogeneous geological data large-scale acquisition method and system

Abstract

The invention discloses a multi-source heterogeneous geological data large-scale acquisition method and system. According to the method, a GIS vector, a CAD drawing, a reconnaissance database and image data are marked to generate a data source portrait; the acquisition terminal acquires paper data, a profile map, a borehole histogram and the like to generate an image acquisition packet containing time and position; calling a data adapter to extract and transcode into a unified intermediate representation; inputting the preprocessed data source portrait and the image acquisition packet into a multi-branch convolutional neural network, executing layout segmentation, text detection and recognition and key information extraction, and outputting a field-level structured result and confidence; and aligning the field and the intermediate representation based on the data source portrait and carrying out semantic mapping to generate a standardized data packet and storing the standardized data packet. Secondary Gaussian filtering is adopted in preprocessing, and tau is determined by gray variance mean value grading; when the confidence of the key field is insufficient or conflicts with the portrait constraint, tau re-filtering reasoning preferential output is up-regulated to improve the collection efficiency and quality.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, specifically to a method and system for large-scale acquisition of multi-source heterogeneous geological data. Background Technology

[0002] Currently, with the continuous advancement of urban construction, major engineering projects, and geological disaster prevention, the scale and types of geological data are rapidly increasing, exhibiting typical characteristics of multi-source heterogeneity. On the one hand, digitized data is widely distributed across different units and systems, commonly including GIS vector data, CAD drawings, and exploration databases; on the other hand, a large amount of historical and field data is still preserved in the form of paper archives, scanned copies, and photographs, such as cross-sections, borehole columnar sections, tables, and site signs. These data differ significantly in storage media, organization methods, field systems, coordinate references, coding standards, and update frequencies, and lack unified standards, making data aggregation, sharing, and reuse extremely difficult.

[0003] In existing technologies, the aggregation of multi-source geological data typically employs manual processing and semi-automatic import methods: for GIS and database data, different software tools or scripts are used for extraction, transformation, and loading; for CAD drawings, key attributes are extracted primarily through layer rules or manual interpretation; for paper or image materials, manual input is mainly done after scanning, or text is converted to text using general OCR and then manually processed. Due to the lack of a unified data source labeling and profiling mechanism, the field semantics, unit systems, and coding systems of data from different sources often require repeated sorting and manual configuration, making it difficult to form reusable mapping rules; at the same time, the extraction links of different data sources are highly coupled, and custom development is required when adding new data types, making it difficult to support large-scale parallel acquisition and continuous incremental updates. For image data, affected by factors such as shooting angle, lighting, occlusion, blur, and layout complexity, the accuracy of general OCR fluctuates greatly in terms of table structure restoration, field location, and feature recognition, and most solutions lack field-level confidence output and quality handling strategies, resulting in erroneous data easily entering the database and high subsequent error correction costs.

[0004] Furthermore, heterogeneous data commonly suffers from issues such as "homophones, homonyms, missing fields, inconsistent units, and inconsistent coordinate bases." Without a structured field alignment and semantic mapping mechanism, it is difficult to effectively integrate image analysis results with GIS / database extraction results, ultimately hindering the formation of standardized data packages suitable for unified management and sharing services. Existing platforms and tools often focus on processing single data types or only provide coarse-grained import / export capabilities, lacking an end-to-end closed loop for multi-source heterogeneous scenarios. This includes data source profiling-driven acquisition and orchestration, adapter-based extraction and transcoding, deep learning analysis of image data, field-level confidence control, and field alignment and semantic mapping with intermediate representations. Therefore, a technical solution is urgently needed to achieve large-scale acquisition, unified transcoding, intelligent structuring, and standardized data entry for multi-source heterogeneous geological data, improving acquisition efficiency, reducing labor costs, and ensuring the quality of data entered into the database. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention discloses a method and system for large-scale acquisition of multi-source heterogeneous geological data. The method generates data source profiles from GIS vectors, CAD maps, exploration databases, and image data annotations; the acquisition terminal acquires paper documents, cross-sectional views, borehole columnar views, etc., generating image acquisition packages containing time and location information; a data adapter is called to extract and transcode these into a unified intermediate representation; the preprocessed data source profiles and image acquisition packages are input into a multi-branch convolutional neural network to perform page segmentation, text detection and recognition, and key information extraction, outputting field-level structured results and confidence scores; based on the data source profiles, fields are aligned with the intermediate representations and semantically mapped to generate standardized data packages for storage. Preprocessing employs a quadratic Gaussian filter, where τ is determined by the grayscale variance mean; when the confidence score of a key field is insufficient or conflicts with profile constraints, τ is increased for re-filtering and inference to optimize output, improving acquisition efficiency and quality; it is implemented as follows: A method for large-scale acquisition of multi-source heterogeneous geological data, comprising the following steps:

[0006] S1: Label geological data sources and generate data source profiles; geological data sources include GIS vector data, CAD drawings, exploration databases, and image data;

[0007] S2: Acquire image data by using an image acquisition terminal to photograph paper documents, site signs, cross-sectional views, borehole columnar views, and table lists, and generate an image acquisition package containing the acquisition time, acquisition location, and acquisition device identification.

[0008] S3: Adapter-based extraction and transcoding, calls the data adapter that matches the data source type to perform extraction, and converts the extraction result into a unified intermediate representation;

[0009] S4: Input the preprocessed data source profile and image acquisition package into the trained first convolutional neural network model, output field-level structured results and field-level confidence scores; and perform field alignment and semantic mapping between the field-level structured results and the intermediate representation based on the data source profile to generate standardized data packets;

[0010] S5: Write the standardized data packet into the target geological database to complete the standardized data entry.

[0011] Preferably, step S1 includes: labeling each geological data source with its data source identifier, data type, data format, spatial reference information, coverage, update frequency, and permission level, and labeling the layer with its field semantic category, unit, and encoding system; generating a data source profile based on the above labeling, wherein the data source profile includes at least a field mapping dictionary and constraint parameters for subsequent data collection and arrangement, field alignment, and semantic mapping.

[0012] Preferably, step S3, the adapter-based extraction and transcoding, includes: establishing a data adapter registry and defining a unified adapter interface; selecting a data adapter that matches the data source type based on the data source profile; having the data adapter complete data connection, reading, and extraction according to the unified adapter interface; and performing format transcoding and field normalization processing on the extraction results, encapsulating them into a unified intermediate representation, wherein the intermediate representation includes source identifier, record identifier, record structure, attribute key-value pairs, spatial reference information, and metadata.

[0013] Preferably, the trained first convolutional neural network model is a conditional multi-branch convolutional network, comprising at least: a convolutional feature extraction backbone network, a feature pyramid fusion layer, a layout region segmentation branch, a text detection branch, a text recognition branch, and a key information extraction branch; wherein, the convolutional feature extraction backbone network generates multi-scale feature maps for the images in the image acquisition package, the data source profile is embedded and encoded to generate conditional vectors and injected into the feature pyramid fusion layer and / or the key information extraction branch; the key information extraction branch generates field-level structured results based on the character sequences output by the text recognition branch, the field-level structured results including at least preset database field names, field values, and field location information; the field-level confidence score is a score of the correctness of the field values, obtained by fusing the detection score of the text detection branch, the recognition score of the text recognition branch, and the extraction score of the key information extraction branch.

[0014] Preferably, the data source profile and image acquisition package are preprocessed, including: standardizing and embedding discrete labels of data type, data format, coordinate system, unit system and permission level to generate condition vectors; and denoising and unifying the field names, aliases and semantic categories in the field mapping dictionary to form field constraint parameters. The image acquisition package is preprocessed, including: grouping images into batches according to acquisition time and acquisition location, and performing Gaussian filtering for noise reduction and histogram equalization on the images.

[0015] This application also provides a large-scale acquisition system for multi-source heterogeneous geological data, including

[0016] The data source management module annotates geological data sources and generates data source profiles; geological data sources include GIS vector data, CAD drawings, exploration databases, and image data;

[0017] The image acquisition module acquires image data and uses the image acquisition terminal to photograph paper documents, on-site signs, cross-sectional views, borehole columnar diagrams, and table lists, generating an image acquisition package containing the acquisition time, acquisition location, and acquisition device identification.

[0018] The extraction and transcoding module adapts extraction and transcoding, calls a data adapter that matches the data source type to perform extraction, and converts the extraction result into a unified intermediate representation.

[0019] The neural network image parsing module inputs the preprocessed data source profile and image acquisition package into the trained first convolutional neural network model, outputs field-level structured results and field-level confidence scores; and performs field alignment and semantic mapping between the field-level structured results and the intermediate representation based on the data source profile to generate standardized data packets.

[0020] The standardized data entry module writes standardized data packets into the target geological database, completing the standardized data entry process.

[0021] Preferably, the data source management module includes: labeling each geological data source with its data source identifier, data type, data format, spatial reference information, coverage, update frequency, and permission level; and labeling the layers with their field semantic categories, units, and encoding systems; generating a data source profile based on the above labels, wherein the data source profile includes at least a field mapping dictionary and constraint parameters for subsequent data collection and arrangement, field alignment, and semantic mapping.

[0022] Preferably, the extraction and transcoding module includes: establishing a data adapter registry and defining a unified adapter interface; selecting a data adapter that matches the data source type based on the data source profile; the data adapter completing data connection, reading, and extraction according to the unified adapter interface; and performing format transcoding and field normalization processing on the extraction results, encapsulating them into a unified intermediate representation, wherein the intermediate representation includes source identifier, record identifier, record structure, attribute key-value pairs, spatial reference information, and metadata.

[0023] Preferably, the trained first convolutional neural network model is a conditional multi-branch convolutional network, comprising at least: a convolutional feature extraction backbone network, a feature pyramid fusion layer, a layout region segmentation branch, a text detection branch, a text recognition branch, and a key information extraction branch; wherein, the convolutional feature extraction backbone network generates multi-scale feature maps for the images in the image acquisition package, the data source profile is embedded and encoded to generate conditional vectors and injected into the feature pyramid fusion layer and / or the key information extraction branch; the key information extraction branch generates field-level structured results based on the character sequences output by the text recognition branch, the field-level structured results including at least preset database field names, field values, and field location information; the field-level confidence score is a score of the correctness of the field values, obtained by fusing the detection score of the text detection branch, the recognition score of the text recognition branch, and the extraction score of the key information extraction branch.

[0024] Preferably, the data source profile and image acquisition package are preprocessed, including: standardizing and embedding discrete labels of data type, data format, coordinate system, unit system and permission level to generate condition vectors; and denoising and unifying the field names, aliases and semantic categories in the field mapping dictionary to form field constraint parameters. The image acquisition package is preprocessed, including: grouping images into batches according to acquisition time and acquisition location, and performing Gaussian filtering for noise reduction and histogram equalization on the images.

[0025] Image preprocessing of the image acquisition package includes secondary filtering, and the filtering function... for:

[0026]

[0027]

[0028] in, The baseline scale factor represents the basic smooth scale under nominal imaging quality, with different preset values ​​depending on the data source profile (such as data type / template); The standard deviation of the Gaussian kernel for the current second-order filter. The scale level is obtained by mapping the mean variance of pixel grayscale values ​​in the borehole histogram and is adaptively adjusted by field-level confidence closure; x is the horizontal coordinate of the pixel in the convolution kernel window, y is the vertical coordinate of the pixel in the convolution kernel window, and m is the side length of the convolution kernel in pixels.

[0029] Within the same image acquisition package, for the "breach shaft histogram" set, the grayscale variance of each image is calculated, and the mean is taken. Based on the mean, a hierarchical mapping is performed to obtain the initial scale level. When performing secondary filtering on the image acquisition package, the scale level τ is obtained by mapping the mean of the grayscale variance of the borehole shaft histogram pixels, and the filtering parameters are determined. When the confidence of the key field output by the first convolutional neural network model is lower than the threshold or inconsistent with the field constraint parameters of the data source profile, τ is increased and the filtering and inference are repeated. The result with higher confidence and satisfying the field constraints is selected as the field-level structured output.

[0030] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0031] (1) This invention targets multi-source heterogeneous geological data such as GIS vectors, CAD maps, exploration databases, and image data. First, it generates a data source profile through annotation, solidifying key constraints such as data type, format, coordinate reference, field semantics, units, and coding system, so that the collected objects can be arranged and managed by a unified profile. Then, it adopts an adapter-based extraction and transcoding mechanism to extract and encapsulate data from different sources into an intermediate representation through a unified interface, significantly reducing the dependence on a single toolchain and manual scripts, and improving the access efficiency and scalability of new data sources. Based on the intermediate representation and profile constraints, field alignment and semantic mapping can achieve cross-system standardization, reduce the errors caused by homonyms, homonyms, and inconsistent units, and form a reusable standardized data package to support large-scale parallel acquisition, incremental updates, and traceable management, thereby improving the efficiency of geological data aggregation and data consistency.

[0032] (2) For image data such as paper documents, cross-sectional views, borehole column charts, and table lists, this invention introduces a pre-trained conditional multi-branch convolutional neural network model to complete page segmentation, text detection and recognition, and key information extraction. It directly outputs field-level structured results (field names, field values, and location information) and field-level confidence scores, achieving end-to-end automation "from image to structured field". Compared with solutions that rely solely on general OCR, this invention can maintain more stable extraction quality in scenarios with complex page layouts, varied table structures, and mixed symbols and annotations. It also provides interpretable quality quantification through field-level confidence scores, which can be used for the triage decision-making of automatic database entry, manual review, and re-extraction. Furthermore, the field results and intermediate representations are aligned and semantically mapped under the constraints of the data source profile, so that the image parsing results and GIS / database extraction results can be integrated and consistent, significantly reducing the cost of secondary manual processing and improving the accuracy and usability of key fields.

[0033] (3) This invention introduces a secondary filtering mechanism in the image acquisition package preprocessing, using the Gaussian weight function G(x,y) to generate the convolution kernel, and using... Adaptively determine the smoothing scale, where The baseline scaling factor can be preset according to the data source profile (data type / template). τ is obtained by hierarchical mapping of the mean variance of pixel grayscale values ​​in the borehole histogram and is adaptively adjusted by a field-level confidence loop. When the confidence of a key field output by the first convolutional neural network model is lower than the threshold or inconsistent with the constraints of the profile field, the system automatically increases τ and re-filters and infers, selecting the result with higher confidence and satisfying the constraints as the output. This mechanism of "variance hierarchical + profile prior + confidence loop" enables the preprocessing parameters to adapt to data quality, suppressing noise and illumination disturbances while avoiding detail loss due to excessive smoothing, thereby improving the robustness of structured extraction and the overall reliability of data storage in low-quality image scenarios. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method for large-scale acquisition of multi-source heterogeneous geological data according to the present invention;

[0035] Figure 2 This is a structural diagram of a large-scale acquisition system for multi-source heterogeneous geological data according to the present invention. Detailed Implementation

[0036] Those skilled in the art will understand that, in order to make the above-mentioned objects, features, and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This application illustrates a method for large-scale acquisition of multi-source heterogeneous geological data, including...

[0037] S1: Label geological data sources and generate data source profiles; geological data sources include GIS vector data, CAD drawings, exploration databases, and image data;

[0038] In some embodiments, specifically in this embodiment, the system needs to centrally collect and standardize the geological data along a certain city's rail transit project and store it in a database. The geological data sources to be collected include: GIS vector data source A: Shapefile format, containing "borehole location layer" and "stratum distribution surface layer"; CAD drawing data source B: DWG format, containing "exploration profile" and "site layout plan"; exploration database data source C: relational database (PostgreSQL), containing "borehole information table," "stratification information table," and "sampling / test table"; and image data source D: scanned copies of paper borehole columnar sections, table lists, and profiles, along with on-site photographs.

[0039] The system performs two types of annotation for each data source: "data source-level annotation" and "field / layer-level annotation." Data source-level annotation generates a data source identifier (source_id) for each data source A through D, and annotates the following fields: Data type: GIS vector / CAD / survey database / image; Data format: SHP, DWG, JPG, PDF, etc.; Spatial reference: Coordinate system (e.g., CGCS2000), projection parameters, datum; For image data without coordinates, annotate "Coordinates Unknown"; Coverage: Annotate the coverage area by administrative region or engineering mileage segment (e.g., K12+500~K18+000); Update frequency: One-time import / Weekly update / Monthly update; Access level: Public / Internal / Confidential; Initial quality rating: A / B / C level is given based on indicators such as missing rate, coordinate validity, and image clarity.

[0040] GIS vector data source A: Borehole point layer: fields “BH_ID”, “X”, “Y”, “Z”, “hole depth”, “project section”, etc.; annotation field semantic category (borehole number / plane coordinates / elevation / depth / project attribute), unit (m), value range (depth>0), missing rate; stratigraphic distribution surface layer: fields “STR_CODE”, “STR_NAME”, “age”, “lithology”, etc.; annotation coding system (such as stratigraphic coding table version V1.2).

[0041] CAD drawing data source B: Label layer number and layer semantics (e.g., "TEXT_" is the text annotation layer, "STR_" is the stratigraphic boundary layer, "BH_*" is the borehole annotation layer); Label line type / color and semantic rules (e.g., red dashed lines represent speculative boundaries, black solid lines represent measured boundaries); Label legend items and symbol meanings (e.g., symbols such as "sand", "clay", "pebbles" etc.).

[0042] Exploration database data source C: Borehole information table: labeled with primary key BH_ID, foreign key project segment ID; units and value ranges for fields such as "borehole elevation", "borehole depth", and "drilling date"; Strata information table: labeled with fields such as "stratum sequence", "bottom depth", "lithological description", and "stratigraphic code"; and labeled with the relationship with the borehole table; Test table: labeled with test type, indicator units and standards (such as water content %, standard penetration test blow count, etc.).

[0043] Image data source D: Label the image source category (borehole columnar section / table list / section / site sign); label the resolution and imaging method (scanning / photographing); label the layout type (table type / text and image mixed type / column type); label the "target field set" (e.g., borehole number, borehole depth, bottom depth, lithology, stratigraphic code, etc.) for field constraints in subsequent neural network parsing.

[0044] The system automatically generates a field mapping dictionary (mapping_dict) based on the above annotations to resolve the issue of inconsistent naming of "homophones / hypothetical names". For example, the GIS field "BH_ID", the database field "borehole_id", and the image OCR field "borehole number" are uniformly mapped to the standard field "borehole_id"; "borehole depth", "final borehole depth", and "depth_total" are uniformly mapped to "total_depth"; and field constraint parameters (constraint_params) are generated, including: unit unification rules (cm→m, mm→m); coding standardization rules (stratigraphic coding table version, lithological enumeration table); coordinate validity rules (coordinate range, projection consistency); and a list of key fields and their priorities (such as borehole_id, total_depth, and strata_bottom_depth as key fields).

[0045] The data source profile is structured and stored on disk. Ultimately, the system encapsulates the aforementioned "data source level annotations + field / layer level annotations + mapping dictionary + constraint parameters" into a data source profile record and stores it (e.g., JSON or table structure). Fields include:

[0046] source_id, source_type, format, srid / coordinate system, coverage, update_policy, acl_level, quality_grade;

[0047] schema_profile (list of layers / tables / fields and semantic labels, units, missing rate, range of values);

[0048] mapping_dict(standard field name ↔ set of aliases);

[0049] constraint_params (unit / encoding / coordinate / key field threshold rules).

[0050] In S3, the system selects the corresponding adapter (SHP adapter, DWG adapter, DB adapter, image adapter) from the adapter registry based on source_type and format. In S4, the "target field set, mapping dictionary and constraint parameters" in the data source profile are used as conditional information to participate in field alignment and semantic mapping, and are used to detect field conflicts and handle confidence, thereby generating standardized data packets and storing them in the database.

[0051] S2: Acquire image data by using an image acquisition terminal to photograph paper documents, site signs, cross-sectional views, borehole columnar views, and table lists, and generate an image acquisition package containing the acquisition time, acquisition location, and acquisition device identification.

[0052] In some embodiments, field personnel collect images of paper and on-site data at a tunnel engineering survey site and data room. The image acquisition terminal is a mobile terminal (such as a mobile phone / tablet) or a dedicated acquisition device equipped with a camera, positioning, and network communication capabilities. The terminal has a built-in acquisition application and is bound to a unique device identifier (device_id). Before starting acquisition, the user selects the acquisition task (project_id) and data type (doc_type) in the acquisition application. Data types include: paper documents, on-site signage, cross-section diagrams, borehole columnar diagrams, and tabular lists. The acquisition application loads the target field set for the task (e.g., borehole number, borehole depth, bottom depth, stratigraphic code, etc.) and layout type prompts (column graph / table / mixed text and image layout) from the data source profile as shooting guidance information.

[0053] For shooting and quality control, field personnel photograph the target data. The acquisition application displays the viewfinder and boundary detection results in real time and performs a rapid quality assessment after shooting: Sharpness detection: determines whether it is blurry based on image gradient / Laplacian variance; Perspective and tilt detection: determines the four corner deviations based on edge line fitting; Brightness and reflection detection: determines exposure and reflection based on histogram and the proportion of bright areas; When the quality indicators do not meet the threshold, the acquisition application prompts for reshooting or automatically activates the fill light / image stabilization mode; For borehole bar charts and table lists, the application prioritizes "document mode" to enhance edge and text contrast.

[0054] The data acquisition time, location, and device identifier are written to the data acquisition application, which generates metadata upon completion of each capture: Capture time (capture_time): obtained from the terminal system clock and written with a UTC timestamp and local time zone information; Capture location (capture_location): obtained through GNSS / base station / WiFi fusion positioning to obtain latitude and longitude and positioning accuracy; when indoor positioning is not possible, a combination of "manually selected location + most recent valid positioning" is used for recording; Capture device identifier (capture_device_id): generated from the terminal hardware ID or the device certificate ID assigned by the application; auxiliary information such as the data acquisition personnel ID (user_id), document type (doc_type), and project segment / mileage range (segment_id) are also recorded.

[0055] Image acquisition package encapsulation and naming rules: The acquisition application encapsulates multiple images and their metadata under the same acquisition task into an image acquisition package image_pkg, and generates a unique identifier pkg_id for the acquisition package. The image acquisition package contains at least: an image file set images[]: each image contains image_id, file path / binary data, resolution, and compression parameters; a metadata set meta[]: for each image, it writes capture_time, capture_location (latitude and longitude, precision), capture_device_id, doc_type, project_id, and segment_id; verification information: a content hash (such as SHA-256) is generated for each image for deduplication and integrity verification; and an associated index: an index is generated according to doc_type and acquisition order to ensure that subsequent parsing can be grouped and processed according to "bar chart / table / section view". Within a single data collection package, borehole bar charts can be aggregated by "same borehole number or same data batch". If there is a correspondence between the field sign and the paper data (e.g., the sign contains the borehole number), the data collection application writes it into the link_key field under the same pkg_id for subsequent alignment.

[0056] The image acquisition package is uploaded and local disaster recovery is implemented. After the acquisition package is generated, the acquisition application encrypts and uploads the image_pkg to the server. If the network is unavailable, it is first cached locally and marked as pending, and the upload will resume automatically after the network is restored. After the server returns a receipt, the acquisition application updates the acquisition package status to uploaded and records the upload time and the server-side package number. In the subsequent S4, the server performs preprocessing on the image acquisition package (including secondary filter parameter calculation and adaptive adjustment), and inputs the preprocessed image into the convolutional neural network model to output field-level structured results and field-level confidence scores. At the same time, the time, location, and device identifier recorded in the acquisition package are used to trace and mark the structured results, generate standardized data packets, and store them in the database.

[0057] S3: Adapter-based extraction and transcoding, calls the data adapter that matches the data source type to perform extraction, and converts the extraction result into a unified intermediate representation;

[0058] In some embodiments, the system needs to extract and transcode four types of data sources (GIS vectors, CAD drawings, survey databases, and image data) into a unified intermediate representation to support subsequent field alignment and semantic mapping with image parsing results. In step S1, the system generates a data source profile for each data source, which includes at least data type, format, spatial reference, field mapping dictionary, unit, and encoding constraint parameters.

[0059] A data adapter registry and a unified adapter interface are established on the server side. The server-side AdapterRegistry is used to register adapter capabilities and applicable scope. Each adapter implements the unified adapter interface IAdapter, which includes at least: connect(source_profile): establishes a connection or opens a file; probe(): reads metadata (field / layer / table structure, spatial reference, etc.); extract(extract_spec): extracts data according to extraction rules; transform(record): performs format transcoding and field normalization; emit(): outputs unified intermediate representation entries.

[0060] Registry examples: SHPAdapter: suitable for GIS vector graphics such as Shapefile / GeoJSON; DWGAdapter: suitable for CAD drawings such as DWG / DXF; DBAdapter: suitable for survey databases such as PostgreSQL / MySQL; ImageMetaAdapter: suitable for metadata extraction from image acquisition packages (the image binary itself is processed in S2 / S4).

[0061] Based on the data source profile, the system performs adapter matching and selection. It reads the `source_type` and `format` from the data source profile and, combined with constraints such as spatial reference, permission level, and field mapping dictionary, performs matching and selection: if `source_type` = GIS and `format` = SHP, then `SHPAdapter` is selected; if `source_type` = CAD and `format` = DWG, then `DWGAdapter` is selected; if `source_type` = DB, then `DBAdapter` is selected and the connection string and access credentials are read; if `source_type` = IMAGE, then `ImageMetaAdapter` is selected to extract metadata such as collection time, location, and device identifier, and generate intermediate representation entries for subsequent traceability and alignment. This selection process also generates the extraction specification `extract_spec`, such as: target layer / table, incremental time window, field whitelist, and spatial range culling conditions.

[0062] The extraction and transcoding process is as follows: For GIS vector data (SHPAdapter), the SHPAdapter opens the borehole point layer and stratigraphic distribution layer, reads the original spatial reference, and unifies it according to the coordinate system constraints in the data source image (e.g., unifying it to CGCS2000). For each feature, the following is performed: Geometric transcoding: Point / line / polygon geometry is uniformly represented as WKT or GeoJSON; Attribute normalization: Field aliases are normalized to standard field names according to the mapping dictionary (e.g., BH_ID → borehole_id), and units are converted according to the constraint parameters (cm → m); Quality marking: Missing fields, out-of-range coordinates, and other anomalies are recorded.

[0063] The CAD drawing (DWGAdapter) parses the DWG layer structure and filters target layers (such as stratigraphic boundary layers, borehole annotation layers, and text annotation layers) based on the "layer semantic rules" in the data source image. It performs the following operations on the extracted objects: Primitive encoding: converting line segments / polylines / text entities into a unified primitive structure (geometry + style + attributes); Text and annotation extraction: outputting text entity content as attribute key-value pairs while preserving their spatial position; Legend / symbol mapping: converting symbol categories to standard codes (such as lithology code / stratigraphic code) based on the legend dictionary in the image.

[0064] The exploration database (DBAdapter) connects to the database and determines the extraction scope based on the "primary key / foreign key relationship" in the profile: extracting borehole tables, stratigraphic tables, and test tables and performing correlation. For each record, the following is performed: field type normalization: date, numerical, and enumeration data are standardized; unit and encoding standardization: e.g., bottom depth is standardized to meters, and stratigraphic codes are mapped to the standard version; incremental extraction: if the profile update frequency is "daily," then new or changed records are extracted based on the last_update_time field.

[0065] The unified intermediate representation encapsulation, regardless of the data source, ultimately outputs a Unified Intermediate Representation (UIR) entry from the adapter. This entry includes at least: source_id (source identifier), record_id (record / feature identifier), record_type (point / line / polygon / table_row / text_entity, etc.); geometry (optional, WKT / GeoJSON), sr_info (spatial reference information); attrs (attribute key-value pairs, all standard field names), meta (extraction time, version, quality marker, permission level); and link_key (optional, a key used for alignment with image parsing results, such as borehole_id, section ID, mileage range, etc.). The system writes the UIR entries to intermediate storage (such as a message queue / intermediate library) for subsequent S4 field alignment, semantic mapping, and standardized data packet generation based on the data source profile.

[0066] To avoid duplicate entries, the system calculates a content fingerprint (e.g., hash(source_id+record_id+attrs+geometry)) for each UIR entry. If the fingerprint already exists and the version hasn't changed, the write operation is skipped. When the version changes, the new version is recorded, and the prev_hash is retained for traceability. This mechanism is consistent with the capture_time / capture_location / device_id traceability information in the image acquisition package, facilitating subsequent auditing and quality backtracking.

[0067] S4: Input the preprocessed data source profile and image acquisition package into the trained first convolutional neural network model, output field-level structured results and field-level confidence scores; and perform field alignment and semantic mapping between the field-level structured results and the intermediate representation based on the data source profile to generate standardized data packets;

[0068] In some embodiments, the system receives three types of inputs: (1) the data source profile obtained in step S1 (including field mapping dictionary, unit and encoding rules, key field list, etc.); (2) the image acquisition package generated in step S2 (including borehole columnar diagrams, profile diagrams, table lists, etc., as well as acquisition time, acquisition location, acquisition device identifier); and (3) the unified intermediate representation generated in step S3 (including standardized field key-value pairs and spatial reference information extracted from GIS vectors, CAD drawings, and survey databases). The system performs neural network parsing, field alignment, and semantic mapping according to the following process to finally generate standardized data packets.

[0069] The preprocessed data source profile is generated as follows: (1) Field mapping cleaning: The system uniformly organizes the field mapping dictionary in the data source profile and merges synonymous field names into the same standard field name. For example, "borehole number, borehole ID, BH_ID, borehole_id" are uniformly mapped to the standard field "borehole_id"; "final borehole depth, borehole depth, total_depth" are uniformly mapped to the standard field "total_depth". (2) Constraint parameter extraction: The system extracts unit and coding constraint parameters from the profile, including the depth unit is uniformly "meter", the version number used for stratigraphic coding, lithology enumeration table, etc., and extracts the key field set and its priority (such as borehole_id, total_depth, strata_bottom_depth, strata_code as key fields). (3) Profile condition coding: The system encodes discrete information such as data type (borehole columnar section / table list / section), template category, unit system, coding system, etc. into condition information to guide subsequent model reasoning and field selection rules. The above processing yields the "preprocessed data source profile".

[0070] Image Acquisition Package Preprocessing and Quality Closed Loop (1) Basic Preprocessing: The system performs orientation correction, perspective correction and edge cropping on each image in the image acquisition package to ensure that the layout boundary and text direction are standardized. (2) Adaptive Secondary Filtering: For the image set labeled "drill bar chart" in the acquisition package, the system calculates the statistical index of the gray level fluctuation of the image (e.g., the mean of gray level variance) and maps the statistical index to the smoothing intensity level; the higher the smoothing intensity level, the greater the filtering intensity. The system generates a filtering kernel based on the smoothing intensity level and the preset basic scale parameters in the image, and performs a first filtering on the image to suppress noise and illumination disturbance. (3) Confidence Closed Loop Triggering Reprocessing: The system inputs the image after the first filtering into the neural network for initial inference. If the confidence of the key field is lower than the threshold, or if the output field conflicts with the data source image constraints (unit, encoding, value range), the system increases the smoothing intensity level, performs a second filtering on the same image and re-infers; the system selects the result with higher confidence of the key field and meets the image constraints as the final image input and output for structured processing from the two inference results. The above processing yields the "preprocessed image acquisition package".

[0071] The first convolutional neural network model inference and output definition: The system inputs the "preprocessed image acquisition package" and "preprocessed data source image condition information" into the trained first convolutional neural network model for inference. The model completes layout understanding and field extraction in one inference and outputs two types of results: (1) Field-level structured results: The structured content is output in units of fields. Each field contains at least three elements: standard field name, field value, and field location information. The field location information is used to indicate the position of the field value in the image. It can be expressed by page number, region box coordinates, or table row and column index. For example, the output "borehole_id=ZK-12, located in the upper title area of ​​page 1"; the output "total_depth=32.5, located in the 3rd row and 2nd column of the basic information table on page 1". (2) Field-level confidence: A confidence score is output for each field to indicate the credibility of the field value. The confidence score is obtained by the model's internal comprehensive judgment of text detection, text recognition and key information extraction. The higher the value, the more reliable the field. The system will use the confidence level for subsequent conflict resolution, review and diversion, and closed-loop reprocessing.

[0072] Field alignment (interconnection with intermediate representation): The system aligns the field-level structured results output by the neural network with the intermediate representation generated in step S3. The alignment process includes: (1) Unified naming: Based on the preprocessed field mapping dictionary, the image parsing fields and intermediate representation fields are uniformly converted into the same set of standard field names; (2) Association matching: Based on the key field (preferably borehole_id) and auxiliary conditions (collection location, engineering section, mileage range, data batch number, etc.), the image parsing results are associated with the corresponding records in the intermediate representation; if there is a conflict of boreholes with the same name, the matching range is further narrowed by the constraints of the collection location and engineering section; (3) Alignment output: After alignment, a set of fields of "same entity" is formed, for example, the spatial point (from GIS / database) and the hierarchical field (from image bar chart) corresponding to the same borehole are merged into the same record.

[0073] Semantic mapping and normalization: After completing field alignment, the system performs semantic mapping and normalization processing based on the constraint parameters in the data source profile: (1) Unit normalization: The field values ​​are uniformly converted to the units specified in the profile, such as the depth being uniformly "meters"; conversion marks are recorded for fields where unit conversion occurs; (2) Coding normalization: Stratigraphic names, lithological descriptions, symbol categories, etc. are mapped to the standard codes specified in the profile version; (3) Rule verification: The value range and logical consistency verification are performed on key fields, such as "the bottom depth of the layer must be less than or equal to the borehole depth" and "the layer sequence must be increasing"; an exception mark is generated when the condition is not met.

[0074] Conflict resolution and quality handling: When the image parsing field and the intermediate representation field conflict, the system adopts the handling strategy of "image constraint priority + confidence level drive": (1) If the image field has a high confidence level and meets the image constraint, the image parsing value is used first, and the intermediate representation value is kept as an alternative; (2) If the image field has a low confidence level or is inconsistent with the image constraint, the intermediate representation value is used first, and the image field is marked as "to be reviewed"; (3) If neither of them meets the constraint, a "re-sampling / review task" is generated, and the conflicting field name, image positioning information, confidence level and source record identifier are recorded to facilitate manual quick location and verification.

[0075] Standardized data packet generation and output: The system ultimately generates standardized data packets, which include at least the following: (1) Source information: collection packet number, collection time, collection location, collection device identifier, and source data source identifier; (2) Standard field set: field values, units, encoding versions, field location information, and field-level confidence levels organized by standard field names; (3) Quality markers: missing markers, conflict markers, anomaly markers, and review / re-collection markers; (4) Lineage records: field source type (image parsing or intermediate representation), corresponding original record identifier, and processing link version. The standardized data packets are then passed to step S5 to be written into the target geological database, completing the standardized database entry.

[0076] S5: Write the standardized data packet into the target geological database to complete the standardized data entry.

[0077] In this embodiment, the system has generated a standardized data package std_pkg in step S4. The target geological database is a "geological data center database," which adopts an integrated deployment of relational and spatial databases (e.g., including spatial feature tables and attribute business tables), and a standard field system and encoding dictionary table are pre-established. The system writes std_pkg into the target geological database according to the following process, completing the standardized data entry.

[0078] Preparation for data entry and target table location: (1) The system parses the "entity type" and "key fields" in std_pkg to determine whether the data entry object belongs to the drilling class, layering class, test class, map index class or document data class, and selects the corresponding target table set accordingly, such as: the main drilling table Borehole; the layering detail table Strata; the test result table TestResult; the image data index table DocIndex; the spatial point table BoreholeGeom (or the Borehole table contains a geometry field). (2) The system matches the field names to the target table fields according to the mapping dictionary between the field set in std_pkg and the data source profile; fields that cannot be directly matched are written to the extended field table ExtField or written to the extended attribute table in the form of "key-value pairs".

[0079] Transactional Inbound and Idempotent Control (1) The system uses the package number pkg_id in std_pkg as the inbound batch number to start a database transaction; (2) The system constructs an idempotent key (e.g., "borehole_id + data source identifier + version number") for key fields (preferably borehole_id). Before inbound, it checks whether the idempotent key already exists: if it does not exist, it performs an add write; if it already exists and the version has not changed, it skips the write and records the "duplicate package" log; if it already exists but the version has changed, it performs an update write and retains the historical version record (see Part 5).

[0080] The main table and the detailed table are written as follows: (1) Write to the borehole main table Borehole: The system writes the borehole number, borehole elevation, borehole depth, engineering section and other fields in std_pkg into the Borehole table; if std_pkg contains spatial coordinates, the point geometry is written into the spatial field or into the BoreholeGeom table and associated with the Borehole table through borehole_id. (2) Write to the layered detailed table Strata: The system writes the fields such as "layer top depth / layer bottom depth / stratum code / lithological description" into the Strata table layer by layer according to the layer sequence, and associates each layer record with borehole_id; if std_pkg contains field location information and confidence level, it is written into the quality extension column or the quality supplementary table StrataQC for subsequent traceability. (3) Foreign key consistency verification: Before committing the transaction, the system verifies that all records in the Strata table can be found in the Borehole table with the corresponding borehole_id to avoid isolated details.

[0081] Quality marking, review diversion and data entry status write-back: (1) If the quality_flags in std_pkg shows "pending review" or "conflict unresolved", the system will write the corresponding record to the "Pending Review Queue" and record the status field (e.g., status=review_pending) in the main table. At the same time, the system can still choose to enter the field value into the database but mark it as "temporary value / candidate value". (2) If there is a "re-collection" mark in std_pkg, the system will only enter the archive index (DocIndex) and traceability information into the database, and will not write the key business fields into the formal business table. The system will also generate a re-collection task form (including collection location, image positioning information, conflict field name, etc.). (3) The system will write back the data entry result (success / partial success / pending review / failure reason) to the data collection task management module for statistics and closed-loop management.

[0082] Versioning and historical retention: (1) When data with the same borehole_id is updated, the system uses the version field version_no for management: the old record is copied to the history table (e.g., Borehole_His, Strata_His), and the effective time and expiration time are recorded; the new record is written to the main table, and version_no and update time are updated; (2) At the same time, the lineage table Lineage is written: the source pkg_id, source data source identifier, processing link version, and field source (image parsing / intermediate representation) summary information of the record are recorded to achieve traceability.

[0083] Submitting Transactions and Index Updates (1) After all target tables have been written and the consistency check has passed, the system submits the database transaction; (2) The system updates the spatial index and the full-text index: for example, it updates the spatial index for borehole locations and the full-text search index for document index tables to ensure that subsequent retrieval and map display can take effect immediately; (3) The system outputs the database entry log and audit records, including pkg_id, table name written, number of written records, number of updated records, number of records to be reviewed, and anomaly statistics.

[0084] For a standardized data package std_pkg (pkg_id=PKG20250101-001), the system completes the following: adding 1 record to the Borehole table (borehole_id=ZK-12); adding 12 hierarchical records to the Strata table; writing 12 confidence and location information records to the StrataQC table; writing 1 lineage record to the Lineage table; if there are two fields with insufficient confidence, the corresponding fields are written to the ReviewQueue and marked as review_pending in the main table, pending review before being updated and stored in the database.

[0085] Preferably, step S1 includes: labeling each geological data source with its data source identifier, data type, data format, spatial reference information, coverage, update frequency, and permission level, and labeling the layer with its field semantic category, unit, and encoding system; generating a data source profile based on the above labeling, wherein the data source profile includes at least a field mapping dictionary and constraint parameters for subsequent data collection and arrangement, field alignment, and semantic mapping.

[0086] Preferably, step S3, the adapter-based extraction and transcoding, includes: establishing a data adapter registry and defining a unified adapter interface; selecting a data adapter that matches the data source type based on the data source profile; having the data adapter complete data connection, reading, and extraction according to the unified adapter interface; and performing format transcoding and field normalization processing on the extraction results, encapsulating them into a unified intermediate representation, wherein the intermediate representation includes source identifier, record identifier, record structure, attribute key-value pairs, spatial reference information, and metadata.

[0087] In some embodiments, the trained first convolutional neural network model is a conditional multi-branch convolutional network, comprising at least: a convolutional feature extraction backbone network, a feature pyramid fusion layer, a layout region segmentation branch, a text detection branch, a text recognition branch, and a key information extraction branch; wherein, the convolutional feature extraction backbone network generates multi-scale feature maps for the images in the image acquisition package, the data source profile is embedded and encoded to generate conditional vectors and injected into the feature pyramid fusion layer and / or the key information extraction branch; the key information extraction branch generates field-level structured results based on the character sequences output by the text recognition branch, the field-level structured results including at least preset database field names, field values, and field location information; the field-level confidence score is a score of the correctness of the field values, obtained by fusing the detection score of the text detection branch, the recognition score of the text recognition branch, and the extraction score of the key information extraction branch.

[0088] In some embodiments, the first neural network comprises the following structure:

[0089] The input and conditionalization interface allows for preprocessed single or batch images (e.g., uniformly scaled to a fixed size while maintaining aspect ratio, with edge padding if necessary). Image channels are RGB or grayscale (grayscale can be copied to three channels) and are normalized.

[0090] The data source profile input (conditional information) is preprocessed and not directly used as an "image tensor" input. Instead, it is first encoded into a conditional vector to guide network branch selection and field extraction constraints. The conditional vector includes: data type / template category, target field set, field alias set, unit system and encoding system version, etc. The conditional vector is generated by a lightweight "conditional encoder": after embedding discrete labels, they are concatenated and then passed through two fully connected layers (or a 1×1 convolutional equivalent structure) to obtain a fixed-length cond_vec.

[0091] Conditional injection (key structural points): CNN1 introduces conditional injection at the "feature fusion layer / key information extraction branch". Typical implementations can be chosen from the following: Conditional normalization: use cond_vec to generate scaling / translation parameters for each layer's features and perform conditional normalization on the features; FiLM modulation: perform "scaling + bias" on the feature channels; Attention gating: cond_vec generates channel attention weights and performs gating enhancement on the features.

[0092] Sharing a backbone network, CNN1 uses a general convolutional backbone to extract multi-scale features. Typical options include: lightweight deployment: MobileNetV3 / MobileNetV4; accuracy-first: ResNet-50 / ResNet-101. The output is a multi-layer feature map (e.g., four scales): C2, C3, C4, C5 (resolution decreases stepwise, semantics are enhanced stepwise).

[0093] The feature fusion neck (FPN / PAN) takes into account fine-grained features such as small text, thin lines, and table lines. CNN1 uses feature pyramid fusion after the backbone: FPN top-down fusion: upsampling C5 and fusion with C4, C3, and C2 step by step; optional PAN bottom-up enhancement: further strengthening the localization information (more stable for text boxes and table lines). The fused output features are denoted as P2, P3, P4, and P5, and can be combined into a unified high-resolution shared feature map P. Conditional injection is recommended to be performed here (on P2~P5 or P) to make the parsing of different template / field sets more stable.

[0094] Multi-branch task heads (Heads), CNN1 should include at least the following four types of branches (consistent with your previous claims): Page layout segmentation branch: Purpose: To segment the image according to semantic regions, providing "structural priors" for subsequent detection and recognition. Structure: U-Net-style decoder (upsampling + skip connections), outputting multiple types of masks: table areas, text areas, legend / symbol areas, wireframe / grid area, background area (the number of categories can be adjusted according to business needs). Output: Pixel-level probability map of each type of region + (optional) page layout hierarchy features (e.g., containment / adjacency hints between regions).

[0095] The text detection branch aims to locate text lines / blocks (adapting to slanted and dense text). Structure: A detection head based on a shared feature map P outputs candidate text regions (rectangular or quadrilateral): one type outputs a "text probability map"; the other outputs a "geometric regression map" (position, angle / boundary offset, etc.). Post-processing: Based on thresholding and connected component / geometric merging, a set of text boxes, `text_boxes`, is obtained.

[0096] The text recognition branch aims to identify character sequences (the fundamental source of field values) for each text box. ROI feature extraction involves performing ROI alignment / cropping on the shared feature map for the text boxes; sequence modeling uses a convolutional sequence encoder (2D convolution extraction followed by 1D feature sequence conversion); decoding uses CTC decoding or lightweight attention decoding to obtain the character sequence and character-by-character confidence; output includes the recognition result `text_str` and recognition score `rec_score` for each text box.

[0097] The Key Information Extraction Branch (KIE: Field Extraction Head) aims to map "identified text / symbols" to "preset database fields," outputting field-level structured results and confidence scores. Inputs include: a set of text boxes, the ROI features of each text box, the identified character sequence, layout mask features, and cond_vec (target field set / field alias / unit encoding constraints). Structure (typically two subnets): Field Classification Subnet: Predicts which field each text box belongs to (or belongs to "non-field"), outputting the field_id probability; Field Aggregation Subnet: When a field may correspond to multiple text boxes (e.g., lithological description spanning multiple lines), aggregates the subboxes, generating the field value v_k, and outputting the field location information loc_k (page number, box coordinates, table row and column indexes, etc.). Output: A set of field-level structured result triples {(f_k, v_k, loc_k)}.

[0098] The generation mechanism for field-level confidence scores: CNN1 outputs a field-level confidence score c_k (0 to 1) for each field. It is recommended to use an interpretable mechanism of "multi-source score fusion": Detection score: the probability of detecting the text box associated with this field (from the text detection branch); Recognition score: the recognition score of the text box associated with this field (from the text recognition branch); Extraction score: the probability of field classification / aggregation (from the key information extraction branch); Constraint consistency score (from the data source profile): whether the field value meets unit / encoding / value range constraints (satisfaction earns points, conflicts reduce points). The fused c_k is used as the basis for subsequent decisions on "conflict resolution / review and diversion / preprocessing closed-loop rerun". Input: Preprocessed image acquisition package + preprocessed data source profile (cond_vec and constraint parameters); CNN1 output: Field-level structured results (field name, field value, location information) + field-level confidence score; Post-processing: Use the mapping dictionary of the data source profile to align the fields to a unified standard field and perform semantic mapping (unit conversion, encoding mapping, value range verification) to generate standardized data packets.

[0099] In some embodiments, the data source profile and image acquisition package are preprocessed, including: standardizing and embedding discrete labels of data type, data format, coordinate system, unit system and permission level to generate condition vectors, and denoising and unifying the field names, aliases and semantic categories in the field mapping dictionary to form field constraint parameters; and performing image preprocessing on the image acquisition package, including: grouping images into batches according to acquisition time and acquisition location, and performing Gaussian filtering for noise reduction and histogram equalization on the images.

[0100] This application also provides a large-scale acquisition system for multi-source heterogeneous geological data, such as Figure 2The system hardware components include: Image acquisition terminals (field / office terminals) in the form of mobile phones / tablets / dedicated acquisition devices (including cameras, supplementary lighting, image stabilization, GNSS or positioning modules). Functions include: capturing / scanning paper documents, signs, cross-sectional views, borehole columnar sections, and table lists; generating image acquisition packages and uploading them. Multi-source data access gateways (acquisition access servers / edge gateways, optional) in the form of field edge servers or internal network access gateways. Functions include: connecting to terminal uploads, accessing internal network data sources; providing breakpoint resume, encrypted transmission, permission verification, and cache forwarding.

[0101] Data source server cluster (existing business systems / third-party systems), form: GIS server (such as ArcGIS / GeoServer), CAD drawing server / file server, survey database server (PostgreSQL / MySQL / Oracle), document / image library server. Function: Provides storage and access to GIS vectors, CAD drawings, survey structured data, and image materials. Acquisition orchestration and adapter engine server (application server), form: application server or containerized service cluster. Key components: task orchestration module (acquisition task map, parallel scheduling), adapter execution module (GIS / CAD / DB / document adapter), intermediate representation generation module (unified intermediate representation UIR), AI inference server (GPU server / accelerator card server), form: GPU server (NVIDIA / domestic GPUs are both acceptable) or server with AI accelerator card. Function: Deploys the first convolutional neural network model to complete page segmentation, text detection and recognition, and key information extraction; outputs field-level structured results and field-level confidence scores; supports confidence score closed-loop re-inference. Message queue / streaming bus (optional but recommended), form: Kafka / RabbitMQ, etc. Function: To achieve decoupling, peak shaving and reliable delivery in the "collection package upload - preprocessing - inference - alignment mapping - database entry" chain.

[0102] Intermediate data storage and metadata service (intermediate library) format: a combination of relational database / document library / object storage. Function: stores unified intermediate representation, acquisition package index, field mapping dictionary, handles lineage, model version, etc. Target geological database (standard library / results library) format: spatial database (PostGIS / Oracle Spatial, etc.) + relational database (business tables). Function: receives standardized data packets and completes standardized data entry; supports versioning, traceability, and service publishing. Management and display terminal (management console) format: PC browser / workstation. Function: data source labeling and profiling configuration, acquisition task distribution, quality review, data entry auditing, and data service publishing. Network and security equipment (optional depending on deployment environment): VPN / firewall / gateway / certificate service, etc., for cross-network access, access control, and auditing.

[0103] The hardware connection methods (topology) are as follows: 1) Image acquisition link (S2): The image acquisition terminal connects to the multi-source data access gateway via cellular network / wireless LAN / private network; the access gateway connects to the acquisition orchestration and adapter engine server via HTTPS / leased line / VPN; the original image files are usually written to object storage / image library, and the acquisition packet metadata is written to intermediate data storage. 2) Multi-source data extraction link (S1, S3): The acquisition orchestration and adapter engine server connects to the survey database server via intranet database connection (JDBC / ODBC); connects to the CAD drawing server / file server via file sharing protocol (SMB / NFS) or API; connects to the GIS server via GIS service interface (WMS / WFS / REST); the extraction results are written to intermediate data storage in a unified intermediate representation and can be delivered to subsequent modules via message queue. 3) AI parsing link (S4): The acquisition orchestration and adapter engine server / preprocessing service calls the AI ​​inference server via RPC / HTTP / gRPC; the AI ​​inference server pulls images from object storage or receives image byte streams, and outputs field-level structured results and confidence scores to intermediate data storage; when the confidence score closure loop is triggered, a "reprocessing-reinference" loop is formed between the preprocessing service and the AI ​​inference service (still implemented through RPC / message queue).

[0104] 4) Standardized data entry link (S4, S5) field alignment and semantic mapping service (usually deployed in the data acquisition and orchestration server cluster): Read from intermediate data storage: image parsing fields and confidence scores, unify intermediate representation, and after generating standardized data packets from data source profiles and constraint parameters, connect to the target geological database through the database write interface (SQL / batch import) and submit for data entry; write data entry results, lineage and audit logs to the metadata service for querying in the management console.

[0105] The management and auditing process involves connecting the management and display terminals via Web / HTTPS to the orchestration and adapter engine server and metadata service. The management terminal can view task status, data quality, conflicting fields, review queues, and maintain permissions and configurations.

[0106] One of two deployment models is adopted: Cloud / Centralized Deployment: Terminal → Public Network / VPN → Access Gateway → Central Server Cluster (Orchestration + AI Inference + Middleware Library) → Target Geological Database. The other is Intranet / Edge Collaborative Deployment: Terminal → Field Edge Gateway (Cache / Initial Screening) → Intranet Orchestration and Adapter Server → AI Inference Server (can be on Intranet GPU) → Target Geological Database (Intranet).

[0107] The data source management module annotates geological data sources and generates data source profiles; geological data sources include GIS vector data, CAD drawings, exploration databases, and image data;

[0108] The image acquisition module acquires image data and uses the image acquisition terminal to photograph paper documents, on-site signs, cross-sectional views, borehole columnar diagrams, and table lists, generating an image acquisition package containing the acquisition time, acquisition location, and acquisition device identification.

[0109] The extraction and transcoding module adapts extraction and transcoding, calls a data adapter that matches the data source type to perform extraction, and converts the extraction result into a unified intermediate representation.

[0110] The neural network image parsing module inputs the preprocessed data source profile and image acquisition package into the trained first convolutional neural network model, outputs field-level structured results and field-level confidence scores; and performs field alignment and semantic mapping between the field-level structured results and the intermediate representation based on the data source profile to generate standardized data packets.

[0111] The standardized data entry module writes standardized data packets into the target geological database, completing the standardized data entry process.

[0112] Preferably, the data source management module includes: labeling each geological data source with its data source identifier, data type, data format, spatial reference information, coverage, update frequency, and permission level; and labeling the layers with their field semantic categories, units, and encoding systems; generating a data source profile based on the above labels, wherein the data source profile includes at least a field mapping dictionary and constraint parameters for subsequent data collection and arrangement, field alignment, and semantic mapping.

[0113] Preferably, the extraction and transcoding module includes: establishing a data adapter registry and defining a unified adapter interface; selecting a data adapter that matches the data source type based on the data source profile; the data adapter completing data connection, reading, and extraction according to the unified adapter interface; and performing format transcoding and field normalization processing on the extraction results, encapsulating them into a unified intermediate representation, wherein the intermediate representation includes source identifier, record identifier, record structure, attribute key-value pairs, spatial reference information, and metadata.

[0114] Preferably, the trained first convolutional neural network model is a conditional multi-branch convolutional network, comprising at least: a convolutional feature extraction backbone network, a feature pyramid fusion layer, a layout region segmentation branch, a text detection branch, a text recognition branch, and a key information extraction branch; wherein, the convolutional feature extraction backbone network generates multi-scale feature maps for the images in the image acquisition package, the data source profile is embedded and encoded to generate conditional vectors and injected into the feature pyramid fusion layer and / or the key information extraction branch; the key information extraction branch generates field-level structured results based on the character sequences output by the text recognition branch, the field-level structured results including at least preset database field names, field values, and field location information; the field-level confidence score is a score of the correctness of the field values, obtained by fusing the detection score of the text detection branch, the recognition score of the text recognition branch, and the extraction score of the key information extraction branch.

[0115] Preferably, the data source profile and image acquisition package are preprocessed, including: standardizing and embedding discrete labels of data type, data format, coordinate system, unit system and permission level to generate condition vectors; and denoising and unifying the field names, aliases and semantic categories in the field mapping dictionary to form field constraint parameters. The image acquisition package is preprocessed, including: grouping images into batches according to acquisition time and acquisition location, and performing Gaussian filtering for noise reduction and histogram equalization on the images.

[0116] Image preprocessing of the image acquisition package includes secondary filtering, and the filtering function... for:

[0117]

[0118]

[0119] in, The baseline scale factor represents the basic smooth scale under nominal imaging quality, with different preset values ​​depending on the data source profile (such as data type / template); The standard deviation of the Gaussian kernel for the current second-order filter. The scale level is obtained by mapping the mean variance of pixel grayscale values ​​in the borehole histogram and is adaptively adjusted by field-level confidence closure; x is the horizontal coordinate of the pixel in the convolution kernel window, y is the vertical coordinate of the pixel in the convolution kernel window, and m is the side length of the convolution kernel in pixels.

[0120] Within the same image acquisition package, for the "breach shaft histogram" set, the grayscale variance of each image is calculated, and the mean is taken. Based on the mean, a hierarchical mapping is performed to obtain the initial scale level. When performing secondary filtering on the image acquisition package, the scale level τ is obtained by mapping the mean of the grayscale variance of the borehole shaft histogram pixels, and the filtering parameters are determined. When the confidence of the key field output by the first convolutional neural network model is lower than the threshold or inconsistent with the field constraint parameters of the data source profile, τ is increased and the filtering and inference are repeated. The result with higher confidence and satisfying the field constraints is selected as the field-level structured output.

[0121] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0122] (4) This invention targets multi-source heterogeneous geological data such as GIS vectors, CAD maps, exploration databases, and image data. First, it generates a data source profile through annotation, solidifying key constraints such as data type, format, coordinate reference, field semantics, units, and coding system, so that the collected objects can be arranged and managed by a unified profile. Then, it adopts an adapter-based extraction and transcoding mechanism to extract and encapsulate data from different sources into an intermediate representation through a unified interface, significantly reducing the dependence on a single toolchain and manual scripts, and improving the access efficiency and scalability of new data sources. Based on the intermediate representation and profile constraints, field alignment and semantic mapping can achieve cross-system standardization, reduce the errors caused by homonyms, homonyms, and inconsistent units, and form a reusable standardized data package to support large-scale parallel acquisition, incremental updates, and traceable management, thereby improving the efficiency of geological data aggregation and data consistency.

[0123] (5) For image data such as paper documents, cross-sectional views, borehole column charts, and table lists, this invention introduces a pre-trained conditional multi-branch convolutional neural network model to complete page segmentation, text detection and recognition, and key information extraction. It directly outputs field-level structured results (field names, field values, and location information) and field-level confidence scores, achieving end-to-end automation "from image to structured field". Compared with solutions that rely solely on general OCR, this invention can maintain more stable extraction quality in scenarios with complex page layouts, varied table structures, and mixed symbols and annotations. It also provides interpretable quality quantification evidence through field-level confidence scores, which can be used for the triage decision-making of automatic database entry, manual review, and re-extraction. Furthermore, the field results and intermediate representations are aligned and semantically mapped under the constraints of the data source profile, so that the image parsing results and GIS / database extraction results can be integrated and consistent, significantly reducing the cost of secondary manual processing and improving the accuracy and usability of key fields.

[0124] (6) This invention introduces a secondary filtering mechanism in the image acquisition package preprocessing, using the Gaussian weight function G(x,y) to generate the convolution kernel, and using... Adaptively determine the smoothing scale, where The baseline scaling factor can be preset according to the data source profile (data type / template). τ is obtained by hierarchical mapping of the mean variance of pixel grayscale values ​​in the borehole histogram and is adaptively adjusted by a field-level confidence loop. When the confidence of a key field output by the first convolutional neural network model is lower than the threshold or inconsistent with the constraints of the profile field, the system automatically increases τ and re-filters and infers, selecting the result with higher confidence and satisfying the constraints as the output. This mechanism of "variance hierarchical + profile prior + confidence loop" enables the preprocessing parameters to adapt to data quality, suppressing noise and illumination disturbances while avoiding detail loss due to excessive smoothing, thereby improving the robustness of structured extraction and the overall reliability of data storage in low-quality image scenarios.

[0125] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products, and therefore this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0126] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method for large-scale acquisition of multi-source heterogeneous geological data, characterized in that, Including the following steps: S1: Label geological data sources and generate data source profiles; geological data sources include GIS vector data, CAD drawings, exploration databases, and image data; S2: Acquire image data by using an image acquisition terminal to photograph paper documents, site signs, cross-sectional views, borehole columnar views, and table lists, and generate an image acquisition package containing the acquisition time, acquisition location, and acquisition device identification. S3: Adapter-based extraction and transcoding, calls the data adapter that matches the data source type to perform extraction, and converts the extraction result into a unified intermediate representation; S4: Input the preprocessed data source profile and image acquisition package into the trained first convolutional neural network model, output field-level structured results and field-level confidence scores; and perform field alignment and semantic mapping between the field-level structured results and the intermediate representation based on the data source profile to generate standardized data packets; S5: Write the standardized data packet into the target geological database to complete the standardized data entry.

2. The method for large-scale acquisition of multi-source heterogeneous geological data according to claim 1, characterized in that, Step S1 includes: labeling each geological data source with its data source identifier, data type, data format, spatial reference information, coverage, update frequency, and permission level; and labeling the layers with their field semantic categories, units, and encoding systems; generating a data source profile based on the above labels, wherein the data source profile includes at least a field mapping dictionary and constraint parameters for subsequent data collection and arrangement, field alignment, and semantic mapping.

3. The method for large-scale acquisition of multi-source heterogeneous geological data according to claim 1, characterized in that, Step S3, the adapter-based extraction and transcoding, includes: establishing a data adapter registry and defining a unified adapter interface; selecting a data adapter that matches the data source type based on the data source profile; having the data adapter complete data connection, reading, and extraction according to the unified adapter interface; and performing format transcoding and field normalization processing on the extraction results, encapsulating them into a unified intermediate representation, which includes source identifier, record identifier, record structure, attribute key-value pairs, spatial reference information, and metadata.

4. The method for large-scale acquisition of multi-source heterogeneous geological data according to claim 1, characterized in that, The trained first convolutional neural network model is a conditional multi-branch convolutional network, comprising at least: a convolutional feature extraction backbone network, a feature pyramid fusion layer, a layout region segmentation branch, a text detection branch, a text recognition branch, and a key information extraction branch; wherein, the convolutional feature extraction backbone network generates multi-scale feature maps for the images in the image acquisition package, the data source profile is embedded and encoded to generate conditional vectors and injected into the feature pyramid fusion layer and / or the key information extraction branch; the key information extraction branch generates field-level structured results based on the character sequences output by the text recognition branch, the field-level structured results including at least preset database field names, field values, and field location information; the field-level confidence score is a score of the correctness of the field values, obtained by fusing the detection score of the text detection branch, the recognition score of the text recognition branch, and the extraction score of the key information extraction branch.

5. The method for large-scale acquisition of multi-source heterogeneous geological data according to claim 1, characterized in that, The data source profile and image acquisition package are preprocessed, including: standardizing and embedding discrete labels of data type, data format, coordinate system, unit system and permission level to generate condition vectors; and denoising and unifying the field names, aliases and semantic categories in the field mapping dictionary to form field constraint parameters. The image acquisition package is preprocessed, including: grouping images into batches according to acquisition time and location, and performing Gaussian filtering for noise reduction and histogram equalization on the images.

6. A large-scale acquisition system for multi-source heterogeneous geological data, characterized in that, include The data source management module annotates geological data sources and generates data source profiles; Geological data sources include GIS vector data, CAD maps, exploration databases, and image data; The image acquisition module acquires image data and uses the image acquisition terminal to photograph paper documents, on-site signs, cross-sectional views, borehole columnar diagrams, and table lists, generating an image acquisition package containing the acquisition time, acquisition location, and acquisition device identification. The extraction and transcoding module adapts extraction and transcoding, calls a data adapter that matches the data source type to perform extraction, and converts the extraction result into a unified intermediate representation. The neural network image parsing module inputs the preprocessed data source profile and image acquisition package into the trained first convolutional neural network model, outputs field-level structured results and field-level confidence scores; and performs field alignment and semantic mapping between the field-level structured results and the intermediate representation based on the data source profile to generate standardized data packets. The standardized data entry module writes standardized data packets into the target geological database, completing the standardized data entry process.

7. A large-scale acquisition system for multi-source heterogeneous geological data according to claim 6, characterized in that, The data source management module includes: labeling each geological data source with its data source identifier, data type, data format, spatial reference information, coverage, update frequency, and permission level; and labeling layers with their field semantic categories, units, and encoding systems; generating a data source profile based on the above labels, wherein the data source profile includes at least a field mapping dictionary and constraint parameters for subsequent data collection and arrangement, field alignment, and semantic mapping.

8. A large-scale acquisition system for multi-source heterogeneous geological data according to claim 6, characterized in that, The extraction and transcoding module includes: establishing a data adapter registry and defining a unified adapter interface; selecting a data adapter that matches the data source type based on the data source profile; the data adapter completes data connection, reading and extraction according to the unified adapter interface; and performs format transcoding and field normalization processing on the extraction results, encapsulating them into a unified intermediate representation, which includes source identifier, record identifier, record structure, attribute key-value pairs, spatial reference information and metadata.

9. A large-scale acquisition system for multi-source heterogeneous geological data according to claim 6, characterized in that, The trained first convolutional neural network model is a conditional multi-branch convolutional network, comprising at least: a convolutional feature extraction backbone network, a feature pyramid fusion layer, a layout region segmentation branch, a text detection branch, a text recognition branch, and a key information extraction branch; wherein, the convolutional feature extraction backbone network generates multi-scale feature maps for the images in the image acquisition package, the data source profile is embedded and encoded to generate conditional vectors and injected into the feature pyramid fusion layer and / or the key information extraction branch; the key information extraction branch generates field-level structured results based on the character sequences output by the text recognition branch, the field-level structured results including at least preset database field names, field values, and field location information; the field-level confidence score is a score of the correctness of the field values, obtained by fusing the detection score of the text detection branch, the recognition score of the text recognition branch, and the extraction score of the key information extraction branch.

10. A large-scale acquisition system for multi-source heterogeneous geological data according to claim 6, characterized in that, The data source profile and image acquisition package are preprocessed, including: standardizing and embedding discrete labels of data type, data format, coordinate system, unit system and permission level to generate condition vectors; and denoising and unifying the field names, aliases and semantic categories in the field mapping dictionary to form field constraint parameters. The image acquisition package is preprocessed, including: grouping images into batches according to acquisition time and location, and performing Gaussian filtering for noise reduction and histogram equalization on the images.

Images