Intelligent analysis system and method for dmo file based on feature recognition and electronic equipment

Abstract

The application discloses a feature recognition-based DMO file intelligent analysis system and method, an electronic device and a storage medium, relates to the technical field of data processing, and comprises a user interface layer, a business logic layer, a data access layer and the like.The user interface layer is used for providing file selection, progress display and state feedback functions.The business logic layer comprises a DMO analysis engine, a feature recognition module and a data verification module.The DMO analysis engine is used for realizing the reading and analysis of batch DMO files by adopting a multi-thread parallel processing mechanism.The feature recognition module is used for identifying measurement features in DMO files and extracting parameters.The data verification module is used for verifying the analyzed data.The data access layer is used for storing the verified data into a database, structurally processing the data and outputting analysis results.The application accurately identifies measurement features in DMO files by using an improved feature recognition algorithm, improves the analysis efficiency by adopting a multi-thread technology and supports batch file processing.

Description

Technical Field

[0001] This relates to the field of data processing technology, specifically a feature-based intelligent parsing system for DMO files, a feature-based intelligent parsing method for DMO files, electronic devices, and storage media. Background Technology

[0002] With the development of modern industrial measurement technology, the DMO file format has been widely used in the field of precision measurement. However, existing technologies have the following problems:

[0003] Inefficient parsing: Traditional parsing methods use single-threaded processing, which cannot effectively handle large batches of DMO files, resulting in slow processing speed and low resource utilization.

[0004] Inaccurate encoding recognition: Poor automatic recognition capability for file encoding, prone to garbled characters, affecting data accuracy.

[0005] Incomplete feature extraction: Existing systems have low accuracy in recognizing complex geometric features, failing to meet the requirements of high-precision measurement.

[0006] Data management is cumbersome: the lack of a unified data management interface leads to low retrieval and analysis efficiency, impacting work productivity.

[0007] Lack of intelligent analysis: It is impossible to perform intelligent analysis and early warning of measurement data, making it difficult to detect quality problems in a timely manner.

[0008] Poor scalability: The system architecture is poorly designed and cannot adapt to new measurement requirements and feature types. Summary of the Invention

[0009] The purpose of this invention is to provide a feature-based intelligent parsing system for DMO files, a feature-based intelligent parsing method for DMO files, an electronic device, and a storage medium, aiming to solve the technical problems of current traditional parsing methods that use single-threaded processing, cannot effectively process large batches of DMO files, have slow processing speed, and low resource utilization.

[0010] This invention provides the following solution:

[0011] According to one aspect of the present invention, a feature-based intelligent parsing system for DMO files is provided, the system comprising:

[0012] User interface layer, business logic layer, and data access layer;

[0013] The user interface layer provides file selection, progress display, and status feedback functions;

[0014] The business logic layer includes the DMO parsing engine, feature recognition module, and data verification module;

[0015] The DMO parsing engine is used to read and parse batches of DMO files using a multi-threaded parallel processing mechanism.

[0016] The feature recognition module is used to identify measurement features in the DMO file;

[0017] And extract the parameters;

[0018] The data verification module is used to verify the parsed data;

[0019] The data access layer is used to store verified data in the database, perform structured processing on the data, and output the parsed results.

[0020] Furthermore, including:

[0021] The feature recognition module includes: feature recognition using a deep learning model;

[0022] Feature recognition using a deep learning model includes a dense layer, a dropout layer, and an output layer connected in sequence, used to classify and identify measured features;

[0023] Measurement features include point data, circle data, parallel lines, and edge points.

[0024] Furthermore, including:

[0025] The DMO parsing engine also includes: an encoding recognition and conversion module;

[0026] The encoding recognition and conversion module is used to automatically detect the encoding format of DMO files and convert it.

[0027] The encoding formats include UTF-8, GBK, and Latin1.

[0028] Furthermore, including:

[0029] The DMO parsing engine also includes:

[0030] It adopts a coroutine-based asynchronous I / O processing mechanism, using a thread pool to perform file I / O operations and a process pool to handle CPU-intensive parsing.

[0031] Furthermore, including:

[0032] The data validation module also includes a mechanism for performing multi-level validation;

[0033] Multi-level verification includes format integrity verification, numerical range verification, feature relationship verification, and tolerance compliance verification.

[0034] Furthermore, including:

[0035] The data access layer also includes: for using relational databases;

[0036] The database includes the DMO_FILES table and the METADATA table;

[0037] The DMO_FILES table stores basic file information by time range;

[0038] The METADATA table stores measurement data and feature information, and the database caches hot data through Redis.

[0039] Furthermore, including:

[0040] The business logic layer also includes an intelligent data analysis module;

[0041] The intelligent data analysis module is used to perform anomaly detection and time series trend analysis using statistical learning algorithms;

[0042] The statistical learning algorithms include the Isolation Forest algorithm and the ARIMA model.

[0043] According to a second aspect of the present invention, a method for intelligent parsing of DMO files based on feature recognition is provided, comprising:

[0044] The DMO file to be parsed is obtained through the user interface layer, and a multi-threaded parallel processing mechanism is initiated.

[0045] Perform encoding detection and conversion on DMO files, read file content and parse basic information;

[0046] A feature recognition algorithm is used to identify measurement features in the file and extract feature parameters.

[0047] Perform multi-level data validation on the extracted parameters;

[0048] The validated data is stored in the database, processed in a structured manner, and the parsed results are output.

[0049] According to three aspects of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0050] The memory stores a computer program, which, when executed by the processor, causes the processor to perform steps of a feature-based intelligent parsing method for DMO files.

[0051] According to four aspects of the present invention, a computer-readable storage medium is provided that stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a feature-based intelligent parsing method for DMO files.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] This application uses an improved feature recognition algorithm to accurately identify measurement features in DMO files.

[0054] This application improves parsing efficiency by employing multi-threading technology and supports batch file processing.

[0055] This application enhances compatibility by automatically identifying and processing various file encodings (such as UTF-8, GBK, Latin1).

[0056] This application establishes a complete data verification mechanism, including verification of format integrity, numerical range, characteristic relationships, and tolerance compliance, forming a multi-level data verification system to improve data accuracy.

[0057] This application uses a relational database to store the parsed results, supports the automatic creation of table structures, and achieves data integrity through foreign key associations. Attached Figure Description

[0058] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0059] Figure 1 This is a structural diagram of a feature-based intelligent parsing system for DMO files provided by one or more embodiments of the present invention.

[0060] Figure 2 This is a flowchart of a feature recognition-based intelligent parsing method for DMO files provided by one or more embodiments of the present invention.

[0061] Figure 3 This is a flowchart of a feature-based intelligent parsing method for DMO files according to a specific embodiment of the present invention.

[0062] Figure 4 This is the main window of a feature-based intelligent parsing system for DMO files, according to a specific embodiment of the present invention.

[0063] Figure 5This is the data management window of a feature-based intelligent parsing system for DMO files, according to a specific embodiment of the present invention.

[0064] Figure 6 This is a data details window of a feature-based intelligent parsing system for DMO files, according to a specific embodiment of the present invention.

[0065] Figure 7 This is a block diagram of an electronic device structure based on a feature recognition-based intelligent parsing method for DMO files, provided by one or more embodiments of the present invention. Detailed Implementation

[0066] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Figure 1 This is a structural diagram of a feature-based intelligent parsing system for DMO files provided by one or more embodiments of the present invention.

[0068] like Figure 1 As shown, the system includes:

[0069] User interface layer, business logic layer, and data access layer;

[0070] The user interface layer provides file selection, progress display, and status feedback functions.

[0071] The business logic layer includes the DMO parsing engine, feature recognition module, and data verification module;

[0072] The DMO parsing engine is used to read and parse batches of DMO files using a multi-threaded parallel processing mechanism.

[0073] The feature recognition module is used to identify measurement features in the DMO file;

[0074] And extract the parameters;

[0075] The data verification module is used to verify the parsed data;

[0076] The data access layer is used to store verified data in the database, perform structured processing on the data, and output the parsed results.

[0077] Furthermore, including:

[0078] The feature recognition module includes: feature recognition using a deep learning model;

[0079] Feature recognition using a deep learning model includes a dense layer, a dropout layer, and an output layer connected in sequence, used to classify and identify measured features;

[0080] Measurement features include point data, circle data, parallel lines, and edge points.

[0081] Specifically, existing systems employ traditional rule matching or simple machine learning models, which have low accuracy in recognizing complex geometric features; the feature recognition models have poor generalization ability and cannot adapt to new measurement feature types, requiring manual redevelopment of rules, which is costly and time-consuming; the extraction of feature parameters (coordinate values, tolerance data) relies on manual assistance, which is prone to omissions and errors, and cannot meet the needs of high-precision measurement.

[0082] The deep learning model (128-dimensional dense layer + Dropout regularization) significantly improves the recognition accuracy of 4 core features and reduces the recognition error of complex features.

[0083] Through transfer learning techniques, the adaptation cycle for new feature types is shortened and the adaptation cost is reduced.

[0084] It achieves fully automated feature parameter extraction without manual intervention, improving parameter extraction efficiency and reducing the missed extraction rate.

[0085] Furthermore, including:

[0086] The DMO parsing engine also includes: an encoding recognition and conversion module;

[0087] The encoding recognition and conversion module is used to automatically detect the encoding format of DMO files and convert it.

[0088] The encoding formats include UTF-8, GBK, and Latin1.

[0089] Specifically, the existing system lacks automatic encoding recognition function, requiring manual specification of the encoding format. If the encoding is mismatched, it will result in garbled data and poor accuracy.

[0090] It only supports a single encoding (such as UTF-8) and cannot be compatible with GBK and Latin1 encoded DMO files generated by different devices in industrial scenarios, resulting in insufficient compatibility.

[0091] The encoding conversion process lacks a verification mechanism, which can easily lead to character loss after conversion, affecting subsequent feature extraction.

[0092] The automatic identification accuracy through encoding reaches 99%, requiring no manual intervention, completely solving the problem of garbled characters and significantly improving the original accuracy of data;

[0093] It is compatible with three mainstream encoding types, covering more than 95% of the DMO file output formats of industrial measurement equipment, and significantly improving encoding compatibility.

[0094] After encoding conversion, character integrity verification is automatically performed, reducing the character loss rate to 0.001% and ensuring the effectiveness of subsequent feature extraction.

[0095] Furthermore, including:

[0096] The DMO parsing engine also includes:

[0097] It adopts a coroutine-based asynchronous I / O processing mechanism, using a thread pool to perform file I / O operations and a process pool to handle CPU-intensive parsing.

[0098] Specifically, the traditional single-threaded processing mode cannot handle a large number of DMO files, resulting in slow processing speed and low resource utilization.

[0099] Processing large files (such as 100MB+) using the "full read" mode can easily lead to memory overflow and system lag.

[0100] I / O operations (file reading) and CPU-intensive operations (parsing / feature recognition) are executed sequentially, blocking each other and further reducing processing efficiency.

[0101] Thread pools handle I / O operations, and process pools handle CPU-intensive tasks, achieving "I / O-computation parallelism" and improving the efficiency of batch file processing;

[0102] The streaming parsing mechanism eliminates the need to read the entire file, reducing memory usage for large file processing and completely avoiding memory overflow issues.

[0103] Asynchronous scheduling of coroutines reduces thread switching overhead, improves CPU utilization, and significantly optimizes hardware resource utilization.

[0104] Furthermore, including:

[0105] The data validation module also includes a mechanism for performing multi-level validation;

[0106] Multi-level verification includes format integrity verification, numerical range verification, feature relationship verification, and tolerance compliance verification.

[0107] Specifically, the existing system only performs simple format checks (such as file extension verification), and cannot detect deeper problems such as numerical anomalies (such as coordinate values ​​exceeding the measurement range) and contradictory feature relationships (such as parallel line spacing not meeting design standards), resulting in a high data processing error rate (1%).

[0108] Without tolerance conformity verification, data that does not meet tolerance requirements can flow into downstream processes, easily leading to missed quality issues.

[0109] The verification results were not logged, the abnormal data could not be traced, and the problem investigation efficiency was low.

[0110] By using four layers of verification to cover all dimensions of data issues, the data processing error rate is reduced and the error interception rate is increased.

[0111] By directly filtering out non-conforming data based on tolerance compliance verification, the rate of missed detection of downstream quality problems is reduced, thus reducing the cost of quality rework.

[0112] Abnormal data is automatically logged (including file ID, exception type, and verification node), which shortens the troubleshooting time and improves the efficiency of tracing the source.

[0113] Furthermore, including:

[0114] The data access layer also includes: for using relational databases;

[0115] The database includes the DMO_FILES table and the METADATA table;

[0116] The DMO_FILES table stores basic file information by time range;

[0117] The METADATA table stores measurement data and feature information, and the database caches hot data through Redis.

[0118] Specifically, the existing system uses a single data table to store all data, resulting in low retrieval efficiency when querying large amounts of data (such as 100,000+ measurement records) (a single query takes more than 10 seconds).

[0119] Without a caching mechanism, high-frequency queries (such as measurement data from the last 7 days) require repeated access to the database, resulting in high database pressure and high response latency.

[0120] Basic file information and measurement feature information are stored together, resulting in low data structuring, requiring additional processing for subsequent analysis, and low data reuse efficiency.

[0121] The DMO_FILES table is partitioned by time (e.g., "dmo_files_202506"), which improves the efficiency of historical data queries and shortens the time of a single query.

[0122] Redis caches frequently accessed data (such as high-frequency query records from the last 7 days), achieving a cache hit rate of 90%, reducing database access pressure by 70%, and shortening query response latency to within 50ms;

[0123] The dual-table structure (file information + measurement features) enables structured data storage, eliminating the need for additional data processing in subsequent intelligent analysis and improving data reuse efficiency.

[0124] Furthermore, including:

[0125] The business logic layer also includes an intelligent data analysis module;

[0126] The intelligent data analysis module is used to perform anomaly detection and time series trend analysis using statistical learning algorithms;

[0127] The statistical learning algorithms include the Isolation Forest algorithm and the ARIMA model.

[0128] Specifically, the existing system lacks intelligent analysis capabilities, requiring manual review of each measurement data to detect anomalies, resulting in low anomaly detection efficiency and a high rate of missed detections.

[0129] The inability to predict trends in measurement data (such as the trend of changes in a certain characteristic deviation value) makes it difficult to predict quality risks in a timely manner, resulting in a delayed response to quality issues.

[0130] The analysis results lack visualization output, and data trends and anomaly distributions cannot be presented intuitively, resulting in weak decision support capabilities.

[0131] The Isolation Forest algorithm achieves an anomaly detection accuracy of over 95%, improving anomaly identification efficiency and shortening the anomaly detection time for 1000 files.

[0132] The ARIMA model can predict measurement data trends for the next 10 time points, with a lead time of up to 24 hours for quality risk prediction, thus avoiding the generation of batches of non-conforming products.

[0133] The analysis results are output in the form of "anomaly list + trend chart", which allows decision-makers to quickly locate problems, formulate countermeasures, and improve decision response speed.

[0134] Specifically, existing technologies for DMO parsing systems suffer from poor scalability due to unreasonable system architecture design and high coupling between functional modules. By decoupling the interface, logic, and data modules, functions can be flexibly added / iterated, significantly improving scalability.

[0135] The lack of a unified user interaction interface leads to fragmented processes for file selection, progress monitoring, and result feedback, resulting in cumbersome data management and low operational efficiency. By providing a unified interface for file import, progress monitoring, and result viewing, the operation process is simplified, improving data management efficiency.

[0136] To address the problem of data processing (parsing and verification) being disconnected from storage, resulting in a lack of structured processing flow, chaotic data flow, and low efficiency in subsequent retrieval and analysis, a standardized flow link from parsing to storage is established. This lays a structured foundation for subsequent intelligent analysis and reduces the time spent on data flow.

[0137] Figure 2This is a flowchart of a feature recognition-based intelligent parsing method for DMO files provided by one or more embodiments of the present invention.

[0138] like Figure 2 As shown, the method includes:

[0139] Step S1: Obtain the DMO file to be parsed through the user interface layer and start the multi-threaded parallel processing mechanism;

[0140] Step S2: Perform encoding detection and conversion on the DMO file, read the file content and parse the basic information;

[0141] Step S3: Use a feature recognition algorithm to identify measurement features in the file and extract feature parameters;

[0142] Step S4: Perform multi-level data validation on the extracted parameters;

[0143] Step S5: Store the verified data in the database, perform structured processing, and output the parsing results.

[0144] Specifically, the user interface layer provides two methods: batch selection of folders and drag-and-drop import of files. It automatically identifies files with the .dmo extension and filters out non-target files (such as .txt and .log), reducing manual screening costs.

[0145] After obtaining the file list, immediately initialize the thread pool (4 IO threads by default) and the process pool (processing processes matching the number of CPU cores) to allocate resources for the subsequent parallel processing of S2-S4 and avoid the delay of "initialization during processing";

[0146] The interface displays the file import progress in real time (e.g., 100 / 100 files have been imported) and the thread / process startup status (e.g., 4 IO threads are ready), ensuring process transparency.

[0147] By calling the encoding recognition algorithm (traversing UTF-8→GBK→Latin1 priority), and following the logic of "attempting decoding - confirming encoding if no UnicodeDecodeError", the file encoding is automatically detected and converted to the unified UTF-8 format to avoid garbled characters.

[0148] It adopts a "streaming read" mechanism (reading file content in 10MB / blocks), which eliminates the need to load the entire file into memory, making it suitable for large file processing;

[0149] Based on regular expressions, key information (such as “MEAS_DATE:20250603” to extract the measurement date and “PROJ_CODE:C101” to extract the project number) is matched and automatically associated with file IDs for storage, avoiding manual data entry.

[0150] An improved feature recognition algorithm (combining deep learning models and rule matching) is invoked—first, the starting row of features is located by matching the "FEAT / " identifier, and then the feature type (POINT / CIRCLE / CPARLN / EDGEPT) is classified and identified by a deep learning model (dense layer + dropout layer), balancing recognition speed and accuracy;

[0151] Structured storage: The validated data is split and stored in a relational database according to "file information - measurement features"—the DMO_FILES table (split by time, such as "dmo_files_202506") stores basic file information (file name, measurement date, project number), and the METADATA table stores feature parameters (feature ID, coordinate value, deviation value). The two tables are linked by "file ID".

[0152] Cache optimization: Use Redis to cache frequently queried data from the past 7 days (such as measurement results from the C101 project for the past 7 days), and set it to automatically expire after 24 hours to reduce database pressure;

[0153] Results Output: Outputs "dual-dimensional results"—basic dimension (structured parameter table, supports Excel / JSON export) and analytical dimension (feature distribution chart, pass rate statistics, and percentage of abnormal data), intuitively presenting the analytical value.

[0154] Figure 3 This is a flowchart of a feature-based intelligent parsing method for DMO files according to a specific embodiment of the present invention.

[0155] like Figure 3 As shown, in one specific embodiment,

[0156] Specifically, it includes:

[0157] DMO file input: The system receives externally input DMO files through file reading, loads the file contents into memory, and prepares for subsequent processing.

[0158] File encoding detection detects the file's encoding format (such as UTF-8, GBK, etc.) and converts it to an encoding that is uniformly supported by the system (ensuring no garbled characters in subsequent processing).

[0159] File format verification verifies whether DMO files conform to preset format specifications (such as structure, syntax, identifier header, etc.), filters out files with incorrect formats, and avoids errors in subsequent processes.

[0160] Feature recognition identifies the core feature information contained in the file (such as data type, identifier fields, etc.) and classifies the features according to rules (to facilitate subsequent targeted processing).

[0161] Parameter extraction involves extracting key parameters from the classified features, and calculating these parameters if necessary.

[0162] Data standardization involves converting extracted parameters into a unified standard format (such as standardization of data type, precision, and units) to ensure data consistency.

[0163] Database storage involves writing standardized data into a database (such as a relational database or NoSQL database) to achieve long-term data storage and management.

[0164] Data analysis involves statistical analysis and mining of DMO data in the database (such as trend analysis and anomaly detection) to extract valuable information.

[0165] Results output: The results of data analysis are output in the form of visual reports, files, interfaces, etc., for use by users or other systems.

[0166] It mainly includes three levels:

[0167] 1. User Interface Layer: Provides file selection, progress display, and status feedback functions.

[0168] 2. Business Logic Layer: Includes DMO parsing engine, feature recognition module, and data verification module.

[0169] 3. Data Access Layer: Responsible for data storage and structured processing.

[0170] It supports intelligent recognition of complex geometric features through a feature classification model based on deep learning.

[0171] By employing transfer learning techniques, the model's adaptability to new feature types can be improved.

[0172] And to achieve automatic extraction and verification of feature parameters.

[0173] Through the parsing engine, based on the asynchronous I / O processing mechanism of coroutines, parallel file processing and streaming parsing, as well as memory-optimized large file processing strategies are achieved.

[0174] Through an intelligent data analysis system, anomaly detection algorithms based on statistical learning are used, along with measurement data trend analysis and early warning mechanisms, and an adaptive data quality assessment system.

[0175] Specifically, in another embodiment,

[0176] It includes a feature-based intelligent parsing system for DMO files.

[0177] It includes: a file reading module, an encoding recognition and conversion module, a feature recognition and processing module, a data verification module, and a database storage module.

[0178] The specific workflow is as follows:

[0179] 1. File Input

[0180] Supports batch file import;

[0181] Automatically identify file encoding;

[0182] Perform encoding conversion (if necessary).

[0183] 2. Data Analysis

[0184] Extract basic file information (date, time, project number, etc.);

[0185] Identify measurement features (points, circles, lines, etc.);

[0186] Extract coordinate values, tolerance data, and deviation values, etc.

[0187] 3. Feature Processing

[0188] The system supports processing the following feature types:

[0189] POINT: Point data;

[0190] CIRCLE: Round Data;

[0191] CPARLN: Parallel lines;

[0192] EDGEPT: Edge point;

[0193] 4. Data Validation

[0194] Format integrity verification;

[0195] Numerical range verification;

[0196] Feature relationship verification;

[0197] Tolerance compliance verification;

[0198] 5. Data storage

[0199] The parsing results are stored using a relational database, and mainly include the following table structure:

[0200] DMO_FILES: Stores basic file information;

[0201] METADATA: Stores measurement data and feature information.

[0202] The encoding recognition algorithm includes:

[0203] Python

[0204] def detect_encoding(file_content):

[0205] encodings = ['utf-8', 'gbk', 'latin1']

[0206] For encoding in encodings:

[0207] try:

[0208] decoded_content = file_content.decode(encoding)

[0209] return encoding

[0210] except UnicodeDecodeError:

[0211] continue

[0212] return None

[0213] ```

[0214] Feature recognition algorithms include:

[0215] Python

[0216] def identify_feature(line):

[0217] if line.startswith('FEAT / '):

[0218] feature_type = line.split(',')[1].strip()

[0219] return {

[0220] 'POINT': process_point,

[0221] 'CIRCLE': process_circle,

[0222] 'CPARLN': process_parallel_line,

[0223] 'EDGEPT': process_edge_point

[0224] }.get(feature_type, None)

[0225] ```

[0226] 7. Exception Handling

[0227] The system implements a complete exception handling mechanism:

[0228] File reading error;

[0229] Encoding conversion error;

[0230] Data format error;

[0231] Feature recognition anomaly;

[0232] Database operation error;

[0233] 8. Performance optimization;

[0234] Multi-threaded parallel processing is employed;

[0235] Implement a data caching mechanism;

[0236] Optimize database indexes;

[0237] Batch data processing;

[0238] Database optimization design;

[0239] This system employs the following database optimization strategies:

[0240] Table partitioning strategy:

[0241] Data is divided into tables based on time range;

[0242] Use partitioned tables to improve query efficiency;

[0243] An automated table management mechanism, which includes:

[0244] CREATE TABLE dmo_files_#{year_month} ( id BIGINT PRIMARY KEY AUTO_INCREMENT, file_name VARCHAR(255) NOT NULL, measurement_date DATE NOT NULL, project_code VARCHAR(50), operator_id VARCHAR(50), metadata_count INT DEFAULT0, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, INDEX idx_measurement_date(measurement_date), INDEX idx_project_code (project_code)) PARTITION BY RANGE(TO_DAYS(measurement_date)) ( PARTITION p_current VALUES LESS THAN (TO_DAYS(CURRENT_DATE)), PARTITION p_future VALUES LESS THAN MAXVALUE);

[0245] Caching strategy:

[0246] Use Redis to cache frequently accessed data;

[0247] Implement an intelligent cache preheating mechanism;

[0248] Automatic expiration of cached data;

[0249] In another specific embodiment, it includes: a feature recognition system based on deep learning.

[0250] This embodiment implements a feature recognition system based on deep learning, specifically including:

[0251] Feature classification model: It adopts a multi-layer neural network structure to support the recognition of various geometric features;

[0252] Parameter extraction module: Enables automatic parameter extraction and validation for different feature types;

[0253] Model training system: Supports incremental learning and model updates to improve recognition accuracy;

[0254] class FeatureRecognitionSystem: def __init__(self): self.model =self._load_model() self.feature_types = { 'POINT': PointProcessor(), 'CIRCLE': CircleProcessor(), 'CPARLN': ParallelLineProcessor(), 'EDGEPT':EdgePointProcessor()} def _load_model(self): """Load the pre-trained deep learning model"""model = tf.keras.models.Sequential([ tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dense(len(FEATURE_TYPES), activation='softmax') ])return model

[0255] def predict_feature_type(self, feature_data): """Predict feature type"""preprocessed_data = self._preprocess_data(feature_data) prediction =self.model.predict(preprocessed_data) return FEATURE_TYPES[np.argmax(prediction)] def extract_parameters(self, feature_data, feature_type): """Extract feature parameters""" processor = self.feature_types.get(feature_type) if processor:return processor.extract_parameters(feature_data) return None.

[0256] In another specific embodiment, it includes: a coroutine-based asynchronous I / O processing system.

[0257] This embodiment implements a high-performance file processing system, specifically including:

[0258] Asynchronous I / O processing: Using coroutines to implement non-blocking I / O operations improves system throughput;

[0259] Parallel processing: Supports parallel processing of multiple files, making full use of system resources;

[0260] Memory management: Enables streaming processing and reduces memory usage;

[0261] class AsyncFileProcessor: def __init__(self): self.io_pool =ThreadPoolExecutor(max_workers=4) self.process_pool = ProcessPoolExecutor(max_workers=cpu_count()) async def process_files(self, file_paths): """Asynchronously process multiple files""" tasks = [self.process_single_file(path) for path in file_paths] return await asyncio.gather(*tasks) async def process_single_file(self, file_path): """Asynchronously process a single file""" # Use thread pool for IO operations content = await self.loop.run_in_executor( self.io_pool, self._read_file, file_path) # Use process pool for CPU-intensive processing features = await self.loop.run_in_executor(self.process_pool, self._process_content, content) return features.

[0262] In another specific embodiment, it includes: an intelligent analysis system based on statistical learning.

[0263] This embodiment implements an intelligent data analysis system, specifically including:

[0264] Anomaly detection: Anomaly detection is achieved using the Isolation Forest algorithm;

[0265] Trend analysis: Time series analysis was performed using the ARIMA model;

[0266] Quality assessment: Implementing an adaptive data quality assessment mechanism;

[0267] class IntelligentAnalysisSystem: def __init__(self): self.anomaly_detector = IsolationForest( contamination=0.1, random_state=42 ) self.trend_analyzer = ARIMA() def detect_anomalies(self, measurements): """Detect anomalies""" # Use the Isolation Forest algorithm to detect anomalies predictions = self.anomaly_detector.fit_predict(measurements) return np.where(predictions == -1)[0] def analyze_trends(self, time_series_data): """Analyze data trends""" # Use the ARIMA model for time series analysis model = self.trend_analyzer.fit(time_series_data) forecast =model.forecast(steps=10) return forecast.

[0268] Alternatives to the present invention include:

[0269] Using traditional machine learning algorithms to replace deep learning models will reduce recognition accuracy.

[0270] Using multithreading instead of coroutines will reduce system resource utilization;

[0271] It uses encoding recognition, but lacks flexibility and cannot adapt to various encoding scenarios;

[0272] Relational databases can be used instead of distributed storage, but scalability will be limited.

[0273] Offline analysis is used instead of real-time analysis, but instant early warning cannot be achieved;

[0274] Using simple data format checks or manual verification may not cover complex verification needs, such as feature relationship verification.

[0275] Figure 7This is a block diagram of an electronic device structure based on a feature recognition-based intelligent parsing method for DMO files, provided by one or more embodiments of the present invention.

[0276] like Figure 7 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0277] The memory stores a computer program that, when executed by the processor, causes the processor to perform steps of a feature-based intelligent parsing method for DMO files.

[0278] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a feature-based intelligent parsing method for DMO files.

[0279] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0280] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature-based intelligent parsing system for DMO files, characterized in that, include: User interface layer, business logic layer, and data access layer; The user interface layer provides file selection, progress display, and status feedback functions; The business logic layer includes the DMO parsing engine, feature recognition module, and data verification module; The DMO parsing engine is used to read and parse batches of DMO files using a multi-threaded parallel processing mechanism. The feature recognition module is used to call an improved feature recognition algorithm, first to locate the starting line of the feature by matching feature identifiers according to rules, and then to classify and recognize the measurement features in the DMO file by using a deep learning model, and extract feature parameters; the measurement features include point data, circle data, parallel lines and edge points; The data verification module is used to verify the parsed data; The data access layer is used to store verified data in the database, perform structured processing on the data, and output the parsed results.

2. The intelligent parsing system for DMO files based on feature recognition according to claim 1, characterized in that, The feature recognition module includes: performing feature recognition using a deep learning model; The feature recognition using a deep learning model includes a dense layer, a dropout layer, and an output layer connected in sequence, used to classify and recognize the measured features.

3. The intelligent parsing system for DMO files based on feature recognition according to claim 1, characterized in that, The DMO parsing engine also includes: an encoding recognition and conversion module; The encoding recognition and conversion module is used to automatically detect and convert the UTF-8, GBK, and Latin1 encoding formats of DMO files, and uniformly convert them to the UTF-8 standard encoding format.

4. The intelligent parsing system for DMO files based on feature recognition according to claim 1, characterized in that, The DMO parsing engine also includes: It adopts a coroutine-based asynchronous I / O processing mechanism, using a thread pool to perform file I / O operations and a process pool to handle CPU-intensive parsing.

5. The intelligent parsing system for DMO files based on feature recognition according to claim 1, characterized in that, The data verification module also includes a function for performing multi-level verification; Multi-level verification includes format integrity verification, numerical range verification, feature relationship verification, and tolerance compliance verification. Among them, feature relationship verification verifies whether there are contradictions in the feature relationships, and tolerance compliance verification verifies whether the tolerances meet the requirements.

6. The intelligent parsing system for DMO files based on feature recognition according to claim 1, characterized in that, The data access layer also includes: a method for using a relational database; The database includes the DMO_FILES table and the METADATA table; The DMO_FILES table stores basic file information by time range. The METADATA table stores measurement data and feature information, and the database caches hot data through Redis.

7. The intelligent parsing system for DMO files based on feature recognition according to claim 1, characterized in that, The business logic layer also includes an intelligent data analysis module; The intelligent data analysis module is used to perform anomaly detection and time series trend analysis using statistical learning algorithms; The statistical learning algorithms include the Isolation Forest algorithm and the ARIMA model.

8. A method for intelligent parsing of DMO files based on feature recognition, characterized in that, Includes the following steps: The DMO file to be parsed is obtained through the user interface layer, and a multi-threaded parallel processing mechanism is started. Perform encoding detection and conversion on DMO files, read file content and parse basic information; A feature recognition algorithm is used to identify measurement features in the file and extract feature parameters. Specifically, the improved feature recognition algorithm is invoked to first locate the starting row of features by matching feature identifiers according to rules, and then the feature type is classified and identified by a deep learning model for classifying and recognizing the measurement features; the measurement features include point data, circle data, parallel lines, and edge points; Perform multi-level data validation on the extracted parameters; The validated data is stored in the database, processed in a structured manner, and the parsed results are output.

9. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the feature recognition-based intelligent parsing method for DMO files as described in claim 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by an electronic device. When the computer program is run on the electronic device, it causes the electronic device to perform the steps of the feature recognition-based intelligent parsing method for DMO files as described in claim 8.

Images