Model file processing method and device and computer equipment
Through state machine model and multi-threaded concurrent processing technology, the internal structure of Simulink model files is directly parsed, and the environmental dependency, compatibility and performance efficiency problems in the existing technology are solved, and flexible, efficient analysis and in-depth analysis of model files are realized. ability.
Patent Information
- Application Number
- CN202411437458.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2024-11-29
AI Technical Summary
The existing Simulink model file analysis technology has challenges in terms of environmental dependency, compatibility, performance efficiency and functional depth, which limits the flexible resolution, accuracy, efficiency and security of model files, especially when processing large-scale models, the resource consumption is high. , CPU usage is too high, and there is a lack of unified description standards and an open tool ecosystem.
Through the state machine model, state identification of the model file, labels and attributes are determined, and nested levels are tracked based on the data stack structure, subsystem extraction and multi-threaded concurrent processing are realized, directly parsing the internal structure of the model file, improving parsing efficiency and accuracy .
It realizes efficient and accurate analysis of large-scale model files, reduces system resource consumption, improves the parsing accuracy and efficiency of model files, enhances cross-platform compatibility and applicability, supports model files of different versions and structures, and simplifies In-depth analysis and optimization capabilities.
Smart Images

Figure CN119047424A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of computer-aided design and model file processing, and in particular to a model file processing method, device and computer equipment. Background Art
[0002] When parsing model files, existing technologies are often highly dependent on specific software environments, especially the MATLAB / Simulink ecosystem. This dependence not only limits the portability and cross-platform capabilities of the technology, but also increases the entry threshold and usage costs for users. With the continuous update and iteration of Simulink software, the model file format (such as the evolution of .Slx from the XML-based format) has also changed, which may cause compatibility issues when facing different versions of model files. The lack of extensive support for historical versions and future new versions limits the wide applicability and long-term vitality of the technology.
[0003] For large-scale and complex Simulink models, existing parsing technologies may not be ideal in terms of processing speed and resource consumption. High memory usage, excessive CPU usage, or long parsing time during the parsing process are particularly prominent when dealing with ultra-large-scale system models, affecting work efficiency and user experience. Many existing parsing tools and libraries focus mainly on basic model structure and parameter extraction, and lack support for in-depth logic analysis, custom module processing, and advanced simulation of model behavior. This limits the ability to perform in-depth analysis and optimization of specific application areas such as intelligent transportation systems and aerospace. Due to the lack of a unified model description standard and an open tool ecosystem, parsing tools developed by different research teams and companies are often incompatible and difficult to share and integrate.
[0004] In summary, the existing Simulink model file parsing technology faces a series of challenges in terms of environmental dependence, compatibility, security and privacy, performance efficiency, functional depth and standardization. Innovative technologies are urgently needed to overcome these limitations and improve the accuracy, efficiency and security of flexible model parsing. Summary of the invention
[0005] Based on this, the purpose of this application is to provide a model file method, device and computer equipment that improve the accuracy, efficiency and security of flexible model parsing to solve the technical problems mentioned in the above background technology.
[0006] In a first aspect, the present application provides a model file processing method, including:
[0007] Acquire a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each of the tags;
[0008] Tracking the nesting level of each of the tags based on the data stack structure, and determining the actual tag in the corresponding actual nesting level when parsing any attribute in the model file;
[0009] Subsystem extraction is performed on the model file according to the actual tag, so that the subsystem is processed concurrently by multiple threads.
[0010] In one embodiment, the model file includes a byte stream; after obtaining the model file, the following steps further include: identifying the beginning field in the byte stream to confirm the file type of the model file; and reading a specific tag in the model file to determine the corresponding version information.
[0011] In one embodiment, the nesting level of each tag is tracked based on the data stack structure, including: for the tag of the current nesting level, when the tag is detected to be in the starting state, the tag is pushed into the corresponding data stack, and the next nesting level of the current nesting level is entered; when the tag is detected to be in the ending state, the top element of the data stack is popped out, and the previous nesting level of the current nesting level is entered.
[0012] In one embodiment, when parsing any attribute in the model file, determining the actual label in the corresponding actual nested level includes: when parsing any attribute in the model file, determining the corresponding actual label according to the top element of the data stack at the current moment; and taking the depth of the data stack at the current moment as the actual nested level corresponding to the actual label.
[0013] In one embodiment, the actual tags include system tags and block tags; the subsystem extraction of the model file according to the actual tags includes: determining whether the currently parsed block tag structure is the subsystem to be extracted according to the block tag and the field information associated with the block tag; if so, extracting the subsystem of the model file through the state machine model and the system tag.
[0014] In one embodiment, the actual tag also includes a line tag; the subsystem extraction of the model file through the state machine model and the system tag includes: when the system tag is identified as a starting state, the model file is switched to system parsing, and when the block tag associated with the system tag is identified as a termination state, block parsing is performed; when the line tag associated with the block tag is identified as a starting state, switching to line parsing, and when the system tag is identified as a termination state and the previous state is the termination state of the line tag, block association is performed; the content of the subsystem between the starting state and the termination state of the system tag is extracted.
[0015] In one embodiment, to enable the subsystem to process multiple threads concurrently, the method includes: creating a thread pool, and determining whether the number of tasks of the subsystem is less than the number of threads in the waiting state in the thread pool; the number of tasks is less than the maximum number of threads limited to concurrency by the thread pool; if so, performing multi-thread concurrent processing on the subsystem based on the number of threads in the waiting state; if not, performing multi-thread concurrent processing on the subsystem according to the task queue and the updated thread pool.
[0016] In one embodiment, subsystem multi-threaded concurrent processing is performed based on the task queue and the updated thread pool, including: placing unprocessed tasks of multiple subsystems into the task queue; updating the thread pool according to the length of the task queue and the busy status of the threads in the thread pool to obtain a new thread pool; returning to the process of determining whether the number of unprocessed subsystem tasks is less than the number of waiting threads in the new thread pool, until all subsystem multi-threaded concurrent processing is completed.
[0017] In a second aspect, the present application also provides a model file processing device, including:
[0018] A state machine model module is used to obtain a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each of the tags;
[0019] A data stack structure module, used for tracking the nesting level of each of the tags based on the data stack structure, and determining the actual tag in the corresponding actual nesting level when parsing any attribute in the model file;
[0020] The multi-thread processing module is used to extract subsystems from the model file according to the actual tags, so that the subsystems can be processed in a multi-threaded concurrent manner.
[0021] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:
[0022] Acquire a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each of the tags;
[0023] Tracking the nesting level of each of the tags based on the data stack structure, and determining the actual tag in the corresponding actual nesting level when parsing any attribute in the model file;
[0024] Subsystem extraction is performed on the model file according to the actual tag, so that the subsystem is processed concurrently by multiple threads.
[0025] The above-mentioned model file processing method, device and computer equipment use a state machine model to identify the state of the model file, determine multiple tags and the attributes associated with each tag, and track the nesting level of each tag based on the data stack structure, so that when parsing any attribute in the model file, the actual tag in the corresponding actual nesting level is determined, and the model file is extracted from the subsystem according to the actual tag, so that the subsystem can be processed concurrently by multiple threads. Therefore, this solution overcomes the limitations of the prior art that rely on specific software environments and general parsing libraries, and realizes flexible and efficient model information extraction by directly parsing the inherent structure of model files in different data formats. The state machine model and multi-threaded concurrent processing strategy ensure that even large-scale model files can realize the identification of actual tags in actual nesting levels and efficient parsing of extracted subsystems, which improves the accuracy and efficiency of model file parsing and reduces system resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 An application environment diagram of a model file processing method in an embodiment;
[0027] Figure 2 A schematic diagram of a flow chart of a model file processing method in one embodiment;
[0028] Figure 3 A schematic diagram of a flow chart of a general parsing method for a simulink model file in one embodiment;
[0029] Figure 4 A version correspondence diagram of an Mdl file in an embodiment;
[0030] Figure 5 A diagram showing the version information content of an XML file in an embodiment;
[0031] Figure 6 is a diagram of example text content when the system extracts in one embodiment;
[0032] Figure 7 A state machine state transition diagram of a model file in one embodiment;
[0033] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0035] Among them, Simulink is a graphical programming environment widely used in multi-domain simulation and model design. Simulink model files represent the dynamic behavior of the system. They can contain a series of components, signals and parameters: Simulink contains two model file formats, Mdl and Slx. Mdl files are binary formats with fast loading speeds, and are usually used for actual simulation runs; Slx files are actually compressed files. After decompression, they are in xml text format, which is easy to read and edit, and suitable for model sharing and version control. MATLAB is a high-performance language and interactive environment for numerical computing, widely used in engineering and scientific fields.
[0036] The model file processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The server 104 is used to obtain the model file from the terminal 102, and identify the state of the model file through the state machine model, determine multiple tags and the attributes associated with each tag; track the nesting level of each tag based on the data stack structure, and determine the actual tag in the corresponding actual nesting level when parsing any attribute in the model file. The server 104 is also used to extract the subsystem of the model file according to the actual tag so that the subsystem multi-threaded concurrent processing. Among them, the terminal 102 can be but not limited to various personal computers, laptops, smart phones, tablets, drone equipment, smart car-mounted equipment and portable wearable devices. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, which can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platforms.
[0037] In one embodiment, Figure 2 As shown, a model file processing method is provided, which is applied to Figure 1 The server in the example is used as an example, and a model file processing system is integrated in the server to implement the following steps in the above method:
[0038] Step S202: Acquire a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each tag.
[0039] The model files include but are not limited to Simulink model files in the MATLAB environment. A state machine model is a computational model that uses a set of states and transitions between these states to describe the behavior of a system. The concept of state machines is very common in computer science, mathematics, and engineering, and is used to design systems and programs to ensure that their behavior is predictable and controllable.
[0040] Specifically, the server includes a general file parsing module, which is the core parsing module and is mainly used to parse the model file and obtain the complex attribute parameters, module relationships and other information inside the model. The state machine model is used to parse the acquired model file byte by byte. Through the pre-designed state transition logic, such as tag start, tag end, attribute name, attribute value, text content, etc., the tag start characterizes the tag as the starting state, that is, each state represents a specific stage in the parsing process. Therefore, the server dynamically converts the state of the byte stream input by the model file through the state machine model, thereby identifying the specific tag type and its associated attributes. This makes it possible to parse and reconstruct the structure and association between tags and attributes in the model file directly from the byte stream without completely loading the file into the memory to build the DOM tree.
[0041] In one embodiment, before the code generation of the model file, the server ensures that the model design not only complies with the internal design rules, but also meets the specific requirements of the target hardware, including the code style and performance indicators, thereby simplifying the subsequent hardware embedded implementation steps.
[0042] In one embodiment, the server can deeply analyze the model structure and data flow characteristics, realize fusion modeling between multi-source data, and further identify and solve bottlenecks that affect the effectiveness of simulation, providing an optimization path for system performance.
[0043] Step S204: Track the nesting level of each tag based on the data stack structure, and determine the actual tag in the corresponding actual nesting level when parsing any attribute in the model file.
[0044] The labels include but are not limited to system labels, line labels and block labels. For example, system labels may include block labels and line labels, and block labels may nest system labels to form label structures at different levels.
[0045] Specifically, the server pushes and pops tags into and out of the data stack in advance to manage the nested levels, so that when dynamically parsing any attribute in the model file, the state machine model and the data stack structure are combined to focus on identifying different actual tags and their attributes, and accurately track the actual tag level to which the current attribute parameters belong. Therefore, by controlling the parsing process through intelligent state machine model logic and dynamically managing the nested levels of tags in combination with the data stack structure, the parsing of complex model file structures can be completed efficiently and accurately under the premise of low memory usage, and the step-by-step reconstruction and relationship mapping of model elements can be achieved. This method is particularly suitable for processing large or deeply nested file structures, and effectively improves parsing efficiency and resource utilization.
[0046] Step S206, extracting subsystems from the model file according to actual tags, so that the subsystems can be processed concurrently by multiple threads.
[0047] Specifically, the server also includes a multi-threaded concurrent model. In view of the challenges of large-scale model file processing, it is particularly important to adopt a divide-and-conquer strategy. By carefully analyzing the subsystem structure of the model file, the original model file is intelligently split according to the subsystem boundary. The subsystem includes multiple byte streams in the model file that implement, for example, partial simulation functions. Therefore, the server determines the location of each starting state of the actual label in the model file and the corresponding terminal state, and then extracts the byte streams between the starting state and the terminal state as subsystems through the state machine model to obtain multiple subsystems. This strategy can not only greatly reduce the burden of a single process, but also make full use of the powerful parallel processing capabilities of modern multi-core server processors. Therefore, efficient multi-threading technology is used to assign independent parsing tasks to each subsystem, so that these tasks are executed concurrently on different processor cores without affecting each other, thereby significantly accelerating the parsing process of the overall function of the model file.
[0048] In one embodiment, each thread is responsible for parsing its assigned subsystem and generating corresponding intermediate parsing results. After completing the concurrent parsing of each subsystem, it enters the key summary phase to integrate the parsing results of each thread to build a unified and high-level model view.
[0049] In one embodiment, the server considers possible future structural changes and new parsing requirements of different models, and can be expanded and customized to meet the needs of different application scenarios. The models include a state machine model and a multi-threaded concurrent model.
[0050] In one embodiment, the server can efficiently set up test scenarios and verify system performance by accurately parsing the details of each model, such as setting up modeling and testing technologies related to various industries, thereby accelerating the system integration process. At the same time, it performs automated checks on design specifications, such as naming specifications, structural rationality, parameter consistency, etc., to improve the standardization level and quality consistency of the model.
[0051] In one embodiment, the server can achieve cross-team collaborative design and improve the maintainability and reusability of the model by integrating the documents and annotations in the model and combining them with custom code analysis.
[0052] In one embodiment, the server integrates the present application with the platform through third-party tools, promotes effective interaction between model file information and a diverse tool set, supports multi-dimensional analysis and decision-making in complex system design, and cross-platform operability.
[0053] In the above-mentioned model file processing method, the state of the model file is identified through the state machine model, multiple tags and the attributes associated with each tag are determined, and the nesting level of each tag is tracked based on the data stack structure, so that when any attribute in the model file is parsed, the actual tag in the corresponding actual nesting level is determined, and the model file is extracted from the subsystem according to the actual tag, so that the subsystem can be processed concurrently by multiple threads. Therefore, by directly parsing the internal structure of model files in different data formats, flexible and efficient model information extraction is achieved, and the state machine model and multi-threaded concurrent processing strategy ensure that even large-scale model files can realize the identification of actual tags in the actual nesting level and efficient parsing of the extracted subsystem, which improves the accuracy and efficiency of model file parsing and reduces system resource consumption.
[0054] In addition, since the present invention can be applied in different fields, it can help improve the efficiency of design verification, the quality of teaching and training, the agility of software development, the smoothness of system integration, and the speed of scientific research and innovation, bringing extensive benefits and value across fields.
[0055] For example, it is applied to high-tech engineering design verification and optimization: Simulink models play a core role in high-tech fields such as aerospace, automotive electronics, and robotics that have strict requirements on safety and accuracy. This invention focuses on accelerating the top-level subsystem extraction and connection analysis of complex models, greatly improving the efficiency and accuracy of engineering design verification. This not only ensures the correctness of the control logic and the reliability of the design, but also significantly reduces the cost and time of relying on physical prototype testing, and promotes a faster and more economical design iteration process.
[0056] Application in teaching and training innovation: In the higher education system, Simulink is widely used as a teaching tool in control system and complex system design courses. This invention helps students intuitively understand the complex system architecture and quickly grasp the relationship between top-level subsystems by simplifying the high-level analysis of the model, effectively enhancing students' practical ability and the efficiency of theoretical knowledge conversion, significantly improving teaching interactivity and course completion quality, and accelerating the learning process from theory to practice.
[0057] Efficiency improvement in software development and testing: In response to the demand for rapid iteration in the software development industry, the fast parsing technology provided by the present invention has become the key to optimizing Simulink model version management and testing. It not only accelerates the compatibility check between the old and new model versions, but also efficiently extracts top-level subsystem information, identifies potential software defects in advance, promotes continuous code optimization, significantly reduces software release risks, and improves product quality and market response speed.
[0058] Seamless connection for system integration: In the integration projects of automation, Internet of Things and communication systems, efficient interconnection between multiple subsystems is the key to successful deployment. This invention greatly simplifies the system integration work by quickly parsing the top-level subsystem relationship and optimizing the configuration process, improves the interoperability between components, ensures the overall coordination of the system, greatly shortens the project deployment cycle, and improves customer satisfaction and market competitiveness.
[0059] Accelerator for scientific research simulation and simulation: Simulink is an indispensable tool for simulation and simulation in the field of scientific research. This invention helps researchers to quickly analyze the top-level system structure, efficiently extract key data, accelerate the simulation verification process of scientific hypotheses, promote the speed and quality of research discoveries, provide strong technical support for the output of scientific research results, and improve the overall efficiency and innovation ability of scientific research.
[0060] Through these comprehensive applications, parsing Simulink model files not only enhances the intrinsic quality of the model, but also systematically improves the rigor and consistency of the design through an automated design specification checking mechanism, significantly reduces the time cost of manual review, accelerates project progress, and ensures that every step from design to implementation is accurate and efficient.
[0061] In one embodiment, after obtaining the model file, the method further includes: identifying the head field in the byte stream to confirm the file type of the model file; and reading a specific tag in the model file to determine the corresponding version information.
[0062] The model files include Mdl files or Slx files, and different types of files can be presented in the form of byte streams.
[0063] Specifically, Figure 3 As shown, Figure 3 The present invention is a flowchart of a general parsing method for simulink model files. When the server obtains the model file, it first performs configuration loading and parsing, and identifies the file type of the model file, such as identifying an Mdl file or an Slx file. If an unknown file is identified, the process ends directly. Then the server performs version identification and model structure analysis on the Mdl file or Slx file in turn until the multi-threaded concurrent processing of the corresponding subsystem is completed. Among them, the server includes a file type identification module and a file version identification module. Through the file type identification and file version identification modules, the universality and compatibility of simulink model file parsing are guaranteed.
[0064] The main function of the file type identification module is to read the byte stream in a part of the model file content through the pre-reading mechanism, and automatically identify the file format in combination with the text content identifier. For example, the file format identifier is described as follows: Mdl file identification is determined by whether the beginning string is "Model{", and Slx file identification is determined by whether the first four bytes are \x50\x4B\x30\x04. After the R2021B version, the description of the model parameters of the Mdl file format has changed to xml format, so it is necessary to determine the xml format of the Mdl file after determining the Slx file type, and identify it through the identifier "<?xml", and finally determine the json-like format of the Mdl.
[0065] The main function of the file version identification module is to obtain the version information of the file by reading the specific tag of the model file for use by the subsequent parsing interface. For example, the version information location and identification are described as follows: The Mdl file version information is described in the third line of the Mdl text, starting with the tag version, where the version information is described like this: 8.7, 8.8, 10.9, etc. Different version numbers correspond to different matlab versions, and the corresponding relationship is as follows Figure 4 As shown, Figure 4 This is a version correspondence diagram of an Mdl file. The Slx file version description information is located in coreProperties.xml and is obtained through the XML tag "cp:version". Its content format is as follows Figure 5 As shown, Figure 5 This is a sample version information content diagram of an XML file, which is consistent with the Mdl mapping to facilitate subsequent use.
[0066] In this embodiment, the file type and version information are determined by directly parsing the byte stream. Compared with the traditional reliance on a general parsing library, direct byte stream parsing significantly reduces the intermediate processing steps and improves parsing efficiency. In addition, by dynamically identifying version information, data feature analysis of model files of different file formats is achieved, and the general data format of the model file can be extracted, which solves the compatibility problem of model files of different versions and improves the versatility of the algorithm.
[0067] In one embodiment, the nesting level of each tag is tracked based on the data stack structure, including: for the tag of the current nesting level, when the tag is detected to be in the starting state, the tag is pushed into the corresponding data stack, and the next nesting level of the current nesting level is entered; when the tag is detected to be in the ending state, the top element of the data stack is popped out, and the previous nesting level of the current nesting level is entered.
[0068] Specifically, the "last in, first out" feature of the data stack structure is used to track the nesting level of tags. For tags at the current nesting level, when a tag's starting state is encountered, the tag-related information (such as the tag name) is pushed into the data stack, indicating that the next nesting level of the current nesting level has been entered. Conversely, when the tag is detected to be in the terminal state, the top element of the data stack is popped out, indicating that the parsing returns to the previous nesting level. Therefore, the top element of the data stack always represents the deepest level of the tag currently being parsed.
[0069] In this embodiment, the nesting level is managed by pushing and popping tags into and out of the data stack, so that during the dynamic parsing process, the nesting position of each tag can be accurately tracked and adapted to different inputs and structural changes. Therefore, the accuracy, efficiency and reliability of nested structure processing are improved, the code logic is simplified, and the maintainability and fault tolerance of the system are enhanced.
[0070] In one embodiment, when parsing any attribute in the model file, the actual label in the corresponding actual nested level is determined, including: when parsing any attribute in the model file, the corresponding actual label is determined according to the top element of the data stack at the current moment; the depth of the data stack at the current moment is used as the actual nested level corresponding to the actual label.
[0071] Specifically, the server needs to make an attribution determination when parsing any attribute parameter. By checking the top element of the data stack, it can be clear which level of label the attribute belongs to. The depth of the data stack at the current moment directly reflects the current nesting depth, ensuring that the attribute parameter can be correctly associated with the actual label structure to which it belongs. Therefore, through systematic data stack management and nesting level determination, the accuracy, efficiency and maintainability of the parsing process are improved, which is suitable for processing model files with complex nested structures.
[0072] In one embodiment, subsystem extraction is performed on a model file according to actual labels, including: determining whether a currently parsed block label structure is a subsystem to be extracted according to block labels and field information associated with the block labels; if so, extracting subsystems from the model file through a state machine model and system labels.
[0073] Among them, the actual labels include system labels and block labels; Figure 6 As shown, Figure 6 This is a sample text content diagram when extracting a subsystem; system labels are like System labels, and block labels are like Block labels.
[0074] Specifically, the subsystem extraction steps can be referred to Figure 6 The server identifies whether the currently parsed block tag structure is a subsystem by looking at the Block tag and whether the BlockType in the field information associated with the block tag is SubSystem. If so, the server combines the state machine model and when switching to the terminal state of the Block tag of the current subsystem, it can extract the subsystem content between the starting state ("System{") and the terminal state ("}") of the system tag, and finally distribute the subsystem content to the task thread.
[0075] In this embodiment, the block tag and related field information are used to determine whether the parsed block tag structure is a subsystem to be extracted, which helps to more accurately identify and locate the subsystem and reduce the possibility of misjudgment. Combining the state machine model and system tags, the subsystem extraction in the model file can be automatically processed, reducing the need for manual intervention and improving processing efficiency.
[0076] In one embodiment, subsystem extraction is performed on a model file through a state machine model and a system tag, including: when a system tag is identified as a starting state, the model file is switched to system parsing, and when a block tag associated with the system tag is identified as a termination state, block parsing is performed; when a line tag associated with a block tag is identified as a starting state, switching to line parsing, and when a system tag is identified as a termination state and a previous state is the termination state of the line tag, block association is performed; and the content of the subsystem between the starting state and the termination state of the system tag is extracted.
[0077] The actual label also includes a line label; a line label is a line label. Figure 7 As shown, Figure 7 The state machine state transition diagram for the model file.
[0078] Specifically, the server pre-reads the model file, and when the system tag is identified as the starting state, the model file is switched to system parsing; the model file is continued to be pre-read and when the block tag is identified as the starting state, the model file is switched to block parsing, and according to the block tag and the field information associated with the block tag, it is determined whether the currently parsed block tag structure is the subsystem to be extracted. If so, the process of returning to the process of identifying the system tag as the starting state continues until it is determined that the currently parsed block tag structure is the subsystem to be extracted; if not, the block tag structure is identified in a loop until the block tag associated with the system tag is identified as the termination state, and block parsing is performed.
[0079] The server continues to pre-read the model file. When it recognizes that the line tag associated with the block tag is in the starting state, it switches to line parsing and loops to identify the line tag structure until it recognizes the tag of the ending state and the line tag in the current state is in the starting state, then it exits the loop. When the server recognizes that the system tag is in the ending state and the previous state is the ending state of the line tag, it performs block association and ends the parsing process when it determines that the current parsing level is 0. Otherwise, it returns to the process of identifying the block tag associated with the system tag as the ending state and continues. Finally, the server extracts the content of the subsystem between the starting state and the ending state of the system tag.
[0080] In this embodiment, by identifying specific tags and switching different parsing models according to the start and end states of these tags, adaptability is achieved, and the most appropriate parsing strategy can be used according to different contexts. By distinguishing between system parsing, block parsing, and line parsing, the system can break down complex parsing tasks into smaller, more manageable parts, thereby improving the efficiency and accuracy of parsing.
[0081] In one embodiment, to enable multi-threaded concurrent processing of the subsystem, the method includes: creating a thread pool and determining whether the number of tasks of the subsystem is less than the number of waiting threads in the thread pool; if so, performing multi-threaded concurrent processing of the subsystem based on the number of waiting threads; if not, performing multi-threaded concurrent processing of the subsystem based on the task queue and the updated thread pool.
[0082] Among them, the number of tasks is less than the maximum number of concurrent threads limited by the thread pool.
[0083] Among them, in the concurrent parsing scenario of processing ultra-large-scale models, when the thread pool is not used to manage threads, frequent creation and destruction of threads will introduce significant resource overhead problems. This is because the initialization and termination operations of threads themselves consume considerable computing resources, which may lead to the attenuation of program performance. Especially under high load conditions, uncontrolled thread generation not only increases the burden of system scheduling, but also may cause fierce resource competition and frequent context switching, both of which are the culprits for slowing down system response speed and reducing overall efficiency. The thread pool mechanism is introduced to solve the above dilemma. The thread pool pre-creates and maintains a certain number of threads, and limits the maximum number of concurrent threads, effectively curbing the situation of excessive resource competition. In this way, when a task arrives, there is no need to experience the delay of thread creation, and an idle thread is directly selected from the pool for execution, which significantly improves the task response speed.
[0084] Specifically, the server also includes a thread pool model. When the system starts, the size of the thread pool is initialized according to system resources (such as the number of CPU cores) and the expected workload, and a fixed number of worker threads are created to wait for task assignment as waiting threads. When a new subsystem task arrives, it is determined whether the number of subsystem tasks is less than the number of waiting threads in the thread pool. If so, the thread pool model assigns all tasks to the waiting threads. If not, it indicates that there are threads that are busy, and the unprocessed tasks of multiple subsystems are placed in the task queue for processing.
[0085] The thread that completes the current task will return to the waiting state so that it can be reassigned a new task, thereby realizing thread reuse. And when the system is shut down or the thread pool is no longer needed, the thread will be destroyed for resource cleanup. In addition, the thread pool model will dynamically adjust the size of the thread pool according to system resources and performance requirements. For example, the number of threads will be reduced when the load is low and increased when the load is high.
[0086] In this embodiment, by implementing the thread pool strategy, not only can the degree of concurrency be reasonably controlled and the performance bottleneck caused by improper thread management be avoided, but also the system can be ensured to run with higher efficiency and more stable performance when processing resource-intensive tasks such as super-large models. At the same time, after the task is completed, the thread is not destroyed, but returns to the pool to wait for the next reuse, which reduces system overhead and enhances the stability and execution efficiency of the program.
[0087] In one embodiment, subsystem multi-threaded concurrent processing is performed based on the task queue and the updated thread pool, including: placing unprocessed tasks of multiple subsystems into the task queue; updating the thread pool according to the length of the task queue and the busy status of the threads in the thread pool to obtain a new thread pool; returning to the process of determining whether the number of unprocessed subsystem tasks is less than the number of waiting threads in the new thread pool until all subsystem multi-threaded concurrent processing is completed.
[0088] Specifically, the server uses a first-in-first-out or priority queue to manage pending tasks. When tasks are encapsulated as task objects and submitted to the thread pool model, it is also a thread pool manager. The thread pool manager adds tasks to the task queue, and the worker thread periodically checks the task queue, obtains and executes tasks in the queue, such as using conditional variables or semaphores to notify the worker thread of the arrival of new tasks. The server monitors the length of the task queue to evaluate the system load and performance bottlenecks, and dynamically adjusts the thread pool size using performance counters and feedback mechanisms based on the length of the task queue and the busy state of the thread to obtain a new thread pool. Next, the server continues to execute the process of whether the number of unprocessed subsystem tasks is less than the number of waiting threads in the new thread pool, and the thread takes the task out of the task queue and executes the logic of the task until all subsystems are completed. Multi-threaded concurrent processing.
[0089] Similarly, if the server uses a priority queue to manage pending tasks, the order in which tasks are processed can be adjusted according to their priorities. To ensure that the operation of the task queue is thread-safe, the server also uses mutexes or other synchronization mechanisms to prevent race conditions. The server also provides a mechanism to cancel tasks that have not been processed for a long time or to process timed-out tasks, and after the task is completed, send feedback to the thread pool manager for task statistics and performance monitoring.
[0090] In this embodiment, by dynamically adjusting the size of the thread pool according to the length of the task queue and the busy state of the threads in the thread pool, system resources can be used more effectively. This can prevent the thread pool from being too crowded or idle, thereby improving processing efficiency and system throughput, and ensuring that all subsystem multi-threaded concurrent processing can be completed efficiently in the future.
[0091] In one embodiment, the server also includes a configuration management module, which is intended to enhance the flexibility and adaptability of the system and provides a mechanism for customizing parsing content and parsing rules. It mainly includes configuration file formats: using formats such as JSON, XML or YAML to define a standardized configuration file structure to ensure the readability and flexibility of the configuration. It also includes a configuration parser: implementing a configuration parser to read and parse the configuration file and converting user-defined configuration items into an internal format recognizable by the system. It also includes a configuration manager class: designing a configuration manager class, which is responsible for loading, verifying, applying and updating the configuration to ensure the correctness and effectiveness of the configuration.
[0092] In one embodiment, the server also includes an exception module. The exception management module is a core component of the system robustness, focusing on improving the resilience of the program and user experience. It comprehensively covers various possible abnormal situations, including but not limited to key links such as unrecognizable file types, file version information parsing errors, failures in the file parsing process, thread initialization failures, and memory allocation failures. Through refined exception capture and intelligent processing strategies, the module ensures that when encountering unexpected situations, the program can respond calmly and avoid crashes, while providing users with clear and friendly error prompts. This not only strengthens the robustness of the software, but also greatly improves the user's confidence and satisfaction when operating in complex or changing environments. It is a key line of defense to ensure the smooth operation of the application.
[0093] In one embodiment, the server also includes a log module, which is a key component in the software for recording runtime information, which helps developers monitor program status, debug problems, and analyze runtime behavior. The following is a detailed description of log levels and log formats.
[0094] (1) Log levels include DEBUG (debug), which is used to provide detailed technical information and is usually used in the development phase or for problem diagnosis. The DEBUG level records the internal state of the program and variable values to help developers understand the execution flow of the program. Examples: recording variable values, function calls, and internal decision points of the program. It also includes INFO (information), which is used to record the normal operating status and key milestones of the program. The INFO level is suitable for production environments and provides an overview of program operation. Examples: program startup, configuration loading completion, and normal operating status updates. It also includes WARNING (warning), which is used to indicate potential problems or unexpected events, but these events will not cause the program to stop running. The WARNING level alerts users or developers to issues that may require attention. Examples: configuration errors, resource usage approaching the upper limit, performance degradation, etc. It also includes ERROR (error), which is used to record serious error events that may cause program malfunction or failure. The ERROR level usually requires immediate attention and processing. Examples: inability to access critical resources, violation of business rules, system crash, etc.
[0095] (2) The log format includes a timestamp. Format: Usually the ISO 8601 format is used, such as YYYY-MM-DDTHH:MM:SS.sssZ, which indicates the exact time of the log entry. Function: Ensure the time order and time zone information of the log entries. Also includes the log level. Format: Indicated in uppercase letters, such as DEBUG, INFO, WARNING, ERROR. Function: Quickly identify the severity of the log entry. Also includes the message content. Format: A concise and clear description that includes enough information to understand what happened. Function: Provide a detailed description and context of the event. Also includes the thread or process identifier. Format: Optional, used to distinguish the source of the log in a multi-threaded or distributed system. Function: Identify the specific thread or process that generates the log in a concurrent environment. Also includes the log source. Format: Optional, including class name, method name or module name. Function: Quickly locate the code location where the log is generated. Also includes the error code or exception information (only applicable to the ERROR level). Format: Optional, including error code, exception type or stack trace. Function: Provide detailed information required for error handling and diagnosis. It also includes the unique identifier of the log entry. Format: optional, such as log ID or sequence number. Purpose: To track a specific log entry in a large number of logs.
[0096] In one embodiment, when the requirements for parsing performance are not high, a mature and stable XML parsing library can be used to parse the SLX model file, thereby reducing the complexity of the technical solution implementation. At the same time, for the extraction and processing of text strings in the byte stream, regular expressions can be used to quickly match and obtain them.
[0097] In one embodiment, the API provided by MATLAB / Simulink is used. If the user already has MATLAB authorization, the API provided by MATLAB / Simulink can be directly used to parse the model file, so that the ecosystem and tool set of MATLAB can be fully utilized.
[0098] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0099] Based on the same inventive concept, the embodiment of the present application also provides a model file processing device for implementing the model file processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model file processing device embodiments provided below can refer to the limitations of the model file processing method above, and will not be repeated here.
[0100] In one embodiment, a model file processing device is provided, comprising: a state machine model module, a data stack structure module and a multi-thread processing module, wherein:
[0101] The state machine model module is used to obtain a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each of the tags.
[0102] The data stack structure module is used to track the nesting level of each of the tags based on the data stack structure, and determine the actual tag in the corresponding actual nesting level when parsing any attribute in the model file.
[0103] The multi-thread processing module is used to extract subsystems from the model file according to the actual tags, so that the subsystems can be processed in a multi-threaded concurrent manner.
[0104] Each module in the above model file processing can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.
[0105] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store model files. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model file processing method is implemented.
[0106] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0107] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0108] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0109] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.
[0110] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.
[0111] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0112] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A model file processing method, characterized in that: The method comprises: Acquire a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each of the tags; Tracking the nesting level of each of the tags based on the data stack structure, and determining the actual tag in the corresponding actual nesting level when parsing any attribute in the model file; Subsystem extraction is performed on the model file according to the actual tag, so that the subsystem is processed concurrently by multiple threads.
2. The method according to claim 1, characterized in that The model file includes a byte stream; after obtaining the model file, the following further includes: Identify the beginning field in the byte stream to confirm the file type of the model file; Read the specific tag in the model file to determine the corresponding version information.
3. The method according to claim 1, characterized in that: Tracking the nesting level of each tag based on the data stack structure includes: For a tag of the current nesting level, when it is detected that the tag is in the starting state, the tag is pushed into the corresponding data stack, and the next nesting level of the current nesting level is entered; When it is detected that the tag is in the termination state, the top element of the data stack is popped out, and the process enters the previous nesting level of the current nesting level.
4. The method according to claim 3, characterized in that When parsing any attribute in the model file, determining the actual label in the corresponding actual nesting level includes: When parsing any attribute in the model file, determining the corresponding actual label according to the top element of the data stack at the current moment; The depth of the data stack at the current moment is used as the actual nesting level corresponding to the actual tag.
5. The method according to claim 1, characterized in that The actual label includes a system label and a block label; and extracting the subsystem from the model file according to the actual label includes: Determining whether the currently parsed block tag structure is a subsystem to be extracted according to the block tag and field information associated with the block tag; If so, subsystem extraction is performed on the model file through the state machine model and the system tag.
6. The method according to claim 5, characterized in that The actual label also includes a line label; and the subsystem extraction of the model file through the state machine model and the system label includes: When the system tag is identified as a starting state, the model file is switched to system parsing, and when the block tag associated with the system tag is identified as a termination state, block parsing is performed; When it is recognized that the line tag associated with the block tag is in the starting state, switching to line parsing, and when it is recognized that the system tag is in the ending state and the previous state is the ending state of the line tag, executing block association; The content of the subsystem between the start state and the end state of the system tag is extracted.
7. The method according to claim 1, characterized in that The method of enabling the subsystem to process multi-threaded concurrently includes: Create a thread pool, and determine whether the number of tasks of the subsystem is less than the number of threads in the waiting state in the thread pool; the number of tasks is less than the maximum number of threads limited by the thread pool for concurrency; If so, performing multi-thread concurrent processing on the subsystem based on the number of threads in the waiting state; If not, perform subsystem multi-threaded concurrent processing based on the task queue and the updated thread pool.
8. The method according to claim 7, characterized in that The subsystem multi-threaded concurrent processing according to the task queue and the updated thread pool includes: Putting unprocessed tasks of the plurality of subsystems into a task queue; The thread pool is updated according to the length of the task queue and the busy state of the threads in the thread pool to obtain a new thread pool; Return to the process of determining whether the number of unprocessed subsystem tasks is less than the number of waiting threads in the new thread pool until all subsystem multi-threaded concurrent processing is completed.
9. A model file processing device, characterized in that: The device comprises: A state machine model module is used to obtain a model file, and perform state recognition on the model file through a state machine model to determine multiple tags and attributes associated with each of the tags; A data stack structure module, used for tracking the nesting level of each of the tags based on the data stack structure, and determining the actual tag in the corresponding actual nesting level when parsing any attribute in the model file; The multi-thread processing module is used to extract subsystems from the model file according to the actual tags, so that the subsystems can be processed in a multi-threaded concurrent manner.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Data exchange system of nested tag structure
CN120162466A
A data exchange system with a nested tag structure
CN120162466B
