State machine-based streaming data parsing method, system, device and storage medium

Abstract

The application provides a state machine-based streaming data analysis method, system, device and storage medium. The method comprises: receiving packet streaming data; initializing a state machine and performing character-by-character analysis on the streaming data; when the analysis state of the state machine is a text state, if the analyzed character is a label start symbol, switching the analysis state of the state machine to a label state; when the analysis state of the state machine is the label state, storing the analyzed character in a label buffer until the analyzed character is a label end symbol, ending the character analysis of the current streaming data; querying a predefined configuration mapping table based on the string content of the label buffer to determine a label type; performing a data block operation based on the label type and dynamically appending the recognized non-label character to the current active data block. The application improves the accurate analysis rate of streaming data and the multiplicity and maintainability of the scheme.

Description

State machine-based streaming data parsing methods, systems, devices, and storage media Technical Field

[0001] This application belongs to the field of computer science and technology, and in particular relates to a streaming data parsing method, system, device and storage medium based on state machine. Background Technology

[0002] Streaming data processing technology is widely used in scenarios such as real-time communication, online analysis, and AI result delivery. However, existing technologies still have significant limitations when processing streaming data containing structured tags. During streaming transmission, data is split into multiple segments. Existing parsing methods typically employ a line-by-line reception mechanism, lacking the ability to uniformly identify and reassemble tag content, making it difficult to handle scenarios where the server pushes data packets in segments. Specifically, the main technical problems of existing parsing methods are as follows:

[0003] First, existing parsing methods are difficult to flexibly adapt to newly added tag types and cannot expand the range of tag support without changing the core parsing logic. This means that the parsing process needs to be modified every time a new tag is added or the data block structure is adjusted, resulting in poor maintainability and reusability. Second, when tag identifiers or content are transmitted across multiple stream data segments, common processing methods based on regular expression matching or one-time caching cannot correctly identify and concatenate them, leading to missing tag identification or disordered content concatenation, which in turn leads to parsing failure. Summary of the Invention

[0004] The purpose of this application is to provide a state machine-based method, system, device, and storage medium for parsing streaming data, which solves the technical problems in existing parsing methods, such as incomplete tag recognition, disordered content splicing, and poor reusability caused by the need to modify the parsing process when adding new tag types, due to packet splitting during the transmission of streaming data.

[0005] Firstly, this application provides a streaming data parsing method based on a state machine, the method comprising:

[0006] Receive streaming data including tag start and tag end characters;

[0007] The state machine is initialized, and the streaming data is parsed character by character based on the initialized state machine. The parsing state of the state machine includes a text state and a tag state. When the parsing state of the state machine is in the text state, if the parsed character is the tag start character, the parsing state of the state machine is switched to the tag state. When the parsing state of the state machine is in the tag state, the parsed character is stored in the tag buffer until the parsed character is the tag end character, at which point the character parsing of the current streaming data ends.

[0008] The tag type is determined by querying a predefined configuration mapping table based on the string content of the tag buffer;

[0009] Data block operations are performed based on the label type, and the identified non-label characters are dynamically appended to the currently active data block.

[0010] In some embodiments, the step of ending character parsing of the current streaming data when the parsing state of the state machine is the tag state, until the parsed character is the tag end character, includes:

[0011] If the parsed current streaming data segment does not include a tag terminator, the character content in the tag buffer is retained and the tag state remains unchanged;

[0012] When a new streaming data segment is received, the new character continues to be appended to the tag buffer in the tag state;

[0013] Repeat the above operation until the parsed character is a tag terminator, then end the character parsing of the current streaming data.

[0014] In some embodiments, determining the tag type by querying a predefined configuration mapping table based on the string content of the tag buffer includes:

[0015] The string content in the tag buffer is matched with a predefined configuration mapping table, and the tag type corresponding to the string content in the tag buffer is determined according to the matching result; wherein, the configuration mapping table is a parsing logic storage independent of the state machine, used to store the correspondence between tag types and operation types.

[0016] In some embodiments, performing data block operations based on the tag type and dynamically appending the identified non-tag characters to the currently active data block includes:

[0017] Create a data block stack to manage the hierarchical relationship of nested active data blocks;

[0018] When the tag type is identified as open tag, a corresponding new active data block is created and the new active data block is pushed onto the top of the data block stack. At the same time, the new active data block is set as the current active data block.

[0019] When a closed label is detected, the top data block of the data block stack is popped, and the new top data block after popping is restored as the currently active data block.

[0020] Non-label characters are always appended to the currently active data block to maintain the hierarchical relationship and content ownership between nested active data blocks.

[0021] In some embodiments, after performing the data block operation based on the tag type, the method further includes:

[0022] The hierarchical relationship of nested active data blocks is detected in real time. When an error in the nesting order of tags in a nested active data block is detected, the nested active data block is processed according to the nearest matching principle and / or the hierarchical backtracking principle. The nearest matching principle matches the closing tag with the nearest matching open tag at the top of the stack, and the hierarchical backtracking principle backtracks down along the unclosed tag stack to find the first matching open tag. The unclosed tag stack is used to record open tags that have been identified but have not yet been matched with a corresponding closing tag.

[0023] If there are still unclosed tags when the streaming data parsing ends, the remaining open tags in the unclosed tag stack will be automatically used to generate corresponding closed tags, or the characters parsed by the streaming data will be assigned to the active data block of the previous valid level.

[0024] In some embodiments, after performing the data block operation based on the tag type, the method further includes:

[0025] Output the currently active data block, clear the label buffer, and switch the parsing state of the state machine to the text state. In the text state, continue to parse the input streaming data character by character.

[0026] In some embodiments, the method further includes: performing collaborative parsing and synchronization control on multiple independent streaming data sources, with the following specific steps:

[0027] Create and maintain an independent state machine instance and data block context stack for each streaming data source;

[0028] Define a synchronization tag in the streaming data source; wherein the synchronization tag is used to identify a synchronization point that needs to be reached by multiple streaming data sources;

[0029] When any streaming data source parses a synchronization tag, it records that the streaming data source has reached the corresponding synchronization point and pauses the output of the active data block corresponding to the streaming data source;

[0030] When all specified streaming data sources reach the same synchronization point, a collaborative callback function is triggered to execute subsequent collaborative rendering.

[0031] Secondly, this application provides a streaming data tag parsing system, comprising:

[0032] The data receiving module is used to receive streaming data;

[0033] A state machine parser is used to parse streaming data according to the above method;

[0034] The data block management module is used to dynamically splice and manage tag content;

[0035] The front-end rendering module is used to output the parsing results to the front-end rendering through a callback interface.

[0036] Thirdly, this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the above-described method.

[0037] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0038] As described above, the state machine-based streaming data parsing method, system, device, and storage medium of this application have the following beneficial effects:

[0039] This application, based on a state machine state switching mechanism and a tag buffer mechanism, ensures that streaming data can still be completely spliced ​​together when it is split into multiple data segments, enabling accurate parsing of the streaming data. This solves the problem of missing tag identification or content splicing errors caused by multiple data segments crossing packets in the prior art, which leads to parsing failure. At the same time, this application also sets a predefined configuration mapping table, so that when new tag types of streaming data appear in the system, there is no need to modify the core parsing logic code, only the entries in the configuration mapping table need to be changed, thus improving the reusability and maintainability of the solution. Attached Figure Description

[0040] Figure 1 shows a flowchart of a state machine-based streaming data parsing method provided in an embodiment of this application.

[0041] Figure 2 shows a flowchart of a state machine-based streaming data parsing method provided in an exemplary embodiment of this application.

[0042] Figure 3 shows a flowchart of a method for performing data block operations based on label type and dynamically appending identified non-label characters to the currently active data block, as provided in an exemplary embodiment of this application.

[0043] Figure 4 shows a flowchart of a method for collaborative parsing and synchronization control of multiple independent streaming data sources, provided in an exemplary embodiment of this application.

[0044] Figure 5 shows a structural diagram of a streaming data tag parsing system provided in an exemplary embodiment of this application.

[0045] Figure 6 shows the splicing process of the active data block corresponding to the label name ANALYSIS in an exemplary embodiment of this application.

[0046] Figure 7 shows the structure of the electronic device.

[0047] Explanation of reference numerals in the instruction manual:

[0048] 1. Data receiving module; 2. State machine parser; 3. Data block management module; 4. Front-end rendering module; 20. Electronic device; 21. Processor; 22. Memory; 23. Output interface; 24. Communication interface; 25. Antenna. Detailed Implementation

[0049] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0050] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0051] In modern computer networks, streaming data transmission is a common data exchange method, especially in real-time data processing and big data technologies. In many practical applications, such as real-time data streams, logging systems, and network communications, data is frequently transmitted in streaming form. For example, in IoT systems, devices may continuously send sensor data; or in online chat applications, servers may push chat messages in real time. This data is typically transmitted as a stream via SSE, WebSocket, or TCP protocols. However, during transmission, data loss due to incorrect identification or misinterpretation leading to content splicing errors often occur. Furthermore, when new data streams appear in the system, incompatibility with the core parsing logic often necessitates modifications to the parsing logic, resulting in wasted resources and high maintenance costs.

[0052] To address the aforementioned technical problems, this application proposes a state machine-based streaming data parsing method, system, device, and storage medium. Based on the state machine's state transition mechanism and tag buffer mechanism, it ensures that streaming data, even when split into multiple data segments, can still be completely reassembled, enabling accurate parsing. This solves the parsing failure problem in existing technologies caused by missing tag identification or content splicing errors due to multiple data segments crossing packets. Furthermore, this application sets a predefined configuration mapping table, so that when new tag types of streaming data appear in the system, there is no need to modify the core parsing logic code; only the entries in the configuration mapping table need to be changed, improving the reusability and maintainability of the solution.

[0053] As shown in Figure 1, this embodiment provides a streaming data parsing method based on a state machine, which includes:

[0054] Step 100: Receive streaming data including tag start character and tag end character.

[0055] In some embodiments, the streaming data is obtained from a server. This server can be an SSE server, a WebSocket server, or a TCP persistent connection server.

[0056] In some embodiments, the streaming data is transmitted via SSE, WebSocket, or TCP long connection protocols.

[0057] In some embodiments, the tag start character can be represented by "[" and the tag end character can be represented by "]".

[0058] Step 200: Initialize the state machine. Based on the initialized state machine, parse the streaming data character by character. The parsing state of the state machine includes a text state and a tag state. When the parsing state of the state machine is in the text state, if the parsed character is the tag start character, then switch the parsing state of the state machine to the tag state. When the parsing state of the state machine is in the tag state, store the parsed character in the tag buffer until the parsed character is the tag end character, then end the character parsing of the current streaming data.

[0059] In some embodiments, the initial parsing state of the state machine is set to a text state by setting the state variable of the state machine parser to a text state identifier, while clearing the label buffer. This ensures that the state machine parser starts working from a non-label state, thus solving the label misjudgment problem caused by the unclear starting point of streaming data parsing in the prior art. At the same time, parsing is performed starting from the text state to complete the switching between the text state and the label state, ensuring that the label content is fully recognized, avoiding misjudgment of the label content, and reducing the accuracy of the parsing results.

[0060] In some embodiments, when the parsing state of the state machine is in the tag state, the character parsing of the current streaming data ends when the parsed character is a tag end character, including: if the parsed current streaming data segment does not include a tag end character, then the character content in the tag buffer is retained and the tag state remains unchanged; when a new streaming data segment is received, the new character is continued to be appended to the tag buffer in the tag state; the above operation is repeated until the parsed character is a tag end character, at which point the character parsing of the current streaming data ends.

[0061] For example, a streaming data segment is a fragment of streaming data.

[0062] This embodiment completes the tag content splicing of cross-packet streaming data segments by setting tag start and tag end characters, thereby ensuring that streaming data can still be completely parsed in the case of cross-packet transmission and improving the parsing accuracy of streaming data.

[0063] This embodiment detects the tag start character in text mode and switches to tag mode to continue character parsing, breaking through the data packet boundary limitation and realizing unified recognition of tag content in streaming data. It ensures that streaming data can still be completely spliced ​​when it is split into multiple data fragments, and achieves accurate parsing of streaming data. Compared with the data splicing method of using regular expression matching or one-time caching in the existing technology, this embodiment improves the success rate of cross-packet parsing of streaming data fragments from 85% to nearly 100%.

[0064] Specifically, as shown in Figure 2, step 211: initialize the parsed state of the state machine to the text state.

[0065] Step 212: In this text state, parse the streaming data character by character.

[0066] Step 213: Determine whether the parsed character is a tag start character.

[0067] Step 214: If it is a tag starter, switch the parsing state of the state machine to the tag state and start the tag buffer. If it is not a tag starter, return to step 212.

[0068] Step 215: In the tag state, store the parsed characters into the tag buffer.

[0069] Step 216: Determine whether the parsed character is a tag terminator.

[0070] Step 217: If yes, end the character parsing of the current streaming data. If no, return to step 215.

[0071] In this embodiment, the state machine successfully collects the complete string between "[" and "]", such as "[ANALYSIS]" or "[ / ANALYSIS]", in the tag state, thus avoiding the omission or misidentification of tag content in streaming data.

[0072] Step 300: Determine the tag type by querying the predefined configuration mapping table based on the string content of the tag buffer.

[0073] In some embodiments, the process of determining the tag type by querying a predefined configuration mapping table based on the string content of the tag buffer includes: matching the string content in the tag buffer with the predefined configuration mapping table, and determining the tag type corresponding to the string content in the tag buffer based on the matching result; wherein, the configuration mapping table is a parsing logic storage independent of the state machine, used to store the correspondence between tag types and operation types.

[0074] Specifically, the parser maintains an extensible mapping relationship internally, and this configuration mapping table defines "what kind of tag type" corresponds to "what kind of operation".

[0075] For example, the state machine is the core logic engine of the parser, and the parser is the functional carrier of the state machine. The parser adapts to data input from various streaming communication protocols. The parser adapts to different protocol data input formats through a protocol adapter layer, including data fragmentation, encoding conversion, and error detection.

[0076] This embodiment achieves zero-code intrusion extension by setting a configuration mapping table. When adding streaming data with new tag types, only an entry needs to be added to the configuration mapping table, without modifying the state machine parsing code. This improves the reusability and maintainability of the solution. Existing technologies may take 3+ hours to process streaming data with new tag types, while this embodiment only takes about 5 minutes, significantly reducing the processing time and improving the operational efficiency of this application.

[0077] Step 400: Perform data block operations based on the label type, and dynamically append the identified non-label characters to the currently active data block.

[0078] In some embodiments, as shown in FIG3, the data block operation based on the tag type, and the dynamic appending of the identified non-tag characters to the currently active data block, includes:

[0079] Step 411: Create a data block stack to manage the hierarchical relationship of nested data blocks.

[0080] For example, before the parser parses the streaming data, a data block stack is created, which is initially an empty stack.

[0081] Step 412: When the tag type is identified as open tag, create a corresponding new active data block and push the new active data block onto the top of the data block stack, and set the new active data block as the current active data block.

[0082] Step 413: When the label type is identified as a closed label, pop the top data block of the data block stack and restore the new top data block after popping to the currently active data block.

[0083] Step 414: Non-label characters are always appended to the current active data block to maintain the hierarchical relationship and content ownership between nested active data blocks.

[0084] For example, the opening tag can be represented by "[ANALYSIS]", and the closing tag can be represented by "[ / ANALYSIS]". The opening tag can also be represented by "[SUMMARY]", and the closing tag can also be represented by "[ / SUMMARY]". In this embodiment, the opening and closing tags can be dynamically expanded to multiple strings, which will not be elaborated on here. Among them, ANALYSIS and SUMMARY correspond to the tag names.

[0085] This embodiment achieves accurate parsing of complex nested structures through the creation and maintenance of data block stacks. At the same time, this embodiment ensures accurate character attribution in streaming data by dynamically switching the current active block, with a content attribution accuracy rate of 99%+.

[0086] In some embodiments, performing data block operations based on the label type and dynamically appending the identified non-label characters to the current active data block further includes: outputting the current active data block, clearing the label buffer, and switching the parsing state of the state machine to the text state, in which the input streaming data is parsed character by character.

[0087] In some embodiments, after performing data block operations based on the tag type, the method further includes: real-time detection of the hierarchical relationship of nested active data blocks; when an error in the nesting order of tags in a nested active data block is detected, processing the nested active data block according to the nearest matching principle and / or the hierarchical backtracking principle; wherein, the nearest matching principle is to match the closing tag with the nearest matching open tag at the top of the stack, and the hierarchical backtracking principle is to backtrack down along the unclosed tag stack to find the first matching open tag; wherein, the unclosed tag stack is used to record open tags that have been identified but have not yet matched with the corresponding closing tag; if there are still unclosed tags at the end of the streaming data parsing, the remaining open tags in the unclosed tag stack are automatically used to generate the corresponding closing tags, or the characters parsed by the streaming data are assigned to the active data block of the previous valid level.

[0088] Specifically, tag nesting order refers to the correct matching relationship between opening and closing tags in the hierarchical structure. Its core meaning is: the inner tag must close before the outer tag closes, forming a strict tree structure. If this rule is violated, it is called a nesting order error.

[0089] In some embodiments, an unclosed label refers to a closed label that has not been found to match the open label.

[0090] This embodiment monitors the nesting order of opening tags in real time through an unclosed tag stack. When a mismatch is detected between the closing tag and the top tag of the stack, fault tolerance is triggered to avoid tag misalignment. Simultaneously, this embodiment automatically generates corresponding closing tags for the remaining opening tags in the unclosed tag stack, ensuring that all characters can find their corresponding data blocks. This avoids data errors and tag misalignment caused by unclosed tags, enabling this embodiment to output structurally complete renderable content even in abnormal scenarios such as tag misalignment, nesting breaks, and stream interruptions, improving data integrity to 85%.

[0091] Specifically, as shown in Figure 2, step 421: Determine whether the tag type of the string content in the tag buffer is an open tag or a closed tag.

[0092] Step 422: If the tag type of the string content in the tag buffer is an open tag, the parser will trigger the pre-registered onBlockOpen callback function and pass the tag name as a parameter.

[0093] For example, the label name could be ANALYSIS, STRATEGY, SUMMARY, etc.

[0094] Step 423: If the tag type of the string content in the tag buffer is a closed tag, the parser will trigger the pre-registered onBlockClose callback function and pass the tag name as a parameter.

[0095] For example, the label name could be ANALYSIS, STRATEGY, SUMMARY, etc.

[0096] Step 424: If the tag type of the string content in the tag buffer is a non-tag character, the parser will treat the character content between "[" and "]" as ordinary text, and the parser will trigger the onBlockContent callback function to append it to the current active data block.

[0097] Step 425: Output the currently active data block, clear the label buffer, and switch the parsing state of the state machine to the text state. Return to step 212.

[0098] In some embodiments, as shown in Figure 4, the method further includes: performing collaborative parsing and synchronization control on multiple independent streaming data sources, with the specific steps as follows:

[0099] Step 511: Create and maintain an independent state machine instance and data block context stack for each streaming data source.

[0100] This embodiment ensures that the parsing states of each streaming data source do not interfere with each other by setting up an independent state machine instance and data block stacks for each streaming data source. The independent data block context stack avoids the overlapping of active data blocks corresponding to multiple streaming data sources, reducing memory usage by 40%.

[0101] Step 512: Define a synchronization label in the streaming data source; wherein the synchronization label is used to identify a synchronization point that needs to be reached by multiple streaming data sources.

[0102] In some embodiments, the number and location of synchronization points can be arbitrarily configured to support complex business scenarios.

[0103] This embodiment achieves cross-stream synchronization through tag semantics rather than timestamps, avoiding timing errors caused by network latency.

[0104] Step 513: When any streaming data source resolves to the synchronization tag, record that the streaming data source has reached the corresponding synchronization point and pause the output of the active data block corresponding to the streaming data source.

[0105] This embodiment ensures that incompletely synchronized active data blocks are not rendered prematurely by pausing the output of the corresponding active data blocks from the streaming data sources that have reached the corresponding synchronization point, thus avoiding the problem of content misalignment that may occur between active data blocks from different streaming data sources.

[0106] Step 514: When all specified streaming data sources reach the same synchronization point, trigger the collaborative callback function to execute subsequent collaborative rendering.

[0107] In this embodiment, subsequent operations are performed only when all streaming data sources reach the synchronization point, ensuring the integrity of the streaming data.

[0108] The parsing method in this embodiment further supports collaborative processing of multiple independent streaming data sources, enabling synchronous data aggregation in complex scenarios, and improving the parsing accuracy from 68% in traditional solutions to 98%.

[0109] The scope of protection for the state machine-based streaming data parsing method described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0110] This application also provides a state machine-based streaming data parsing system. The state machine-based streaming data parsing system can implement the state machine-based streaming data parsing method described in this application. However, the implementation device of the state machine-based streaming data parsing method described in this application includes, but is not limited to, the structure of the state machine-based streaming data parsing system listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0111] As shown in Figure 5, this embodiment provides a streaming data tag parsing system, including: a data receiving module 1 for receiving streaming data; a state machine parser 2 for parsing streaming data according to the streaming data tag parsing method mentioned above; a data block management module 3 for dynamically concatenating and managing character content; and a front-end rendering module 4 for outputting the parsing results to the front-end rendering through a callback interface.

[0112] Here, ANALYSIS, STRATEGY, and SUMMARY represent the tag names of the activity data blocks. The analysis area is the front-end display area corresponding to the ANALYSIS block, the strategy area is the front-end display area corresponding to the STRATEGY block, and the summary area is the front-end display area corresponding to the SUMMARY block.

[0113] Figure 6 shows a schematic diagram of the splicing and restoration of streaming data corresponding to the active data block with the label name ANALYSIS in various modules of the parsing system after the data is split.

[0114] This embodiment directly binds the parsing results to the front-end display module to achieve real-time streaming rendering.

[0115] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0116] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0117] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] In one possible embodiment, as shown in FIG7, the electronic device 20 further includes: an output interface 23 for outputting results; a communication interface 24 for transmitting communication signals; and an antenna 25 for transmitting or receiving signals.

[0119] It should be noted that the processor 21 in this embodiment can be an image processing chip or an integrated circuit chip, capable of processing image signals. In implementation, each step of the above method embodiment can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0120] Those skilled in the art will understand that the structure shown in Figure 7 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0121] In one exemplary embodiment, this embodiment also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0122] In one exemplary embodiment, this embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0123] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0124] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0125] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0126] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A streaming data parsing method based on a state machine, characterized in that, The method includes: receiving streaming data including a tag start character and a tag end character; initializing a state machine and parsing the streaming data character by character based on the initialized state machine; wherein the parsing state of the state machine includes a text state and a tag state; when the parsing state of the state machine is in the text state, if the parsed character is a tag start character, the parsing state of the state machine is switched to the tag state; when the parsing state of the state machine is in the tag state, the parsed character is stored in a tag buffer until the parsed character is a tag end character, at which point the character parsing of the current streaming data ends; querying a predefined configuration mapping table based on the string content of the tag buffer to determine the tag type; performing data block operations based on the tag type and dynamically appending the identified non-tag characters to the current active data block; the step of performing data block operations based on the tag type and dynamically appending the identified non-tag characters to the current active data block includes: creating a data block stack for managing nested active data. The hierarchical relationship of blocks; when an open tag is detected, a corresponding new active data block is created and pushed onto the top of the data block stack, and the new active data block is set as the current active data block; when a closed tag is detected, the top data block of the data block stack is popped, and the popped new top data block is restored as the current active data block; non-tag characters are always appended to the current active data block to maintain the hierarchical relationship and content ownership between nested active data blocks; after performing data block operations based on the tag type, the method further includes: real-time detection of the hierarchical relationship of nested active data blocks; when an error in the nesting order of tags in a nested active data block is detected, the nested active data block is processed according to the nearest matching principle and / or the hierarchical backtracking principle; if there are still unclosed tags at the end of the streaming data parsing, the remaining open tags in the unclosed tag stack are automatically generated into corresponding closed tags, or the characters parsed by the streaming data are assigned to the active data block of the previous valid level.

2. The method according to claim 1, characterized in that, When the parsing state of the state machine is in the tag state, the character parsing of the current streaming data ends when the parsed character is a tag end character, including: if the parsed current streaming data segment does not include a tag end character, then the character content in the tag buffer is retained and the tag state remains unchanged; when a new streaming data segment is received, the new character is appended to the tag buffer in the tag state; the above operation is repeated until the parsed character is a tag end character, at which point the character parsing of the current streaming data ends.

3. The method according to claim 2, characterized in that, The step of querying a predefined configuration mapping table based on the string content of the tag buffer to determine the tag type includes: matching the string content in the tag buffer with the predefined configuration mapping table, and determining the tag type corresponding to the string content in the tag buffer according to the matching result; wherein, the configuration mapping table is a parsing logic storage independent of the state machine, used to store the correspondence between tag types and operation types.

4. The method according to claim 1, characterized in that, The nearest matching principle is to match the closed label with the nearest matching open label at the top of the stack, and the hierarchical backtracking principle is to backtrack down along the unclosed label stack to find the first matching open label; the unclosed label stack is used to record open labels that have been identified but have not yet been matched with the corresponding closed label.

5. The method according to claim 2, characterized in that, After performing data block operations based on the tag type, the method further includes: outputting the currently active data block, clearing the tag buffer, and switching the parsing state of the state machine to the text state. In the text state, the input streaming data is parsed character by character.

6. The method according to claim 1, characterized in that, The method further includes: performing collaborative parsing and synchronization control on multiple independent streaming data sources, with the following specific steps: creating and maintaining an independent state machine instance and data block context stack for each streaming data source; defining synchronization tags in the streaming data sources; wherein, the synchronization tags are used to identify synchronization points that need to be reached by multiple streaming data sources; when any streaming data source parses a synchronization tag, recording that the streaming data source has reached the corresponding synchronization point, and pausing the output of the active data block corresponding to the streaming data source; when all specified streaming data sources have reached the same synchronization point, triggering a collaborative callback function to execute subsequent collaborative rendering.

7. A streaming data tag parsing system, characterized in that, include: The data receiving module is used to receive streaming data; A state machine parser for parsing streaming data according to any one of claims 1 to 6; The data block management module is used to dynamically splice and manage tag content; The front-end rendering module is used to output the parsing results to the front-end rendering through a callback interface.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images