Data processing method and device, equipment, medium and product

Abstract

The invention provides a data processing method and device, equipment, a medium and a product, and the method comprises the steps: receiving a data stream transmitted in a dialogue interaction scene, and carrying out the streaming transmission of the data stream according to a data segment mode; in the streaming transmission process of the data fragments, integrity detection processing is carried out on the received data fragments, an integrity detection result is obtained, and the integrity detection result is used for indicating at least one target data fragment which is received and can form a data unit; constructing the at least one target data fragment into a data unit, and performing data analysis processing on the data unit; and outputting the analyzed data unit in the dialogue interaction scene. According to the data processing method and device, the data unit obtained after integrity detection can be processed in the streaming transmission process, and therefore the efficiency and accuracy of the data processing process are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, medium and product, and specifically to a data processing method, a data processing apparatus, a computer device, a computer-readable storage medium and a computer program product. Background Art

[0002] In dialogue interaction scenarios such as between users and intelligent robots, users and vehicle-mounted devices, and users and smart home devices, the server usually streams data to the front end (i.e., terminal device) in the form of data fragments, and the front end needs to receive, parse and display this data in real time.

[0003] Currently, in the process of data streaming, the front end often performs data parsing after receiving a data fragment. Since the data fragment is streamed, the data fragment currently being parsed may have incomplete data, resulting in data parsing errors, which in turn affects the efficiency of data processing. Summary of the invention

[0004] Embodiments of the present application provide a data processing method, apparatus, device, medium and product, which can process data units obtained after integrity detection during streaming transmission, thereby improving the efficiency and accuracy of the data processing process.

[0005] On the one hand, an embodiment of the present application provides a data processing method, the method comprising:

[0006] receiving a data stream transmitted in a dialog interaction scenario, where the data stream is streamed in a data segment manner;

[0007] During the streaming transmission of the data segments, integrity detection processing is performed on the received data segments to obtain integrity detection results, where the integrity detection results are used to indicate at least one target data segment that has been received and can constitute a data unit;

[0008] constructing at least one target data segment into a data unit, and performing data parsing processing on the data unit;

[0009] Output the parsed data unit in the dialogue interaction scenario.

[0010] On the one hand, an embodiment of the present application provides a data processing device, the device comprising:

[0011] A receiving module, used to receive a data stream transmitted in a dialogue interaction scenario, where the data stream is streamed in the form of data segments;

[0012] A processing module, used for performing integrity detection processing on the received data fragments during the streaming transmission of the data fragments to obtain an integrity detection result, where the integrity detection result is used to indicate at least one target data fragment that has been received and can constitute a data unit;

[0013] The processing module is further used to construct at least one target data segment into a data unit and perform data parsing processing on the data unit;

[0014] The output module is used to output the parsed data units in the dialogue interaction scenario.

[0015] In a possible implementation, any data segment is composed of at least one character; the processing module performs integrity detection processing on the received data segment to obtain an integrity detection result, which is used to perform the following operations:

[0016] During the streaming transmission of the data segments, each received data segment i is subjected to integrity detection processing; the integrity detection processing is used to detect whether the data segment i contains a preset identifier;

[0017] If the data segment i does not contain the preset identifier, the data segment i is stored in the buffer;

[0018] Perform integrity check on data segment i+1 until an integrity check result is obtained.

[0019] In a possible implementation, the processing module performs integrity detection processing on the data segment i+1 until an integrity detection result is obtained, which is used to perform the following operations:

[0020] If the data fragment i+1 includes a preset identifier, both the data fragment i and the data fragment i+1 are used as target data fragments that can constitute a data unit, and an integrity check result is obtained based on the data fragment i and the data fragment i+1; or,

[0021] If the data segment i+1 does not contain the preset identifier, the data segment i+1 is stored in the buffer, and the data segment i+2 is subjected to integrity detection processing until an integrity detection result is obtained;

[0022] The data fragments in the buffer are stored in sequence.

[0023] In a possible implementation, at least one data segment transmitted by streaming is stored in a buffer for sequential storage, and any data segment is composed of at least one character; the integrity detection result is used to indicate at least one target data segment that can constitute each data unit in at least one data unit, and the at least one data unit includes a first data unit; the processing module performs integrity detection processing on the received data segment to obtain an integrity detection result, which is used to perform the following operations:

[0024] Acquire M data fragments received through streaming transmission, where M is a positive integer; wherein the M data fragments refer to: data fragments received within a preset time period, or all data fragments allowed to be stored in the buffer;

[0025] Detect whether each data fragment j among the M data fragments contains a preset identifier, where j is a positive integer and 1≤j≤M;

[0026] If the data segment j contains a preset identifier, obtaining at least one data segment to be spliced ​​that is located before the data segment j in the storage order from the buffer;

[0027] At least one data segment to be spliced ​​and data segment j are used as at least one target data segment that can constitute a first data unit.

[0028] In a possible implementation manner, the at least one data unit further includes a second data unit; and the processing module is further configured to perform the following operations:

[0029] The first data unit is cleared from the buffer, and each other data fragment after the first data unit is cleared from the buffer is detected to determine whether the preset identifier is included;

[0030] If it is detected that the data segment k contains a preset identifier, at least one data segment to be spliced ​​that is located before the data segment k in the storage order is obtained from the buffer, where k is a positive integer and j+1≤k≤M;

[0031] At least one data segment to be spliced ​​and the data segment k are used as at least one target data segment that can constitute a second data unit.

[0032] In a possible implementation, the processing module is further configured to perform the following operations:

[0033] In a dialog interaction scenario, a data detection rule for performing integrity detection processing is obtained, wherein the data detection rule defines: at least one type of data unit to be constructed, and an identifier to be detected corresponding to each type of data unit;

[0034] Obtaining a target type of the data unit to be constructed, and determining a preset identifier to be detected corresponding to the data unit of the target type based on the data detection rule;

[0035] Performing detection processing on a preset identifier for each data segment i;

[0036] The data unit includes any one of row data, column data and paragraph data.

[0037] In a possible implementation, the processing module performs data parsing processing on the data unit to perform the following operations:

[0038] Using a preset parsing method, the data unit is parsed to obtain a parsing result of the data unit; the parsing result includes a parsed parsing script object in the data unit;

[0039] The preset parsing methods include: data parsing tools, parsing code files, regular expressions, and any method in the parsing library.

[0040] In a possible implementation, the processing module uses a preset parsing method to perform data parsing on the data unit to obtain a parsing result of the data unit, which is used to perform the following operations:

[0041] Get the target data format corresponding to the data unit;

[0042] Determine a target parsing library that matches a target data format; wherein different data formats are matched with different parsing libraries;

[0043] The target parsing library is used to perform data parsing processing on the data unit of the target data format to obtain the parsing result of the data unit.

[0044] In a possible implementation, the output module outputs the parsed data unit in the dialog interaction scenario to perform the following operations:

[0045] Outputting the parsed data unit in a unit display mode in a dialog interaction scenario, wherein the unit display mode includes: any one of a whole row display, a whole column display, and a whole paragraph display; or,

[0046] In the dialogue interaction scenario, the parsed data units are output in an animated display mode that displays word by word.

[0047] In a possible implementation, the parsed data unit includes a parsing script object of the data unit; the parsed data unit is output in a word-by-word animation display mode in a dialog interaction scenario, and the processing module is used to perform the following operations:

[0048] Set animation attribute parameters for the parsing script object of the data unit, where the animation attribute parameters are used to indicate the attribute parameters of the data unit during the animation output process;

[0049] The parsed script object provided with the animation attribute parameters is animatedly rendered, and the parsed data unit is outputted in a word-by-word animation display mode in a dialog interaction scene.

[0050] In a possible implementation, the processing module sets animation attribute parameters for the parsing script object of the data unit to perform the following operations:

[0051] Using an animation control tool, dynamically setting a first attribute parameter of a cascading style sheet of a parsed script object, the first attribute parameter comprising: at least one of an animation name, an animation duration, a timing function, an animation delay, a direction, a fill mode, and a play state;

[0052] Adopting an animation control tool, dynamically setting a second attribute parameter of a cascading style sheet of a parsed script object, the second attribute parameter including: at least one of an animation step number and an end position;

[0053] Using an animation control tool, dynamically setting a third attribute parameter of the cascading style sheet of the parsed script object, the third attribute parameter including: at least one of display width and display length;

[0054] The first property parameter, the second property parameter, and the third property parameter are combined into animation property parameters of the parsed script object.

[0055] In a possible implementation, the conversation interaction scenario includes a voice interaction scenario between a target object and a large language model, and the data unit is row data; the processing module is further used to perform the following operations:

[0056] Receive task processing instructions from the target object in the dialogue interaction scenario;

[0057] Sending the task processing instruction to the server for analysis and processing, where the analysis and processing is used to indicate the data stream to be replied obtained by analyzing the task processing instruction;

[0058] Receive the data stream transmitted by the server and output each line of data in the data stream word by word through a large language model.

[0059] On the one hand, an embodiment of the present application provides a computer device, which includes a processor and a memory; a computer program is stored in the memory; when the computer program is executed by the processor, the above-mentioned data processing method is executed.

[0060] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned data processing method is executed.

[0061] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned data processing method is executed.

[0062] In an embodiment of the present application, a data stream transmitted in a dialogue interaction scenario is received, and the data stream is streamed in the form of data segments; during the streaming of the data segments, the received data segments are subjected to integrity detection processing to obtain integrity detection results, and the integrity detection results are used to indicate at least one target data segment that has been received and can constitute a data unit; at least one target data segment is constructed as a data unit, and the data unit is subjected to data parsing processing; and the data unit after parsing processing is output in the dialogue interaction scenario. It can be seen that the present application supports the construction of a complete data unit by performing integrity detection on the data segments during the streaming of the data segments. Since the data unit is generated after the integrity detection processing is performed, when the data is parsed on the complete data unit, the erroneous parsing caused by incomplete data can be avoided, thereby improving the accuracy of the data processing process during the streaming process; in addition, since a data unit is constructed by at least one data segment, the method of parsing the data unit as a unit can improve the efficiency of data parsing compared to directly parsing the data segment. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 is a schematic diagram of a data segment and a data unit provided in an embodiment of the present application;

[0065] Figure 2 It is a schematic diagram of the principle of a data processing solution provided in an embodiment of the present application;

[0066] Figure 3 It is a schematic diagram of the architecture of a data processing system provided in an embodiment of the present application;

[0067] Figure 4It is a flowchart of a data processing method provided in an embodiment of the present application;

[0068] Figure 5 It is a schematic diagram of storing data fragments in a buffer provided by an embodiment of the present application;

[0069] Figure 6 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0070] Figure 7 This is a schematic diagram of a data processing flow for a dialogue interaction scenario provided in an embodiment of the present application;

[0071] Figure 8 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0072] Fig. 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] 1. Introduction to the key terms involved in this application.

[0075] (1) Dialogue interaction scenario.

[0076] The so-called dialogue interaction scenario is a real-time communication interaction scenario, which mainly involves real-time communication between two or more objects, such as the interaction between people and machines (such as chat robots, vehicle-mounted devices, smart home devices, etc.) through natural language. This interaction can simulate the conversation between people and is widely used in: customer service, virtual assistants, education, medical care, smart home and other fields. In the dialogue interaction scenario, real-time data transmission is often carried out by streaming data transmission.

[0077] (2) Streaming data transmission.

[0078] Streaming Data Transmission: Streaming transmission is a data transmission method in which data is transmitted from the sender (such as a server) to the receiver (such as a terminal device) in real time in the form of a continuous stream. Streaming data transmission allows real-time data transmission and is suitable for application scenarios that require low latency and high real-time performance. Specifically, streaming data transmission has the following characteristics:

[0079] 1) Real-time: data is transmitted and consumed immediately after it is generated;

[0080] 2) Segmented processing: Data is often divided into small blocks (such as data fragments) for transmission, and the receiving end can process them one by one without waiting for the entire data set to be transmitted;

[0081] 3) Low latency: Suitable for delay-sensitive application scenarios, such as online video playback and real-time calls.

[0082] Since streaming data transmission involves data transmission between the sender (such as a server) and the receiver (such as a terminal device), a communication connection needs to be established between the server and the terminal device through a protocol. Common streaming data transmission protocols include: WebSocket, Server-Sent Events (SSE), etc. Among them, ①WebSocket: is a full-duplex communication protocol that allows a persistent connection to be established between the terminal device and the server, and bidirectional data transmission on the connection; WebSocket is suitable for applications that require real-time communication, such as chat applications, real-time data push, etc. ②Server-Sent Events (SSE): is a one-way communication protocol that allows the server to push real-time updates to the terminal device. The terminal device requests to establish a connection through HTTP (HyperText Transfer Protocol), and the server continuously sends data to the terminal device through the connection; SSE is suitable for applications that require real-time data updates, such as news push, stock quotes, etc.

[0083] (3) Data fragments and data units.

[0084] A data segment refers to the smallest data unit in the process of streaming data transmission, and the data segment is obtained by dividing a long segment of streaming data into small blocks (or small segments). For example, if the data stream is an audio stream, the data segment here can be a voice segment of a preset frame length; for another example, if the data stream is a video stream, the data segment here can be a frame of image; for another example, if the data stream is a text stream, the data segment here can be a text segment consisting of at least one character. In the present application, in a dialogue interaction scenario, data segments are transmitted in a streaming data transmission manner, for example, data segment 1, data segment 2, data segment 3, etc. are transmitted in sequence; and the lengths of different data segments are the same or different, for example, data segment 1 and data segment 2 both contain 2 characters; or, data segment 1 contains 1 character, and data segment 2 contains 2 characters.

[0085] A data unit refers to a data unit with a complete structure generated after integrity detection processing. In the present application, a data unit is composed of at least one data fragment, and the number of data fragments contained in different data units is the same or different. For example, a row of data can be regarded as a data unit, a column of data can be regarded as a data unit, and a paragraph of data can be regarded as a data unit; based on this, data units can include but are not limited to: row data, column data, paragraph data and other types of data units. In the present application, the data unit is mainly taken as an example of row data.

[0086] See also Figure 1 , Figure 1 is a schematic diagram of a data segment and a data unit provided in an embodiment of the present application. Figure 1 As shown, during the streaming transmission of data fragments, it is possible to detect in turn whether each data fragment contains a preset identifier (such as a line break character). If a data fragment with a line break character (such as data fragment 2) is detected, then the data fragment 2 and at least one data fragment received before the data fragment 2 (such as data fragment 1) are formed into a complete data unit (i.e., data unit 1); similarly, if a data fragment with a line break character (such as data fragment 4) is detected, then the data fragment 4 and data fragment 3 are formed into another complete data unit (i.e., data unit 2). And so on, during the streaming transmission of data fragments, at least one data unit can be obtained by performing integrity detection processing on the data fragments in turn.

[0087] (4) Integrity detection and processing.

[0088] Integrity detection processing refers to a detection processing method for data integrity performed on data fragments during the streaming transmission of data fragments, that is, integrity detection processing is a data detection method for detecting whether the data is complete. Specifically, the integrity detection processing is used to detect whether each data fragment of the streaming transmission contains a preset identifier. If it does, the currently received data fragments are spliced ​​into a complete data unit; if it does not, it indicates that the data is not complete, and it is necessary to continue to receive the next data fragment and detect whether it contains the preset identifier until a complete data unit is obtained; based on this, each data fragment after the integrity detection processing can constitute a complete data unit. Among them, for different data units to be constructed, the preset identifiers to be detected may be different. For example, if the row data needs to be used as the data unit, the preset identifiers to be detected may include: line feed characters (such as the Enter key), separators (such as "}", "]" and other symbols), etc.; if the paragraph data needs to be used as the data unit, the preset identifier to be detected may be a paragraph marker (such as a period). It should be understood that during the streaming transmission of data fragments, a complete data unit can be constructed based on each data fragment after performing integrity detection processing, that is, the integrity of the data unit can be guaranteed (for example, the data unit is a complete row of data), thereby improving the accuracy of data processing.

[0089] 2. Introduction of this application plan.

[0090] This application provides a data processing solution that is applicable to the field of AI (Artificial Intelligence), and is specifically applicable to conversation interaction scenarios between AI models (such as LLM (Large Language Model)) and users, such as conversation interaction scenarios including but not limited to: real-time conversation scenarios, machine translation scenarios, question-and-answer scenarios, and other streaming scenarios, which can improve the data processing efficiency and accuracy in streaming scenarios. Specifically, this solution can detect each data fragment of the streaming transmission through integrity detection to form a data unit with integrity and parse it, reducing the situation of parsing errors. Please refer to Figure 2 , Figure 2 This is a schematic diagram of the principle of a data processing solution provided by this solution. Figure 2 The principles of the data processing solution provided in this application are roughly as follows:

[0091] (1) Receive the data stream transmitted in the dialogue interaction scenario, which is streamed in the form of data segments. The dialogue interaction scenario often involves the interaction between the server (i.e., the data source server) and the terminal device. The server here can provide the terminal device with the data stream to be output, and the server and the terminal device can establish a communication connection through the WebSocket / SSE method.

[0092] (2) During the streaming transmission of data segments, each data segment can be transmitted to the buffer of the terminal device for sequential storage. Furthermore, the received data segments can be subjected to integrity detection processing to obtain integrity detection results, which are used to indicate at least one target data segment that has been received and can constitute a data unit. The above-mentioned integrity detection processing refers to: detecting whether at least one data segment received in the buffer can constitute a complete data unit (such as row data, column data, etc.). Specifically, if the data unit is row data, the above-mentioned integrity detection processing can be performed by detecting whether each received data segment contains a preset identifier (such as a line break or a separator); if a line break exists in a data segment i, the current data segment i and at least one data segment received before the data segment i are used as integrity detection results.

[0093] (3) Constructing at least one target data segment into a data unit, for example, sequentially splicing each target data segment into a data unit. According to the above steps, at least one data unit (such as line data: line1, line2, line3) can be constructed during the streaming process; optionally, each of the constructed data units can be sequentially stored in a queue to facilitate subsequent sequential parsing of each data unit.

[0094] (4) Each data unit stored in the queue is retrieved row by row, and data parsing processing (eg, data format conversion) is performed on each data unit.

[0095] (5) Outputting the parsed data unit in the dialogue interaction scenario. For example, the animation attribute parameters of the data unit can be set through JavaScript, and the data unit with the animation attribute parameters set can be animated to display the data unit in a typewriter effect (i.e., outputting the data unit word by word).

[0096] The data processing scheme provided in the present application mainly involves a data processing and display method in a dialogue interaction scenario. On the one hand, the streamed data fragments can be stored in a buffer and the data fragments can be checked for integrity, so as to construct a complete data unit based on at least one data fragment that has passed the integrity check. Since the data unit is generated after the integrity check is performed, when the complete data unit is parsed, the erroneous parsing caused by incomplete data can be avoided, thereby improving the accuracy of the data processing process during streaming, and the method of parsing and processing in units of data units can improve the efficiency of data parsing; on the other hand, in the output process of the data unit, the data unit can be displayed in a typewriter effect of word-by-word output in a manner allowing the setting of its animation attributes, thereby improving the flexibility of data display.

[0097] It should be noted that the data processing solution provided in this application also needs to specifically explain the following two points:

[0098] 1. The relevant data involved in the data processing process of this application (such as data streams, data fragments, data units, etc.). When the above embodiments of this application are applied to specific products or technologies, the permission or consent of the object must be obtained, and the relevant data collection, use and processing process must comply with the relevant laws, regulations and standards of the region, comply with the principles of legality, legitimacy and necessity, and do not involve the acquisition of data types prohibited or restricted by laws and regulations. In some optional embodiments, the relevant data involved in the embodiments of this application are obtained after the object has been authorized separately. In addition, when obtaining the separate authorization of the object, the purpose of the relevant data involved is indicated to the object.

[0099] 2. It is understood that the term "at least one" in this application means one or more, and the meaning of "multiple" means two or more; for example: at least one data fragment means one, two or more data fragments. The terms "first", "second", etc. are used to distinguish the same or similar items with basically the same role and function. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0100] The data processing system provided by this application is introduced in detail below.

[0101] See also Figure 3 , Figure 3 Schematic diagram of the architecture of a data processing system provided in an embodiment of the present application. Figure 3As shown, the architecture diagram of the data processing system may include at least: a server 304 and a terminal device cluster; wherein the terminal device cluster includes: a first terminal device 301, a second terminal device 302, a third terminal device 303 and other multiple terminal devices, and any terminal device has data processing capabilities (such as the ability to parse data) and data output capabilities (such as the ability to output text). It should be understood that the number of terminal devices included in the terminal device cluster is only for example, and the embodiment of the present application does not limit the number and type of terminal devices. Among them, any terminal device in the terminal device cluster can be directly or indirectly connected to the server 304 through the network; the above-mentioned network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI (Wireless Fidelity, a standard wireless local area network), and other networks that implement wireless communication.

[0102] Each terminal device in the terminal device cluster can be: a mobile phone, a tablet computer, a laptop computer, a desktop computer, a gaming device, a vehicle-mounted device, an aircraft, a wearable device (such as a smart watch, a smart bracelet, a pedometer, etc.), a virtual reality device (such as a VR (Virtual Reality) device, an AR (Augmented Reality) device), an intelligent robot, etc. It can be understood that the types of each terminal device in the terminal device cluster can be the same or different, for example: the first terminal device 301 can be a laptop computer, the second terminal device 302 can be a desktop computer, and the third terminal device 303 can be a tablet computer. This application does not limit the number and type of terminal devices in the terminal device cluster. In addition, the server 304 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0103] Next, taking any terminal device (such as the first terminal device 301) in the above data processing system as an example, the data interaction process between the first terminal device 301 and the server 304 is described accordingly:

[0104] ① The first terminal device 301 receives a task processing instruction issued by the target object in a dialogue interaction scenario (such as a human-machine dialogue scenario, i.e., a dialogue scenario between a user and an intelligent robot), and sends the task processing instruction to the server 304. For example, the task processing instruction here may include: querying today's weather, querying today's temperature, querying the introduction of XXX attractions, etc.

[0105] ② The server 304 parses the task processing instruction to obtain reply data about the task processing instruction. For example, the parsing process here includes but is not limited to: at least one of data format conversion and feature extraction process.

[0106] ③ In the dialogue interaction scenario, the server 304 transmits the reply data to the first terminal device 301 in a streaming data transmission manner. The server 304 and the first terminal device 301 may establish a communication connection via WebSocket or SSE to perform streaming data transmission.

[0107] ④ The first terminal device 301 receives the data stream transmitted in the dialogue interaction scenario, and the data stream is streamed in the form of data segments.

[0108] ⑤ During the streaming transmission of the data segments, each data segment may be transmitted to the buffer of the first terminal device 301 for sequential storage. Optionally, integrity detection processing is performed on the received data segments to obtain integrity detection results, which are used to indicate at least one target data segment that has been received and can constitute a data unit; further, at least one target data segment is sequentially spliced ​​into a data unit.

[0109] ⑥ The first terminal device 301 performs data parsing on the data unit and outputs the parsed data unit in the dialogue interaction scene. For example, the animation attribute parameters of the data unit can be set through JavaScript, and the data unit with the animation attribute parameters set can be animated to display the data unit in a typewriter effect (i.e., the data unit is output word by word).

[0110] It should be noted that the above data processing process is only used as an example and does not limit the specific execution process of the first terminal device and the server. Optionally, in the above dialogue interaction scenario, the server can perform integrity detection processing on the data fragment to construct the data unit to be parsed, and the server directly transmits the constructed data unit to the first terminal device; or, the complete process of the above data processing can be executed by the first terminal device or the server alone.

[0111] In a possible implementation, the data processing system provided in the embodiment of the present application can be deployed in the node of the blockchain. For example, the first terminal device 301, the second terminal device 302, the third terminal device 303, and the server 304 in the above system can all be regarded as the node devices of the blockchain to jointly constitute the blockchain network. Therefore, the data processing process executed in the embodiment of the present application can be executed on the blockchain, which can not only ensure the fairness and justice of the data processing process, but also make the above process traceable, and ensure the data security during the data processing process, thereby improving the security and reliability of the entire data processing process.

[0112] Based on the data processing system provided in the embodiment of the present application, on the one hand, it is possible to store the streamed data fragments in a buffer and perform integrity checks on the data fragments, so as to construct a complete data unit based on at least one data fragment that has passed the integrity check. Since the data unit is generated after the integrity check process is performed, when the complete data unit is parsed, the erroneous parsing situation caused by incomplete data can be avoided, thereby improving the accuracy of the data processing process during streaming, and the method of parsing and processing in units of data units can improve the efficiency of data parsing; on the other hand, in the process of outputting the data unit, it is allowed to set its animation attributes in a way that the data unit is displayed in a typewriter effect of word-by-word output, thereby improving the flexibility of data display.

[0113] It can be understood that the data processing system described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. It is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0114] The specific embodiments involved in the data processing scheme of this application are described in detail below with reference to the accompanying drawings.

[0115] See also Figure 4 , Figure 4 is a flow chart of a data processing method provided in an embodiment of the present application. The data processing method can be performed by a computer device (such as Figure 3 Any terminal device shown in the figure) is executed. Figure 4 As shown, the data processing method includes but is not limited to the following steps S401-S404:

[0116] S401: Receive a data stream transmitted in a dialogue interaction scenario, where the data stream is streamed in a data segment format.

[0117] The dialogue interaction scenario involved in the embodiment of the present application is a real-time communication interaction scenario, which mainly involves real-time communication between two or more objects, such as a scenario in which a person and a machine (such as an intelligent robot, a vehicle-mounted device, an intelligent home device, etc.) interact through natural language. Specifically, in a dialogue interaction scenario (such as a dialogue scenario between a user and an intelligent robot), the above-mentioned data stream refers to the reply data obtained after the task processing instruction issued by the computer device (such as an intelligent robot) is processed for the target object (such as a user). In addition, since the dialogue interaction scenario between the user and the intelligent robot is a scenario in which a dialogue is generated in real time, the data stream output in the dialogue interaction scenario is also transmitted in real time; that is, the data stream output in the dialogue interaction scenario is a real-time data stream, and the so-called real-time data stream refers to a dynamic data stream that can continuously generate new data as time goes by (such as whenever a new task processing instruction is detected by the user). Further, the above-mentioned data stream can be streamed in the form of data segments, for example, data segment 1 can be transmitted at the first moment, data segment 2 can be transmitted at the second moment, and so on.

[0118] In one possible implementation, the reply data (i.e., data stream) output by a computer device (such as a server) after replying to the task processing instruction generated for the target object is streamed to the terminal device in the form of data fragments to ensure low latency and real-time performance during data transmission. Specifically, each of the above data fragments is streamed based on a preset data format, and the preset data format here may include, for example, any type of data format such as Markdown, JSON (JavaScript Object Notation), etc. Among them, the so-called Markdown: is a lightweight markup language that uses concise syntax to format text; Markdown is a data format commonly used for writing documents and blog posts; the so-called JSON: is a lightweight data exchange format that is easy for humans to read and write, and is also easy for machines to parse and generate; and JSON is a data format commonly used for data transmission between servers and terminal devices. Based on this, in an embodiment of the present application, the server can stream: Markdown data or JSON data to the terminal device, and the terminal device parses and displays the data.

[0119] Optionally, each data segment in the streaming process may be composed of at least one character, and the lengths of different data segments transmitted at different times may be the same or different, which is not specifically limited in this application. For example, data segment 1 transmitted at the first moment and data segment 2 transmitted at the second moment both include 2 characters; for another example, data segment 1 transmitted at the first moment includes 1 character, and data segment 2 transmitted at the second moment includes 2 characters.

[0120] S402. During the streaming transmission of the data segments, integrity check processing is performed on the received data segments to obtain an integrity check result, where the integrity check result is used to indicate at least one target data segment that has been received and can constitute a data unit.

[0121] Integrity detection processing refers to a detection and processing method for data integrity performed on data fragments during the streaming transmission of data fragments, that is, integrity detection processing is a data detection method for detecting whether the data is complete. The basis for judging whether the data is complete here refers to whether it can constitute a complete data unit (such as row data, that is, a row of data); after the integrity detection processing is performed on the received data fragments, the integrity detection result can be obtained. The integrity detection result refers to the result obtained when at least one target data fragment that can constitute a data unit is detected. The data unit refers to a data structure that meets the data integrity condition. For example, if the data unit is a row of data, then the data integrity condition here is: the last data fragment of at least one data fragment constituting the data unit has a line break or separator. Therefore, by performing the integrity detection processing, it can be ensured that a complete data unit is constructed. The following is a specific description of the two detection methods for the integrity detection processing of data fragments.

[0122] Method 1: Perform integrity check on each received data fragment in real time.

[0123] In a possible implementation, the process of performing integrity check on the received data fragments to obtain integrity check results includes the following steps (1)-(3):

[0124] (1) During the streaming transmission of data segments, each received data segment i is subjected to integrity check processing, wherein the integrity check processing is used to detect whether the data segment i contains a preset identifier, where i is a positive integer. The data segment i here refers to any data segment received at the current moment, and any data segment is composed of at least one character (such as characters of any data type such as Chinese, English, or numbers); then the essence of the integrity check processing performed on the data segment i is to detect whether the data segment i contains a preset identifier, wherein the preset identifier can be understood as an identifier used to identify the end of a complete data unit, such as a preset identifier may include: a line break character, a separator, or other preset end identifiers, etc.

[0125] (2) If data segment i does not contain a preset identifier, then data segment i is stored in a buffer. The buffer here is a temporary storage area for storing streaming data segments until a complete data unit (such as a complete line of data) is formed. For example, the buffer can be a string variable or other data structure. Among them, if data segment i does not contain a preset identifier, then the current data segment i has not yet reached the end condition that can constitute a complete data unit, then it is necessary to continue to receive the next data segment i+1, and store the current data segment i in the buffer; if data segment i contains a preset identifier, then the current data segment i has reached the end condition that can constitute a complete data unit, then the data segment i and at least one data segment received before the data segment i can be used as at least one target data segment that can constitute a data unit (that is, to obtain an integrity detection result).

[0126] (3) Perform integrity check on data fragment i+1 until an integrity check result is obtained. Data fragment i+1 refers to the next data fragment received after data fragment i; therefore, the integrity check performed on data fragment i+1 is the same as the integrity check performed on data fragment i mentioned in the above step, that is, detecting whether data fragment i+1 contains a preset identifier.

[0127] Specifically, the integrity check processing process performed on data fragment i+1 is as follows: ① If data fragment i+1 contains a preset identifier, data fragment i and data fragment i+1 are used as at least one target data fragment that can constitute a data unit, and an integrity check result is obtained based on data fragment i and data fragment i+1 (for example, data fragment i and data fragment i+1 are used as integrity check results, or, the identifier of data fragment i and the identifier of data fragment i+1 are used as integrity check results); ② If data fragment i+1 does not contain the preset identifier, data fragment i+1 is stored in the buffer, and integrity check processing is performed on data fragment i+2, and so on. The above steps are repeated until it is detected that the received data fragment contains the preset identifier, and the integrity check result can be obtained.

[0128] As can be seen from the above, the buffer is used to store each data fragment that does not contain a preset identifier; and each data fragment is stored in the buffer in order; for example, data fragment 1 is transmitted at the first moment, and data fragment 2 is transmitted at the second moment, then data fragment 1 is stored in the buffer first, and then data fragment 2 is stored in the buffer. Until it is detected that any data fragment contains a preset identifier, the integrity detection result is obtained based on the data fragment and each data fragment stored in the current buffer (that is, each data fragment received before the current data fragment). It should be understood that the integrity detection process performed in the above method 1 can be executed cyclically, and each time the above integrity detection process is executed, a data unit can be constructed; then, multiple data units can be constructed by executing multiple integrity detection processes.

[0129] In the integrity detection processing shown in the above method 1, each received data fragment is supported to be integrity checked to see whether it contains a preset identifier (such as a newline character or a separator); if not, it is stored in a buffer and the next data fragment is detected again until a data fragment containing the preset identifier is detected, and the integrity detection result can be obtained.

[0130] Method 2: The data segments are stored in a buffer, and at least one stored data segment is obtained from the buffer for integrity detection.

[0131] In one possible implementation, a buffer is deployed in a computer device, and the buffer is used to store at least one data fragment; therefore, each data fragment transmitted by streaming can be stored in the buffer for sequential storage, so as to obtain at least one data fragment from the buffer for integrity detection processing. It should be noted that in this case, it is necessary to ensure that each data fragment stored in the buffer can constitute at least one data unit; that is, the integrity detection result is used to indicate: at least one target data fragment can constitute each data unit in at least one data unit, and at least one data unit includes the first data unit; specifically, the computer device performs integrity detection processing on the received data fragments, and the process of obtaining the integrity detection result also includes the following steps (1)-(4):

[0132] (1) Obtain M data fragments received through streaming, where M is a positive integer; wherein the M data fragments here refer to: data fragments received within a preset time period, or all data fragments allowed to be stored in the buffer. For example, a certain number of data fragments may be obtained from the buffer periodically (i.e., within a preset time period, such as within 1 minute) for integrity detection processing; for another example, each received data fragment is stored in the buffer in sequence until the number of data fragments stored in the buffer reaches a storage data threshold M (M is a positive integer, such as 10) of the buffer, and then the M data fragments in the buffer are subjected to integrity detection processing.

[0133] (2) Detect whether each data fragment j in the M data fragments contains a preset identifier, where j is a positive integer and 1≤j≤M. As mentioned above, the preset identifier here can be understood as an identifier used to identify the end of a complete data unit. For example, the preset identifier may include: a line break, a separator, or other preset end identifiers, etc.

[0134] (3) If data segment j contains a preset identifier, at least one data segment to be spliced ​​that is stored before data segment j is obtained from the buffer. Figure 5 , Figure 5 FIG. 1 is a schematic diagram of storing data fragments in a buffer provided by an embodiment of the present application. Figure 5 As shown, six data fragments are stored in the buffer: data fragment 1, data fragment 2, data fragment 3, data fragment 4, data fragment 5, and data fragment 6; starting from data fragment 1, each data fragment j is traversed in turn to detect whether each data fragment j contains a preset identifier (such as a newline character). If data fragment j (such as data fragment 3) contains it, data fragment 1 and data fragment 2 are used as at least one data fragment to be spliced ​​that is located before data fragment j (such as data fragment 3) in storage order.

[0135] Optionally, if data fragment j does not contain a preset identifier, the data fragment j+1 in the buffer is checked to see whether it contains a preset identifier; until a data fragment containing a preset identifier is detected, the integrity check result can be obtained, that is, at least one target data fragment that can constitute a data unit is detected.

[0136] (4) Taking at least one data segment to be spliced ​​and data segment j as at least one target data segment that can constitute the first data unit. For example, taking the above-mentioned data segment 1, data segment 2, and data segment 3 as at least one target data segment that can constitute the first data unit.

[0137] Furthermore, the first data unit is cleared from the buffer, and each of the other data fragments after the first data unit is cleared from the buffer is detected to see whether it contains a preset identifier; if it is detected that data fragment k contains the preset identifier, at least one data fragment to be spliced ​​that is stored before data fragment k is obtained from the buffer; at least one data fragment to be spliced ​​and data fragment k are used as at least one target data fragment that can constitute the second data unit. That is, after obtaining the first detection result that can constitute the first data unit, the remaining data fragments in the buffer (such as data fragment 4, data fragment 5, data fragment 6) can be subjected to integrity detection, and the integrity detection process is similar to the process shown in the above steps (1)-(4); if it is detected that data fragment 6 contains a preset identifier (such as a line break), data fragment 4, data fragment 5, and data fragment 6 can be used as at least one target data fragment that can constitute the second data unit.

[0138] In the integrity detection processing shown in the above-mentioned method 2, the received data fragments are first stored in the buffer in order, and the M data fragments stored in the buffer (or all the data fragments stored in the buffer) are periodically checked for integrity. If a data fragment containing a preset identifier is detected, the data fragments stored in the buffer in an order before the data fragment are taken together with the data fragment as an integrity detection result obtained by this integrity detection processing.

[0139] In summary, the integrity detection processing performed by the above two methods can both realize the integrity detection processing of data fragments, thereby obtaining integrity detection results that can constitute a complete data unit; in actual applications, the appropriate integrity detection method can be flexibly selected according to the scenario requirements to perform the above integrity detection processing, and the embodiments of the present application do not make specific limitations on this.

[0140] In a possible implementation, a data detection rule for performing integrity detection processing is obtained in a dialogue interaction scenario, and the data detection rule defines: at least one type of data unit to be constructed, and an identifier to be detected corresponding to each type of data unit; the target type of the data unit to be constructed is obtained, and the preset identifier to be detected corresponding to the data unit of the target type is determined based on the data detection rule; the detection processing on the preset identifier is performed for each data fragment i; wherein the data unit includes: any one of row data, column data, and paragraph data. That is to say, for different types of data units (composed of at least one data fragment), the detected identifiers are different during the integrity detection processing of the data fragment; for example, the detection processing on whether the row data is complete is performed by detecting line breaks and separators; and for example, the detection processing on whether the paragraph data is complete is performed by detecting paragraph markers. In actual application scenarios, the type of data unit to be constructed can be selected as needed. For example, in the embodiment of the present application, the data unit is row data as an example for illustration in the dialogue interaction scenario.

[0141] It should be understood that during the streaming transmission of data segments, the integrity check processing of the data segments needs to be performed cyclically, that is, after obtaining one integrity check result, the next integrity check result can be obtained, and so on. By cyclically performing the above integrity check processing, multiple integrity check results can be obtained to trigger the construction of multiple data units. For the sake of ease of explanation, the embodiments of the present application take any data unit as an example for subsequent explanation.

[0142] S403: construct at least one target data segment into a data unit, and perform data parsing processing on the data unit.

[0143] Among them, since the data fragments are streamed, the integrity check of the data fragments is also performed on each data fragment in sequence according to the transmission order during the streaming process. Therefore, the at least one target data fragment that can constitute the data unit indicated by the integrity check result refers to the data fragments that are transmitted continuously. The so-called data fragments that are transmitted continuously refer to: any two data fragments are adjacent data fragments. For example, at least one target data fragment includes data fragment 1, data fragment 2, and data fragment 3. These three data fragments are several data fragments that are transmitted continuously. Based on this, at least one target data fragment (such as data fragment 1, data fragment 2, data fragment 3) can be directly constructed (such as spliced ​​in sequence) as a complete data unit (such as a complete row of data).

[0144] Specifically, since the original data (i.e., data unit) transmitted over the network is a text string, such as a string in JSON format or a string in Markdown format, it is essentially a string type of data, and this string type of data does not support or is not convenient for operations on its content or attributes (such as subsequent setting of animation attributes, etc.); therefore, in the embodiment of the present application, it is necessary to perform data parsing on this string (such as a JSON string, a Markdown string) to obtain a parsed script object that is easy to operate. For example, a JSON object can be obtained after data parsing of a string in JSON format. Among them, the so-called JSON format string: JSON data represented in text form, usually used for data transmission and storage, and needs to be converted into an operable JavaScript object (i.e., JSON object) through data parsing. JSON object: A JavaScript object after parsing JSON data (such as a text string), which contains the structure and content of the JSON data, and can directly operate and access its properties for subsequent animation output. The following is a detailed description of JSON data and JavaScript objects through Table 1:

[0145] Table 1. Comparison between JSON data and JSON objects

[0146]

[0147] In summary, a data unit (such as JSON data) is a strictly formatted text, which can be considered as static data. The JavaScript object (parsed script object) obtained after parsing the JSON data is a dynamic data that supports direct definition and operation in the code.

[0148] In a possible implementation, the process of parsing the data unit includes: using a preset parsing method to parse the data unit, and obtaining the parsing result of the data unit, which includes the parsing script object (such as JSON object) after being parsed in the data unit; wherein the preset parsing method includes: data parsing tools, parsing code files, regular expressions, and any one of the parsing libraries. For example, ① you can directly use the developed data parsing tools such as OpenCV, Librosa, etc. to conveniently parse the data unit. This parsing method based on the data parsing tool is convenient and efficient; ② you can adaptively write a parsing code file with data parsing function to adapt and parse the data unit to be parsed. This parsing method based on the parsing code file can adapt to different parsing requirements and parsing files, and has high accuracy; ③ regular expressions can be used to extract data in a specific format (such as date, email address, etc.), which is suitable for parsing some data with relatively standard data formats; ④ finally, based on data in different data formats, a matching parsing library can be used to perform data parsing. This parsing method has high accuracy. In practical applications, different parsing methods can be flexibly selected according to needs to perform parsing of data units, thereby achieving efficient and accurate data parsing.

[0149] Specifically, the computer device uses a preset parsing method to perform data parsing processing on the data unit to obtain the parsing result of the data unit, which may include the following steps (1)-(3):

[0150] (1) Obtain the target data format corresponding to each data unit. For example, the target data format may include: Markdown, JSON (JavaScript Object Notation), etc. For example, a complete row of data (such as a complete row of JSON data) can be represented as the following data structure: {"name":"Molecule Man","age":29,"secretIdentity":"Dan Jukes","powers":["Radiation resistance","Turningtiny","Radiation blast"]}; It can be seen that JSON data is usually a structured data grammatically represented by curly braces {}.

[0151] (2) Determine a target parsing library that matches the target data format; different data formats are matched with different parsing libraries. For example, if the target data format is Markdown, the target parsing library that matches the Markdown format is the Remarkable library; for another example, if the target data format is JSON, the target parsing library that matches the JSON format is the JSON.parse() library.

[0152] (3) Use the target parsing library to parse the data unit in the target data format to obtain the parsing result of the data unit. In specific implementation, the parsing results corresponding to data units in different data formats are different. For example, the parsing result corresponding to the data unit in JSON format is a JavaScript object, and the parsing result corresponding to the data unit in Markdown format is an HTML (HyperText Markup Language) string. There are two specific cases as follows:

[0153] ① You can use the JSON.parse() library to parse the data unit in JSON format to convert it into a JavaScript object. The JavaScript object is a data structure in JavaScript that represents a set of key-value pairs and is used to store and manipulate structured data. Its syntax is usually expressed using curly braces {}, for example:

[0154]

[0155] Typical uses: storing data, processing logic, and converting to and from JSON data format.

[0156] ②Use the render method of the Remarkable library to parse the Markdown format data unit into an HTML string. The HTML string is a text string that is used to represent the HTML tags of the web page structure. It is usually stored in plain text and is used to dynamically generate and manipulate the content of the web page. Its syntax usually contains HTML tags, such as:

[0157] const htmlString=" Hello, World! ";

[0158] Typical use: Used as a template for web page content and dynamically inserted into the page.

[0159] S404: Outputting the parsed data unit in the dialogue interaction scenario.

[0160] In a possible implementation, if the data unit is of text type, the output method for the data unit may be to output the data unit in a text display mode; if the data unit is of audio type, the output method for the data unit may be to output the data unit in an audio playback mode; if the data unit is of video type, the output method for the data unit may be to output the data unit in a video playback mode. In the embodiment of the present application, an example is given for explanation by taking the data unit of text type as an example.

[0161] Specifically, since the data unit (such as a string in JSON format) can obtain an operable parsing script object (such as a JSON object) after being parsed, and the JSON object is a relatively complex data structure, which usually includes text content to be displayed on the page, the text content can be output by outputting the parsing script object. In a possible implementation, outputting the parsed data unit in the dialogue interaction scenario mainly includes the following two methods:

[0162] Method 1: Output the parsed data unit in a unit display mode in a conversation interaction scenario, wherein the unit display mode here includes: any of the whole row display, whole column display, and whole paragraph display. Specifically, after rendering the parsed data unit (such as a JSON object or an HTML string), the parsed data can be output. For example, if the data unit is row data, the data can be directly output as a whole row in the conversation interaction scenario; for another example, if the data unit is column data, the data can be directly output as a whole column in the conversation interaction scenario; this data output method is more convenient and quick.

[0163] Method 2: Output the parsed data unit in a dialog interaction scenario in a word-by-word animation display mode. Specifically, output the parsed data unit in a word-by-word animation display mode, which specifically includes the following steps (1)-(2):

[0164] (1) Animation attribute parameters are set for the parsing script object of the data unit, and the animation attribute parameters are used to indicate the attribute parameters of the data unit during the animation output process. In specific implementation, JavaScript can be used to dynamically set the relevant attributes of the CSS (Cascading Style Sheets) of the data unit to display data line by line, reduce the frequency of JavaScript operating DOM, and improve performance.

[0165] Optionally, an animation control tool (such as JavaScript) is used to dynamically set the first attribute parameter of the cascading style sheet of the parsing script object, and the first attribute parameter (such as animation attribute) includes at least one of: animation name, animation duration, timing function, animation delay, direction, fill mode and play state; the animation control tool is used to dynamically set the second attribute parameter (such as steps function) of the cascading style sheet of the parsing script object, and the second attribute parameter includes at least one of: animation steps and end position; the animation control tool is used to dynamically set the third attribute parameter (such as width attribute) of the cascading style sheet of the parsing script object, and the third attribute parameter includes display width, display length and other parameters related to display size; the first attribute parameter, the second attribute parameter, and the third attribute parameter set above are combined into the animation attribute parameter of the parsing script object. In this implementation, it is supported to dynamically set the animation attribute parameter for the parsing script object of the data unit, so that the data can be displayed word by word according to the typewriter effect, reducing the number of redrawing and rearrangement of the browser, thereby improving the data rendering performance.

[0166] (2) Animation rendering is performed on the parsing script object with animation attribute parameters set, and the parsed data unit is output in a word-by-word animation display mode in the dialogue interaction scene. For example, the animation attribute of the element is dynamically set through JavaScript to achieve a typewriter effect of displaying data line by line.

[0167] In an embodiment of the present application, a data segment streamed in a dialogue interaction scenario is received, and during the streaming of the data segment, the received data segment is subjected to integrity detection processing to obtain an integrity detection result, which is used to indicate at least one target data segment that has been received and can constitute a data unit; at least one target data segment is constructed as a data unit, and data parsing processing is performed on the data unit; and the parsed data unit is output in the dialogue interaction scenario. It can be seen that, on the one hand, the present application can store the streamed data segment in a buffer, and perform integrity detection on the data segment, so as to construct a complete data unit based on at least one data segment that passes the integrity detection. Since the data unit is generated after the integrity detection processing is performed, when the data is parsed for this complete data unit, the erroneous parsing caused by incomplete data can be avoided, thereby improving the accuracy of the data processing process during the streaming process, and the method of parsing the data unit as a unit can improve the efficiency of data parsing; on the other hand, during the output process of the data unit, the present application allows the setting of its animation attributes to display the data unit in a typewriter effect of word-by-word output, thereby improving the flexibility of data display.

[0168] See also Figure 6 , Figure 6 is a flow chart of another data processing method provided by an embodiment of the present application. The data processing method can be performed by a computer device (such as Figure 3 Any terminal device shown in the figure) is executed. Figure 6 As shown, the data processing method includes but is not limited to the following steps S601-S607:

[0169] S601: Receive a data stream transmitted in a dialog interaction scenario, where the data stream is streamed in a data segment format.

[0170] In a possible implementation, the dialogue interaction scenario refers to a human-computer dialogue scenario (such as a dialogue scenario between a target object and an intelligent robot), and the terminal device (such as an intelligent robot) can respond to the task processing instruction issued by the target object and send the task processing instruction to the server, and then receive the data stream transmitted by the server. Among them, the server and the terminal device (such as an intelligent robot) can use the WebSocket protocol or the SSE protocol to achieve real-time data transmission between the server and the terminal device; furthermore, the data fragments can be streamed in the dialogue interaction scenario.

[0171] In a possible implementation, the reply data (i.e., data stream) output by the computer device (such as a server) after replying to the task processing instruction generated by the target object is streamed to the terminal device in the form of data segments to ensure low latency and real-time performance during data transmission. Specifically, each of the above data segments is streamed based on a preset data format, and the preset data format here can include, for example, any type of data format such as Markdown, JSON (JavaScript Object Notation), etc.

[0172] Based on this, during the streaming transmission of data fragments, the received data fragments can be processed for integrity detection to obtain integrity detection results to construct a complete data unit. Specifically, in the process of performing integrity detection processing, there are mainly two detection methods: method 1, performing integrity detection processing on each received data fragment in real time; method 2, storing the data fragments in a buffer, and obtaining at least one stored data fragment from the buffer for integrity detection processing. The following describes the two different detection methods in detail.

[0173] Method 1: Perform integrity check processing on each received data fragment in real time (S6021-S6041 or S6021-S6051).

[0174] S6021. During the streaming transmission of data segments, perform integrity check processing on each received data segment i.

[0175] The integrity check process here is used to detect whether the data fragment i contains a preset identifier, where i is a positive integer. The data fragment i here refers to any data fragment received at the current moment, and any data fragment is composed of at least one character; then the essence of the integrity check process performed on the data fragment i is to detect whether the data fragment i contains a preset identifier, where the preset identifier can be understood as an identifier used to identify the end of a complete data unit, for example, the preset identifier can include: a line break, a separator, or other preset end identifiers, etc.

[0176] S6031. If the data segment i does not include a preset identifier, the data segment i is stored in a buffer.

[0177] S6041. Perform integrity check on data segment i+1 until an integrity check result is obtained.

[0178] Specifically, if the data fragment i+1 does not contain the preset identifier, the data fragment i+1 is stored in the buffer; and the data fragment i+2 is subjected to integrity check processing (the detection process is similar to the detection process of the data fragment i), and so on, until it is detected that the data fragment contains the preset identifier, and the integrity check result can be obtained.

[0179] S6051. If the data fragment i+1 contains a preset identifier, both the data fragment i and the data fragment i+1 are used as target data fragments that can constitute a data unit, and an integrity check result is obtained based on the data fragment i and the data fragment i+1.

[0180] For example, data segment i and data segment i+1 may be combined into the integrity detection result; or, the identifier of data segment i and the identifier of data segment i+1 may be used as the integrity detection result.

[0181] In the integrity detection processing shown in the above method 1, each received data fragment is supported to be integrity checked to see whether it contains a preset identifier (such as a newline character or a separator); if not, it is stored in a buffer and the next data fragment is detected again until a data fragment containing the preset identifier is detected, and the integrity detection result can be obtained.

[0182] Method 2: The data segments are stored in a buffer, and at least one stored data segment is obtained from the buffer for integrity detection processing (S6022-S6042).

[0183] S6022. During the streaming transmission of the data segments, each received data segment is stored in sequence in a buffer.

[0184] The buffer here is a temporary storage area for temporarily storing fragment data. The so-called temporary storage area means that when any data fragment is assembled into a data unit, the data fragment is cleared from the buffer; and new data fragments are received in real time for temporary storage of data. For example, the storage structure in the buffer can be specifically shown in Table 2 below:

[0185] Table 2. Storage structure of buffer

[0186] Data fragment Receiving time Number of characters patch-1 t1 1 patch-2 t2 2 patch-3 t3 1

[0187] As shown in Table 1 above, at least one data fragment and the receiving time of each data fragment are stored in sequence in the buffer; in addition, each data fragment is composed of at least one character, and the number of characters contained in different data fragments is the same or different; for example, data fragments patch-1 and patch-3 both contain 1 character, but data fragment patch-2 contains 2 characters.

[0188] S6032. Obtain M data fragments from the buffer, and perform integrity check processing on each data fragment j among the M data fragments.

[0189] Optionally, the M data fragments may be all the data fragments stored in the buffer (such as the maximum number of data fragments allowed to be stored); or, the M data fragments may be data fragments obtained from the buffer periodically (for example, within a preset time period such as 1 minute or 2 minutes), and this application is not limited to this.

[0190] In one possible implementation, the process of performing integrity check processing on each data fragment j (j is a positive integer and 1≤j≤M) among the M data fragments is as follows: check whether each data fragment j among the M data fragments contains a preset identifier (such as a newline character, a separator, etc.); if the data fragment j contains the preset identifier, execute the subsequent step S5042; otherwise, if the data fragment j does not contain the preset identifier, perform integrity check processing on the data fragment j+1 until the integrity check result is obtained.

[0191] S6042: If the data segment j contains a preset identifier, obtain from the buffer at least one data segment to be spliced ​​that is located before the data segment j in a storage order, and obtain an integrity detection result.

[0192] like Figure 5As shown, six data fragments are stored in the buffer: data fragment 1, data fragment 2, data fragment 3, data fragment 4, data fragment 5, and data fragment 6; starting from data fragment 1, each data fragment j is traversed in turn to detect whether each data fragment j contains a preset identifier (such as a newline character). If data fragment j (such as data fragment 3) contains it, data fragment 1 and data fragment 2 are used as at least one data fragment to be spliced ​​that is located before data fragment j (such as data fragment 3) in storage order, and then the integrity detection result is obtained.

[0193] In the integrity detection process shown in the above method 2, firstly, each received data fragment is stored in the buffer in order, and the integrity detection is performed regularly on the M data fragments stored in the buffer (or all the data fragments stored in the buffer). If a data fragment containing a preset identifier is detected, each data fragment stored in the buffer in the order before the data fragment and the data fragment are taken together as an integrity detection result obtained by the integrity detection process. In addition, it should be noted that the specific process of the integrity detection process in this application can be referred to in detail. Figure 4 The relevant process in step S402 in the embodiment will not be repeated here in the embodiment of the present application.

[0194] Based on this, complete data units can be constructed one by one in sequence according to the above method 1 or the above method 2; further, each data unit can be stored in a queue in order, so as to facilitate the subsequent data parsing of each data unit in the queue in order. Among them, the so-called queue (Queue): is a data structure that follows the First In First Out (FIFO) principle. In an embodiment of the present application, the queue can be used to store complete data units (such as row data) to ensure that each row of data can be processed in order; for example, the queue can be an array, or other data structure with first-in-first-out and data storage functions. For example, the storage structure in the queue can be specifically shown in Table 3 below:

[0195] Table 3. Queue storage structure

[0196] Data Unit Creation time Number of fragments Line1 T1 3 Line2 T2 3 Line3 T3 4

[0197] As shown in Table 1 above, the queue stores at least one data unit (such as line data: line1, line2, line3), the creation time of each data unit and the number of fragments it contains in order; in addition, each data unit is composed of at least one character, and the number of characters contained in different data fragments is the same or different; for example, line data line1 and line2 both contain 3 data fragments, but line data line3 contains 4 data fragments. Subsequently, the line data in the queue can be processed in order according to the creation time of each data unit and the first-in-first-out (FIFO) principle.

[0198] S605: Construct at least one target data segment into a data unit, and perform data parsing processing on the data unit.

[0199] Specifically, since the data segments are streamed, the integrity check of the data segments is also performed sequentially on each data segment in the streaming process according to the transmission order, so the at least one target data segment that can constitute the data unit indicated by the integrity check result refers to each data segment that is continuously transmitted. Therefore, the at least one target data segment indicated by the integrity check result (such as data segment 1, data segment 2, data segment 3) can be directly spliced ​​into a data unit in sequence.

[0200] In a possible implementation, the process of data parsing a data unit is as follows: a preset parsing method (e.g., a data parsing tool, a parsing code file, a regular expression, and any one of the parsing libraries) is used to parse the data unit to obtain a parsing result of the data unit, which includes a parsed parsing script object (e.g., a JSON object) in the data unit. Specifically, the process of data parsing through the parsing library is as follows: obtain a target data format (e.g., Markdown, JSON, etc.) corresponding to the data unit, determine a target parsing library that matches the target data format, wherein different data formats match different parsing libraries; for example, the target parsing library that matches the Markdown format is the Remarkable library, and the target parsing library that matches the JSON format is the JSON.parse() library; the target parsing library is used to parse the data unit in the target data format to obtain a parsing result of the data unit, which includes a parsing script object (e.g., a JavaScript object).

[0201] For example, if the data unit is a JSON string, the specific parsing process of parsing the JSON string to obtain a JavaScript object is as follows:

[0202] const jsonstring = '{"name":"Alice","age":25,"isStudent":false}'; / / JSON string (data unit);

[0203] constjsObject = JSON.parse(jsonstring); / / Convert to JavaScript object (parse script object);

[0204] console.log(jsObject);

[0205] / / Output:{name:'Alice',age:25,isstudent:false}.

[0206] S606: Configure animation attribute parameters for the parsed data unit.

[0207] Specifically, the parsed data unit may be a parsed script object (such as a JavaScript object). In a possible implementation, the specific process of configuring the animation attribute parameters for the parsed data unit includes the following steps (1)-(4):

[0208] (1) Using an animation control tool (such as a JavaScript tool), dynamically set the first attribute parameter (animation attribute parameter) of the cascading style sheet (CSS) of the parsed script object, the first attribute parameter including at least one of: animation name, animation duration, timing function, animation delay, direction, fill mode, and play state. A CSS keyframe animation (@keyframes) can be defined to describe the style change from the start point to the end point; and use JavaScript to dynamically set the animation attribute of the parsed script object of the data unit to trigger the animation.

[0209] (2) using an animation control tool to dynamically set a second attribute parameter (steps attribute parameter) of a cascading style sheet (CSS) of a parsed script object, the second attribute parameter including at least one of the number of animation steps and an end position;

[0210] (3) using an animation control tool to dynamically set a third attribute parameter (width attribute parameter) of a cascading style sheet (CSS) of the parsed script object, wherein the third attribute parameter includes at least one of a display width and a display length;

[0211] (4) The first attribute parameter, the second attribute parameter, and the third attribute parameter are combined into animation attribute parameters of the parsed script object.

[0212] Based on the above steps, JavaScript is allowed to dynamically set the animation / steps / width and other animation-related properties of the CSS of each row of data, so that the output speed and animation effect can be flexibly controlled.

[0213] S607: Display the parsed data unit word by word in a typewriter effect in the dialogue interaction scenario.

[0214] Specifically, after dynamically configuring the above-mentioned animation attribute parameters (such as CSS animation / steps / width and other animation-related attributes) for the parsed data unit (such as a JavaScript object), the data unit configured with the above-mentioned animation attribute parameters can be animated by a renderer, so that the parsed data unit can be displayed word by word according to a typewriter effect in a dialogue interaction scenario.

[0215] The specific implementation process in the embodiments of the present application is illustrated below with reference to specific scenarios.

[0216] See also Figure 7 , Figure 7 Schematic diagram of a data processing flow for a dialogue interaction scenario provided by an embodiment of the present application. Figure 7As shown, the dialogue interaction scenario is assumed to be a human-computer dialogue interaction scenario, and the human-computer dialogue interaction scenario mainly involves: the specific interaction between the target object (such as a user), the terminal device (such as an intelligent robot), and the server. Specifically, the data processing flow of the dialogue interaction scenario is as follows: ① The target object can wake up the intelligent robot (such as through a wake-up instruction: such as a preset wake-up instruction such as hello, hello, etc.), and after waking up, it can ask a question query to the intelligent robot according to its own needs. For example, the question query can be: Please help me introduce the relevant information of the famous attraction XXX in detail, and then send the question query to the intelligent robot as a task processing instruction; ② The intelligent robot can receive the task processing instruction and send the task processing instruction to the server. Among them, the intelligent robot and the server can establish a communication connection through the WebSocket or SSE protocol, and stream data through the WebSocket / SSE protocol. ③ After receiving the task processing instruction, the server can parse the task processing instruction to obtain the question query, and can analyze and process the question query through a large language model (LLM), generate reply data corresponding to the question query, and stream the reply data to the intelligent robot; wherein, since the user and the intelligent robot are in real-time dialogue, the user can continue to issue different task processing instructions, and the server can also continue to generate different reply data, and the server can stream data with the intelligent robot. ④ The intelligent robot receives the data stream (such as reply data) transmitted by the server, and the data stream is streamed in the form of data fragments; and during the streaming of the data fragments, the received data fragments are processed for integrity detection to obtain integrity detection results, so that complete row data (i.e., data units) are sequentially constructed according to the above method of this application. ⑤ The animation attribute of each row of data is dynamically set through JavaScript to achieve the typewriter effect of displaying data line by line; for example, JavaScript can be used to dynamically set animation-related attributes such as animation / steps / width of CSS to achieve flexible control of output speed and dynamic effects; and each row of data can be output word by word in the display interface according to the typewriter effect.

[0217] In the embodiments of the present application, on the one hand, by processing data in units of lines, frequent parsing operations are reduced, which can reduce the consumption of CPU and memory resources; on the other hand, the use of a buffer to store data fragments is allowed to ensure that the data parsed each time is complete, thereby reducing the number of parsing errors; on the other hand, JavaScript is used to dynamically control CSS animation to achieve a typewriter effect, thereby reducing the frequency of JavaScript operating the DOM, reducing the number of browser redrawing and reflow times, and improving rendering performance; on the other hand, CSS animation properties are dynamically set through JavaScript, and the output speed and animation effect can be flexibly controlled, thereby improving the user experience.

[0218] The following is an explanation of the relevant devices of the data processing solution provided in the embodiments of the present application.

[0219] It should be noted that, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0220] See also Figure 8 , Figure 8 800 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The data processing device 800 can be used to execute the corresponding steps in the data processing method provided in an embodiment of the present application. In specific implementation, the data processing device 800 can specifically include:

[0221] The receiving module 801 is used to receive a data stream transmitted in a dialogue interaction scenario, where the data stream is streamed in the form of data segments;

[0222] The processing module 802 is used to perform integrity detection processing on the received data fragments during the streaming transmission of the data fragments to obtain an integrity detection result, where the integrity detection result is used to indicate at least one target data fragment that has been received and can constitute a data unit;

[0223] The processing module 802 is further used to construct at least one target data segment into a data unit and perform data parsing processing on the data unit;

[0224] The output module 803 is used to output the parsed data unit in the dialogue interaction scenario.

[0225] In a possible implementation, any data segment is composed of at least one character; the processing module 802 performs integrity detection processing on the received data segment to obtain an integrity detection result for performing the following operations:

[0226] During the streaming transmission of the data segments, each received data segment i is subjected to integrity detection processing; the integrity detection processing is used to detect whether the data segment i contains a preset identifier;

[0227] If the data segment i does not contain the preset identifier, the data segment i is stored in the buffer;

[0228] Perform integrity check on data segment i+1 until an integrity check result is obtained.

[0229] In a possible implementation, the processing module 802 performs integrity detection processing on the data segment i+1 until an integrity detection result is obtained, which is used to perform the following operations:

[0230] If the data fragment i+1 includes a preset identifier, both the data fragment i and the data fragment i+1 are used as target data fragments that can constitute a data unit, and an integrity check result is obtained based on the data fragment i and the data fragment i+1; or,

[0231] If the data segment i+1 does not contain the preset identifier, the data segment i+1 is stored in the buffer, and the data segment i+2 is subjected to integrity detection processing until an integrity detection result is obtained;

[0232] The data fragments in the buffer are stored in sequence.

[0233] In a possible implementation, at least one data segment transmitted by streaming is stored in a buffer for sequential storage, and any data segment is composed of at least one character; the integrity detection result is used to indicate at least one target data segment that can constitute each data unit in at least one data unit, and the at least one data unit includes a first data unit; the processing module 802 performs integrity detection processing on the received data segment to obtain an integrity detection result, which is used to perform the following operations:

[0234] Acquire M data fragments received through streaming transmission, where M is a positive integer; wherein the M data fragments refer to: data fragments received within a preset time period, or all data fragments allowed to be stored in the buffer;

[0235] Detect whether each data fragment j among the M data fragments contains a preset identifier, where j is a positive integer and 1≤j≤M;

[0236] If the data segment j contains a preset identifier, obtaining at least one data segment to be spliced ​​that is located before the data segment j in the storage order from the buffer;

[0237] At least one data segment to be spliced ​​and data segment j are used as at least one target data segment that can constitute a first data unit.

[0238] In a possible implementation manner, the at least one data unit further includes a second data unit; and the processing module 802 is further configured to perform the following operations:

[0239] The first data unit is cleared from the buffer, and each other data fragment after the first data unit is cleared from the buffer is detected to determine whether the preset identifier is included;

[0240] If it is detected that the data segment k contains a preset identifier, at least one data segment to be spliced ​​that is located before the data segment k in the storage order is obtained from the buffer, where k is a positive integer and j+1≤k≤M;

[0241] At least one data segment to be spliced ​​and the data segment k are used as at least one target data segment that can constitute a second data unit.

[0242] In a possible implementation, the processing module 802 is further configured to perform the following operations:

[0243] In a dialog interaction scenario, a data detection rule for performing integrity detection processing is obtained, wherein the data detection rule defines: at least one type of data unit to be constructed, and an identifier to be detected corresponding to each type of data unit;

[0244] Obtaining a target type of the data unit to be constructed, and determining a preset identifier to be detected corresponding to the data unit of the target type based on the data detection rule;

[0245] Performing detection processing on a preset identifier for each data segment i;

[0246] The data unit includes any one of row data, column data and paragraph data.

[0247] In a possible implementation, the processing module 802 performs data parsing processing on the data unit to perform the following operations:

[0248] Using a preset parsing method, the data unit is parsed to obtain a parsing result of the data unit; the parsing result includes a parsed parsing script object in the data unit;

[0249] The preset parsing methods include: data parsing tools, parsing code files, regular expressions, and any method in the parsing library.

[0250] In a possible implementation, the processing module 802 uses a preset parsing method to perform data parsing on the data unit to obtain a parsing result of the data unit, which is used to perform the following operations:

[0251] Get the target data format corresponding to the data unit;

[0252] Determine a target parsing library that matches a target data format; wherein different data formats are matched with different parsing libraries;

[0253] The target parsing library is used to perform data parsing processing on the data unit of the target data format to obtain the parsing result of the data unit.

[0254] In a possible implementation, the output module 803 outputs the parsed data unit in the dialog interaction scenario, and is used to perform the following operations:

[0255] Outputting the parsed data unit in a unit display mode in a dialog interaction scenario, wherein the unit display mode includes: any one of a whole row display, a whole column display, and a whole paragraph display; or,

[0256] In the dialogue interaction scenario, the parsed data units are output in an animated display mode that displays word by word.

[0257] In a possible implementation, the parsed data unit includes a parsing script object of the data unit; the parsed data unit is output in a word-by-word animation display mode in a dialog interaction scenario, and the processing module 802 is used to perform the following operations:

[0258] Set animation attribute parameters for the parsing script object of the data unit, where the animation attribute parameters are used to indicate the attribute parameters of the data unit during the animation output process;

[0259] The parsed script object provided with the animation attribute parameters is animatedly rendered, and the parsed data unit is outputted in a word-by-word animation display mode in a dialog interaction scene.

[0260] In a possible implementation, the processing module 802 sets animation attribute parameters for the parsing script object of the data unit to perform the following operations:

[0261] Using an animation control tool, dynamically setting a first attribute parameter of a cascading style sheet of a parsed script object, the first attribute parameter comprising: at least one of an animation name, an animation duration, a timing function, an animation delay, a direction, a fill mode, and a play state;

[0262] Adopting an animation control tool, dynamically setting a second attribute parameter of a cascading style sheet of a parsed script object, the second attribute parameter including: at least one of an animation step number and an end position;

[0263] Using an animation control tool, dynamically setting a third attribute parameter of the cascading style sheet of the parsed script object, the third attribute parameter including: at least one of display width and display length;

[0264] The first property parameter, the second property parameter, and the third property parameter are combined into animation property parameters of the parsed script object.

[0265] In a possible implementation, the dialogue interaction scenario includes a speech interaction scenario between a target object and a large language model, and the data unit is row data; the processing module 802 is further configured to perform the following operations:

[0266] Receive task processing instructions from the target object in the dialogue interaction scenario;

[0267] Sending the task processing instruction to the server for analysis and processing, where the analysis and processing is used to indicate the data stream to be replied obtained by analyzing the task processing instruction;

[0268] Receive the data stream transmitted by the server and output each line of data in the data stream word by word through a large language model.

[0269] In the embodiments of the present application, on the one hand, it is possible to store the streamed data segments in a buffer and perform integrity checks on the data segments, thereby constructing a complete data unit based on at least one data segment that has passed the integrity check. Since the data unit is generated after the integrity check is performed, when the complete data unit is parsed, the erroneous parsing caused by incomplete data can be avoided, thereby improving the accuracy of the data processing process during streaming, and the method of performing parsing processing in units of data units can improve the efficiency of data parsing. On the other hand, during the output of the data unit, it is allowed to set its animation attributes in such a way that the data unit is displayed in a typewriter effect with word-by-word output, thereby improving the flexibility of data display.

[0270] See also Fig. 9 , Fig. 9: is a structural diagram of a computer device provided in an embodiment of the present application. The computer device 900 is used to execute the steps performed by the computer device in the aforementioned method embodiment. The computer device 900 may include a device (for example, one or more of a server, a node, a terminal device, etc.), and may also include components inside the device (for example, a chip, a software module or a hardware module, etc.). The computer device may include at least one processor 901 and a communication interface 902. Further, optionally, the computer device may also include at least one memory 903 and a bus 904. In addition, the processor 901, the communication interface 902 and the memory 903 are connected via a bus 904. Wherein:

[0271] (1) The processor 901 is a module that performs arithmetic operations and / or logical operations, and may specifically be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a coprocessor (assisting the central processing unit to complete corresponding processing and applications), a micro controller unit (MCU), or a combination of multiple processing modules.

[0272] (2) The communication interface 902 may be used to provide information input or output for at least one processor 901. And / or, the communication interface 902 may be used to receive data sent from the outside and / or send data to the outside, and may be a wired link interface including an Ethernet cable, etc., or a wireless link interface (Wi-Fi, Bluetooth, general wireless transmission, vehicle-mounted short-range communication technology, and other short-range wireless communication technologies, etc.). The communication interface 902 may be used as a network interface.

[0273] (3) The memory 903 is used to provide a storage space, in which data such as an operating system and a computer program (including program instructions) can be stored. The memory 903 can be a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), etc., or a combination of multiple thereof.

[0274] In a specific implementation, the processor 901 executes the computer program stored in the memory 903 to perform the following operations:

[0275] receiving a data stream transmitted in a dialog interaction scenario, where the data stream is streamed in a data segment manner;

[0276] During the streaming transmission of the data segments, integrity detection processing is performed on the received data segments to obtain integrity detection results, where the integrity detection results are used to indicate at least one target data segment that has been received and can constitute a data unit;

[0277] constructing at least one target data segment into a data unit, and performing data parsing processing on the data unit;

[0278] Output the parsed data unit in the dialogue interaction scenario.

[0279] In a possible implementation, any data segment is composed of at least one character; the processor 901 performs integrity detection processing on the received data segment to obtain an integrity detection result, which is used to perform the following operations:

[0280] During the streaming transmission of the data segments, each received data segment i is subjected to integrity detection processing; the integrity detection processing is used to detect whether the data segment i contains a preset identifier;

[0281] If the data segment i does not contain the preset identifier, the data segment i is stored in the buffer;

[0282] Perform integrity check on data segment i+1 until an integrity check result is obtained.

[0283] In a possible implementation, the processor 901 performs integrity detection processing on the data segment i+1 until an integrity detection result is obtained, which is used to perform the following operations:

[0284] If the data fragment i+1 includes a preset identifier, both the data fragment i and the data fragment i+1 are used as target data fragments that can constitute a data unit, and an integrity check result is obtained based on the data fragment i and the data fragment i+1; or,

[0285] If the data segment i+1 does not contain the preset identifier, the data segment i+1 is stored in the buffer, and the data segment i+2 is subjected to integrity detection processing until an integrity detection result is obtained;

[0286] The data fragments in the buffer are stored in sequence.

[0287] In a possible implementation, at least one data segment transmitted by streaming is stored in a buffer for sequential storage, and any data segment is composed of at least one character; the integrity detection result is used to indicate at least one target data segment that can constitute each data unit in at least one data unit, and the at least one data unit includes a first data unit; the processor 901 performs integrity detection processing on the received data segment to obtain an integrity detection result, which is used to perform the following operations:

[0288] Acquire M data fragments received through streaming transmission, where M is a positive integer; wherein the M data fragments refer to: data fragments received within a preset time period, or all data fragments allowed to be stored in the buffer;

[0289] Detect whether each data fragment j among the M data fragments contains a preset identifier, where j is a positive integer and 1≤j≤M;

[0290] If the data segment j contains a preset identifier, obtaining at least one data segment to be spliced ​​that is located before the data segment j in the storage order from the buffer;

[0291] At least one data segment to be spliced ​​and data segment j are used as at least one target data segment that can constitute a first data unit.

[0292] In a possible implementation manner, the at least one data unit further includes a second data unit; and the processor 901 is further configured to perform the following operations:

[0293] The first data unit is cleared from the buffer, and each other data fragment after the first data unit is cleared from the buffer is detected to determine whether the preset identifier is included;

[0294] If it is detected that the data segment k contains a preset identifier, at least one data segment to be spliced ​​that is located before the data segment k in the storage order is obtained from the buffer, where k is a positive integer and j+1≤k≤M;

[0295] At least one data segment to be spliced ​​and the data segment k are used as at least one target data segment that can constitute a second data unit.

[0296] In a possible implementation manner, the processor 901 is further configured to perform the following operations:

[0297] In a dialog interaction scenario, a data detection rule for performing integrity detection processing is obtained, wherein the data detection rule defines: at least one type of data unit to be constructed, and an identifier to be detected corresponding to each type of data unit;

[0298] Obtaining a target type of the data unit to be constructed, and determining a preset identifier to be detected corresponding to the data unit of the target type based on the data detection rule;

[0299] Performing detection processing on a preset identifier for each data segment i;

[0300] The data unit includes any one of row data, column data and paragraph data.

[0301] In a possible implementation, the processor 901 performs data parsing processing on the data unit to perform the following operations:

[0302] Using a preset parsing method, the data unit is parsed to obtain a parsing result of the data unit; the parsing result includes a parsed parsing script object in the data unit;

[0303] The preset parsing methods include: data parsing tools, parsing code files, regular expressions, and any method in the parsing library.

[0304] In a possible implementation, the processor 901 performs data parsing on the data unit in a preset parsing manner to obtain a parsing result of the data unit, which is used to perform the following operations:

[0305] Get the target data format corresponding to the data unit;

[0306] Determine a target parsing library that matches a target data format; wherein different data formats are matched with different parsing libraries;

[0307] The target parsing library is used to perform data parsing processing on the data unit of the target data format to obtain the parsing result of the data unit.

[0308] In a possible implementation, the processor 901 outputs the parsed data unit in the dialog interaction scenario, and is used to perform the following operations:

[0309] Outputting the parsed data unit in a unit display mode in a dialog interaction scenario, wherein the unit display mode includes: any one of a whole row display, a whole column display, and a whole paragraph display; or,

[0310] In the dialogue interaction scenario, the parsed data units are output in an animated display mode that displays word by word.

[0311] In a possible implementation, the parsed data unit includes a parsed script object of the data unit; the parsed data unit is output in a word-by-word animation display manner in a dialog interaction scenario, and the processor 901 is configured to perform the following operations:

[0312] Set animation attribute parameters for the parsing script object of the data unit, where the animation attribute parameters are used to indicate the attribute parameters of the data unit during the animation output process;

[0313] The parsed script object provided with the animation attribute parameters is animatedly rendered, and the parsed data unit is outputted in a word-by-word animation display mode in a dialog interaction scene.

[0314] In a possible implementation, the processor 901 sets animation attribute parameters for the parsing script object of the data unit to perform the following operations:

[0315] Using an animation control tool, dynamically setting a first attribute parameter of a cascading style sheet of a parsed script object, the first attribute parameter comprising: at least one of an animation name, an animation duration, a timing function, an animation delay, a direction, a fill mode, and a play state;

[0316] Adopting an animation control tool, dynamically setting a second attribute parameter of a cascading style sheet of a parsed script object, the second attribute parameter including: at least one of an animation step number and an end position;

[0317] Using an animation control tool, dynamically setting a third attribute parameter of the cascading style sheet of the parsed script object, the third attribute parameter including: at least one of display width and display length;

[0318] The first property parameter, the second property parameter, and the third property parameter are combined into animation property parameters of the parsed script object.

[0319] In a possible implementation, the dialogue interaction scenario includes a voice interaction scenario between a target object and a large language model, and the data unit is row data; the processor 901 is further configured to perform the following operations:

[0320] Receive task processing instructions from the target object in the dialogue interaction scenario;

[0321] Sending the task processing instruction to the server for analysis and processing, where the analysis and processing is used to indicate the data stream to be replied obtained by analyzing the task processing instruction;

[0322] Receive the data stream transmitted by the server and output each line of data in the data stream word by word through a large language model.

[0323] Among them, after the computer device executes the various method steps shown above through the processor, the effects achieved can be referred to the description of the aforementioned embodiments, and will not be repeated in the embodiments of the present application.

[0324] According to one aspect of the present application, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the processor executes the above-mentioned computer program, the method in the corresponding embodiment of the foregoing text can be executed, so it will not be repeated here. For technical details not disclosed in the computer-readable storage medium embodiment involved in the present application, please refer to the description of the method embodiment of the present application. As an example, the computer program can be deployed on a computer device, or executed on multiple computer devices located in one location, or, it can be executed on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0325] According to one aspect of the present application, an embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device can execute the method in the corresponding embodiment above, so it will not be repeated here. For technical details not disclosed in the computer program product embodiment involved in the present application, please refer to the description of the method embodiment of the present application.

[0326] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer programs. When a computer program is loaded and executed on a computer device, the process or function described in the embodiment of the present application is generated in whole or in part; the computer device may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program may be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium; the computer program may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer device can access or a data processing device such as a server or data center that includes one or more available media integrations; wherein the available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0327] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Receiving a data stream transmitted in a dialog interaction scenario, wherein the data stream is streamed in a data segment manner; During the streaming transmission of the data segments, integrity detection processing is performed on the received data segments to obtain integrity detection results, where the integrity detection results are used to indicate at least one target data segment that has been received and can constitute a data unit; constructing the at least one target data segment into a data unit, and performing data parsing processing on the data unit; The parsed data unit is output in the dialog interaction scenario.

2. The method according to claim 1, characterized in that Any data fragment is composed of at least one character; the integrity detection process is performed on the received data fragment to obtain the integrity detection result, including: During the streaming transmission of the data segments, an integrity check process is performed on each received data segment i; the integrity check process is used to detect whether the data segment i contains a preset identifier; If the data segment i does not include a preset identifier, storing the data segment i in a buffer; Perform integrity check on data segment i+1 until an integrity check result is obtained.

3. The method according to claim 2, characterized in that The performing integrity detection processing on the data segment i+1 until an integrity detection result is obtained includes: If the data fragment i+1 includes a preset identifier, both the data fragment i and the data fragment i+1 are used as target data fragments that can constitute a data unit, and an integrity detection result is obtained based on the data fragment i and the data fragment i+1; or, If the data segment i+1 does not include the preset identifier, the data segment i+1 is stored in the buffer, and the data segment i+2 is subjected to integrity detection processing until an integrity detection result is obtained; Wherein, each data fragment in the buffer is stored in sequence.

4. The method according to claim 1, characterized in that At least one data segment transmitted by streaming is stored in a buffer for sequential storage, and any of the data segments is composed of at least one character; the integrity detection result is used to indicate at least one target data segment that can constitute each data unit in at least one data unit, and the at least one data unit includes a first data unit; the integrity detection processing is performed on the received data segment to obtain the integrity detection result, including: Acquire M data fragments received through streaming transmission, where M is a positive integer; wherein the M data fragments refer to: data fragments received within a preset time period, or all data fragments allowed to be stored in the buffer; Detect whether each data fragment j in the M data fragments contains a preset identifier, where j is a positive integer and 1≤j≤M; If the data segment j includes a preset identifier, obtaining at least one data segment to be spliced ​​that is located before the data segment j in a storage order from the buffer; The at least one data segment to be spliced ​​and the data segment j are used as at least one target data segment that can constitute a first data unit.

5. The method according to claim 4, characterized in that The at least one data unit further includes a second data unit; and the method further includes: clearing the first data unit from the buffer, and detecting whether each of the other data fragments after the first data unit is cleared from the buffer contains a preset identifier; If it is detected that the data segment k contains a preset identifier, obtaining at least one data segment to be spliced ​​that is located before the data segment k in a storage order from the buffer, where k is a positive integer and j+1≤k≤M; The at least one data segment to be spliced ​​and the data segment k are used as at least one target data segment that can constitute a second data unit.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire a data detection rule for performing integrity detection processing in a dialog interaction scenario, wherein the data detection rule defines: at least one type of data unit to be constructed, and an identifier to be detected corresponding to each type of data unit; Acquire a target type of a data unit to be constructed, and determine a preset identifier to be detected corresponding to the data unit of the target type based on the data detection rule; Performing detection processing on the preset identifier for each data segment i; The data unit includes any one of row data, column data and paragraph data.

7. The method according to claim 1, characterized in that The performing data parsing processing on the data unit includes: Using a preset parsing method, the data unit is subjected to data parsing processing to obtain a parsing result of the data unit; the parsing result includes a parsed parsing script object in the data unit; The preset parsing method includes: any one of a data parsing tool, a parsing code file, a regular expression, and a parsing library.

8. The method according to claim 7, characterized in that The adopting a preset parsing method to perform data parsing processing on the data unit to obtain a parsing result of the data unit includes: Obtaining a target data format corresponding to the data unit; Determine a target parsing library that matches the target data format; wherein different data formats match different parsing libraries; The target parsing library is used to perform data parsing processing on the data unit of the target data format to obtain the parsing result of the data unit.

9. The method according to claim 1, characterized in that Outputting the parsed data unit in the dialogue interaction scenario includes: In the dialog interaction scenario, the parsed data unit is output in a unit display mode, wherein the unit display mode includes: any one of a whole row display, a whole column display, and a whole paragraph display; or, The parsed data unit is outputted in the dialog interaction scenario in an animated display manner that displays word by word.

10. The method according to claim 9, characterized in that The parsed data unit includes a parsed script object of the data unit; and outputting the parsed data unit in a word-by-word animation display manner in the dialogue interaction scene includes: Setting animation attribute parameters for the parsing script object of the data unit, wherein the animation attribute parameters are used to indicate attribute parameters possessed by the data unit during animation output; Animation rendering is performed on the parsed script object provided with the animation attribute parameters, and the parsed data unit is outputted in the dialog interaction scene in a word-by-word animation display manner.

11. The method according to claim 10, characterized in that The step of setting the animation attribute parameters for the parsing script object of the data unit includes: Using an animation control tool, dynamically setting a first property parameter of the cascading style sheet of the parsed script object, the first property parameter comprising: at least one of an animation name, an animation duration, a timing function, an animation delay, a direction, a fill mode, and a play state; Using an animation control tool, dynamically setting a second attribute parameter of the cascading style sheet of the parsed script object, wherein the second attribute parameter includes at least one of the number of animation steps and the end position; Using an animation control tool, dynamically setting a third attribute parameter of the cascading style sheet of the parsed script object, the third attribute parameter comprising: at least one of display width and display length; The first property parameter, the second property parameter, and the third property parameter are combined into animation property parameters of the parsed script object.

12. The method according to claim 10 or 11, characterized in that The dialogue interaction scenario includes a voice interaction scenario between a target object and a large language model, and the data unit is row data; the method further includes: Receiving a task processing instruction issued by a target object in the dialogue interaction scenario; Sending the task processing instruction to the server for analysis and processing, wherein the analysis and processing is used to indicate that the data stream to be replied is obtained by analyzing the task processing instruction; The data stream transmitted by the server is received, and each line of data in the data stream is output word by word through the large language model.

13. A data processing device, characterized in that: include: A receiving module, used to receive a data stream transmitted in a dialogue interaction scenario, wherein the data stream is streamed in the form of data segments; a processing module, configured to perform integrity detection processing on the received data fragments during the streaming transmission of the data fragments, and obtain an integrity detection result, wherein the integrity detection result is used to indicate at least one target data fragment that has been received and can constitute a data unit; The processing module is further used to construct the at least one target data segment into a data unit and perform data parsing processing on the data unit; An output module is used to output the parsed data unit in the dialogue interaction scenario.

14. A computer device, characterized in that: include: Memory and processor; a memory storing one or more computer programs; A processor, configured to load the one or more computer programs to implement the data processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the data processing method according to any one of claims 1 to 12.

Images