Streaming data processing method, device, electronic device, and storage medium

By combining the streaming calculation operator and mapping operator in streaming data processing, the features of streaming data are extracted, and the problem that streaming operation algorithms in the prior art cannot meet high real-time and accuracy is solved, and more accurate and efficient are achieved streaming data processing.

CN115801744BActive Publication Date: 2024-11-26BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211202599.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-28
Publication Date
2024-11-26
Estimated Expiration
2042-09-28

AI Technical Summary

Technical Problem

Existing streaming algorithms cannot meet the high requirements for real-time and accuracy, especially in the fields of speech recognition and video processing, making it difficult to effectively process streaming data.

Method used

The stream calculation operator and the mapping operator are used to process the stream data, and the data frame results at the sampling time are determined through the stream calculation operator, and the features are extracted by the mapping operator, thereby improving the accuracy and real-timeness of the stream calculation.

Benefits of technology

By adding mapping operators, more features can be extracted from the input streaming data, ensuring the accuracy of streaming operation results, while avoiding operation interruptions, and improving the timeliness and smoothness of streaming operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115801744B_ABST
    Figure CN115801744B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method, device, electronic device, and storage medium for processing streaming data, and relates to the field of data processing, and in particular to the field of streaming data processing. A specific implementation scheme is: inputting the first streaming data obtained according to the streaming data into a stream computing operator, and determining the first result of a data frame whose sampling time in the first streaming data is a target time according to the stream computing operator; inputting the second streaming data obtained according to the streaming data into a mapping operator, and determining the second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator; and determining the streaming operation result corresponding to the streaming data according to the first result and the second result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to streaming data processing technology. Background Art

[0002] Streaming computing can generate data while calculating the corresponding results, and is widely used in speech recognition and video processing. With the progress of the times and the development of science and technology, the real-time and accuracy requirements of streaming computing are getting higher and higher, and the current streaming computing algorithms can no longer meet user needs. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device, and storage medium for processing streaming data that can improve the accuracy and real-time performance of streaming data processing.

[0004] According to a first aspect of the present disclosure, a method for processing streaming data is provided, for performing streaming operations on streaming data including at least two data frames, wherein the streaming operations include a streaming operation operator and a mapping operator; the processing method includes:

[0005] Inputting first streaming data obtained according to the streaming data into a stream computing operator, and determining a first result of a data frame whose sampling time in the first streaming data is a target time according to the stream computing operator;

[0006] Inputting second streaming data obtained according to the streaming data into a mapping operator, and determining a second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator;

[0007] A streaming operation result corresponding to the streaming data is determined according to the first result and the second result.

[0008] According to a second aspect of the present disclosure, a method for processing streaming data is provided, the processing method comprising:

[0009] Inputting streaming data including at least two data frames into a streaming processing model; the streaming processing model comprises at least one streaming operation layer, each streaming operation layer comprises a streaming calculation layer, a mapping layer and a connection layer;

[0010] Inputting the first streaming data obtained according to the streaming data into the stream computing layer, and determining the first result of the data frame whose sampling time is the target time according to the stream computing operator included in the computing layer;

[0011] Inputting the second streaming data obtained according to the streaming data into a mapping layer, and determining a second result of a data frame whose sampling time is the target time according to a mapping operator included in the mapping layer;

[0012] A streaming operation result corresponding to the streaming data is determined according to the first result, the second result, and the connection operator included in the connection layer.

[0013] According to a third aspect of the present disclosure, a streaming data processing device is provided, which is used to perform streaming operations on streaming data including at least two data frames, wherein the streaming operations include a streaming operation operator and a mapping operator; the processing device includes:

[0014] A first calculation module, configured to input first streaming data obtained according to the streaming data into a stream calculation operator, and determine a first result of a data frame whose sampling time in the first streaming data is a target time according to the stream calculation operator;

[0015] A second calculation module, configured to input second streaming data obtained according to the streaming data into a mapping operator, and determine a second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator;

[0016] A determination module is used to determine a streaming operation result corresponding to the streaming data according to the first result and the second result.

[0017] According to a fourth aspect of the present disclosure, a streaming data processing device is provided, the processing device comprising:

[0018] An input module, used for inputting streaming data including at least two data frames into a streaming processing model; the streaming processing model comprises at least one streaming operation layer, each streaming operation layer comprises a streaming calculation layer, a mapping layer and a connection layer;

[0019] A computing module is used to trigger the stream computing layer to use a stream computing operator to calculate the first streaming data obtained according to the streaming data, and obtain a first result of the data frame whose sampling time is the target time; trigger the mapping layer to use a mapping operator to calculate the second streaming data obtained according to the streaming data, and obtain a second result of the data frame whose sampling time is the target time; and trigger the connection layer to determine the streaming operation result corresponding to the streaming data according to the first result and the second result.

[0020] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

[0024] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to any one of the above items.

[0025] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements any of the above methods when executed by a processor.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0028] Figure 1a A flowchart of a method for processing streaming data provided by an exemplary embodiment of the present disclosure;

[0029] Figure 1b An exemplary embodiment of the present disclosure provides a Figure 1a Flowchart of step 102;

[0030] Figure 2a A schematic diagram of a stream computing architecture provided for an exemplary embodiment of the present disclosure;

[0031] Figure 2b A schematic diagram of another stream computing architecture provided for an exemplary embodiment of the present disclosure;

[0032] Figure 3 A flowchart of another method for processing streaming data provided by an exemplary embodiment of the present disclosure;

[0033] Figure 4 A flowchart of another method for processing streaming data provided by an exemplary embodiment of the present disclosure;

[0034] Figure 5 A schematic diagram of a module of a streaming data processing device provided by an exemplary embodiment of the present disclosure;

[0035] Figure 6 A module schematic diagram of another streaming data processing device provided by an exemplary embodiment of the present disclosure;

[0036] Figure 7 The present invention provides a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0038] The embodiment of the present disclosure provides a method for processing streaming data, which is used to perform at least one level of streaming operation on streaming data, wherein each level of streaming operation includes a streaming operation operator and a mapping operator. Figure 1a , the processing method comprises the following steps:

[0039] Step 101: input first streaming data obtained according to streaming data into a stream computing operator, and determine a first result of a data frame whose sampling time in the first streaming data is a target time according to the stream computing operator.

[0040] The data source continuously generates data to form a stream, and the stream operation is to generate data while calculating the corresponding results. The stream data in step 101 is the currently generated data, which includes multiple frames of data obtained by continuous sampling. For example, if the currently generated stream data is {x 1 ,x 2 ,…,x t1}, then for {x 1 ,x 2 ,…,x t1} performs streaming operations; then generates new streaming data {x t1+1 ,…,x t2}, then for {x t1+1 ,…,x t2} performs streaming operations. 1 ,x 2 ,…,x t1 The number of frames contained in the data frame is the same as {x t1+1 ,…,x t2 The number of frames contained in the data frames can be the same or different.

[0041] The stream computing operator can be set according to actual needs. For example, the stream computing operator can be determined according to the type of streaming data and the accuracy requirement. The computing framework of the stream computing operator can be but is not limited to any one of the Strom operator, Heron operator, Spark streaming operator, Flink operator and Apache Kafka operator. The algorithm can be but is not limited to single convolution, multiple convolutions, mask algorithm, etc., and the embodiments of the present disclosure do not specifically limit this.

[0042] Step 102: input the second streaming data obtained according to the streaming data into a mapping operator, and determine a second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator.

[0043] The mapping operator is used to extract features from the second streaming data and determine the extracted features as the second result.

[0044] In one embodiment, when a first-level streaming operation is used to perform streaming operations on streaming data, see Figure 2a , the streaming data input into the streaming operation are respectively determined as the first streaming data and the second streaming data. In this embodiment, the first streaming data is the same as the second streaming data.

[0045] In one embodiment, when performing streaming operations on streaming data using at least two stages of cascaded streaming operations, see Figure 2b Taking the cascaded three-level streaming operation as an example, for the first-level streaming operation, the streaming data input into the first-level streaming operation is respectively determined as the first streaming data and the second streaming data, and the first streaming data of the first-level streaming operation is the same as the second streaming data; for the second-level streaming operation, the operation results of the first-level streaming operation are respectively determined as the first streaming data and the second streaming data, and the first streaming data of the second-level streaming operation is the same as the second streaming data; for the third-level streaming operation, the operation results of the second-level streaming operation are respectively determined as the first streaming data and the second streaming data, and the first streaming data of the third-level streaming operation is the same as the second streaming data.

[0046] It should be noted that Figure 2b The following only takes three-level streaming operations as an example. The number of streaming operations can be set according to actual needs.

[0047] In one embodiment, when streaming data is streamed using at least two levels of cascaded streaming operations, for a first level of streaming operations, the streaming data input into the first level of streaming operations are respectively determined as first streaming data and second streaming data, and the first streaming data of the first level of streaming operations are the same as the second streaming data; for the Lth level of streaming operations, the streaming data input into the first level of streaming operations are respectively determined as first streaming data and second streaming data. 1 Level stream operation, the Lth 1The result of the first-level stream operation is determined as the first stream data, and the Lth 2 The data of the first-level streaming operation is determined as the second streaming data. 2 <L 1 , L 2 and L 1 are all positive integers, L 2 The level flow operation is L 1 The superior level of streaming operations.

[0048] From the perspective of information theory, due to the existence of DPI (data processing inequality), in the process of forward transmission, as the number of levels increases, the features obtained by each level of streaming operation will decrease step by step. The addition of mapping operators can transfer the features of the upper level to the lower level, ensuring that the lower-level streaming operation contains more features than the upper-level streaming operation, thereby enabling the lower-level streaming operation to obtain more features, and thus making the calculation results more accurate.

[0049] In one embodiment, the mapping operator includes a mapping sub-operator and a time control sub-operator. Figure 1b , step 102 specifically includes:

[0050] Step 102-1: perform operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result.

[0051] The mapping sub-operator can be set according to actual needs. For example, the mapping sub-operator can be determined according to the type and accuracy requirements of the streaming data. For example, the mapping sub-operator includes convolution operations, multiplication operations, normalization operations, etc., or the output result of the mapping sub-operator is the same as the input second streaming data.

[0052] The number of mapping operators included in the stream operation can be one or more. When multiple mapping operators are included, the mapping operators can be the same or different. Multiple mapping operators can be learned.

[0053] Step 102-2: Determine the target result corresponding to the dependent data frame in the initial operation result according to the time control sub-operator, and determine the result other than the target result in the initial operation result as the second result.

[0054] The dependent data frame is determined according to the stream computing operator.

[0055] During the flow calculation process, the calculation of the current data frame depends on the data frames adjacent to it, including n frames of data before the current data frame and / or m frames of data after the current data frame. The dependent data frame is the data frame on which the calculation of the current data frame depends. Among them, n and m are positive integers, and the values ​​can be determined according to actual needs.

[0056] For example, the stream computing operator relies on the last x frames of the input streaming data as the data frame, and the time control sub-operator determines the last x frames in the initial operation result as the target result, and determines the results in the initial operation result other than the target result as the second result.

[0057] In one embodiment, after step 102 - 2 , the target result is further stored for performing streaming operations on the newly generated streaming data.

[0058] When determining the target result, first determine whether there is a target result corresponding to the previous streaming data. If the judgment result is yes, it means that the current streaming data is not the first input (that is, it is not the first packet of data), then the target result corresponding to the previous streaming data and the initial operation result corresponding to the current streaming data are spliced ​​in time sequence, and the target result corresponding to the current streaming data is determined from the spliced ​​result. If the judgment result is no, it means that the current streaming data is the first input, then there is no need to perform the splicing operation, and the target result is determined directly from the initial operation result.

[0059] In the disclosed embodiments, it is possible to ensure that the sampling times of the first result and the second result of each level of streaming operation obtained for each input of streaming data are the same, thereby avoiding operation interruptions, thereby ensuring the smoothness of the streaming operation and improving the timeliness of the streaming operation.

[0060] In one embodiment, the target data frame in the second streaming data that matches the sampling time of the dependent data frame is stored for performing streaming operations on the newly generated streaming data.

[0061] When the second streaming data is operated according to the mapping sub-operator, first determine whether there is a target data frame of the previous streaming data. If the judgment result is yes, it means that the current streaming data is not input for the first time (that is, it is not the first packet of data), then the target data frame of the previous streaming data is spliced ​​with the second streaming data in time sequence, and the mapping sub-operator is used to operate the spliced ​​result to obtain the initial operation result, and then the target result is directly determined from the initial operation result. If the judgment result is no, it means that the current second streaming data is input for the first time, then there is no need to perform a splicing operation, and the mapping sub-operator is directly used to operate the input second streaming data to obtain the initial operation result, and then the target result is directly determined from the initial operation result.

[0062] Step 103: Determine a streaming operation result corresponding to the streaming data according to the first result and the second result.

[0063] In step 103, the first result and the second result may be added, concatenated, multiplied, etc. to obtain a stream operation result. Since the sampling time of the data frame corresponding to the first result and the second result is the same, the first result and the second result can be effectively operated.

[0064] If new streaming data is generated later, steps 101 to 103 are repeatedly executed to obtain streaming operation results corresponding to the new streaming data.

[0065] In the disclosed embodiment, a mapping operator is added on the basis of the stream computing operator, and the mapping operator can extract more features from the input stream data as the second result, so that the stream computing result determined according to the first result and the second result is more accurate. And the sampling time of the data frame corresponding to the first result and the second result is the same, so the first result and the second result can be effectively calculated, avoiding the interruption of the calculation caused by the different sampling time of the first result and the second result, thereby further improving the accuracy of the stream computing result, ensuring the smooth progress of the stream computing, and improving the timeliness.

[0066] The method for processing streaming data in the embodiment of the present disclosure is applicable to processing various types of streaming data, such as video data, voice data, etc. The processing process is further described below using voice data as an example.

[0067] Figure 3 A flowchart of another method for processing streaming data provided by an exemplary embodiment of the present disclosure, which is used to perform streaming calculations on voice streaming data, wherein the number of stream operations in the figure is Figure 2a The processing is further described by taking the first level shown as an example. Figure 3 , the processing method comprises the following steps:

[0068] Step 301: Input the voice streaming data generated for the first time into the streaming operation.

[0069] The voice streaming data includes the data frames {x 1 ,x 2 ,…,x t1}.

[0070] Step 302: determine the voice streaming data generated for the first time as the first streaming data and input it into the stream computing operator, and determine the first result of the data frame whose sampling time is the target time in the first streaming data according to the stream computing operator.

[0071] Assuming that the dependent data frame of the stream computing operator is the last x frames of the input voice streaming data, the target time is the sampling time corresponding to the sampling times t1-x+1 to t1, and the first result is the time corresponding to the data frame {x 1 ,x2 ,…,x t1-x The result of {y 1 ,y 2 ,…,y t1-x}.

[0072] Step 303: store the data frame {x t1-x+1 ,…,x t1}.

[0073] {x t1-x+1 ,…,x t1} is used to perform streaming operations on the voice streaming data generated next time.

[0074] Step 304: determine the voice streaming data generated for the first time as the second streaming data and input it into a mapping operator, and determine a second result of a data frame whose sampling time is the target time in the second streaming data according to the mapping operator.

[0075] In step 304, the mapping operator includes mapping sub-operators for the data frame {x 1 ,x 2 ,…,x t1} to perform the operation and obtain the initial operation result {h 1 ,h 2 ,…,h t1}, the time control operator will {h 1 ,h 2 ,…,h t1 The last x frames of results in {h t1-x+1 ,…,h t1} is determined as the target result, and {h 1 ,h 2 ,…,h t1-x} is determined as the second result.

[0076] Step 305: store the target result {h t1-x+1 ,…,h t1}.

[0077] {h t1-x+1 ,…,h t1} is used to perform streaming operations on the voice streaming data generated next time.

[0078] Step 306: Determine the streaming operation result of the voice streaming data generated for the first time according to the first result and the second result.

[0079] For the first generated voice streaming data, the first result is {y 1 ,y 2 ,…,y t1-x}, the second result is {h 1 ,h 2,…,h t1-x}, taking the sum operation of the first result and the second result as an example, the streaming operation result of the voice streaming data generated for the first time is {h 1 +y 1 ,h 2 +y 2 ,…,h t1-x +y t1-x}, since the sampling time of the data frames corresponding to the first result and the second result is the same, the first result and the second result can be effectively added.

[0080] Step 307: Input the voice streaming data generated for the second time into the level streaming operation.

[0081] The second generated voice streaming data includes the data frames {x t1+1 ,x t1+2 ,…,x t2}.

[0082] Step 308: Determine the voice streaming data generated for the second time as the first streaming data and input it into the stream computing operator, and determine the first result of the data frame whose sampling time is the target time in the first streaming data according to the stream computing operator.

[0083] When the stream computing operator is used to compute the second generated voice streaming data, {x t1-x+1 ,…,x t1} and the second generated voice streaming data {x t1+1 ,x t1+2 ,…,x t2} are spliced ​​in time sequence to obtain {x t1-x+1 ,…,x t2}, using stream computing operators to compute {x t1-x+1 ,…,x t2} to calculate {y t1-x+1 ,…,y t2}, the dependent data frame of the stream computing operator is the last x frames of the input voice streaming data, then the target time is the sampling time corresponding to the sampling times t2-x+1 to t2, and the first result is the time corresponding to the data frame {x t1-x+1 ,…,x t2-x The result of {y t1-x+1 ,…,y t2-x}.

[0084] Step 309: store the data frame {x t2-x+1 ,x t2}.

[0085] {x t2-x+1 ,x t2} is used to perform streaming operations on the voice streaming data generated next time.

[0086] In one embodiment, before executing step 309, it is first determined whether the second generated voice streaming data is the last input, that is, whether it is the last packet data. If the second generated voice streaming data is the last packet data, step 309 does not need to be executed.

[0087] Step 310: determine the voice streaming data generated for the second time as the second streaming data and input it into a mapping operator, and determine a second result of a data frame whose sampling time is the target time in the second streaming data according to the mapping operator.

[0088] In step 310, the mapping operator includes mapping sub-operators for the data frame {x t1+1 ,x t1+2 ,…,x t2} to perform the operation and obtain the initial operation result {h t1+1 ,h t1+2 ,…,h t2}, the time control operator stores the target result {h t1-x+1 ,…,h t1} and {h t1+1 ,h t1+2 ,…,h t2} are spliced ​​in time sequence to obtain {h t1-x+1 ,…,h t2}, and {h t1-x+1 ,…,h t2 The last x frames of results in {h t2-x+1 ,…,h t2} is determined as the target result, and {h t1-x+1 ,…,h t2-x} is determined as the second result.

[0089] Step 311: store the target result {h t2-x+1 ,…,h t2}.

[0090] {h t2-x+1 ,…,h t2} is used to perform streaming operations on the voice streaming data generated next time.

[0091] In one embodiment, before executing step 311, it is first determined whether the second generated voice streaming data is the last input, that is, whether it is the last packet data. If the second generated voice streaming data is the last packet data, step 311 does not need to be executed.

[0092] Step 312: Determine the streaming operation result of the voice streaming data generated for the second time according to the first result and the second result.

[0093] For the second generated voice streaming data, the first result is {y t1-x+1 ,…,y t2-x}, the second result is {h t1-x+1 ,…,h t2-x}, taking the sum operation of the first result and the second result as an example, the streaming operation result of the voice streaming data generated for the second time is {h t1-x+1 +y t1-x+1 ,…,h t2-x}, since the sampling time of the data frames corresponding to the first result and the second result is the same, the first result and the second result can be effectively added.

[0094] When new voice streaming data is generated, the above steps are repeated by analogy until the voice streaming data is the tail packet data.

[0095] If the type of the streaming data is other data such as a video stream, the process of performing streaming operations on it is similar to steps 301 to 312. The difference is that the stream computing operator and the mapping operator can select operators that are compatible with the type of the streaming data. The specific implementation process will not be traced here.

[0096] Figure 4 A flowchart of another method for processing streaming data provided by an exemplary embodiment of the present disclosure, wherein the method processes streaming data by using a pre-trained streaming processing model, see Figure 4 The processing method comprises the following steps:

[0097] Step 401: Input streaming data including at least two data frames into a streaming processing model.

[0098] The stream processing model includes at least one stream operation layer, and each stream operation layer includes a stream calculation layer, a mapping layer and a connection layer.

[0099] The streaming processing model is obtained by training the initial streaming processing model using training samples. The specific process of model training can be found in the relevant technical description and will not be repeated here.

[0100] The embodiment of the present disclosure adopts a streaming processing model that can be built by itself, or a residual network can be used, and the present disclosure does not make any special limitations on this.

[0101] Step 402: input the first streaming data obtained according to the streaming data into the stream computing layer, and determine the first result of the data frame whose sampling time is the target time according to the stream computing operator included in the computing layer.

[0102] The specific process of determining the first result is similar to step 301 and will not be repeated here.

[0103] Step 403: input the second streaming data obtained according to the streaming data into a mapping layer, and determine a second result of a data frame whose sampling time is the target time according to a mapping operator included in the mapping layer.

[0104] The specific process of determining the first result is similar to step 302 and will not be repeated here.

[0105] Step 404: Determine a streaming operation result corresponding to the streaming data according to the first result, the second result, and the connection operator included in the connection layer.

[0106] The specific process of determining the stream operation result is similar to step 303 and will not be repeated here.

[0107] In one embodiment, the mapping operator includes a mapping sub-operator and a time control sub-operator; determining the second result of the data frame whose sampling time in the second streaming data is the target time according to the mapping operator, including: performing operations on the second streaming data according to the mapping sub-operator to obtain an initial operation result; determining the target result corresponding to the dependent data frame in the initial operation result according to the time control sub-operator, and determining the results in the initial operation result other than the target result as the second result; the dependent data frame is determined according to the stream calculation operator.

[0108] In one embodiment, the second streaming data is operated according to the mapping sub-operator, including: determining whether there is a target result corresponding to the second streaming data input last time; if the determination result is yes, then combining the target result of the second streaming data input last time with the second streaming data input this time, and operating the combined result according to the mapping sub-operator.

[0109] In one embodiment, when the stream processing model includes at least two cascaded stream computing layers, the stream computing result output by the stream computing layer of the upper layer of the two adjacent stream computing layers is used as the first stream data input to the stream computing layer of the lower layer;

[0110] And / or, when the streaming processing model includes at least two cascaded streaming operation layers, the streaming operation result of the streaming operation of the upper layer of two adjacent streaming operation layers is used as the second streaming data input to the streaming operation layer of the next layer, or the input data of the streaming operation layer of any layer before the streaming operation layer of the next layer is used as the second streaming data input to the streaming operation layer of the next level.

[0111] Experiments have shown that as the number of streaming operation layers increases, the network degrades. When the network degrades, the shallow network can achieve better training results than the deep network.

[0112] From the perspective of information theory, due to the existence of DPI (data processing inequality), in the process of forward transmission, as the number of levels increases, the features obtained by each level of streaming operations will decrease step by step. The addition of mapping operators can transfer the features of the upper streaming operation layer in the streaming processing model to the lower streaming operation layer, ensuring that the lower streaming operation layer contains more features than the upper streaming operation layer, thereby making the calculation results more accurate.

[0113] Corresponding to the aforementioned embodiment of the method for processing streaming data, the present disclosure also provides an embodiment of an apparatus for processing streaming data.

[0114] Figure 5 A schematic diagram of a module of a streaming data processing device provided by an exemplary embodiment of the present disclosure, wherein the processing device is used to perform streaming operations on streaming data including at least two data frames, wherein the streaming operations include a streaming operation operator and a mapping operator; see Figure 5 , the processing device comprises:

[0115] A first calculation module 51 is used for inputting first streaming data obtained according to the streaming data into a stream calculation operator, and determining a first result of a data frame whose sampling time in the first streaming data is a target time according to the stream calculation operator;

[0116] A second calculation module 52 is used to input the second streaming data obtained according to the streaming data into a mapping operator, and determine a second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator;

[0117] The determination module 53 is used to determine the streaming operation result corresponding to the streaming data according to the first result and the second result.

[0118] Optionally, the mapping operator includes a mapping sub-operator and a time control sub-operator;

[0119] The second calculation module is specifically used for:

[0120] Performing operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result;

[0121] The target result corresponding to the dependent data frame in the initial operation result is determined according to the time control sub-operator, and the result other than the target result in the initial operation result is determined as the second result; the dependent data frame is determined according to the stream computing operator.

[0122] Optionally, when the second streaming data is operated according to the mapping sub-operator, the second computing module is specifically used to:

[0123] Determine whether there is a target result corresponding to the second streaming data input last time;

[0124] If the judgment result is yes, the target result of the second streaming data input last time is combined with the second streaming data input this time, and the combined result is calculated according to the mapping sub-operator.

[0125] Optionally, when at least two levels of streaming operations are used to perform streaming operations on the streaming data, the first computing module is specifically configured to use the streaming operation result of the upper level of the streaming operations in the two adjacent levels of streaming operations as first streaming data input to the next level of streaming operations;

[0126] And / or, when at least two levels of streaming operations are used to perform streaming operations on the streaming data, the second computing module is specifically used to use the operation result of the streaming operation of the previous level in two adjacent streaming operations as the second streaming data input to the streaming operation of the next level, or to use the input data of the streaming operation of any level before the streaming operation of the next level as the second streaming data input to the streaming operation of the next level.

[0127] Figure 6 A schematic diagram of a module of another streaming data processing device provided by an exemplary embodiment of the present disclosure, the processing device comprising:

[0128] An input module 61, used for inputting streaming data including at least two data frames into a streaming processing model; the streaming processing model comprises at least one streaming operation layer, each streaming operation layer comprises a streaming calculation layer, a mapping layer and a connection layer;

[0129] The computing module 62 is used to trigger the stream computing layer to use the stream computing operator to calculate the first streaming data obtained according to the streaming data, and obtain a first result of the data frame whose sampling time is the target time, trigger the mapping layer to use the mapping operator to calculate the second streaming data obtained according to the streaming data, and obtain a second result of the data frame whose sampling time is the target time, and trigger the connection layer to determine the streaming operation result corresponding to the streaming data according to the first result and the second result.

[0130] Optionally, the mapping operator includes a mapping sub-operator and a time control sub-operator;

[0131] When determining the second result, the calculation module is specifically used to:

[0132] Performing operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result;

[0133] The target result corresponding to the dependent data frame in the initial operation result is determined according to the time control sub-operator, and the result other than the target result in the initial operation result is determined as the second result; the dependent data frame is determined according to the stream computing operator.

[0134] Optionally, when the second streaming data is operated according to the mapping sub-operator, the calculation module is specifically used to:

[0135] Determine whether there is a target result corresponding to the second streaming data input last time;

[0136] If the judgment result is yes, the target result of the second streaming data input last time is combined with the second streaming data input this time, and the combined result is calculated according to the mapping sub-operator.

[0137] Optionally, when the stream processing model includes at least two cascaded stream computing layers, the stream computing result output by the stream computing layer of the upper layer of the two adjacent stream computing layers is used as the first stream data input to the stream computing layer of the lower layer;

[0138] And / or, when the streaming processing model includes at least two cascaded streaming operation layers, the streaming operation result of the streaming operation of the upper layer of two adjacent streaming operation layers is used as the second streaming data input to the streaming operation layer of the next layer, or the input data of the streaming operation layer of any layer before the streaming operation layer of the next layer is used as the second streaming data input to the streaming operation layer of the next level.

[0139] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0140] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of the streaming data involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0142] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0143] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0144] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0145] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a processing method for streaming data. For example, in some embodiments, the processing method for streaming data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or a communication unit 709. When the computer program is loaded into RAM 803 and executed by the computing unit 701, one or more steps of the processing method for streaming data described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform a processing method for streaming data in any other appropriate manner (e.g., by means of firmware).

[0146] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0148] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0150] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0151] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0153] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing streaming data, for performing streaming operations on streaming data containing at least two data frames, wherein the streaming operations include a streaming operation operator and a mapping operator; the processing method include: Inputting first streaming data obtained according to the streaming data into a stream computing operator, and determining a first result of a data frame whose sampling time in the first streaming data is a target time according to the stream computing operator; Inputting second streaming data obtained according to the streaming data into a mapping operator, and determining a second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator; Determine a streaming operation result corresponding to the streaming data according to the first result and the second result; The mapping operator includes a mapping sub-operator and a time control sub-operator; Determining, according to the mapping operator, a second result of a data frame whose sampling time in the second streaming data is the target time, includes: Performing operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result; The target result corresponding to the dependent data frame in the initial operation result is determined according to the time control sub-operator, and the result other than the target result in the initial operation result is determined as the second result; the dependent data frame is determined according to the stream computing operator.

2. The method for processing streaming data according to claim 1, wherein the second streaming data is operated according to the mapping sub-operator, include: Determine whether there is a target result corresponding to the second streaming data input last time; If the judgment result is yes, the target result of the second streaming data input last time is combined with the second streaming data input this time, and the combined result is calculated according to the mapping sub-operator.

3. According to the method for processing streaming data of claim 1 or 2, when at least two levels of streaming operations are used to perform streaming operations on the streaming data, the streaming operation result of the upper level of the two adjacent levels of streaming operations is used as the first streaming data input to the next level of streaming operations; And / or, when at least two levels of streaming operations are used to perform streaming operations on the streaming data, the operation result of the streaming operation at the upper level in two adjacent levels of streaming operations is used as the second streaming data input to the streaming operation at the next level, or the input data of the streaming operation at any level before the streaming operation at the next level is used as the second streaming data input to the streaming operation at the next level.

4. A method for processing streaming data, the processing method include: Input streaming data including at least two data frames into the streaming processing model; The stream processing model includes at least one stream operation layer, each stream operation layer includes a stream calculation layer, a mapping layer and a connection layer; Inputting the first streaming data obtained according to the streaming data into the stream computing layer, and determining the first result of the data frame whose sampling time is the target time according to the stream computing operator included in the computing layer; Inputting the second streaming data obtained according to the streaming data into a mapping layer, and determining a second result of a data frame whose sampling time is the target time according to a mapping operator included in the mapping layer; Determine a streaming operation result corresponding to the streaming data according to the first result, the second result, and a connection operator included in the connection layer; Wherein, the mapping operator includes a mapping sub-operator and a time control sub-operator; Determining, according to the mapping operator, a second result of a data frame whose sampling time in the second streaming data is the target time, includes: Performing operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result; The target result corresponding to the dependent data frame in the initial operation result is determined according to the time control sub-operator, and the result other than the target result in the initial operation result is determined as the second result; the dependent data frame is determined according to the stream computing operator.

5. The method for processing streaming data according to claim 4, wherein the second streaming data is operated according to the mapping sub-operator, include: Determine whether there is a target result corresponding to the second streaming data input last time; If the judgment result is yes, the target result of the second streaming data input last time is combined with the second streaming data input this time, and the combined result is calculated according to the mapping sub-operator.

6. The method for processing streaming data according to claim 4 or 5, when the streaming processing model includes at least two cascaded streaming computing layers, the streaming computing result output by the upper streaming computing layer of two adjacent streaming computing layers is used as the first streaming data input to the lower streaming computing layer; And / or, when the streaming processing model includes at least two cascaded streaming operation layers, the streaming operation result of the streaming operation of the upper layer of two adjacent streaming operation layers is used as the second streaming data input to the streaming operation layer of the next layer, or the input data of the streaming operation layer of any layer before the streaming operation layer of the next layer is used as the second streaming data input to the streaming operation layer of the next layer.

7. A streaming data processing device, for performing streaming operations on streaming data containing at least two data frames, wherein the streaming operations include a streaming operation operator and a mapping operator; the processing device include: A first calculation module, configured to input first streaming data obtained according to the streaming data into a stream calculation operator, and determine a first result of a data frame whose sampling time in the first streaming data is a target time according to the stream calculation operator; A second calculation module, configured to input second streaming data obtained according to the streaming data into a mapping operator, and determine a second result of a data frame whose sampling time in the second streaming data is the target time according to the mapping operator; A determination module, configured to determine a streaming operation result corresponding to the streaming data according to the first result and the second result; Wherein, the mapping operator includes a mapping sub-operator and a time control sub-operator; The second calculation module is specifically used for: Performing operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result; Determine a target result corresponding to a dependent data frame in the initial operation result according to the time control sub-operator, and determine a result in the initial operation result other than the target result as the second result; The dependent data frame is determined according to the stream computing operator.

8. The streaming data processing device according to claim 7, when the second streaming data is operated according to the mapping sub-operator, the second computing module is specifically used to: Determine whether there is a target result corresponding to the second streaming data input last time; If the judgment result is yes, the target result of the second streaming data input last time is combined with the second streaming data input this time, and the combined result is calculated according to the mapping sub-operator.

9. The streaming data processing device according to claim 7 or 8, when at least two levels of streaming operations are used to perform streaming operations on the streaming data, the first computing module is specifically used to use the streaming operation result of the upper level of the two adjacent levels of streaming operations as the first streaming data input to the next level of streaming operations; And / or, when at least two levels of streaming operations are used to perform streaming operations on the streaming data, the second computing module is specifically used to use the operation result of the streaming operation of the previous level in two adjacent streaming operations as the second streaming data input to the streaming operation of the next level, or to use the input data of the streaming operation of any level before the streaming operation of the next level as the second streaming data input to the streaming operation of the next level.

10. A streaming data processing device, the processing device include: An input module, configured to input streaming data including at least two data frames into a streaming processing model; The stream processing model includes at least one stream operation layer, each stream operation layer includes a stream calculation layer, a mapping layer and a connection layer; A computing module, used to trigger the stream computing layer to use a stream computing operator to compute the first stream data obtained according to the stream data, to obtain a first result of a data frame whose sampling time is a target time, trigger the mapping layer to use a mapping operator to compute the second stream data obtained according to the stream data, to obtain a second result of the data frame whose sampling time is the target time, and trigger the connection layer to determine a stream computing result corresponding to the stream data according to the first result and the second result; Wherein, the mapping operator includes a mapping sub-operator and a time control sub-operator; When determining the second result, the calculation module is specifically used to: Performing operation on the second streaming data according to the mapping sub-operator to obtain an initial operation result; Determine a target result corresponding to a dependent data frame in the initial operation result according to the time control sub-operator, and determine a result in the initial operation result other than the target result as the second result; The dependent data frame is determined according to the stream computing operator.

11. The streaming data processing device according to claim 10, when the second streaming data is operated according to the mapping sub-operator, the computing module is specifically used to: Determine whether there is a target result corresponding to the second streaming data input last time; If the judgment result is yes, the target result of the second streaming data input last time is combined with the second streaming data input this time, and the combined result is calculated according to the mapping sub-operator.

12. The streaming data processing device according to claim 10 or 11, when the streaming processing model includes at least two cascaded streaming computing layers, the streaming computing result output by the upper streaming computing layer of two adjacent streaming computing layers is used as the first streaming data input to the lower streaming computing layer; And / or, when the streaming processing model includes at least two cascaded streaming operation layers, the streaming operation result of the streaming operation of the upper layer of two adjacent streaming operation layers is used as the second streaming data input to the streaming operation layer of the next layer, or the input data of the streaming operation layer of any layer before the streaming operation layer of the next layer is used as the second streaming data input to the streaming operation layer of the next layer.

13. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.