Method and system for processing data

Abstract

Methods and systems for processing of data are disclosed. Specifically, disclosed is a method (300) of processing data by applying a first algorithm in a system comprising a server and a client device, the method comprising, at the client device: continuously receiving (S302) data; dividing (S304) the received data into M subsets of data, Mgt; 1, each subset of data comprises or is sourced from sensor data captured by a sensor at a plurality of time points during a capture time interval; determining (S306), for each subset, whether the processing of the subset should be performed by the client device or by the server, and upon determining that the processing should be performed by the client device, processing (S308) the subset into processed data by applying a first algorithm to the subset, and transmitting (S310) the processed data to the server, and upon determining that the processing should be performed by the server, transmitting (S310) the subset of data to the server.

Description

Technical Field

[0001] The present invention relates to data processing, and in particular, to the distribution of data processing tasks between a client-side device and a server-side device. Background Art

[0002] Load sharing between client devices and servers is a method used to optimize the performance and responsiveness of applications by distributing computing tasks between client-side devices and server-side devices. This concept has evolved as the demand for network-based services has increased and the capabilities of client devices have increased.

[0003] In the early days of computing, most of the processing tasks were performed on the server side because client devices did not have the necessary capabilities. However, as client devices have become more powerful, it has become possible to perform some of the processing tasks on the client side, which can help reduce the load on the server.

[0004] In particular, cloud computing has given rise to new considerations. Cloud servers are typically charged based on usage, including not only the storage used, but also the processing power consumed. Therefore, it is not always cost-effective to move all processing tasks to the server. Distributing some of the processing load to the client can help reduce costs, but this needs to be balanced with other factors (e.g., the capabilities of the client device).

[0005] In any computing environment, the security of data is an important issue. For example, client-side processing may be more secure when the data to be processed is captured by the client device itself (e.g., by a camera). In this case, processing the data directly on the client device can reduce the risk of the data being intercepted and manipulated during transmission to the server.

[0006] The decision between client-side processing and server-side processing should be made based on careful consideration of the trade-offs between security, performance, and cost. Making such a decision can be a difficult task, and in some cases, making the best choice between client-side processing and server-side processing may be impossible.

[0007] Therefore, there is a need for improvement in this regard. Summary of the invention

[0008] In view of the above, it would be beneficial to address or at least mitigate one or more of the above mentioned disadvantages as set out in the accompanying independent patent claims.

[0009] According to a first aspect of the present invention, there is provided a method for processing data by applying a first algorithm in a system comprising a server and a client device.

[0010] The method includes: at a client device: continuously receiving data; dividing the received data into M subsets of data, M>1, each subset of data including or originating from sensor data captured by a sensor at multiple time points during a capture time interval; determining for each subset whether processing of the subset should be performed by the client device or by a server, and when it is determined that the processing should be performed by the client device, processing the subset into processed data by applying a first algorithm to the subset, and transmitting the processed data to the server, and when it is determined that the processing should be performed by the server, transmitting the subset of data to the server.

[0011] Determining for each subset whether processing of the subset should be performed by the client device or by the server includes: determining a duration indicating a processing time for applying a first algorithm to a subset of data; determining a time ratio between the duration and a capture time interval; when the time ratio is less than or equal to a first threshold time ratio, determining that the client device should perform processing of each subset in the M subsets of data; when the time ratio exceeds the first threshold time ratio, determining N subsets that are part of the M subsets, N≤M, where N decreases as the time ratio increases, and determining that the client device should perform processing of N subsets out of the M subsets.

[0012] The method further includes, at the server: receiving data from the client device, and upon determining that the data received from the client device has not been processed by the client device, processing the data received from the client device by applying a first algorithm to the data received from the client device.

[0013] A client device, which may also be referred to as an edge device or just a client, may in some cases include a sensor that captures sensor data. The client device may be, for example, a camera, and the sensor may be an image sensor or a microphone. In other examples, the client device is connected to the sensor.

[0014] The data received at the client device may be any type of sensor data, such as video data, audio data, radar data, gyroscope data, GPS data, etc. In some examples, the data received at the client device may be derived from such sensor data, for example, the output of an analysis of the sensor data, or data that otherwise originates from the sensor data, for example, in the form of metadata.

[0015] Data is continuously received at a client device and divided into subsets of data, wherein each subset of data includes sensor data captured at multiple time points during a capture time interval, or is derived from the sensor data. The capture time interval may be, for example, 1 second, 3 seconds, 5 seconds, 8 seconds, 10 seconds, etc. or any other suitable interval based on the application. In the case where the sensor data is video data, the capture interval may correspond to one or more groups of pictures (GOPs) used for encoding of the video stream, each GOP including multiple image frames (e.g., 15 or 30 image frames) from the video data. The capture interval may be predetermined or variable.

[0016] "Processed data" as defined in this specification refers to data that has undergone some form of transformation or addition that makes it different from its original, unprocessed state. Such transformation may involve the inclusion of supplemental information such as metadata, hash values, or results from analysis of sensor data. In these cases, the processed data includes the original, unprocessed data and the added information. Alternatively, the processed data may be a modified version of the original data, such as by encoding or encrypting the original data.

[0017] The inventors have recognized that, in the case where data is continuously received by a client device and a subset of the received data is required to be processed by applying a first algorithm, it may be difficult to perform all processing at the client device, depending on the time required for the client device to perform the processing. For example, if the time required to process a subset of data is longer than the time required to receive the next subset of data, there will be a backup of data waiting to be processed. This in turn may cause the buffer of the client device to fill up, and if the buffer becomes completely full, further received data will be lost. An alternative is to transmit the unprocessed data to a server to be processed at the server. However, due to reasons such as efficiency, cost and security as described above, it may be advantageous for the client device to process as many subsets of data as possible. In order to solve this problem, the inventors have implemented a method for load sharing of data processing, wherein the load sharing percentage is determined by the client device based on an estimate of the time (duration) that may be required to process a subset of data and the capture time interval of the subset of data.

[0018] The load sharing may be determined based on a time ratio between the duration (indicating a processing time to apply the first algorithm to a subset of the data) and the capture time interval. It should be noted that the time ratio may be determined continuously such that the load sharing is updated based on, for example, complexity of received data, changes in available buffer memory at the client device, changes in available processing power at the client device, etc.

[0019] The time ratio may be compared to a threshold to determine load sharing. For example, if the time ratio is below a threshold time ratio, it may be determined by the client device that the client device should (temporarily) perform processing of a subset of each data. The threshold time ratio may be a value less than or equal to 1, which may indicate that the processing time does not result in a need for backup of the next subset of data, as described above.

[0020] In an example, if the time ratio exceeds a threshold time ratio, the client device can determine that a portion of a subset of sensor data (e.g., every second subset, every third subset, 2 / 3 subset, 3 / 5 subset, etc.) should be processed by the client device. The larger the time ratio, the smaller the portion of the subset that should be processed by the client device. Therefore, the portion can be determined so that the buffer memory of the client device is not filled with subsets of data waiting to be processed by the client device.

[0021] Therefore, the client device performs processing according to the decision made by the client device and transmits the processed data to the server.When it is determined by the client device that processing should be performed by the server, a subset of the data (i.e., a subset of unprocessed data) is transmitted to the server.

[0022] On the server side, the server receives data from the client device, a subset of processed data or a subset of unprocessed data. The server is configured to identify whether the received data is processed, and if a set of received data is not processed, the server will perform processing of such data. The server can, for example, be configured to check a value in a header of the received data indicating whether the data is processed. The server can check whether a value of a parameter (variable, attribute, feature, element, etc.) in the received data exists. In other embodiments, the server can identify whether the data is processed by identifying the format or encryption status of the data.

[0023] In some examples, determining the duration includes determining a processing time for applying the first algorithm to the subset of data and adding a margin to the determined processing time. Advantageously, such a margin can act as a safety margin in the event that the determined processing time is too short, if for some reason the complexity of the processing of the subset of data suddenly increases, if the available processing power at the client device suddenly decreases, etc.

[0024] In some embodiments, the margin is determined based on the bit size of the subset of data, wherein the margin increases as the bit size increases. For some processes, the processing time and / or the required processing power may increase as the bit size increases, and therefore, it may be advantageous to increase the safety margin depending on the bit size.

[0025] In some embodiments, N (the portion) is determined by determining the number of subsets of data that can be stored in a buffer memory of the client device, wherein N increases as the number of subsets of data that can be stored in a buffer memory of the client device increases. Thus, for a smaller buffer in which a smaller subset of data can be temporarily stored before being processed by the client device, a smaller portion of the subset can be processed by the client device than if the buffer memory is larger. Advantageously, the portion is determined to best utilize the available buffer memory.

[0026] In an example, the data received at the client device includes one of video data or audio data, wherein the data received at the client device is encoded using a compression standard. Advantageously, load sharing of processing of the encoded data can be achieved, such that encoding of the captured video data or audio data can still be performed prior to processing. Thus, systems (e.g., legacy systems) that implement immediate encoding of captured sensor data can be supported.

[0027] In some cases, the method further includes, at the server, using data received from the client device to stream the media content using a streaming technology, the streaming technology being one of: HTTP live streaming, the HLS standard, or the Dynamic Adaptive Streaming over HTTP (DASH) standard, wherein the segments in the streamed media content originate from a subset of data in a subset of the M data. Techniques for load sharing of data processing can be advantageously applicable to DASH or HLS streaming applications. For example, processing can include calculating a hash value, which in turn can be used to verify the authenticity of each segment of the media content being streamed. Using the techniques described herein, preparation for such verification of the authenticity of the content being streamed can be advantageously distributed between the client device and the server device.

[0028] In some embodiments, a first algorithm is used to calculate a hash of a subset of data, wherein the processed data includes the subset of data and the hash calculated based on the subset of data. Advantageously, hash values ​​can be calculated for all types of data, and the hash values ​​are then used to verify the authenticity of the data. For example, hash values ​​can be calculated for both raw video data and compressed video data, or for encrypted data and raw data. Hash values ​​can be calculated using any suitable algorithm such as SHA-256, MD5, CRC32, SHA-1, etc. The processing time for calculating a hash value for a subset of a particular data can advantageously be estimated based on the bit size of the subset of data, which in turn makes it possible for the client device to make accurate decisions about load sharing and / or use a small or zero margin to utilize the available processing power at the client device.

[0029] In some embodiments, the first algorithm is used to encrypt a subset of the data, and the processed data includes the encrypted subset of the data. Similar to the hash algorithm, the processing time for encrypting a particular subset of data is closely related to the bit size of the subset of data, which in turn makes it possible for the client device to make accurate decisions about load sharing and / or use a small margin or zero margin to utilize available processing power at the client device.

[0030] In an example, the first algorithm is a sensor data analysis algorithm, wherein the processed data includes a subset of the data and an output from the sensor data analysis algorithm. The sensor data analysis algorithm may require considerable processing power, which may increase the cost of running a cloud-based server if the algorithm is applied to the subset of the data at the server. At the same time, the available processing power on the client device may not be sufficient to run all analysis of the received data on the client. Advantageously, the techniques described herein may provide a cost-effective way to distribute data processing.

[0031] In an embodiment, the step of determining the duration includes applying a moving average algorithm to historical data including a plurality of data points, each data point indicating a processing time of applying a first algorithm to previously received data. Monitoring the processing time required to process a subset of sensor data can facilitate estimating the time required to process a subset of future sensor data. Implementing a moving average algorithm allows for adaptive changes to the estimated processing time in response to changes in processing time, which may occur due to changes in data complexity, fluctuations in bit size, and other factors. This approach facilitates continually updating and optimizing the estimated processing time based on the latest data trends.

[0032] In some embodiments, the moving average algorithm is a weighted moving average algorithm. When estimating processing time, there are several benefits to using a weighted moving average algorithm. This algorithm assigns different weights to different observations, which means that recent data can be considered more relevant and, therefore, given a greater weight in the calculation. Since it allows the algorithm to adapt to recent changes more quickly, this may be particularly useful in cases where data patterns change over time. For example, if the complexity or bit size of sensor data suddenly increases, the weighted moving average algorithm will adapt to these changes faster than a simple moving average algorithm, thereby providing a more accurate and timely estimate of the required processing time. Since it can more accurately predict the resources required to process incoming data, this can in turn help optimize the performance of the system, thereby reducing the risk of buffer overflows or other system overloads, and thus improving load sharing.

[0033] In some embodiments, the step of determining the duration includes using the bit size of the subset of data. For example, a lookup table may be used. This may be a low complexity way of estimating the processing time.

[0034] According to a second aspect of the present invention, there is provided a system for processing data by applying a first algorithm, the system comprising a server and a client device.

[0035] The client device includes: one or more processors; and one or more non-transitory computer-readable media storing first computer-executable instructions, which, when executed by the one or more processors, cause the client device to perform actions, the actions including: continuously receiving data; dividing the received data into M subsets of data, M>1, each subset of data including or derived from sensor data captured by a sensor at multiple time points during a capture time interval; determining for each subset whether processing of the subset should be performed by the client device or by the server, and when determining that processing should be performed by the client device, processing the subset into processed data by applying a first algorithm to the subset; data, and transmits the processed data to a server, and when it is determined that the processing should be performed by the server, transmits a subset of the data to the server; wherein the determination includes: determining a duration, which indicates a processing time for applying a first algorithm to a subset of the data; determining a time ratio between the duration and a capture time interval; when the time ratio is less than or equal to a threshold time ratio, determining that the client device should perform processing on each subset of M subsets of data; when the time ratio exceeds the threshold time ratio, determining N subsets that are part of the M subsets, N≤M, wherein N decreases as the time ratio increases, and determining that the client device should perform processing on N subsets of the M subsets.

[0036] The server includes: one or more processors; and one or more non-transitory computer-readable media storing second computer-executable instructions, which, when executed by the one or more processors, cause the server to perform actions, the actions including: receiving data from a client device; and processing the data received from the client device by applying a first algorithm to the data received from the client device when it is determined that the data received from the client device has not been processed by the client.

[0037] In some embodiments, the first algorithm is used to compute a hash of a subset of the data, and the processed data includes the subset of the data and the hash computed based on the subset of the data.

[0038] In some embodiments, the client device is a surveillance camera.

[0039] The second aspect may generally have the same features and advantages as the first aspect.

[0040] Further features and advantages of the present invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, with reference to the accompanying drawings.

[0041] In general, unless expressly defined herein, all terms used in the claims should be interpreted according to their ordinary meaning in the art. Unless expressly stated otherwise, all references to "a / an / the [element, device, part, means, step, etc.]" should be openly interpreted as referring to at least one instance of an element, device, part, means, step, etc.

[0042] Unless explicitly stated, the steps of any method disclosed herein do not have to be performed in the exact order disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and additional objects, features and advantages of the present invention will be better understood through the following illustrative and non-limiting detailed description of embodiments of the present invention with reference to the accompanying drawings, in which like reference numerals will be used for similar elements, wherein:

[0044] Figure 1 A graph illustrating subsets of data and indicating transfer of processing of the subsets to a server according to an embodiment;

[0045] Figure 2 A system including a client device and a server according to an embodiment is shown, the system implementing load sharing of data processing tasks;

[0046] Figure 3 Shown in Figure 2 A flowchart of a method for applying a first algorithm to process data in a system;

[0047] Figure 4 A flow chart illustrating a method for determining load sharing between a client and a server according to an embodiment. DETAILED DESCRIPTION

[0048] In a system including a client device (client device or edge device) and a server-side device (server), computing tasks can be distributed between the client-side device and the server-side device. For several reasons, it can be beneficial to perform as much processing as possible on the client device. For example, the client device is usually completely under the control and ownership of the user, which inherently makes the processing task cost-effective compared to transferring the processing task to a server that may be charged based on use. In addition, since it is completely controlled by the user, client processing may be safer when the data to be processed is captured by the client device itself. Transferring raw data to the server for processing may cause concerns about data security and privacy, so it is inclined to process data on the client device and transfer it only when absolutely necessary. However, processing complex or large amounts of data can quickly overwhelm the client device, resulting in delays, data loss, or a decline in user experience. The present disclosure proposes a system that can dynamically transfer processing to the server to solve these challenges when the client device lacks sufficient CPU power or when there is a risk of filling up the memory buffer. This method can optimize the use of available resources, prevent data loss, and provide a flexible solution that can adapt to different scenarios and needs.

[0049] Figure 1 Data 100 received at a client device is shown by way of example. Data is continuously received at the client device. Figure 1 , data 100 is illustrated as an image frame from a video. The client device may, for example, be a camera that captures the video. In other examples, data 100 may include or be derived from other types of sensor data such as audio data, radar data, etc. Therefore, in some embodiments, data 100 may include such sensor data. In other embodiments, data 100 may be derived from such sensor data, and, for example, include the output of an analysis of the sensor data or data that otherwise originates from the sensor data in the form of, for example, metadata. The data 100 received at the client device is divided into chunks or subsets 102a to 102c. Splitting the data into smaller parts can improve efficiency during transmission to a server, for example, for storage purposes. Advantageously, such parts can be transmitted in a single HTTP(S) POST or PUT request. Therefore, each subset 102a to 102c of data includes (or is derived from) sensor data captured by a sensor at multiple time points during a capture time interval. For video data, this capture time interval may be 3-10 seconds of video data (image frames). For other types of data, other intervals may be used based on requirements and applications. The capture time interval may vary between subsets 102a to 102c. In some embodiments, the capture time interval is similar between subsets. In some embodiments, the capture time interval is predetermined and therefore the same between subsets.

[0050] In some embodiments, for example, when data 100 includes video data or audio data, the data may be received in an encoding format such as H.264, H.265, AV1, etc. for video data or MP3, AAC, FLAC, etc. for audio data. In other embodiments, data 100 includes raw sensor data.

[0051] In many applications, different processing of the subsets 102a to 102c of data may be required. For example, from a data security perspective, it may be beneficial to calculate a hash value for each subset 102a to 102c in order to be able to verify that the data has not been manipulated, for example after being uploaded to a server. Other processing tasks include encryption, sensor data analysis, sensor data processing, etc. Therefore, the processing includes applying a first algorithm (for achieving hash values, encryption, analysis, etc.) to the subsets 102a to 102c of data.

[0052] Since the data 100 is received continuously, it may be important to perform processing in real time to prevent loss of data. The client device may include a buffer memory for temporarily storing subsets of data awaiting processing, but for complex processing, or when the available processing power at the client device is limited, such a buffer may fill up, resulting in loss of data. Due to the data segmentation strategy, the received data 100 is divided into subsets 102a to 102c of data, and real-time processing at the client device means that the processing time for applying the first algorithm to the subsets 102a to 102c of data divided by the capture time interval of the subsets 102a to 102c of data should be at least less than 1 (threshold time ratio). In order to add some safety margin, the threshold time ratio may be less than 1. Alternatively or additionally, the margin may be added to the processing time.

[0053] Figure 1 Further included is a graph 110 schematically depicting the transfer as a function of time ratio. Figure 4 An embodiment of how to determine whether to transfer is described. The time ratio is based on determining S402 a duration indicating a processing time to apply the first algorithm to the subset 102a to 102c of the data. In some cases, the duration may be related only to an estimated time to process the subset 102a to 102c of the data. In some cases, determining S402 the duration includes determining / estimating the processing time to apply the first algorithm to the subset of the data and adding a margin to the determined processing time.

[0054] The duration of a subset of particular data can be determined by looking at the processing time of a subset of previously processed data S402. For example, determining the duration can include applying a moving average algorithm to historical data including a plurality of data points, each data point indicating the processing time of applying a first algorithm to previously received data. For example, if the historical data includes data points [3.6, 3.2, 2.9, 2.8], the duration (for a moving average with a window of size 4) can be determined to be 3.1. In some embodiments, the moving average algorithm is a weighted moving average algorithm. For example, the weights can be larger for newer data points. For the same historical data as described above, where the first data point is the newest data point, and where the weights are 0.5, 0.3, 0.15, 0.05, the duration can be determined to be 3.4.

[0055] For some types of processing, the processing time can be determined based on the bit size of the subset of data S402. For example, the time required to calculate the hash value generally depends on the size of the data. Most cryptographic hash functions like SHA-256 operate on the data in blocks, processing each block sequentially. As the size of the data increases, the data is divided into more blocks, which requires more time to calculate the hash value. Although factors such as specific hash functions and hardware can affect efficiency, in general, larger data sizes mean longer processing times. Therefore, the processing time of a subset of data can be determined using a lookup table or the like. The same method can also be used for other processing such as encryption, where the time required to encrypt the data depends largely on its size. Many encryption algorithms such as AES work by encrypting data in chunks or blocks. As the size of the data grows, more data blocks need to be encrypted, thereby increasing the processing time.

[0056] In the case where the capture time intervals between subsets 102a to 102c of the data vary and the duration is calculated based on historical data, the capture time intervals are also included in the calculation of the duration S402, so that the historical data is weighted based on the difference between the capture time intervals of the data points in the historical data and the capture time intervals of the current subset.

[0057] Thus, the time ratio is determined by dividing the duration by the capture time interval S402. Typically, the capture time intervals do not differ substantially between the subsets of data, in which case the time ratio may be determined S404 using the capture time interval of any one of the subsets 102a to 102c of data. In other embodiments, the capture time interval of the subset of the most recent data received / determined at the client device is used, or the average of the K most recent subsets of data received / determined at the client device is used, or by any other suitable calculation.

[0058] Figure 1 The graph 110 of graph 110 shows a comparison S406 between the time ratio and a first threshold time ratio 118. If the time ratio is less than or equal to the threshold time ratio 118, it can be determined S408 not to transfer processing to the server, and the client device should perform processing of each of the M subsets of data. This is visualized by the segment 112 of the curve of the graph 110. The first threshold time ratio can be 1 or a value less than 1, such as 0.8 (to add some safety margin to the processing decision).

[0059] If the time ratio exceeds the first threshold time ratio 118, it may be determined S410 that it may be necessary to transfer some of the processing to the server to avoid loss of data. Advantageously, subsets of the data are therefore processed either at the client or at the server to reduce complexity. Depending on the time ratio and how it compares to the first threshold time ratio 118, a portion (MN) of the M subsets are sent to the server without being processed, while the remaining N subsets are processed at the client. Figure 1 As can be seen, as visualized by segment 114 of the curve of graph 110, N decreases as the time ratio increases. The portion can be determined by dividing 1 by the time ratio. For example, if the time ratio is close to 2, then every second subset of data can be processed at the client. If the time ratio is about 1.5, then two of the three subsets of data can be processed at the client. In some embodiments, the time ratio is rounded to the next integer (ceiling function) to determine the rounding value (J) so that every Jth subset is processed by the client.

[0060] In some embodiments (not in Figure 4 ), when the time ratio equals or exceeds the second threshold time ratio 120, it can be determined, as visualized by the segment 116 of the curve of the graph 110, that the server should perform processing on each of the M subsets. Therefore, the second threshold time ratio 120 can act as a safety mechanism in the event that the processing is too complex for the client to handle at all. In other embodiments, as the time ratio increases, the line of the graph 110 becomes a sigmoid function, so that smaller and smaller portions of the subsets are processed by the client, such as 1 in every 50, 1 in every 100, 1 in every 200, etc.

[0061] The gradient of the curve of graph 110 may, for example, depend on the number of subsets of data that can be stored in the client's buffer memory. In the case where a large number of subsets can be stored, more subsets can be processed by the client device. In other words, in some cases, N increases with the number of subsets of data that can be stored in the client device's buffer memory.

[0062] Figure 22 shows a system including a client device 200 and a server 201. The system implements load sharing of data processing tasks, and now the flowchart showing such load sharing will be described. Figure 3 to describe.

[0063] The client device 200 generally includes one or more processors and one or more non-transitory computer-readable media storing first computer-executable instructions that, when executed by the one or more processors, cause the client device to perform actions as described herein.

[0064] The server 201 generally includes one or more processors and one or more non-transitory computer-readable media storing second computer-executable instructions that, when executed by the one or more processors, cause the client device to perform actions as described herein.

[0065] Suitable processors for executing a program of instructions include, by way of example, both general purpose processors and special purpose microprocessors, as well as the single processor or one of multiple processors or cores of any kind of computer.The processor can be supplemented by, or incorporated in, ASICs (Application Specific Integrated Circuits).

[0066] exist Figure 2 , reference numeral 202 visualizes subsets of data, wherein dashed squares indicate subsets of unprocessed data and solid squares indicate subsets of processed data.

[0067] The client continuously receives S302 data 100, the data being divided into M subsets of data, M>1, each subset of data comprising or originating from sensor data captured by a sensor at a plurality of time points during a capture time interval. Thus, the sensor data is captured sequentially over time, and the letters following the reference numeral 202 indicate the order of the subsets of data, where "a" indicates the first and "g" indicates the last.

[0068] For a subset of data (e.g., subsets 202f and 202g), the client device makes a transfer decision 204, that is, determines whether the processing S306 is to be performed by the client 200 or the server 201. The determination includes combining Figure 1 It should be noted that such a time ratio can be calculated for each subset of data or for each K subset of data, where K can be 2, 4, 10, 20, etc. In the latter case, the transfer decision 204 (process all subsets at the client 200 or transfer a part / all subsets to be processed at the server 201) is valid for each subset of data until the next calculation of the time ratio.

[0069] Based on the determination S306, the subset of data is either processed S308 at the client so that the processed subset of data can be transmitted S310 to the server, or the subset of unprocessed data is transmitted S310 to the server 201 for processing. The client includes a processing component 206 configured to apply a first algorithm to the subset of data. The processing component 206 may further include or be connected to a buffer memory (not shown) for temporarily storing the subset of data to be processed. The buffer memory may be referred to as a cache memory and may include any suitable type of memory such as a DRAM or SRAM memory.

[0070] The client 200 is configured to transmit 208, S310 data to the server 201, and the server is configured to receive S312 data. In order to transmit data from the client 200 to the server 201 by wired or wireless means, several components may be included in the transmission 208. A transceiver capable of both sending and receiving data may be used in the process. In a wired environment, cables such as Ethernet cables or fiber optic cables may connect devices, while connectors and switches may help direct data flows. For wireless transmission, antennas may handle the transmission and capture of radio waves, and modems or routers may process and route these data to the intended devices. In addition, protocols and interface controllers may be used to ensure smooth data transmission between devices. Examples of such protocols include HTTP(S), and a subset of each data (whether processed or not) may be transmitted in a single HTTP(S) POST or PUT request. In addition to HTTP(S), there are many other protocols that may be used for computing and networking. For example, FTP or the file transfer protocol is used to transfer files between computers within a network. Other suitable protocols include message queue telemetry transmission (MQTT) or user datagram protocol (UDP). Each of these protocols has different uses in different networking scenarios.

[0071] If you can Figure 2 As seen in , subsets of data do not have to be transmitted in order (based on the time when the underlying sensor data was collected). Figure 2 In the example, the subset 202e of unprocessed data is transmitted 208, S310 before the subset 202d of processed data, even though they are in reverse order. The data 202e, 202d may include an index, for example in a header, for indicating the order to the server.

[0072] A subset of processed data, such as subset 202d, may include data that depends on the processing 206 (application of the first algorithm) performed on the subset. For example, if the first algorithm is used to calculate a hash of a subset of data, the processed data may include the subset of data (i.e., the subset of unprocessed data) and the hash calculated based on the subset of data (e.g., in a header or provided as metadata). If the first algorithm is used to encrypt a subset of data, the processed data may include the encrypted subset of data. If the first algorithm is a sensor data analysis algorithm, the processed data may include the subset of data (i.e., the subset of unprocessed data) and the output from the sensor data analysis algorithm provided as metadata. Other types of processing, such as applying a privacy filter to image data, may result in the subset of processed data including a modified subset of data, such as having blurred faces.

[0073] At the server end 201, data is received S312 from the client 200. The server 201 implements a function 210 for determining S314 whether the data received S312 from the client 200 is processed. Such a function 210 may, for example, include checking the format of the received data, checking a flag or setting value of a header of the received data, checking a metadata portion of a header portion of the received data, etc.

[0074] If it is determined S314 that the received data has not been processed, the server is configured to apply the first algorithm to the received data using the processing component 212. Similar to the client 200, the processing component 212 of the server may further include or be connected to a buffer memory (not shown) for temporarily storing a subset of the data to be processed. The buffer memory may be referred to as a cache memory and may include any suitable type of memory such as a DRAM or SRAM memory.

[0075] Therefore, the data that is determined S314 as not processed by the client is processed at the server 201. Figure 2 As shown in , all M subsets of data have been processed. In some embodiments, the processed subsets of data are stored in memory 214. In the case where the processing involves the calculation of a hash value, the hash value can be stored in memory 214 and subsequently used to verify the authenticity of the subset of data when it is read from memory 214.

[0076] For example, hash values ​​can be stored in a blockchain. In the field of media (video, audio, etc.) authentication, integrating hash values ​​into blockchain systems provides a promising way to ensure the authenticity of media segments. As described above, when a media segment is created or obtained, its hash value is generated (at the client or at the server). This hash value, which is essentially a fixed-size string or number derived from the media data, serves as a unique fingerprint for the segment. Given the properties of cryptographic hash functions, even slight changes in the media segment will result in a completely different hash value. Once the hash value is received at the server 201 or calculated at the server 201, it is recorded on the blockchain. The blockchain (a decentralized and immutable ledger) provides a chronological record of all completed entries. Each entry or block contains a link to the previous block, linking them together. Once a block with a specific hash value is added to the blockchain, it is virtually impossible to change it without changing all subsequent blocks, which will be detected by the participants of the system. Later, if someone wishes to verify the authenticity of the media segment, they can generate a hash value based on the segment in question and check it against the blockchain. If the hash exists on the blockchain and matches the hash generated from the piece, it proves that the media has not been altered since its hash was recorded. This system essentially leverages the immutability of the blockchain to combat the rampant problems of deep fakes, media tampering, and misinformation. By ensuring the integrity of media pieces, whether for journalistic integrity, legal evidence, or historical archiving, stakeholders are able to trust the authenticity of the content.

[0077] In some embodiments, the server 201 or the memory 214 of the server 201 is connected 215 to a streaming component 216 for streaming media content using a streaming technology, and the streaming technology is one of the following streaming technologies: HTTP real-time streaming, HLS standard or dynamic adaptive streaming (DASH) standard based on HTTP, wherein the fragments in the media content being streamed are derived from a subset of data in a subset of M data. Therefore, the technology described herein can be advantageously implemented in a streamed environment so that the processing of the subset of data that will form the fragments in the media content being streamed later has been processed as a unit from the beginning, which can reduce complexity, improve security, ensure performance, and therefore provide a favorable framework for processing and streaming.

[0078] The above embodiments should be understood as illustrative examples of the present invention. It should be understood that any feature described with respect to any one embodiment may be used alone or in combination with other features described, and may also be used in combination with one or more features of any other embodiment or any combination of any other embodiment. In addition, equivalents and modifications not described above may also be adopted without departing from the scope of the present invention defined by the appended claims.

Claims

1. A method for processing data by applying a first algorithm in a system, the system comprising a server and a client device, the method comprising, at the client device: Receive data continuously; dividing the received data into M subsets of data, M>1, each subset of data including or derived from sensor data captured by a sensor at a plurality of time points during a capture time interval; determining for each subset whether the processing of the subset should be performed by the client device or by the server, and processing the subset into processed data by applying the first algorithm to the subset and transmitting the processed data to the server when it is determined that the processing should be performed by the server, and transmitting the subset of data to the server when it is determined that the processing should be performed by the server; Wherein, the determining comprises: determining a duration indicating a processing time of applying the first algorithm to a subset of data; determining a time ratio between the duration and the capture time interval; when the time ratio is less than or equal to a threshold time ratio, determining that the client device should perform processing of each subset of the M subsets of data; and when the time ratio exceeds the threshold time ratio, determining N subsets that are part of the M subsets, N≤M, wherein N decreases as the time ratio increases, and determining that the client device should perform processing of the N subsets of the M subsets; The method further includes, at the server: receiving data from the client device, and when it is determined that the data received from the client device has not been processed by the client device, processing the data received from the client device by applying the first algorithm to the data received from the client device.

2. The method according to claim 1, wherein: Determining a duration includes determining a processing time to apply the first algorithm to a subset of data and adding a margin to the determined processing time.

3. The method according to claim 2, further comprising: The margin is determined based on a bit size of the subset of the data, wherein the margin increases as the bit size increases.

4. The method according to claim 1, wherein: N is determined by determining the number of subsets of data that can be stored in the buffer memory of the client device, wherein N increases as the number of subsets of data that can be stored in the buffer memory of the client device increases.

5. The method according to claim 1, wherein: The data received at the client device includes one of video data and audio data, and wherein the data received at the client device is encoded using a compression standard.

6. The method of claim 5, further comprising using, at the server, the data received from the client device to stream the media content using a streaming technology, the streaming technology being one of: HTTP Live Streaming, the HLS standard, or the Dynamic Adaptive Streaming over HTTP (DASH) standard, wherein The segments of the media content being streamed originate from a subset of data in the M subsets of data.

7. The method according to claim 1, wherein: The first algorithm is for computing a hash of a subset of data, and wherein the processed data comprises the subset of data and the hash computed based on the subset of data.

8. The method according to claim 1, wherein: The first algorithm is used to encrypt a subset of data, and wherein the processed data comprises the encrypted subset of the data.

9. The method according to claim 1, wherein: The first algorithm is a sensor data analysis algorithm, wherein the processed data includes a subset of the data and an output from the sensor data analysis algorithm.

10. The method according to claim 1, wherein: The step of determining the duration includes applying a moving average algorithm to historical data, each data point indicating a processing time of applying the first algorithm to previously received data, the historical data including a plurality of data points.

11. The method according to claim 10, wherein: The moving average algorithm is a weighted moving average algorithm.

12. The method according to claim 1, wherein: The step of determining the duration comprises using a bit size of the subset of said data.

13. A system for processing data by applying a first algorithm, the system comprising a server and a client device, the client device comprising: one or more processors; and One or more non-transitory computer-readable media storing first computer-executable instructions, which, when executed by the one or more processors, cause the client device to perform actions, the actions comprising: continuously receiving data; dividing the received data into M subsets of data, M>1, each subset of data including or derived from sensor data captured by a sensor at multiple time points during a capture time interval; determining for each subset whether the processing of the subset should be performed by the client device or by the server, and when it is determined that the processing should be performed by the client device, processing the subset into processed data by applying the first algorithm to the subset, and transmitting the processed data to the server, and when it is determined that the processing should be performed by the server, transmitting the subset of data to the server; Wherein, the determining comprises: determining a duration indicating a processing time of applying the first algorithm to a subset of data; determining a time ratio between the duration and the capture time interval; when the time ratio is less than or equal to a threshold time ratio, determining that the client device should perform processing of each subset of the M subsets of data; when the time ratio exceeds the threshold time ratio, determining N subsets that are part of the M subsets, N≤M, wherein N decreases as the time ratio increases, and determining that the client device should perform processing of the N subsets of the M subsets; The server includes: one or more processors; and one or more non-transitory computer-readable media storing second computer-executable instructions, which, when executed by the one or more processors, cause the server to perform actions, the actions comprising: receiving data from the client device; Upon determining that the data received from the client device has not been processed by the client device, processing the data received from the client device by applying the first algorithm to the data received from the client device.

14. The system according to claim 13, wherein: The first algorithm is for computing a hash of a subset of data, and wherein the processed data comprises the subset of data and the hash computed based on the subset of data.

15. The system of claim 13, wherein: The client device is a surveillance camera.

Images