Methods and systems for data processing

Abstract

The present invention relates to a method for data processing, wherein the method comprises: receiving a stream of data from a data source, executing a data processing pipeline on the received stream of data, or at least a part thereof, and sending an output of executing the data processing pipeline. The present invention also relates to a system for performing the method and to a corresponding computer program product.

Description

[0001] Methods and systems for data processing

[0002] Field

[0003] The present invention relates generally to the field of data processing. More particularly, it relates to real-time processing of data, preferably audio-visual data.

[0004] Background

[0005] Increasing internet access and speed have allowed novel applications to be developed. Applications around processing of data, particularly audio-visual data, may be of particular relevance. Audio-visual data may, for example, comprise data captured by cameras installed in different locations, such as surveillance cameras, or traffic cameras. It may be of advantage to process data captured by these cameras to extract useful information. However, audiovisual data may typically be large in size compared, for example, to character data and, processing of such data may be more complex, and time-inefficient.

[0006] Additionally, such data may present additional complexities owing to the variety of environments in which cameras may be installed, thus presenting different captured data. Aside from this scene diversity, a desired result of the processing of data captured by the camera may vary. For example, in one scenario, it may be of advantage to, say, switch on a device, in response to a result of the data processing, whereas in another scenario, it may be of advantage to, say, send a message to a device. Yet other complexity may arise from scaling the data processing capabilities, as more and more such cameras are installed.

[0007] Different approaches to processing such data have been explored, including using artificial intelligence and computer vision. These approaches have proved to be quite powerful in processing and analyzing such data. However, these approaches may be difficult to generalize across the diversity of scenes, results, or scaling, as described above. In particular, the approach may only be applicable to input data in a specific format, and for data that may be similar to training data that may have been used for training an underlying model.

[0008] In light of the above, it may be of advantage to have a method to process data, particularly audio-visual data as described above, that may be applied across a variety of scenes, desired results, and scaling. It may also be advantageous for the method to be simple to use, be secure, and be useful across a variety of system architectures.

[0009] Summary

[0010] The present invention seeks to overcome or at least alleviate the shortcomings of the prior art. More particularly, it is an object of the present invention to provide a method and a system for data processing, that may be more efficient, more secure, simpler to use, and more customizable.

[0011] According to a first aspect, the present invention relates to a method comprising: receiving a stream of data from a data source, executing a data processing pipeline on the received stream of data, or at least a part thereof, and sending an output of executing the data processing pipeline. The output may be based, at least in part, on the stream of data, or at least the part thereof. In particular, the stream of data may comprise a sequence of bits, and the data processing pipeline may be configured to process chunks of the sequence to generate the output. For example, the data processing pipeline may process every 1000 contiguous bits to generate the output. Thus, it may be understood, that the output may vary as more data is read in from the data source.

[0012] The stream of data may comprise a sequence of bits, and the output may be based, at least in part, on execution of the data processing pipeline on a subset of the sequence of bits. Thus, generally, the data processing pipeline may be configured so as to operate on sequences of bits. This may be of advantage, in particular, in allowing real-time processing of the stream of data.

[0013] The subset may comprise a plurality of contiguous bits in the sequence. Contiguous bits may be understood to comprise bits that are received one after the other. For example, the sequence of bits received from the data source may be representative of a video such that contiguous-bit subsets correspond to frames of the video. The data processing pipeline may then allow processing of individual frames. Alternatively, the subset of contiguous bits may comprise, for example, a plurality of consecutive frames, based on which the output may be generated. This may be of relevance, for example, in determining a pose of a face in the video.

[0014] The data processing pipeline may comprise a data processing module configured to accept an input and to send an output. The data processing module may, in particular, accept input in a defined format, and send the output in a defined format. For example, when the stream of data is representative of a video, the input format may comprise a defined resolution, such as 220 x 240 in pixel space. The output may, for example, comprise the input together with a binary flag indicating whether or not a face was detected in the video.

[0015] The data processing pipeline may comprise a plurality of data processing modules. The plurality of data processing modules may allow different kinds of processing to be carried out on the stream of data. For example, some data processing modules may allow detection of specific features, such as faces and objects among others, in the stream of data, whereas some other data processing modules may sharpen, blur, re-colorize, among others, images represented by the stream of data. Note that this list may be considered to be only exemplary, but not limiting, of the different processes that may be carried out by any of the plurality of data processing modules. The method further may comprise creating the data processing pipeline.

[0016] Creating the data processing pipeline may comprise selecting, from a collection of available data processing modules, a data processing module. The collection of available data processing modules may comprise one or more data processing modules. The data processing modules may be represented, for example, via strings representative of a result of the data processing carried out by the data processing module. In other words, selecting a data processing module may comprise selecting only a string representative of the data processing module such that the corresponding data processing module may be retrieved based, at least in part, on the string, later on. This may be of advantage in storing data processing pipelines, as described further below, more efficiently. Moreover, it should be understood, that selecting a data processing module may also comprise connecting the data processing module with another data processing module that is already part of the data processing pipeline. Thus, creating of the data processing pipeline may comprise a bottom-up process, such that individual data processing modules may be selected and combined with a data processing module in the data processing pipeline created so far.

[0017] Creating the data processing pipeline may comprise selecting, from a collection of available data processing modules, a plurality of data processing modules.

[0018] Creating the data processing pipeline may further comprise connecting at least one of the plurality of selected data processing modules with at least one other of the plurality of selected data processing modules.

[0019] Selecting the data processing module, or the plurality thereof, may comprise using a graphical user interface.

[0020] The graphical user interface may comprise a distinct draggable element corresponding to each available data processing module, and selecting the data processing module, or the plurality thereof, may comprise dragging and dropping the draggable element, or a plurality thereof, corresponding to the data processing module, or to the plurality thereof. Thus, for example, the graphical user interface may comprise one or more elements each representing a unique, available data processing module. In order to create the data processing pipeline, the user may be able to click on the element of a desired data processing module, and drag it into an area that may be depict the data processing pipeline. By dragging and dropping multiple such elements, and by creating connections between the elements, representative of data flows between the data processing modules, the data processing pipeline, or at least a representation thereof, may be created. It may be understood, that not every data processing module may be connected to every other data processing module. The graphical user interface may be configured appropriately to only allow valid (as may be pre-defined) connections between data processing modules, or the corresponding elements. The provision of a graphical user interface may allow a user to create data processing pipelines more easily and simply in respect of the prior art.

[0021] The method may comprise executing the data processing pipeline on the stream of data, or the part thereof, in real-time. The real-time processing may be of particular advantage when a result of the execution may be needed in real-time for further action. For example, the data source may comprise a feed from a camera, configured to detect motion. The data processing pipeline may, then, be configured for motion detection, and to receive the feed from the camera. An output of the data processing pipeline may, for example, be connected to a light that may be configured to glow when motion is detected. In this case, it may be of advantage to ensure that a time interval between motion happening in front of the camera and the glowing of the light is as small as possible.

[0022] The method may comprise automatically executing the data processing pipeline on the stream of data, or the part thereof. In other words, the data processing pipeline and the data source may be connected such that whenever data is received from the data source, the data processing pipeline is automatically executed on the received data.

[0023] The stream of data, or the part thereof, may be representative of a video stream. That is, the bits corresponding to the stream of data may be displayed as a video. Processing of a video stream using a data processing pipeline as described above may be of advantage in triggering some action based, at least in part, on the video. For example, the video stream may correspond to a recording by a camera, and an appropriate data processing pipeline may be executed depending on a location / purpose of the camera.

[0024] The stream of data, or the part thereof, may be representative of an audio stream. That is, the bits corresponding to the stream of data may be represented as an audio. Processing of an audio stream using a data processing pipeline as described above may be of advantage in triggering some action based, at least in part, on the audio.

[0025] The data processing pipeline may be associated with a unique identifier. The unique identifier may allow referring to the data processing pipeline and / or to retrieving it from a storage component. The storage component may comprise, for example, a local or a cloud storage component. Further, the unique identifier may allow using the same data processing pipeline with multiple data sources, also concurrently. Thus, overall efficiency may be improved, as a data processing pipeline once created may be used repeatedly without having to re-create it.

[0026] The method may comprise receiving the unique identifier.

[0027] The method may comprise retrieving the data processing pipeline based, at least in part, on the received unique identifier. For example, one or more data processing pipelines may be stored in a storage component, and may be retrieved from the storage component using their unique identifiers. As described above, this may improve the efficiency of the processing of data as data processing pipelines can simply be retrieved once they have been created.

[0028] Creating the data processing pipeline further may comprise generating a unique identifier for the created data processing pipeline. The unique identifier may be generated, at least in part, automatically at the creation of a data processing pipeline. Alternatively, or additionally, the unique identifier may be generated, at least in part, manually, by a user creating the data processing pipeline. As may be appreciated, a semi-automatic generation may also be possible, wherein, for example, the user may amend the unique identifier generated automatically.

[0029] The method may further comprise sending the generated unique identifier. The generated unique identifier may be sent, for example, to the storage component as described above. Alternatively, or additionally, the generated unique identifier may be sent to a user interface component to allow the user access to the generated unique identifier. In some embodiments, the data processing pipeline may be configured to send an output of executing the data processing pipeline on the stream of data to a device that may be configured to use the output. In such embodiments, it may also be advantageous to send the unique identifier to the device to enhance security of the data transfer.

[0030] The method may further comprise storing the created data processing pipeline.

[0031] The method may comprise storing the generated unique identifier.

[0032] The method may further comprise providing the data processing module. For example, the data processing module may be stored in a storage component (that may or may not be the same as the storage component described above). In particular, the data processing module may comprise a neural network, wherein different weights of the neural network, preferably after training of the neural network, may be stored in the storage component. Providing the data processing module may then comprise providing the weights stored in the storage component.

[0033] The method may further comprise providing the plurality of data processing modules.

[0034] Executing the data processing pipeline may comprise performing, at least in part, a convolution of the stream of data, or the part thereof. The convolution may be performed with an appropriate function chosen to obtain a desired result. For example, This may be of particular advantage, for example, when a data processing module comprises an artificial intelligence-based computer vision module.

[0035] The output of executing the data processing pipeline may comprise textual data. Textual data may be significantly reduced in size compared to the original stream of data, or the part thereof, on which the textual output is based. The reduced size may be of particular advantage in embodiments where the output is to be sent to another device over a network. Thus, overall efficiency may be improved.

[0036] The textual data may comprise metadata about the stream of data, or at least a part thereof. For example, the stream of data may comprise a video stream from a webcam, and the metadata may comprise information about whether or not a person is looking into the webcam. Generally, the metadata output from the execution of the data processing pipeline may depend on a configuration (i.e., the kind of data processing modules used, and how they are connected to one another) of the data processing pipeline, that may, in turn, depend on a purpose of the data processing.

[0037] The textual data may relate, at least in part, to a summary of the stream of data, or the part thereof.

[0038] Executing the data processing pipeline may comprise detecting a feature in the stream of data, or the part thereof. For example, the data processing pipeline may be configured to detect a face in the stream of data. A result of the detection may generally be either true, when the feature is detected, or false, otherwise. In particular, the data processing pipeline may comprise a data processing module configured to detect the feature. The data processing module may be connected to, for example, two different data processing modules such that the output is sent to one of the data processing modules if the result is true, and to the other one if the result is false.

[0039] The textual data may be based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof. The textual data may then, for example, comprise the text "true" or "false", depending on the result of the detection.

[0040] The method may comprise asking for user input.

[0041] The textual data comprises, at least in part, the user input, or at least a part thereof. This may be of advantage in allowing further customization of the output of executing the data processing pipeline. For example, the user input may describe the output, such as "face detected" or "face not detected". This may be of particular advantage when the output of the execution of the data processing pipeline is to be sent to another device. The other device may only accept input in a defined format and the user input may allow sending the output of the execution in the defined format.

[0042] Executing the data processing pipeline may comprise excluding a feature from the stream of data, or the part thereof. Excluding features may be of particular advantage in embodiments where the stream of data is representative of an audio / video stream, as features relating to the audio / video representation of the stream may be excluded, thus reducing a size of the output.

[0043] Executing the data processing pipeline may comprise excluding a plurality of features from the stream of data, or the part thereof.

[0044] The method may comprise generating a first output based, at least in part, on a first subset of bits. For example, when the stream of data is representative of a video stream, the first subset of bits may comprise a first frame of the video.

[0045] The method may comprise generating a second output based, at least in part, on a second subset of bits. For example, when the stream of data is representative of the video stream described above, the second subset of bits may comprise a second frame of the video.

[0046] The method may comprise determining a time interval between generating the first output and the second output. In other words, for the example described above, the time interval between generating the output for the first frame and for the second frame of the video may be determined.

[0047] The first subset and the second subset may be disjoint.

[0048] The method may comprise determining a time interval between two successive outputs of executing the data processing pipeline.

[0049] In some embodiments, it may be advantageous to determine a time interval between two successive outputs of any of the data processing modules in the data processing pipeline. In particular, when the data processing pipeline comprises some data processing modules that may take long to finish, successive outputs of some of the other data processing modules may also be used to extract information about the stream of data.

[0050] The method may comprise determining a time interval between two successive outputs of the data processing module.

[0051] The method may comprise initializing an accumulator.

[0052] The method may comprise accumulating the time interval described above in the accumulator. Accumulating the time interval described above may be of advantage in determining, for example, a time since the output of executing the data processing pipeline last changed.

[0053] The method may comprise accumulating the time interval described above in the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof. For example, the stream of data may comprise video captured by a camera and it may be desirable to determine the last time someone looked into the camera. This may be determined, at least in part, by accumulating the time intervals between successive outputs of executing the data processing pipeline, wherein the outputs correspond to no one looking into the camera.

[0054] The method may comprise resetting the accumulator.

[0055] The method may comprise resetting the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof. For example, in the scenario described above, the accumulator may be reset when the output of executing the data processing pipeline corresponds to someone looking into the camera.

[0056] The method may comprise triggering an action based, at least in part, on the output. Triggering an action may comprise changing a state of a component and / or a device. In other words, the output of executing the data processing pipeline may be used to perform an action based, at least in part, on the output. For example, as described above, the output of executing the data processing pipeline may relate to detection of motion in front of a camera, and the action triggered may relate to switching on / off of a light.

[0057] The stream of data may comprise a plurality of images. In particular, the plurality of images may correspond to frames of a video.

[0058] The method may comprise sending the output for a plurality of the plurality of images. In other words, the data processing pipeline may be executed for at least two of the plurality of images and the output of the execution for each of the at least two of the plurality of images be sent.

[0059] The method may comprise storing the output of executing the data processing pipeline.

[0060] The data source may be associated with a unique source identifier. The source identifier may be of advantage in allowing the data processing pipeline to be executed on data streams from different data sources. Further, the source identifier may be associated with a data processing pipeline, or to the unique identifier thereof, wherein the data processing pipeline may be configured to accept data from the associated data source. This may be of advantage in improving the efficiency of data processing as appropriate data processing pipelines may be linked with data sources.

[0061] The method may comprise storing the output with the source identifier. Storing the output with the source identifier may be of advantage in keeping track of, among others, a last output associated with a data source, that may be useful, for example, in determining time intervals between successive outputs associated with a data source as described above. Generally, this may allow multiple streams of data to be processed using, for example, the same data processing pipeline.

[0062] The method may comprise storing the output of the data processing module. In other words, intermediate information from processing the stream of data may also be stored.

[0063] The method may comprise storing the output of the data processing module with the source identifier. This may allow, for example, determining the last output of the data processing module for data received from a defined data source. Storing the last output of the data processing module may be of advantage in determining time intervals between successive outputs of the data processing module on the stream of data from the defined data source, as described above.

[0064] The method may comprise deleting the output of the data processing module after a defined time interval has elapsed.

[0065] The method may comprise deleting the output of the data processing module after a defined time interval has elapsed such that a next output of the data processing module with the same source identifier has not been stored. In other words, if no data is received from the same data source within a defined time interval, intermediate information about the data received from the data source earlier may be deleted. This may be of advantage in optimizing memory usage.

[0066] The method may comprise receiving a plurality of streams of data each from a distinct data source.

[0067] The method may comprise pre-processing the stream of data, or at least a part thereof, before executing the data processing pipeline.

[0068] According to a second aspect, the present invention relates to a system configured: to receive a stream of data from a data source, to execute a data processing pipeline on the stream of data, or at least a part thereof, and to send an output of executing the data processing pipeline.

[0069] The system may comprise a data processing component.

[0070] The system may comprise a plurality of data processing components.

[0071] The plurality of data processing components may comprise a first data processing component and a second data processing component. The first data processing component and the second data processing component may be housed in the same device. In other words, the same device may be used, for example, for pre-processing of the stream of data as well as for executing the data processing pipeline. This may, however, require sufficient processing capacity of the device.

[0072] The first data processing component and the second data processing component may be housed in different devices. In other words, different devices may be used, for example, to pre-process the stream of data and to execute the data processing pipeline. This may be of particular advantage when one of the devices may have a lower processing capacity compared to the other.

[0073] The system may comprise a memory component. The memory component may be used to store any of the data processing pipeline, the data processing modules, the identifiers for the pipelines, and the source identifiers as described above.

[0074] Note that also the memory component may comprise a plurality of memory components, that may be distributed over different devices. In particular, for example, the data processing pipelines may be stored in a cloud device and the stream of data may be sent to the cloud device for executing the data processing pipeline.

[0075] The system may comprise a communication unit.

[0076] The communication unit may be configured to communicate with the data processing component.

[0077] The communication unit may be configured to communicate with the data source.

[0078] The system may comprise a user interface component configured to allow the system to communicate with a user. The user interface component may, in particular, comprise a display, and means to allow the user to interact with a graphical user interface presented on the display.

[0079] The data processing pipeline may be associated with a unique identifier.

[0080] The system may be configured to receive the unique identifier.

[0081] The system may be configured to retrieve the data processing pipeline based, at least in part, on the received unique identifier.

[0082] The system may be configured to create the data processing pipeline. Creating the data processing pipeline may comprise generate a unique identifier associated with the created data processing pipeline.

[0083] The system may be configured to store the created data processing pipeline.

[0084] The system may be configured to store the generated unique identifier.

[0085] The system may be configured to send the generated unique identifier.

[0086] The data processing pipeline may comprise a data processing module configured to accept an input and to send an output.

[0087] The data processing pipeline may comprise a plurality of data processing modules.

[0088] The system may be configured to allow selecting, from a collection of available data processing modules, a data processing module, to create the data processing pipeline.

[0089] The system may be configured to allow selecting, from a collection of available data processing modules, a plurality of data processing modules, to create the data processing pipeline.

[0090] The system may be configured to present a graphical user interface, wherein selecting the data processing module, or the plurality thereof, may comprise using the graphical user interface. In particular, as described above, the user interface component may comprise a display and the graphical user interface may be presented on the display.

[0091] The graphical user interface may comprise a distinct draggable element corresponding to each available data processing module, and selecting the data processing module, or the plurality thereof, may comprise dragging and dropping the draggable element, or a plurality thereof, corresponding to the data processing module, or to the plurality thereof.

[0092] The system may be configured to execute the data processing pipeline on the stream of data, or the part thereof, in real-time.

[0093] The system may be configured to automatically execute the data processing pipeline on the stream of data, or the part thereof.

[0094] The stream of data, or the part thereof, may be representative of a video stream.

[0095] The stream of data, or the part thereof, may be representative of an audio stream.

[0096] The system may be further configured to provide the data processing module. The system may be further configured to provide the plurality of data processing modules.

[0097] Executing the data processing pipeline may comprise performing, at least in part, a convolution of the stream of data, or the part thereof.

[0098] The output may comprise textual data.

[0099] The textual data may comprise metadata about the stream of data, or at least a part thereof.

[0100] The textual data may relate, at least in part, to a summary of the stream of data, or the part thereof.

[0101] Executing the data processing pipeline may comprise detecting a feature in the stream of data, or the part thereof.

[0102] The textual data may be based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0103] The system may be configured to ask for user input.

[0104] The textual data may comprise, at least in part, the user input, or at least a part thereof.

[0105] Executing the data processing pipeline may comprise excluding a feature from the stream of data, or the part thereof.

[0106] Executing the data processing pipeline may comprise excluding a plurality of features from the stream of data, or the part thereof.

[0107] The system may be configured to determine a time interval between two successive outputs of executing the data processing pipeline.

[0108] The system may be configured to determine a time interval between two successive outputs of the data processing module.

[0109] The system may be configured to initialize an accumulator.

[0110] The system may comprise accumulating the time interval in the accumulator.

[0111] The method may comprise accumulating the time interval in the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0112] The system may be configured to reset the accumulator. The system may be configured to reset the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0113] The system may be configured to trigger an action based, at least in part, on the output.

[0114] The system may be configured to change a state of a component and / or device based, at least in part, on the output of the execution of the data processing pipeline.

[0115] The stream of data may comprise a plurality of images.

[0116] The system may be configured to send the output for a plurality of the plurality of images.

[0117] The system may be configured to store the output of executing the data processing pipeline.

[0118] The data source may be associated with a unique source identifier.

[0119] The system may be configured to store the output with the source identifier.

[0120] The system may be configured to store the output of the data processing module.

[0121] The system may be configured to store the output of the data processing module with the source identifier.

[0122] The system may be configured to delete the output of the data processing module after a defined time interval has elapsed.

[0123] The system may be configured to delete the output of the data processing module after a defined time interval has elapsed such that a next output of the data processing module with the same source identifier has not been stored.

[0124] The system may be configured to receive a plurality of streams of data each from a distinct data source.

[0125] The system may be configured to pre-process the stream of data, or at least a part thereof, before executing the data processing pipeline.

[0126] The system may be configured to perform the method as described above.

[0127] According to a third aspect, the present invention relates to a computer program product comprising instructions, when run on a system as described above, to perform the method as described above. The computer program product may comprise instructions, when run on a system comprising a data processing component, to perform the method as described above.

[0128] The present invention is also described by the following numbered embodiments.

[0129] Below method embodiments will be discussed. These are abbreviated by the letter "M" followed by a number. Whenever reference is, herein, made to the method embodiments, the following embodiments are meant.

[0130] Ml. A method for data processing, wherein the method comprises: receiving a stream of data from a data source, executing a data processing pipeline on the received stream of data, or at least a part thereof, and sending an output of executing the data processing pipeline.

[0131] M2. The method according to the preceding embodiment, wherein the stream of data comprises a sequence of bits, and wherein the output is based, at least in part, on execution of the data processing pipeline on a subset of the sequence of bits.

[0132] M3. The method according to the preceding embodiment, wherein the subset comprises a plurality of contiguous bits in the sequence.

[0133] M4. The method according to any of the preceding method embodiments, wherein the data processing pipeline comprises a data processing module configured to accept an input and to send an output.

[0134] M5. The method according to the preceding embodiment, wherein the data processing pipeline comprises a plurality of data processing modules.

[0135] M6. The method according to any of the preceding method embodiments, wherein the method further comprises creating the data processing pipeline.

[0136] M7. The method according to the preceding embodiment, wherein creating the data processing pipeline comprises selecting, from a collection of available data processing modules, a data processing module.

[0137] M8. The method according to the preceding embodiment, wherein creating the data processing pipeline comprises selecting, from a collection of available data processing modules, a plurality of data processing modules. M9. The method according to any of the 2 preceding embodiments, wherein selecting the data processing module, or the plurality thereof, comprises using a graphical user interface.

[0138] MIO. The method according to the preceding embodiment, wherein the graphical user interface comprises a distinct draggable element corresponding to each available data processing module, and wherein selecting the data processing module, or the plurality thereof, comprises dragging and dropping the draggable element, or a plurality thereof, corresponding to the data processing module, or to the plurality thereof.

[0139] Mil. The method according to any of the preceding method embodiments, wherein the method comprises executing the data processing pipeline on the stream of data, or the part thereof, in real-time.

[0140] M12. The method according to any of the preceding method embodiments, wherein the method comprises automatically executing the data processing pipeline on the stream of data, or the part thereof.

[0141] M13. The method according to any of the preceding method embodiments, wherein the stream of data, or the part thereof, is representative of a video stream.

[0142] M14. The method according to any of the preceding method embodiments, wherein the stream of data, or the part thereof, is representative of an audio stream.

[0143] M15. The method according to any of the preceding method embodiments, wherein the data processing pipeline is associated with a unique identifier.

[0144] M16. The method according to the preceding embodiment, wherein the method comprises receiving the unique identifier.

[0145] M17. The method according to the preceding embodiment, wherein the method comprises retrieving the data processing pipeline based, at least in part, on the received unique identifier.

[0146] M18. The method according to any of the preceding method embodiments and with the features of embodiment M6, wherein creating the data processing pipeline further comprises generating a unique identifier for the created data processing pipeline.

[0147] M19. The method according to the preceding embodiment, wherein the method further comprises sending the generated unique identifier.

[0148] M20. The method according to any of the preceding method embodiments and with the features of embodiment M6, wherein the method further comprises storing the created data processing pipeline. M21. The method according to any of the preceding method embodiments and with the features of embodiment M18, wherein the method comprises storing the generated unique identifier.

[0149] M22. The method according to any of the preceding method embodiments and with the features of embodiment M7, wherein the method further comprises providing the data processing module.

[0150] M23. The method according to any of the preceding method embodiments and with the features of embodiment M8, wherein the method further comprises providing the plurality of data processing modules.

[0151] M24. The method according to any of the preceding method embodiments, wherein executing the data processing pipeline comprises performing, at least in part, a convolution of the stream of data, or the part thereof.

[0152] M25. The method according to any of the preceding method embodiments, wherein the output of executing the data processing pipeline comprises textual data.

[0153] M26. The method according to the preceding embodiment, wherein the textual data comprises metadata about the stream of data, or the part thereof.

[0154] M27. The method according to any of the 2 preceding embodiments, wherein the textual data relates, at least in part, to a summary of the stream of data, or the part thereof.

[0155] M28. The method according to any of the preceding method embodiments, wherein executing the data processing pipeline comprises detecting a feature in the stream of data, or the part thereof.

[0156] M29. The method according to the preceding embodiment and with the features of embodiment M25, wherein the textual data is based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0157] M30. The method according to any of the preceding method embodiments, wherein the method comprises asking for user input.

[0158] M31. The method according to the preceding embodiment and with the features of embodiment M25, wherein the textual data comprises, at least in part, the user input, or at least a part thereof. M32. The method according to any of the preceding method embodiments, wherein executing the data processing pipeline comprises excluding a feature from the stream of data, or the part thereof.

[0159] M33. The method according to the preceding embodiment, wherein executing the data processing pipeline comprises excluding a plurality of features from the stream of data, or the part thereof.

[0160] M34. The method according to any of the preceding method embodiments and with the features of embodiment M3, wherein the method comprises generating a first output based, at least in part, on a first subset of bits.

[0161] M35. The method according to the preceding embodiment, wherein the method comprises generating a second output based, at least in part, on a second subset of bits.

[0162] M36. The method according to the preceding embodiment, wherein the method comprises determining a time interval between generating the first output and the second output.

[0163] M37. The method according to any of the 2 preceding embodiments, wherein the first subset and the second subset are disjoint.

[0164] M38. The method according to any of the preceding method embodiments, wherein the method comprises determining a time interval between two successive outputs of executing the data processing pipeline.

[0165] M39. The method according to any of the preceding method embodiments and with the features of embodiment M4, wherein the method comprises determining a time interval between two successive outputs of the data processing module.

[0166] M40. The method according to any of the preceding method embodiments, wherein the method comprises initializing an accumulator.

[0167] M41. The method according to the preceding embodiment and with the features of embodiment M36, wherein the method comprises accumulating the time interval in the accumulator.

[0168] M42. The method according to the preceding embodiment and with the features of embodiment M28, wherein the method comprises accumulating the time interval in the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof. M43. The method according to any of the preceding method embodiments and with the features of embodiment M40, wherein the method comprises resetting the accumulator.

[0169] M44. The method according to the preceding embodiment and with the features of embodiment M28, but without the features of the penultimate embodiment, wherein the method comprises resetting the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0170] M45. The method according to any of the preceding method embodiments, wherein the method comprises triggering an action based, at least in part, on the output of executing the data processing pipeline.

[0171] M46. The method according to any of the preceding method embodiments and with the features of embodiment M13, wherein the stream of data comprises a plurality of images.

[0172] M47. The method according to the preceding embodiment, wherein the method comprises sending the output for a plurality of the plurality of images.

[0173] M48. The method according to any of the preceding method embodiments, wherein the method comprises storing the output of executing the data processing pipeline.

[0174] M49. The method according to any of the preceding method embodiments, wherein the data source is associated with a unique source identifier.

[0175] M50. The method according to the preceding embodiment and with the features of embodiment M48, wherein the method comprises storing the output with the source identifier.

[0176] M51. The method according to any of the preceding method embodiments and with the features of embodiment M4, wherein the method comprises storing the output of the data processing module.

[0177] M52. The method according to the preceding embodiment and with the features of embodiment M49, wherein the method comprises storing the output of the data processing module with the source identifier.

[0178] M53. The method according to any of the 2 preceding embodiments, wherein the method comprises deleting the output of the data processing module after a defined time interval has elapsed.

[0179] M54. The method according to the preceding embodiment and with the features of embodiment M52, wherein the method comprises deleting the output of the data processing module after a defined time interval has elapsed such that a next output of the data processing module with the same source identifier has not been stored.

[0180] M55. The method according to any of the preceding method embodiments, wherein the method comprises receiving a plurality of streams of data each from a distinct data source.

[0181] M56. The method according to any of the preceding method embodiments, wherein the method comprises pre-processing the stream of data, or at least a part thereof, before executing the data processing pipeline.

[0182] M57. The method according to any of the preceding method embodiments and with the features of embodiment M45, wherein triggering an action comprises changing a state of a component and / or a device.

[0183] M58. The method according to any of the preceding method embodiments and with the features of embodiment M8, wherein creating the data processing pipeline comprises connecting at least one of the plurality of selected data processing modules with at least one other of the plurality of selected data processing modules.

[0184] Below system embodiments will be discussed. These are abbreviated by the letter "S" followed by a number. Whenever reference is, herein, made to the system embodiments, the following embodiments are meant.

[0185] 51. A system for data processing, wherein the system is configured: to receive a stream of data from a data source, to execute a data processing pipeline on the stream of data, or at least a part thereof, and to send an output of executing the data processing pipeline.

[0186] 52. The system according to the preceding embodiment, wherein the system comprises a data processing component.

[0187] 53. The system according to the preceding embodiment, wherein the system comprises a plurality of data processing components.

[0188] 54. The system according to the preceding embodiment, wherein the plurality of data processing components comprises a first data processing component and a second data processing component.

[0189] 55. The system according to the preceding embodiment, wherein the first data processing component and the second data processing component are housed in the same device. 56. The system according to the penultimate embodiment, wherein the first data processing component and the second data processing component are housed in different devices.

[0190] 57. The system according to any of the preceding system embodiments, wherein the system comprises a memory component.

[0191] 58. The system according to any of the preceding system embodiments, wherein the system comprises a communication unit.

[0192] 59. The system according to the preceding embodiment and with the features of embodiment S2, wherein the communication unit is configured to communicate with the data processing component.

[0193] 510. The system according to any of the 2 preceding embodiments, wherein the communication unit is configured to communicate with the data source.

[0194] 511. The system according to any of the preceding system embodiments, wherein the system comprises a user interface component configured to allow the system to communicate with a user.

[0195] 512. The system according to any of the preceding system embodiments, wherein the data processing pipeline is associated with a unique identifier.

[0196] 513. The system according to the preceding embodiment, wherein the system is configured to receive the unique identifier.

[0197] 514. The system according to the preceding embodiment, wherein the system is configured to retrieve the data processing pipeline based, at least in part, on the received unique identifier.

[0198] 515. The system according to any of the preceding system embodiments, wherein the system is configured to create the data processing pipeline.

[0199] 516. The system according to the preceding embodiment, wherein creating the data processing pipeline comprises generate a unique identifier associated with the created data processing pipeline.

[0200] S17. The system according to any of the 2 preceding embodiments, wherein the system is configured to store the created data processing pipeline. 518. The system according to any of the 2 preceding embodiments, wherein the system is configured to store the generated unique identifier.

[0201] 519. The system according to any of the 3 preceding embodiments, wherein the system is configured to send the generated unique identifier.

[0202] 520. The system according to any of the preceding system embodiments, wherein the data processing pipeline comprises a data processing module configured to accept an input and to send an output.

[0203] 521. The system according to the preceding embodiment, wherein the data processing pipeline comprises a plurality of data processing modules.

[0204] 522. The system according to any of the preceding system embodiments and with the features of embodiment S15, wherein the system is configured to allow selecting, from a collection of available data processing modules, a data processing module, to create the data processing pipeline.

[0205] 523. The system according to the preceding embodiment, wherein the system is configured to allow selecting, from a collection of available data processing modules, a plurality of data processing modules, to create the data processing pipeline.

[0206] 524. The system according to any of the 2 preceding embodiments, wherein the system is configured to present a graphical user interface, and wherein selecting the data processing module, or the plurality thereof, comprises using the graphical user interface.

[0207] 525. The system according to the preceding embodiment, wherein the graphical user interface comprises a distinct draggable element corresponding to each available data processing module, and wherein selecting the data processing module, or the plurality thereof, comprises dragging and dropping the draggable element, or a plurality thereof, corresponding to the data processing module, or to the plurality thereof.

[0208] 526. The system according to any of the preceding system embodiments, wherein the system is configured to execute the data processing pipeline on the stream of data, or the part thereof, in real-time.

[0209] 527. The system according to any of the preceding system embodiments, wherein the system is configured to automatically execute the data processing pipeline on the stream of data, or the part thereof.

[0210] S28. The system according to any of the preceding system embodiments, wherein the stream of data, or the part thereof, is representative of a video stream. 529. The system according to any of the preceding system embodiments, wherein the stream of data, or the part thereof, is representative of an audio stream.

[0211] 530. The system according to any of the preceding system embodiments and with the features of embodiment S22, wherein the system is further configured to provide the data processing module.

[0212] 531. The system according to any of the preceding system embodiments and with the features of embodiment S23, wherein the system is further configured to provide the plurality of data processing modules.

[0213] 532. The system according to any of the preceding system embodiments, wherein executing the data processing pipeline comprises performing, at least in part, a convolution of the stream of data, or the part thereof.

[0214] 533. The system according to any of the preceding system embodiments, wherein the output comprises textual data.

[0215] 534. The system according to the preceding embodiment, wherein the textual data comprises metadata about the stream of data, or at least a part thereof.

[0216] 535. The system according to any of the 2 preceding embodiments, wherein the textual data relates, at least in part, to a summary of the stream of data, or the part thereof.

[0217] 536. The system according to any of the preceding system embodiments, wherein executing the data processing pipeline comprises detecting a feature in the stream of data, or the part thereof.

[0218] 537. The system according to the preceding embodiment and with the features of embodiment S33, wherein the textual data is based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0219] 538. The system according to any of the preceding system embodiments, wherein the system is configured to ask for user input.

[0220] 539. The system according to the preceding embodiment and with the features of embodiment S33, wherein the textual data comprises, at least in part, the user input, or at least a part thereof. 540. The system according to any of the preceding system embodiments, wherein executing the data processing pipeline comprises excluding a feature from the stream of data, or the part thereof.

[0221] 541. The system according to the preceding embodiment, wherein executing the data processing pipeline comprises excluding a plurality of features from the stream of data, or the part thereof.

[0222] 542. The system according to any of the preceding system embodiments, wherein the system is configured to determine a time interval between two successive outputs of executing the data processing pipeline.

[0223] 543. The system according to any of the preceding system embodiments and with the features of embodiment S20, wherein the system is configured to determine a time interval between two successive outputs of the data processing module.

[0224] 544. The system according to any of the preceding system embodiments, wherein the system is configured to initialize an accumulator.

[0225] 545. The system according to the preceding embodiment and with the features of embodiment S42, wherein the system comprises accumulating the time interval in the accumulator.

[0226] 546. The system according to the preceding embodiment and with the features of embodiment S36, wherein the method comprises accumulating the time interval in the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0227] 547. The system according to any of the preceding system embodiments and with the features of embodiment S44, wherein the system is configured to reset the accumulator.

[0228] 548. The system according to the preceding embodiment and with the features of embodiment S36, but without the features of the penultimate embodiment, wherein the system is configured to reset the accumulator, based, at least in part, on a result of the detection of a feature in the stream of data, or the part thereof.

[0229] 549. The system according to any of the preceding system embodiments, wherein the system is configured to trigger an action based, at least in part, on the output.

[0230] S50. The system according to any of the preceding system embodiments, wherein the system is configured to change a state of a component and / or a device based, at least in part, on the output of the execution of the data processing pipeline. 551. The system according to any of the preceding system embodiments and with the features of embodiment S28, wherein the stream of data comprises a plurality of images.

[0231] 552. The system according to the preceding embodiment, wherein the system is configured to send the output for a plurality of the plurality of images.

[0232] 553. The system according to any of the preceding system embodiments, wherein the system is configured to store the output of executing the data processing pipeline.

[0233] 554. The system according to any of the preceding system embodiments, wherein the data source is associated with a unique source identifier.

[0234] 555. The system according to the preceding embodiment and with the features of embodiment S53, wherein the system is configured to store the output with the source identifier.

[0235] 556. The system according to any of the preceding system embodiments and with the features of embodiment S20, wherein the system is configured to store the output of the data processing module.

[0236] 557. The system according to the preceding embodiment and with the features of embodiment S54, wherein the system is configured to store the output of the data processing module with the source identifier.

[0237] 558. The system according to any of the 2 preceding embodiments, wherein the system is configured to delete the output of the data processing module after a defined time interval has elapsed.

[0238] 559. The system according to the preceding embodiment and with the features of embodiment S57, wherein the system is configured to delete the output of the data processing module after a defined time interval has elapsed such that a next output of the data processing module with the same source identifier has not been stored.

[0239] 560. The system according to any of the preceding system embodiments, wherein the system is configured to receive a plurality of streams of data each from a distinct data source.

[0240] S61. The system according to any of the preceding system embodiments, wherein the system is configured to pre-process the stream of data, or at least a part thereof, before executing the data processing pipeline. S62. The system according to any of the preceding system embodiments, wherein the system is configured to perform the method according to any of the preceding method embodiments.

[0241] Below computer program product embodiments will be discussed. These are abbreviated by the letter "P" followed by a number. Whenever reference is, herein, made to the computer program product embodiments, the following embodiments are meant.

[0242] Pl. A computer program product comprising instructions, when run on a system according to any of the preceding system embodiments, to perform the method according to any of the preceding method embodiments.

[0243] P2. The computer program product according to the preceding embodiment, wherein the system comprises a system according to the system embodiment S2.

[0244] Brief Figure Description

[0245] Figure 1 depicts an embodiment of a system according to the present technology,

[0246] Figure 2 depicts an embodiment of a method according to the present technology, and

[0247] Figure 3 depicts other embodiments of the system according to the present technology.

[0248] Detailed Figure Description

[0249] Figure 1 depicts a system 1 for performing one embodiment of a method according to the present invention. The system 1 comprises a first data processing component 101, and a user interface component 102. The first data processing component 101 and the user interface component 102 may communicate, i.e., exchange data, with each other. The system 1 may further comprise a memory component 105, that may be used to store data. The first data processing component 101 may be configured to communicate with the memory component 105. The user interface component 102 may comprise means to accept input from a user. Alternatively, or additionally, the user interface component 102 may comprise means to report an output to a user. For example, the user interface component 102 may comprise any of a touchscreen, a keyboard, a mouse, a joystick, a gamepad, or any other suitable means to input information. The user interface component 102 may comprise a display, a printer, a speaker, or any other suitable means to output information.

[0250] In some embodiments, the system 1 may additionally comprise a second data processing component 104 and a communication unit 103. The communication unit 103 may be configured to allow communication between the first data processing component 101 and the second data processing component 104. The communication unit 103 may thus enable a communication channel between the first data processing component 101 and the second data processing component 104. It may be further understood that, generally, the system 1 may comprise a plurality of any of the user interface component 102, the communication unit 103, and the data processing components 101, 104.

[0251] The first data processing component 101 may typically be comprised in one device, and the second data processing component 104 may typically be comprised in another device. However, in some embodiments, the system 1 may comprise only the first data processing component 101, comprised in one device.

[0252] The system 1 may be configured to execute a data processing pipeline. The system 1 may alternatively, or additionally, be configured to allow a user to create a data processing pipeline and to execute it.

[0253] The data processing pipeline may be stored in the memory component 105. A unique identifier may be associated with the data processing pipeline. The system 1 may be configured to allow retrieval of the data processing pipeline from the memory component 105 based, at least in part, on the unique identifier. In embodiments where the system 1 is used to create the data processing pipeline, the system 1, particularly the first data processing component 101 thereof, may be configured to generate the unique identifier associated with the data processing pipeline. The generated unique identifier may be sent to the user interface component 102. In some embodiments, the system 1 may be configured to prompt the user to provide the unique identifier associated with the data processing pipeline.

[0254] The system 1, particularly the memory component 105 thereof, may comprise one or more available data processing modules. A data processing module may be understood to comprise a module that may accept data in a defined format as input, and output a result of processing that data. The output may also comprise a defined format. At least one of the available data processing modules may be configured to convolve the input data with a defined function. Note further, that while the following disclosure relates to one or more data processing modules, it may be understood that in some embodiments, a monolithic structure may also be used to the same effect.

[0255] Using the user interface component 102, a user may select one or more of the available data processing modules to create a data processing pipeline, such that data may be input at one end of the data processing pipeline, and a result of executing the data processing pipeline may be obtained at the other end. In other words, a data processing pipeline may be understood to comprise one or more data processing modules arranged such that the output from one of the data processing modules is used as the input to another of the data processing modules.

[0256] The user may define a desired order of the plurality, if present, of selected data processing modules based, at least in part, on a desired sequence of operations to be performed. In other words, the system 1 may be configured to allow the user to define a desired order of the plurality of selected data processing modules. Alternatively, or additionally, each of the data processing modules may be configured to be connected to only some of the available data processing modules. In particular, any defined data processing module may only receive input from such a data processing module as outputs the data in the defined input format of the defined data processing module. Similarly, the defined data processing module may only send output to such a data processing module as accepts the data in the format of the output generated by the defined data processing module. Thus, generally, the order of data processing modules may also be based on the defined formats input to / output by each of the data processing modules.

[0257] The system 1 may be configured to receive data from a data source 2. The data source 2 may, for example, send data to the communication unit 103 and the communication unit 103 may further send it to the first data processing component 101. Pre-processing of the data received from the data source 2, comprising checks regarding data fidelity, completeness, and others that may be of advantage in preparing the data for further processing in the data processing component 101 such as lowering a resolution or changing a color space of the data received from the data source 2, may, for example, be carried out by the communication unit 103 and / or a data processing unit provided proximal the data source 2 before sending the data further to the first data processing component 101. Alternatively, or additionally, the preprocessing, or at least a part thereof, may be carried out by the first data processing component 101.

[0258] The data source 2 may be associated with a unique identifier, that may be called a source identifier, for example. The source identifier may be of advantage in determining, for example, a type of data received from the data source 2 or a data processing pipeline to be used for processing data received from the data source 2. Moreover, the source identifier may be of advantage in allowing the system 1 to simultaneously execute a plurality of data processing pipelines each on data received from a distinct data source 2, as will be described further below. The system 1, particularly the first data processing component 101 thereof, may be configured to receive the source identifier.

[0259] The first data processing component 101 may be configured to receive a data processing pipeline identifier. The first data processing component 101 may be configured to retrieve the data processing pipeline corresponding to the data processing pipeline identifier from the memory component 105. For example, a user may provide the data processing pipeline identifier via the user interface component 102. Alternatively, or additionally, the data processing pipeline identifier may be associated with the data source 2. Alternatively, or additionally, a client software that may receive an output 3 of the execution of the data processing pipeline may send the data processing pipeline identifier. The first data processing component 101 may be configured, in particular, to execute the retrieved data processing pipeline on the (pre-processed) data received from the data source 2. More particularly, the data processing pipeline may be executed on the data in real-time.

[0260] The data processing pipeline may be executed on chunks of data at a time. For example, the data source 2 may send a stream of integers, each corresponding to a 32-bit integer. The data processing pipeline may be configured to accept an integer and process it to return, say, a Boolean value. Then, executing the data processing pipeline in real-time may be understood to comprise the generation of Boolean outputs for each of the integers in the stream of integers, such that a stream of Boolean output values is generated. Depending on a size of the chunk of data processed by the data processing pipeline, the time difference between generation of an output and accepting of input may be variable. For example, the time difference between receiving data from the communication unit 103 and sending the output of the execution of the data processing pipeline on the data may be set as a parameter in the data processing pipeline, or may be pre-defined based on a kind of tasks implemented by the data processing pipeline.

[0261] The data source 2 may be configured to send a stream of data representative of audio or video information, or both, for example. Preferably, the stream of data may be representative of video information. The data processing pipeline may be configured, for example, to process data representative of an image frame when the stream of data is representative of video information. In other words, the chunk of data, as described above, may comprise an image frame. Executing the data processing pipeline in real-time may then comprise, for example, generating an output for each image frame of the video information in succession.

[0262] The system 1, particularly the first data processing component 101 thereof, may be configured to represent any of the available data processing modules as a graphical component. The graphical component may be displayed, for example, on a display of the user interface component 102. The graphical component may additionally be draggable. Thus, creating a data processing pipeline may comprise dragging a data processing module, particularly the graphical representation thereof, comprised in the data processing pipeline.

[0263] The system 1 may be further configured to allow creation of a data processing pipeline by dropping a dragged data processing component, particularly the graphical representation thereof. In particular, a plurality of data processing modules, particularly the graphical representations thereof, may be dragged and dropped to create the data processing pipeline.

[0264] Further, the system 1 may be configured to allow establishing directed connections between any of the data processing modules, or the graphical representations thereof. The connections may also be represented graphically, for examples, as arrows. The connections may be of relevance when establishing a path along which data flows in the data processing pipeline. Connections may be permitted between data processing modules, or the graphical representations thereof, subject to format requirements as described above, and the allowed permissions may be stored in the memory component 105, for example.

[0265] Thus, generally, it may be understood that a user may create a data processing pipeline using the system 1, wherein creating the data processing pipeline may comprise dragging and dropping one or more graphical components, each representative of a data processing module, and establishing connections between any of the one or more graphical components.

[0266] The data processing pipeline may be configured, at least in part, to determine / detect the presence of a feature in the data. For example, the data source 2 may send video camera data, and the data processing pipeline may comprise a face-detection module configured to detect a face in the data sent from the video camera.

[0267] In particular, the available data processing modules may comprise a data processing module configured to detect the presence of a feature in the data. The output of such a data processing module may comprise a Boolean value indicating a result of the detection of the feature, viz., true if the feature was detected and false if the feature was not detected, and / or the data input to the data processing module.

[0268] The data processing module, as described above, may be configured to be connected to more than one data processing module, such that the output may be sent to any one of the more than one data processing module. This may be of advantage, for example, when the data processing module is connected to two different data processing modules, and may send the output to one data processing module if the feature is detected, and send the output to the other data processing module if the feature is not detected. Thus, data may be further processed differently based, at least in part, on a result of the detection of the feature in the stream of data. Thus, such a module may allow creating branches in the data processing pipeline, such that one branch may be executed if the feature is detected, and another branch may be executed if the feature is not detected.

[0269] For example, the data processing modules may be configured for any of person detection, face detection, person face detection, garment detection, garment check, face analysis, person analysis, face pose analysis, face gender majority, or any other suitable detection modules. Further, for example, a face pose analysis module may only be connected to a "true" branch from the face detection module. Note that the above list may be considered exemplary, but not limiting, of different data processing modules that may be available. Generally, it may be understood that the data processing modules may be configured for any object or attribute detection.

[0270] The data processing modules described above may comprise, for example, machine learning models. In particular, they may comprise neural networks, wherein the weights may have been obtained by a suitable training algorithm prior to being stored in the memory component 105 or the memory component connected to the second data processing component 104 as described further below. Methods for training neural networks for carrying out feature detection are widely known and are not described further here.

[0271] The result 3 of executing the data processing pipeline may comprise, at least in part, textual data, i.e., the output 3 of the data processing pipeline may comprise, at least in part, textual data. Textual data may be understood to comprise any string of alphanumeric or special characters. The textual data may relate to metadata about the data received from the data source 2. Alternatively, or additionally, the textual data may comprise a summary of the data received from the data source 2. Additionally, or alternatively, the textual data may comprise data relating to a result of the detection of a feature in the data as described above.

[0272] Further, the output 3 may comprise data representative of an audio or a video. Preferably, however, the output 3 comprises textual data.

[0273] The system 1 may be further configured to accept input from a user, and the output 3 may comprise, at least in part, data representative of the user input. For example, the user may input one message to be sent and / or displayed at a display of the user interface component 102 when a face is detected in the data received from the data source 2, and / or another message to be sent and / or displayed when a face is not detected.

[0274] For example, the data source 2 may comprise a camera installed in a room. The camera may send images of the room at some times to the system 1 for further processing. The system 1 may be configured to execute a data processing pipeline that may determine a number of people in the room based, at least in part, on the images captured by the camera. The user may then choose, as part of the data processing pipeline, to send, as the output 3, a message "decrease temperature" if the number of people detected is greater than 10, for example, and a message "increase temperature" otherwise. The message may be sent, for example, to a thermostat connected to the system 1.

[0275] Further exemplarily, another data processing pipeline, determined using its unique identifier as described above, may be configured to send the same messages based on different criteria such as number of people being greater / lesser than 20 or a time of stay inside the room based on the images. Thus, by changing just the data processing pipeline executed, the system 1 may allow simple and efficient change of behavior.

[0276] The system 1, particularly the first data processing component 101 thereof, may be configured to accept an execution plan comprising a start time and a data processing pipeline identifier. In some embodiments, the execution plan may also comprise a stop time. More particularly, the execution plan may comprise a plurality of data processing pipeline identifiers, each associated with a start time. The system 1, particularly the first data processing component 101 thereof, may be configured, based, at least in part, on the execution plan, to execute the data processing pipeline corresponding to a data processing pipeline identifier starting from a time significantly identical to the start time associated with the data processing pipeline identifier.

[0277] The system 1, particularly the first data processing component 101 thereof, may be configured, based, at least in part, on the execution plan, to stop executing the data processing pipeline corresponding to a data processing pipeline identifier starting from a time significantly identical to the stop time associated with the data processing pipeline identifier.

[0278] The data processing pipeline may be configured to determine a time interval between two successive outputs 3 of the data processing pipeline and / or between successive outputs of any of the data processing modules in the data processing pipeline. In other words, the available data processing modules as described above may further comprise, for example, a timer module that may be configured to allow measuring the time interval between successive outputs 3. For example, the timer module may be configured to be connected, at least significantly, close to the end of the data processing pipeline, and to record a time at which it receives an input. The time may, for example, be recorded in the received data and then the time-stamped data be output from the timer module. Then, by using the record of the times at which inputs were received in the timer module, time intervals between successive outputs 3 of the data processing pipeline may be determined.

[0279] The timer module may be of particular advantage in pipelines with branches, for example, where they may allow determining, for example, a time spent in a given branch. For example, it may be advantageous to determine the total amount of time that a face is detected in a video feed from, say, a webcam. Then, the timer module may be attached in a "face detected" branch of the data processing pipeline. At the same time, another timer module attached to the "no face detected" branch that may, for example, record a time at which the non-detection occurred. By comparing the outputs 3 from each of these branches, it may be possible to determine a continuous length of time during which any of the branches was active.

[0280] The system 1 may be configured to send the output 3 to an external device 4. The external device 4 may be configured to trigger an action based, at least in part, on the output 3. The action may comprise a physical action, such as triggering an alarm, or a virtual action, such as deleting data from a memory component or further analysis based, at least in part, on the output 3. Generally, any action that leads to a change of state may be triggered. For example, the external device 4 may comprise an Internet-of-Things (loT) device and based on the output 3, the loT device may be activated / inactivated.

[0281] Alternatively, or additionally, the action may be triggered by the system 1, particularly the data processing components 101, 104 thereof, based, at least in part, on the output 3. For example, as described above, the system 1, particularly the data processing components 101, 104 thereof, may be configured to send the output 3, or at least a part thereof, to the user interface component 102.

[0282] The system 1 may be configured to store the output 3 in the memory component 105. The output 3 may be stored with the source identifier as described above. Thus, the system 1 may be configured to store a record of the outputs 3 generated for data received from the data source 2. This may be of particular advantage in recording a history of output from the execution of the data processing pipeline for data received from different data sources 2.

[0283] The system 1 may also be configured to store intermediate data generated during a single execution of the data processing pipeline, i.e., execution of the data processing pipeline for a chunk of data, together with the source identifier. For example, the system 1 may store the output from each, or at least some, of the data processing modules in the data processing pipeline. The intermediate data may be used when the next chunk of data from the same data source 2 is received. For example, storing the intermediate data may be of particular advantage when using the timer module as described above.

[0284] The intermediate data may be deleted after a defined interval of time has elapsed. The defined interval of time may be based, at least in part, on the data source 2. For example, the defined time interval may be 2 seconds. Further, the intermediate data may be deleted if a next data chunk from the same data source 2 has not been received within a defined time interval. For example, the intermediate data may be deleted if a next chunk of data is not received from the same data source 2 within 2 seconds.

[0285] Any of the functions described above may be carried out in the first data processing component 101. For example, the first data processing component 101 may retrieve the data processing modules from the memory component 105 and execute the data processing pipeline, and send the output 3 to trigger an action.

[0286] In some embodiments, however, at least some of the functions described above may be carried out in the second data processing component 104. The second data processing component 104 may be appropriately configured therefor. For example, the second data processing component 104 may pre-process, at least in part, the data received from the data source 2, as described above, or the second data processing component 104 may communicate with a second memory component, that may serve at least some of the functions described above for the memory component 105, such as storing and / or retrieving the data processing modules, storing and / or retrieving data processing pipelines, and so on.

[0287] Figure 2 depicts one embodiment of the method according to the present invention. The method comprises a first step, Pl, comprising acquiring raw data from the data source 2. In a next step, P2, the raw data may be pre-processed, such as checking for data fidelity, completeness, lowering resolution, or changing color space. Then, in a third step, P3, the pre- processed data is input to the data processing pipeline and the data processing pipeline is executed on the pre-processed data. Finally, in a last step, P4, the output 3 of the data processing pipeline is sent to trigger further action.

[0288] Depending on where any of the steps described above is carried out, different embodiments of the system 1 may be realized, as depicted in Figure 3. The raw data may be acquired at a device DI from the data source 2. For example, the data source 2 may comprise a smartphone camera, a webcam, a security camera, or other suitable data sources 2.

[0289] The raw data may be sent for pre-processing to the first data processing component 101 of the system 1. As depicted in Figure 3, the first data processing component 101 may be located on the same device that acquires raw data, viz., the device DI, or on a different device, say D2. For example, when the device DI comprises a smartphone, a processing capacity of the smartphone DI may be sufficient for pre-processing the raw data. Alternatively, when the processing capacity is not sufficient for pre-processing the raw data such as in the case of a webcam, the raw data may be sent to another device, such as a computer D2, to which the webcam may be connected. Generally, the raw data may be sent over a private connection, such as a wired connection, to ensure data security. The raw data may or may not be encrypted before sending to the other device.

[0290] In the next step, the pre-processed data may be sent to the data processing pipeline. The data processing pipeline may be located on the device DI, on the device D2 used for preprocessing the raw data as described above, or on a device D3 comprising the second data processing component 104 described above. In other words, the pre-processed data may be further processed using the data processing pipeline on any of the raw data acquisition device, the pre-processing device, or a third device. For example, the second data processing component 104 may be comprised in a server D3 that may be privately owned, or in a server D4 that may be contracted from a provider.

[0291] The last step, relating to further action based on the output 3 of the data processing pipeline may also be carried out on either the same device as used to execute the processing pipeline or on another device.

[0292] The data may, generally, be encrypted before sending it to a different device. In particular, the pre-processed data may be encrypted before sending it to the second processing component 104 for executing the data processing pipeline, and / or further to another device for triggering the action. As may be appreciated, no encryption may be necessary if the same device is used for any of the above steps. For example, the data flows depicted by dashed lines in Figure 3 may comprise flow of data over a network, either local or over the internet. Data transmitted over the network may be encrypted. On the other hand, data flows depicted with solid lines may comprise data flow within the same device / trusted network, and so, need not comprise flow of encrypted data. However, in some embodiments, even these may comprise flow of encrypted data. Overall, embodiments of the present invention thus relate to systems and methods for realtime data processing that may be more secure, more efficient, more customizable, and simpler to use.

[0293] Whenever a relative term, such as "about", "substantially" or "approximately" is used in this specification, such a term should also be construed to also include the exact term. That is, e.g., "substantially straight" should be construed to also include "(exactly) straight".

[0294] Whenever steps were recited in the above or also in the appended claims, it should be noted that the order in which the steps are recited in this text may be accidental. That is, unless otherwise specified or unless clear to the skilled person, the order in which steps are recited may be accidental. That is, when the present document states, e.g., that a method comprises steps (A) and (B), this does not necessarily mean that step (A) precedes step (B), but it is also possible that step (A) is performed (at least partly) simultaneously with step (B) or that step (B) precedes step (A). Furthermore, when a step (X) is said to precede another step (Z), this does not imply that there is no step between steps (X) and (Z). That is, step (X) preceding step (Z) encompasses the situation that step (X) is performed directly before step (Z), but also the situation that (X) is performed before one or more steps (Yl), ..., followed by step (Z). Corresponding considerations apply when terms like "after" or "before" are used.

[0295] While in the above, preferred embodiments have been described with reference to the accompanying drawings, the skilled person will understand that these embodiments were provided for illustrative purpose only and should by no means be construed to limit the scope of the present invention, which is defined by the claims.

Claims

Claims1. A method for data processing, wherein the method comprises: receiving a stream of data from a data source, executing a data processing pipeline on the received stream of data, or at least a part thereof, and sending an output of executing the data processing pipeline.

2. The method according to the preceding claim, wherein the method further comprises creating the data processing pipeline.

3. The method according to the preceding claim, wherein creating the data processing pipeline comprises selecting, from a collection of available data processing modules, a data processing module, or a plurality of data processing modules.

4. The method according to the preceding claim, wherein selecting the data processing module, or the plurality thereof, comprises using a graphical user interface, wherein the graphical user interface comprises a distinct draggable element corresponding to each available data processing module, and wherein selecting the data processing module, or the plurality thereof, comprises dragging and dropping the draggable element, or a plurality thereof, corresponding to the data processing module, or to the plurality thereof.

5. The method according to any of the preceding claims and with the features of claim 2, wherein creating the data processing pipeline comprises generating a unique identifier for the created data processing pipeline.

6. The method according to any of the preceding claims and with the features of claim 3, wherein the method comprises providing the data processing module, or the plurality of data processing modules.

7. The method according to any of the preceding claims, wherein the data source is associated with a unique source identifier.

8. The method according to the preceding claim, wherein the method comprises storing the output of executing the data processing pipeline, and wherein the method comprises storing the output with the source identifier.

9. The method according to any of the preceding claims, wherein the method comprises receiving a plurality of streams of data each from a distinct data source.

10. A system for data processing, wherein the system is configured: to receive a stream of data from a data source,to execute a data processing pipeline on the stream of data, or at least a part thereof, and to send an output of executing the data processing pipeline.

11. The system according to the preceding claim, wherein the system comprises a plurality of data processing components, and wherein the plurality of data processing components comprises a first data processing component and a second data processing component.

12. The system according to the preceding claim, wherein the first data processing component and the second data processing component are housed in the same device.

13. The system according to the penultimate claim, wherein the first data processing component and the second processing component are housed in different devices.

14. The system according to any of the claims 10 to 13, wherein the system is configured to change a state of a component and / or a device based, at least in part, on the output of the execution of the data processing pipeline.

15. A computer program product comprising instructions, when run on a system according to any of the claims 10 to 14, to perform the method according to any of the claims 1 to 9.

Images