Data stream processing methods, apparatus, devices and readable storage media

Abstract

This invention discloses a data stream processing method, apparatus, device, and computer-readable storage medium. The method includes: when it is determined that the cache space is insufficient to cache a second data stream received before the first data stream is accessed, storing the portion of the second data stream received before the first data stream that cannot be cached in the cache space into memory space; after the first data stream is accessed, transmitting the received first data stream to a data processing device; and reading the second data stream from its storage space and transmitting it to the data processing device. This invention reduces the latency and storage cost of simultaneous processing of multiple data streams.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data stream processing method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Wide dynamic range (WDR) technology is a technique that allows image sensors to capture multiple frames of image data with different exposure times in high-contrast environments. This data is then transmitted to an image signal processor (ESP) for wide dynamic range calculations and outputs a bright, balanced image. WDR enables video recording equipment to capture clear images even in bright or backlit conditions. WDR requires the synchronous processing of two or more image signal data streams with different exposure times, necessitating buffering of the first data stream received. In some fields, high real-time video performance is required, necessitating low-latency image signal processing. Furthermore, storage media is a significant cost factor for video equipment, making it crucial to reduce the memory requirements of image processing. Summary of the Invention

[0003] The main objective of this invention is to provide a data stream processing method, apparatus, device, and computer-readable storage medium, aiming to provide a data stream processing solution that reduces the latency of simultaneous processing of multiple data streams while lowering storage costs.

[0004] To achieve the above objectives, the present invention provides a data stream processing method, the method comprising the following steps:

[0005] Receive a first data stream and a second data stream, wherein the access time of the first data stream is later than the access time of the second data stream;

[0006] If it is determined that the cache space is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the cache space will be stored in the memory space.

[0007] After receiving the first data stream, the received first data stream is transmitted to the data processing device, and the second data stream is read from the storage space according to the first-in-first-out method and transmitted to the data processing device, wherein the cache space and the memory space are both storage spaces.

[0008] Optionally, the data stream processing method further includes:

[0009] After reading the stored second data stream from its storage space and transmitting it to the data processing device, the second data stream received after the first data stream is accessed is stored in a target space, wherein the target space is the storage space that becomes available due to the removal of the stored second data stream.

[0010] Optionally, the step of storing the portion of the second data stream received before the first data stream was accessed that cannot be cached in the cache space into memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was accessed includes:

[0011] After receiving the second data stream, the received second data stream is cached in the cache space;

[0012] Detect whether the cache size of the cache space has reached the preset cache limit, and detect whether the first data stream has been connected;

[0013] If it is detected that the first data stream is not connected after the cache size in the cache space reaches the preset cache limit, then the second data stream received before the first data stream is connected and after the cache size in the cache space reaches the preset cache limit will be stored in the memory space.

[0014] Optionally, the step of storing the portion of the second data stream received before the first data stream was accessed that cannot be cached in the cache space into memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was accessed includes:

[0015] Obtain the access time difference between the first data stream and the second data stream;

[0016] If the access time difference is greater than a preset threshold, then a preset size of memory space is requested, wherein the memory space is matched with the access time difference;

[0017] After the second data stream is received, the received second data stream is cached in the cache space. After the cache size of the cache space reaches the preset cache limit, the second data stream received before the first data stream is received is stored in the applied memory space.

[0018] Optionally, the first data stream is the data stream with the latest access time among the multiple data streams, and the second data stream is any data stream other than the first data stream among the multiple data streams. Corresponding buffer spaces are pre-set for each data stream other than the first data stream among the multiple data streams.

[0019] The step of storing the portion of the second data stream received before the first data stream was accessed that cannot be cached in the cache space into memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was accessed includes:

[0020] If it is determined that the cache space corresponding to the second data stream is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the corresponding cache space will be stored in memory space.

[0021] Optionally, the data processing device is an image processing circuit, and the step of receiving the first data stream and the second data stream includes:

[0022] The system receives a first data stream and a second data stream from an image sensor, wherein the exposure duration of the first data stream is greater than the exposure duration of the second data stream.

[0023] To achieve the above objectives, the present invention also provides a data stream processing apparatus, the apparatus comprising:

[0024] A receiving module is used to receive a first data stream and a second data stream, wherein the access time of the first data stream is later than the access time of the second data stream;

[0025] The storage module is used to store the portion of the second data stream received before the first data stream was connected that cannot be cached in the cache space into the memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was connected.

[0026] The output module is used to transmit the received first data stream to the data processing device after receiving the first data stream, and to read the second data stream from the storage space according to the first-in-first-out method and transmit it to the data processing device, wherein the cache space and the memory space are both storage spaces.

[0027] Optionally, the storage module is further configured to:

[0028] After reading the stored second data stream from its storage space and transmitting it to the data processing device, the second data stream received after the first data stream is accessed is stored in a target space, wherein the target space is the storage space that becomes available due to the removal of the stored second data stream.

[0029] To achieve the above objectives, the present invention also provides a data stream processing device, the data stream processing device comprising: a memory, a processor, and a data stream processing program stored in the memory and executable on the processor, wherein the data stream processing program, when executed by the processor, implements the steps of the data stream processing method as described above.

[0030] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a data stream processing program, which, when executed by a processor, implements the steps of the data stream processing method described above.

[0031] In this embodiment of the invention, by detecting whether the cache space is sufficient to cache the second data stream received before the first data stream is accessed, if the cache space is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the cache space will be stored in memory space. This ensures that regardless of the access time interval between different data streams, the previously accessed data stream can be transmitted completely or partially through online caching, thereby minimizing data processing latency and storage costs. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating an embodiment of the data stream processing method of the present invention;

[0033] Figure 2 This is a schematic diagram of a data stream storage according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of a data stream storage according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of a data stream access according to an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of a multi-channel image signal data stream processing architecture according to an embodiment of the present invention;

[0037] Figure 6 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention.

[0038] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0039] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0040] Reference Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data stream processing method of the present invention.

[0041] This invention provides embodiments of a data stream processing method. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order. The data stream processing method in these embodiments can be applied to devices such as smartphones, personal computers, cameras, servers, and monitors, and is not limited thereto. The data stream processing method in these embodiments can be implemented by software programs or hardware circuits, and is not limited thereto. In this embodiment, the data stream processing method includes the following steps:

[0042] Step S10: Receive a first data stream and a second data stream, wherein the access time of the first data stream is later than the access time of the second data stream;

[0043] For any two data streams with different access times but requiring synchronous processing, the data stream with the later access time is referred to as the first data stream, and the data stream with the earlier access time is referred to as the second data stream for distinction. A data stream is data sent in a continuous flow, with a period of time between the start and end of transmission. The access time of a data stream refers to the time when the data stream is first received; that is, from the access time, data is continuously sent until all data transmission is complete. In this embodiment, there are no restrictions on the specific application scenario in which the data stream is generated or by whom it is sent. For example, it could be a multi-channel image signal data stream acquired by an image sensor, or a multi-channel audio signal data stream acquired by an audio acquisition device.

[0044] Step S20: If it is determined that the cache space is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the cache space will be stored in the memory space.

[0045] Both cache space and memory space are used for storing data, but the access speed of cache space is higher than that of memory space, and the cost is also higher. In this embodiment, the second data stream that is received first is cached using cache space, and if the cache space cannot hold the data, the remaining portion is stored in memory space. This achieves both improved data processing efficiency and reduced cache space, thus lowering costs.

[0046] In one feasible implementation, the memory space can be implemented using DDR memory. DDR stands for Double Data Rate Synchronous Dynamic Random Access Memory. DDR memory access refers to the process of writing and reading data from DDR memory. This data interaction enables data storage and data transmission.

[0047] It is understandable that if there is enough cache space to cache the second data stream received before the first data stream is accessed, all the second data streams received before the first data stream is accessed can be cached in the cache space.

[0048] For example, such as Figure 2 As shown, data stream i is equivalent to the second data stream, and the latest data stream is equivalent to the first data stream. The second data stream received before the first data stream is accessed can be cached in the buffer space, and no memory space is needed to store the second data stream at this time. Figure 3 As shown, the second data stream received before the first data stream is connected cannot be stored in the cache space, so a certain amount of memory space is borrowed to store the second data stream.

[0049] In one feasible implementation, step S20 includes:

[0050] Step S201: After receiving the second data stream, cache the received second data stream in the cache space;

[0051] It is understandable that the data stream is transmitted sequentially. For the second data stream that is received at the beginning, a buffer space is used to cache it in order to improve the efficiency of transmitting the second data stream to the data processing module later.

[0052] Step S202: Detect whether the cache size of the cache space has reached the preset cache limit, and detect whether the first data stream has been connected;

[0053] During the process of receiving the second data stream and caching it into the cache space, it is possible to detect whether the cache size of the cache space has reached the preset cache limit, and to detect whether the first data stream has been received. The preset cache limit is the maximum cache size of the cache space, which can be preset as needed, but is not limited in this embodiment.

[0054] Step S203: If it is detected that the first data stream is not connected after the cache size of the cache space reaches the preset cache limit, then the second data stream received before the first data stream is connected and after the cache size of the cache space reaches the preset cache limit will be stored in the memory space.

[0055] If the first data stream is not connected after the cache size of the cache space reaches the preset cache limit, it means that the cache space is not enough to cache the second data stream received before the first data stream is connected. In this case, the second data stream received before the first data stream is connected and after the cache size of the cache space reaches the preset cache limit can be stored in the memory space to reduce storage costs.

[0056] In one feasible implementation, step S20 includes:

[0057] Step S211: Obtain the access time difference between the first data stream and the second data stream;

[0058] In some application scenarios where the access time difference between the first data stream and the second data stream can be known in advance, the access time difference between the first data stream and the second data stream can be obtained before accessing the first data stream, so as to determine whether additional memory space is needed to store the second data stream based on the access time difference.

[0059] Step S212: If the access time difference is greater than a preset threshold, then apply for a preset size of memory space, wherein the memory space is matched with the access time difference;

[0060] The preset threshold can be pre-set based on the preset transmission speed of the data stream and the preset buffer space's upper limit. When the access time difference is greater than the preset threshold, it is assumed that the second data stream received before the first data stream is accessed will exceed the preset buffer space's upper limit. The preset size can also be pre-set based on the preset transmission speed of the data stream and the preset buffer space's upper limit. For example, it can be set to the access time difference multiplied by the data stream's transmission speed minus the preset buffer space's upper limit.

[0061] Step S213: After the second data stream is connected, the received second data stream is cached in the cache space, and after the cache size of the cache space reaches the preset cache limit, the second data stream received before the first data stream is connected is stored in the applied memory space.

[0062] After the second data stream is received, it can be initially cached using a buffer to improve the efficiency of transmitting it to the data processing module. During the process of receiving and caching the second data stream, the buffer size is checked to see if it has reached its preset cache limit. If the cache size reaches the preset limit, the second data streams received before the first data stream was received and after the cache size reached the preset limit can be stored in pre-allocated memory space, thereby reducing storage costs.

[0063] In a specific implementation, after the first data stream is received, the second data stream may not have finished transmitting, or the second data stream may have finished transmitting before the first data stream is received. In the case where the second data stream has not finished transmitting after the first data stream is received, this embodiment does not limit the storage method of the second data stream received after the first data stream is received; for example, it may also be stored in memory space.

[0064] Step S30: After receiving the first data stream, the received first data stream is transmitted to the data processing device, and the second data stream is read from the storage space according to the first-in-first-out method and transmitted to the data processing device, wherein the cache space and the memory space are both storage spaces.

[0065] To improve the real-time performance of data processing, the received first data stream can be directly transmitted to the data processing device after it is received. To process the first and second data streams synchronously, while transmitting the first data stream to the data processing device, the second data stream is also read from its storage space and transmitted to the data processing device, enabling the data processing device to process both data streams simultaneously. It is understood that both cache space and memory space are storage spaces. The portion of the second data stream stored in the cache space will be retrieved from the cache space and transmitted to the data processing device, while the portion stored in the memory space will be retrieved from the memory space and transmitted to the data processing device. In a specific implementation, when reading the second data stream from the storage space, it can be read according to the order in which the data was stored, with the earliest stored data being read first.

[0066] In one feasible implementation, to further improve the real-time performance of data processing and reduce latency, for the first data in the first data stream and the second data in the second data stream stored in the memory space, compared to the time required to transmit the first data to the data processing device, the second data can be read from the memory space in advance for a preset duration, and then the second data can be transmitted to the data processing device after reading. Here, the first data and the second data are data that need to be transmitted to the data processing device synchronously, and the preset duration is a pre-set memory space data retrieval delay, that is, the delay of retrieving data from the memory space and transmitting the data to the data processing device.

[0067] In this embodiment, by detecting whether the cache space is sufficient to cache the second data stream received before the first data stream is accessed, if the cache space is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the cache space is stored in memory. This ensures that regardless of the access time interval between different data streams, the data stream that was accessed earlier can be transmitted completely or partially through online caching, thereby minimizing data processing latency and storage costs. It is understandable that, compared to pre-setting all data streams with short access time intervals to be stored in the cache space and all data streams with long access time intervals to be stored in memory space based on the access time interval between each data stream and the data stream with the latest access time, this embodiment executes a corresponding detection process, which can flexibly allocate memory space. The time spent aligning the data streams is the access time interval between the first and second data streams, without introducing additional memory access latency. Therefore, it can align the data streams in the shortest time, achieving the goal of low latency and low memory usage.

[0068] Furthermore, based on the first embodiment described above, a second embodiment of the data processing method of the present invention is proposed. In this embodiment, the data stream processing method further includes:

[0069] Step S40: After reading the stored second data stream from its storage space and transmitting it to the data processing device, the second data stream received after the first data stream is accessed is stored in the target space, wherein the target space is the storage space that becomes available due to the removal of the stored second data stream.

[0070] In this embodiment, to further reduce memory usage, the second data stream received after the first data stream is accessed is stored by alternating between cache and memory.

[0071] After the first data stream is received, in order to process the first and second data streams synchronously, while transmitting the first data stream to the data processing device, the second data stream is read from the storage space where the second data stream is located and transmitted to the data processing device. As the second data stream is read from the storage space, the storage space that originally contained data becomes free. The newly received second data stream can be read and stored in the free storage space at the same time. This means that the second data stream received after the first data stream is received will not require additional storage space to store it, thereby further reducing the memory space occupation and achieving low memory and low latency.

[0072] The following is Figure 4The example shown illustrates the alternating use of cache and memory. Data stream i is equivalent to the second data stream, and the latest data stream is equivalent to the first data stream. When the amount of data that can be received from data stream i within the access time difference between data stream i and the latest data stream exceeds the cache limit, data stream i needs to move in and out of memory (DDR in the diagram) to achieve alignment with the latest data stream. To reduce memory requirements to the difference between the amount of data generated during the access time interval and the cache limit, data stream alignment is achieved through alternating use of cache and memory. For example... Figure 4 As shown, during the time interval t0 to t1, the earlier data stream i is written to the cache for later reading when the latest data stream is received, thus completing data stream alignment. During the time interval t1 to t2, the cache is completely full, and the subsequently received data stream i needs to be written to DDR, waiting to be read later for data stream alignment. During the time interval t2 to t3, both data stream i and the latest data stream are input simultaneously. At this time, it is necessary to read the previously written data stream i from the cache and also write the newly received data stream i to the cache. During the time interval t3 to t4, it is necessary to read the previously written data stream i from DDR and also write the newly received data stream i to DDR. In this process, writing the new data stream i is always later than reading the old data stream i. Therefore, the requested DDR memory space size only needs to be the difference D. The difference D is the difference between the amount of data generated by the access time interval between data stream i and the latest data stream and the upper limit of the cache space.

[0073] In this embodiment, memory space can be flexibly allocated based on the latency differences between multiple input data streams, aligning the data streams in the shortest possible time to achieve low latency and low memory usage. When the latency difference between data streams is less than a critical threshold, no memory space needs to be allocated; the data stream is directly transmitted online using an internal cache, achieving zero memory and low latency processing. This processing latency is the same as the processing latency required for the longest data stream branch, with no additional memory access latency. When the latency difference between data streams exceeds the critical threshold, a memory space is allocated for the earlier-accessed data stream. This memory size matches the latency difference of the data stream exceeding the critical threshold. By using a write-then-read-then-write operation on this memory space, the required memory space is reduced, achieving low memory and low latency processing. This processing latency is also the same as the processing latency required for the longest data stream branch, and no additional memory access latency is introduced.

[0074] Furthermore, based on the first and / or second embodiments described above, a third embodiment of the data stream processing method of the present invention is proposed. In this embodiment, step S20 includes:

[0075] Step S221: If it is determined that the cache space corresponding to the second data stream is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the corresponding cache space will be stored in memory space.

[0076] In this embodiment, multiple data streams can be processed synchronously. The access times of the multiple data streams are different. The data stream that is accessed last is called the first data stream, and any one of the other data streams is called the second data stream. That is, each of the data streams other than the first data stream can be aligned with the first data stream according to the data processing method in the above embodiment, thereby realizing the synchronous processing of multiple data streams, while also reducing memory usage and improving latency.

[0077] For each data stream other than the first data stream in a multi-stream dataset, a corresponding buffer space can be set. That is, if there are N data streams, N-1 buffer spaces can be set. For the second data stream, it can be determined whether the buffer space corresponding to the second data stream is sufficient to buffer the second data stream received before the first data stream is accessed. If it is insufficient, the portion of the second data stream received before the first data stream that cannot be buffered in the buffer space corresponding to the second data stream will be stored in memory.

[0078] Further, in one feasible embodiment, the data processing device is an image processing circuit, and step S10 includes:

[0079] Step S101: Receive the first data stream and the second data stream sent by the image sensor, wherein the exposure duration of the first data stream is greater than the exposure duration of the second data stream.

[0080] In this embodiment, the first data stream and the second data stream can be image signal data streams with different exposure durations, and correspondingly, the data processing device can be an image processing circuit. The data processing methods in the above embodiments can be applied to an image signal processor, which can be deployed in devices such as cameras and surveillance cameras. The image signal processor can use wide dynamic range (WDR) technology to process multiple image signal data streams acquired by the image sensor through different exposure durations. WDR technology is a technique in which, under high contrast conditions, the image sensor acquires multiple frames of image data with different exposure durations through multiple exposures, then transmits them to the image signal processor for wide dynamic range calculation and outputs a bright and balanced image. WDR technology enables video recording equipment to capture clear images even in strong light or backlight conditions.

[0081] In one feasible implementation, the processing architecture for the image signal data stream can be as follows: Figure 5 As shown. The data receiving and distribution module is used to receive multiple data streams with different exposure times transmitted by the image sensor. The latest data stream is directly transmitted to the image processing circuit, while the other data streams are buffered first. If the buffer space is insufficient, DDR memory space is used for storage. When it is needed, the data is read from the buffer space and DDR memory space and transmitted to the image processing circuit.

[0082] Furthermore, embodiments of the present invention also propose a data stream processing apparatus, the apparatus comprising:

[0083] A receiving module is used to receive a first data stream and a second data stream, wherein the access time of the first data stream is later than the access time of the second data stream;

[0084] The storage module is used to store the portion of the second data stream received before the first data stream was connected that cannot be cached in the cache space into the memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was connected.

[0085] The output module is used to transmit the received first data stream to the data processing device after receiving the first data stream, and to read the second data stream from the storage space according to the first-in-first-out method and transmit it to the data processing device, wherein the cache space and the memory space are both storage spaces.

[0086] Furthermore, the storage module is also used for:

[0087] After reading the stored second data stream from its storage space and transmitting it to the data processing device, the second data stream received after the first data stream is accessed is stored in a target space, wherein the target space is the storage space that becomes available due to the removal of the stored second data stream.

[0088] Furthermore, the storage module is also used to: cache the received second data stream into the cache space after the second data stream is received;

[0089] Detect whether the cache size of the cache space has reached the preset cache limit, and detect whether the first data stream has been connected;

[0090] If it is detected that the first data stream is not connected after the cache size in the cache space reaches the preset cache limit, then the second data stream received before the first data stream is connected and after the cache size in the cache space reaches the preset cache limit will be stored in the memory space.

[0091] Furthermore, the storage module is also used for:

[0092] Obtain the access time difference between the first data stream and the second data stream;

[0093] If the access time difference is greater than a preset threshold, then a preset size of memory space is requested, wherein the memory space is matched with the access time difference;

[0094] After the second data stream is received, the received second data stream is cached in the cache space. After the cache size of the cache space reaches the preset cache limit, the second data stream received before the first data stream is received is stored in the applied memory space.

[0095] Furthermore, the first data stream is the data stream with the latest access time among the multiple data streams, and the second data stream is any data stream other than the first data stream among the multiple data streams. Corresponding cache spaces are pre-set for each data stream other than the first data stream among the multiple data streams. The storage module is also used for:

[0096] If it is determined that the cache space corresponding to the second data stream is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the corresponding cache space will be stored in memory space.

[0097] Furthermore, the data processing device is an image processing circuit, and the receiving module is also used for:

[0098] The system receives a first data stream and a second data stream from an image sensor, wherein the exposure duration of the first data stream is greater than the exposure duration of the second data stream.

[0099] Furthermore, embodiments of the present invention also propose a data stream processing device, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention. It should be noted that the data stream processing device in the embodiments of the present invention can be a smartphone, a personal computer, a server, or other devices, and no specific limitations are imposed here.

[0100] like Figure 6As shown, the data stream processing device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0101] Those skilled in the art will understand that Figure 6 The device structure shown does not constitute a limitation on the data stream processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0102] like Figure 6 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data stream processing program. The operating system is a program that manages and controls the device's hardware and software resources, supporting the operation of the data stream processing program and other software or programs. Figure 6 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the data stream processing program stored in the memory 1005 and perform the following operations:

[0103] Receive a first data stream and a second data stream, wherein the access time of the first data stream is later than the access time of the second data stream;

[0104] If it is determined that the cache space is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the cache space will be stored in the memory space.

[0105] After receiving the first data stream, the received first data stream is transmitted to the data processing device, and the second data stream is read from the storage space according to the first-in-first-out method and transmitted to the data processing device, wherein the cache space and the memory space are both storage spaces.

[0106] Furthermore, the processor 1001 can also be used to call the data stream processing program stored in the memory 1005 to perform the following operations:

[0107] After reading the stored second data stream from its storage space and transmitting it to the data processing device, the second data stream received after the first data stream is accessed is stored in a target space, wherein the target space is the storage space that becomes available due to the removal of the stored second data stream.

[0108] Furthermore, the operation of storing the portion of the second data stream received before the first data stream was accessed that cannot be cached in the cache space into memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was accessed includes:

[0109] After receiving the second data stream, the received second data stream is cached in the cache space;

[0110] Detect whether the cache size of the cache space has reached the preset cache limit, and detect whether the first data stream has been connected;

[0111] If it is detected that the first data stream is not connected after the cache size in the cache space reaches the preset cache limit, then the second data stream received before the first data stream is connected and after the cache size in the cache space reaches the preset cache limit will be stored in the memory space.

[0112] Furthermore, the operation of storing the portion of the second data stream received before the first data stream was accessed that cannot be cached in the cache space into memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was accessed includes:

[0113] Obtain the access time difference between the first data stream and the second data stream;

[0114] If the access time difference is greater than a preset threshold, then a preset size of memory space is requested, wherein the memory space is matched with the access time difference;

[0115] After the second data stream is received, the received second data stream is cached in the cache space. After the cache size of the cache space reaches the preset cache limit, the second data stream received before the first data stream is received is stored in the applied memory space.

[0116] Furthermore, the first data stream is the data stream with the latest access time among the multiple data streams, and the second data stream is any data stream other than the first data stream among the multiple data streams. Corresponding buffer spaces are pre-set for each data stream other than the first data stream among the multiple data streams.

[0117] The operation of storing the portion of the second data stream received before the first data stream was accessed that cannot be cached in the cache space into memory space when it is determined that the cache space is insufficient to cache the second data stream received before the first data stream was accessed includes:

[0118] If it is determined that the cache space corresponding to the second data stream is insufficient to cache the second data stream received before the first data stream is accessed, the portion of the second data stream received before the first data stream that cannot be cached in the corresponding cache space will be stored in memory space.

[0119] Furthermore, the data processing device is an image processing circuit, and the operation of receiving the first data stream and the second data stream includes:

[0120] The system receives a first data stream and a second data stream from an image sensor, wherein the exposure duration of the first data stream is greater than the exposure duration of the second data stream.

[0121] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a data stream processing program, which, when executed by a processor, implements the steps of the data stream processing method described below.

[0122] The various embodiments of the data stream processing device and computer-readable storage medium of the present invention can be referred to the various embodiments of the data stream processing method of the present invention, and will not be repeated here.

[0123] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0124] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0126] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of data flow processing, the method comprising: The data stream processing method comprises: receiving a first data stream and a second data stream, wherein the access time of the first data stream is later than that of the second data stream; storing, in the case that it is determined that the cache space is insufficient for caching the second data stream received before the first data stream is accessed, the part of the second data stream received before the first data stream is accessed that cannot be cached in the cache space into a memory space; after the first data stream is accessed, transmitting the received first data stream to a data processing device, and reading out and transmitting the second data stream from the storage space in which it is located to the data processing device in a first-in-first-out manner, wherein the cache space and the memory space both belong to the storage space; wherein the step of storing, in the case that it is determined that the cache space is insufficient for caching the second data stream received before the first data stream is accessed, the part of the second data stream received before the first data stream is accessed that cannot be cached in the cache space into a memory space comprises: after the second data stream is accessed, caching the received second data stream to the cache space; detecting whether the cache amount of the cache space reaches a preset cache upper limit, and detecting whether the first data stream is accessed; if it is detected that the first data stream is not accessed after the cache amount of the cache space reaches the preset cache upper limit, storing the second data stream received before the first data stream is accessed and after the cache amount of the cache space reaches the preset cache upper limit into the memory space.

2. The data flow processing method of claim 1, wherein, The data stream processing method further comprises: after the stored second data stream is read from the storage space in which it is located and transmitted to the data processing device, storing the second data stream received after the first data stream is accessed to a target space, wherein the target space is the storage space that is idled out due to the removal of the stored second data stream.

3. The data flow processing method of claim 1, wherein, The step of storing, in the case that it is determined that the cache space is insufficient for caching the second data stream received before the first data stream is accessed, the part of the second data stream received before the first data stream is accessed that cannot be cached in the cache space into a memory space comprises: obtaining an access time difference between the first data stream and the second data stream; if the access time difference is greater than a preset threshold, applying a memory space of a preset size, wherein the memory space matches the access time difference; after the second data stream is accessed, caching the received second data stream to the cache space, and after the cache amount of the cache space reaches a preset cache upper limit, storing the second data stream received subsequently before the first data stream is accessed into the applied memory space.

4. The data flow processing method of claim 1, wherein, The first data stream is the data stream with the latest access time in a plurality of data streams, the second data stream is any one of the data streams other than the first data stream in the plurality of data streams, and a corresponding cache space is set for each of the data streams other than the first data stream in the plurality of data streams in advance, The step of storing the part of the second data stream received before the first data stream into the memory space in the case that the buffer space is not enough for buffering the second data stream received before the first data stream comprises: The step of storing the part of the second data stream received before the first data stream into the memory space in the case that the buffer space corresponding to the second data stream is not enough for buffering the second data stream received before the first data stream.

5. The data flow processing method of any of claims 1 to 4, wherein, The data processing device is an image processing circuit, and the steps of receiving the first data stream and the second data stream comprise: The first data stream and the second data stream are received from an image sensor, and an exposure time of the first data stream is longer than an exposure time of the second data stream.

6. A data flow processing apparatus, characterized by The data stream processing device comprises: A receiving module configured to receive a first data stream and a second data stream, wherein an access time of the first data stream is later than an access time of the second data stream; A storage module configured to store, in the case that a buffer space is not enough for buffering the second data stream received before the first data stream, a part of the second data stream received before the first data stream into a memory space in the buffer space; An output module configured to, after the first data stream is accessed, transmit the received first data stream to a data processing device, and read out the second data stream from a storage space where the second data stream is stored according to a first-in-first-out manner and transmit the second data stream to the data processing device, wherein the buffer space and the memory space belong to the storage space; The storage module is specifically configured to: After the second data stream is accessed, store the received second data stream into the buffer space; Detect whether a buffer amount of the buffer space reaches a preset buffer upper limit, and detect whether the first data stream is accessed; If it is detected that the first data stream is not accessed after the buffer amount of the buffer space reaches the preset buffer upper limit, store the second data stream received before the first data stream is accessed and after the buffer amount of the buffer space reaches the preset buffer upper limit into the memory space.

7. The data flow processing device of claim 6, wherein, The storage module is further configured to: After the stored second data stream is read out from the storage space and transmitted to the data processing device, store the second data stream received after the first data stream is accessed into a target space, wherein the target space is a storage space that is idle due to the stored second data stream being taken out.

8. A data flow processing device, characterized by The data stream processing device comprises a memory, a processor, and a data stream processing program stored in the memory and executable on the processor, and the data stream processing program, when executed by the processor, implements the steps of the data stream processing method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored thereon a data stream processing program, which, when executed by the processor, implements the steps of the data stream processing method according to any one of claims 1 to 5.

Images