Multi-video stream processing method, device, equipment and computer-readable storage medium

Abstract

A method, device, equipment and computer-readable storage medium for processing multiple video streams. The method includes: in the i-th storage cycle, writing all data corresponding to a frame of the first-class image frame and all data corresponding to N frames of the second-class image frames output by N non-target cameras; in the i-th processing cycle, first reading and processing the data corresponding to the written first-class image frame through the ISP in a first-in-first-out order, and then reading and processing the data corresponding to the written N frames of the second-class image frame after all the data corresponding to the first-class image frame is processed, until all the data corresponding to the N frames of the second-class image frame are processed, wherein the moment when all the data corresponding to the first-class image frame is processed in the i-th processing cycle is after the end moment of the i-th storage cycle, and the difference between the two is a small time interval. Through this embodiment, real-time performance is taken into account while processing multiple video streams.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and computer-readable storage medium for processing multiple video streams. Background Art

[0002] At present, the data collected by vehicle sensors is generally processed, and the vehicle behavior is decided based on the processing results, thereby realizing assisted driving or automatic driving. Among them, an important type of vehicle-mounted sensors is the camera. There are generally two ways to process the data collected by the camera:

[0003] One is full real-time processing, that is, while writing the data output by the camera, the data output by the camera is read and processed by the ISP (Image Signal Processor). Although this ensures real-time performance, the ISP can only process data from one camera, that is, it can only process one video stream.

[0004] The other is completely offline processing, that is, the data output by the camera is first stored in the memory, and then the ISP reads the data corresponding to a complete frame of the image from the memory and processes it. Although this method allows the ISP to process multiple video streams, it cannot guarantee real-time performance. Summary of the invention

[0005] The present application provides a multi-video stream processing method, device, equipment and computer-readable storage medium, which can solve the technical problem in the prior art that it is impossible to process multiple video streams while taking into account real-time performance.

[0006] In a first aspect, an embodiment of the present application provides a method for processing multiple video streams, the method comprising:

[0007] In the i-th storage cycle, a frame of the first type of image output by a target camera All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1;

[0008] In the i-th processing cycle, the written data is read and processed by the image signal processor in a first-in-first-out order. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Small time intervals.

[0009] In combination with the first aspect, in one embodiment, in the i-th processing cycle, the written The time of the corresponding data At the end of the i-th storage cycle Before.

[0010] In combination with the first aspect, in one implementation, after reading and processing The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than or equal to the duration of a single storage cycle.

[0011] In combination with the first aspect, in one implementation, the first type of image frame output by a target camera Writing all the corresponding data includes multiple write actions, each write action is The corresponding part of the data is written as the data to be written, wherein for each writing action, it includes:

[0012] If the capacity of the available storage area in the SRAM memory is greater than or equal to the amount of data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the available storage area, wherein the available storage area includes an area where no data is stored and an area where the stored data has been read;

[0013] If the capacity of the available storage area in the SRAM memory is smaller than the amount of the data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the double data rate synchronous dynamic random access memory DDR.

[0014] In combination with the first aspect, in one implementation, the N frames of the second type of image frames output by the N non-target cameras All corresponding data are written including:

[0015] The N frames of the second type of image frames output by N non-target cameras All corresponding data are written into DDR.

[0016] In a second aspect, an embodiment of the present application provides a multi-video stream processing device, the multi-video stream processing device comprising:

[0017] The writing module is used to write a first-class image frame output by a target camera in the i-th storage cycle. All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1;

[0018] The processing module is used to read and process the written data through the image signal processor in the order of first-in first-out in the i-th processing cycle. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Small time intervals.

[0019] In conjunction with the second aspect, in one implementation, in the i-th processing cycle, the written The time of the corresponding data At the end of the i-th storage cycle Before.

[0020] In conjunction with the second aspect, in one implementation, after reading and processing The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than or equal to the duration of a single storage cycle.

[0021] In a third aspect, an embodiment of the present application provides a multi-video stream processing device, comprising a processor, a memory, and a multi-video stream processing program stored in the memory and executable by the processor, wherein when the multi-video stream processing program is executed by the processor, the steps of the multi-video stream processing method described in the first aspect are implemented.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a multi-video stream processing program is stored, wherein when the multi-video stream processing program is executed by a processor, the steps of the multi-video stream processing method described in the first aspect are implemented.

[0023] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0024] In the embodiment of the present application, in the i-th storage cycle, a frame of the first type of image frame output by a target camera is All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1; in the i-th processing cycle, the written data are first read and processed by the image signal processor in a first-in-first-out order. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Through this embodiment, it is realized that the image signal processor processes multiple video streams in time-sharing manner, and for the first type of image frames, the processing thereof is completed almost at the same time as the storage thereof is completed, which is equivalent to real-time processing of one video stream output by the target camera, that is, real-time performance is taken into account while processing multiple video streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of a scenario of the first embodiment of the multi-video stream processing method of the present application;

[0026] Figure 2 This is a schematic diagram of a scenario of the second embodiment of the multi-video stream processing method of the present application;

[0027] Figure 3 This is a schematic diagram of a scenario of the third embodiment of the multi-video stream processing method of the present application;

[0028] Figure 4 This is a schematic diagram of a scenario of the fourth embodiment of the multi-video stream processing method of the present application;

[0029] Figure 5 This is a schematic diagram of a writing and reading scenario in an embodiment of a multi-video stream processing method of the present application;

[0030] Figure 6 This is a functional module diagram of an embodiment of a multi-video stream processing device of the present application;

[0031] Figure 7 This is a schematic diagram of the hardware structure of a multi-video stream processing device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0034] In a first aspect, an embodiment of the present application provides a method for processing multiple video streams.

[0035] In one embodiment, the multi-video stream processing method includes:

[0036] In the i-th storage cycle, a frame of the first type of image output by a target camera All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1;

[0037] In the i-th processing cycle, the written data is read and processed by the image signal processor in a first-in-first-out order. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Small time intervals.

[0038] In this embodiment, N=4 is taken as an example, that is, in each storage cycle, a frame of the first type of image frame output by a target camera is All corresponding data are written and the 4 second-class image frames output by the 4 non-target cameras are written. All corresponding data are written. Taking the assisted / autonomous driving scenario as an example, the target camera is the vehicle's main front-view camera, and the four non-target cameras are the left front-view camera, the right front-view camera, the left rear-view camera, and the right rear-view camera. Figure 1 , Figure 1 This is a schematic diagram of a scenario of the first embodiment of the multi-video stream processing method of the present application. Figure 1 As shown, in the first storage cycle, Write all the corresponding data and 4 frames All corresponding data are written; in the second storage cycle, Write all the corresponding data and 4 frames All the corresponding data are written, and so on.

[0039] Continue to refer to Figure 1 In the first processing cycle, the written data is read and processed by the image signal processor in a first-in-first-out order. The corresponding data in Always After all the corresponding data processing is completed, the image signal processor reads and processes the 4 frames that have been written. The corresponding data until Time to 4 frames All corresponding data processing is completed, among which, This is the end time of the first processing cycle; similarly, in the second processing cycle, the written data is read and processed by the image signal processor in the order of first in first out. The corresponding data in Always After all the corresponding data processing is completed, the 4 frames written are read and processed by the image signal processor in a first-in-first-out order. The corresponding data until Time to 4 frames All corresponding data processing is completed, among which, This is the end time of the second processing cycle, and so on.

[0040] in, and The difference is a small time interval, which is determined according to the actual situation, such as 50 microseconds, 100 microseconds, etc. This value is related to the rate at which the ISP reads and processes data. The higher the rate at which the ISP reads and processes data, the smaller the value.

[0041] Further, in one embodiment, the i-th processing cycle starts reading and processing the written The time of the corresponding data At the end of the i-th storage cycle Before.

[0042] In this embodiment, refer to Figure 2 , Figure 2 This is a schematic diagram of a second embodiment of the multi-video stream processing method of the present application. Figure 2As shown, the first processing cycle starts reading and processing the written The time of the corresponding data It can be the start time of the first storage cycle The same, that is, at the beginning While the corresponding data is being written, the written data is read and processed through ISP. Corresponding data; in the first processing cycle, read and process the written The time of the corresponding data It can also be at the beginning of the first storage cycle After that, After a part of the corresponding data is written, the written data is read and processed through ISP. The corresponding data.

[0043] In the subsequent processing cycle after the first processing cycle, taking the second processing cycle as an example, the written The time of the corresponding data At the end of the first processing cycle After or with The subsequent processing cycle is similar to the second processing cycle and will not be described in detail here. At the end of the i-th processing cycle After or with same.

[0044] Further, in one embodiment, after reading and processing The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than or equal to the duration of a single storage cycle.

[0045] In this embodiment, it is assumed that the duration of a single storage cycle is 30 ms. The time required for all corresponding data is 20ms, and N frames are read and processed. The time required for all the corresponding data is 10ms, that is, reading and processing The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is equal to the duration of a single storage cycle. Figure 3 , Figure 3 This is a schematic diagram of a scenario of the third embodiment of the multi-video stream processing method of the present application. Figure 3 As shown, in each processing cycle, the ISP is always reading and processing data, wherein it starts to read and process the N frames that have been written. The time of the corresponding data With The time when all corresponding data processing is completed Same; start reading and processing the written The time of the corresponding data With N frames The time when all corresponding data processing is completed same.

[0046] In another embodiment, assuming that the duration of a single storage cycle is 30 ms, the The time required for all corresponding data is 10ms, and N frames are read and processed. The time required for all the corresponding data is 10ms, that is, reading and processing The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than the duration of a single storage cycle. Figure 4 , Figure 4 This is a schematic diagram of a scenario of the fourth embodiment of the multi-video stream processing method of the present application. Figure 4 As shown, in the i-th processing cycle The time when all corresponding data processing is completed The i+1th processing cycle starts reading and processing the written The time of the corresponding data The time difference between them is 20ms, so you can choose any 10ms for N frames during this time. All corresponding data are processed, for example, Figure 4 middle, to is 10ms, then to During, and to During this period, the ISP can be in an idle state. That is, in this case, the ISP does not need to read and process data all the time in each processing cycle.

[0047] In another embodiment, when i is greater than 1, the sum of the time required to read and process all corresponding data and the time required to read and process all data corresponding to N frames is less than or equal to the duration of a single storage cycle.

[0048] In this embodiment, the first processing cycle starts reading and processing the written The time of the corresponding data The start time of the first storage cycle The same, that is, at the beginning While the corresponding data is being written, the written data is read and processed through ISP. For the corresponding data, please refer to Figure 3 or Figure 4 And the corresponding embodiments are not described in detail here.

[0049] In the embodiment of the present application, in the i-th storage cycle, a frame of the first type of image frame output by a target camera is All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1; in the i-th processing cycle, the written data are first read and processed by the image signal processor in a first-in-first-out order. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Through this embodiment, it is realized that the image signal processor processes multiple video streams in time-sharing manner, and for the first type of image frames, the processing thereof is completed almost at the same time as the storage thereof is completed, which is equivalent to real-time processing of one video stream output by the target camera, that is, real-time performance is taken into account while processing multiple video streams.

[0050] Furthermore, in one embodiment, the first type of image frame output by one target camera Writing all the corresponding data includes multiple write actions, each write action is The corresponding part of the data is written as the data to be written, wherein for each writing action, it includes:

[0051] If the capacity of the available storage area in the SRAM memory is greater than or equal to the amount of data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the available storage area, wherein the available storage area includes an area where no data is stored and an area where the stored data has been read;

[0052] If the capacity of the available storage area in the SRAM memory is smaller than the amount of the data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the double data rate synchronous dynamic random access memory DDR.

[0053] In this embodiment, the camera interface receives the All the corresponding data are written. The complete writing process includes multiple writing actions, that is, each writing action is The corresponding part of the data is written as the data to be written. For example, The corresponding total data is approximately 8M, and the amount of data to be written corresponding to each write action is 1M.

[0054] Reference Figure 5 , Figure 5 This is a schematic diagram of a write-read scenario in an embodiment of a multi-video stream processing method of this application. Figure 5 As shown, for each write action, if the capacity of the available storage area in the SRAM memory is greater than or equal to the data amount of the data to be written corresponding to this write action, the data to be written corresponding to this write action is written into the available storage area, and in the processing cycle, the ISP reads the written data from the SRAM memory and processes it; if the capacity of the available storage area in the SRAM memory is less than the data amount of the data to be written corresponding to this write action, the data to be written corresponding to this write action is written into the double data rate synchronous dynamic random access memory DDR, and in the processing cycle, the ISP reads the written data from the DDR and processes it.

[0055] It should be noted that when the write speed is faster than the ISP read speed, the written data can be cached until the ISP processes it; when the ISP read speed is faster than the write speed, resulting in no image data in the SRAM that needs to be processed, the ISP read can be blocked, so that the ISP read is faster than the write and the read data is not incorrect.

[0056] Furthermore, in one embodiment, the N frames of the second type of image frames output by the N non-target cameras All corresponding data are written including:

[0057] The N frames of the second type of image frames output by N non-target cameras All corresponding data are written into DDR.

[0058] In this embodiment, N frames of the second type of image frames output by N non-target cameras All corresponding data are directly written into DDR. During the processing cycle, ISP reads the written data from DDR and processes it.

[0059] In a second aspect, an embodiment of the present application also provides a multi-video stream processing device.

[0060] In one embodiment, referring to Figure 6 , Figure 6 This is a functional module diagram of an embodiment of a multi-video stream processing device of the present application. Figure 6 As shown, the multi-video stream processing device includes:

[0061] The writing module 10 is used to write a first-class image frame output by a target camera in the i-th storage cycle. All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1;

[0062] The processing module 20 is used to read and process the written data in the image signal processor in the order of first-in first-out in the i-th processing cycle. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Small time intervals.

[0063] Further, in one embodiment, the i-th processing cycle starts reading and processing the written The time of the corresponding data At the end of the i-th storage cycle Before.

[0064] Further, in one embodiment, after reading and processing The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than or equal to the duration of a single storage cycle.

[0065] Furthermore, in one embodiment, the first type of image frame output by one target camera Writing all the corresponding data includes multiple write actions, each write action is The corresponding part of the data is written as the data to be written, wherein for each writing action, the writing module 10 is used to:

[0066] If the capacity of the available storage area in the SRAM memory is greater than or equal to the amount of data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the available storage area, wherein the available storage area includes an area where no data is stored and an area where the stored data has been read;

[0067] If the capacity of the available storage area in the SRAM memory is smaller than the amount of the data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the double data rate synchronous dynamic random access memory DDR.

[0068] Furthermore, in one embodiment, the writing module 10 is used to:

[0069] The N frames of the second type of image frames output by N non-target cameras All corresponding data are written into DDR.

[0070] Among them, the functional implementation of each module in the above-mentioned multi-video stream processing device corresponds to each step in the above-mentioned multi-video stream processing method embodiment, and its functions and implementation processes are no longer repeated here.

[0071] In a third aspect, an embodiment of the present application provides a multi-video stream processing device, which may be a device with data processing functions, such as a personal computer (PC), a laptop computer, or a server.

[0072] Reference Figure 7 , Figure 7 Schematic diagram of the hardware structure of a multi-video stream processing device involved in the embodiment of the present application. In the embodiment of the present application, the multi-video stream processing device may include a processor, a memory, a communication interface and a communication bus.

[0073] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0074] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to realize the interconnection of devices inside the multi-video stream processing device, and the interface used to realize the interconnection of the multi-video stream processing device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display (Display), a keyboard (Keyboard), etc.

[0075] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0076] The processor may be a general-purpose processor, and the general-purpose processor may call the multi-video stream processing program stored in the memory and execute the multi-video stream processing method provided by the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the multi-video stream processing program is called may refer to the various embodiments of the multi-video stream processing method of the present application, and will not be repeated here.

[0077] Those skilled in the art will understand that Figure 7 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0078] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0079] The computer-readable storage medium of the present application stores a multi-video stream processing program, wherein when the multi-video stream processing program is executed by a processor, the steps of the multi-video stream processing method as described above are implemented.

[0080] Among them, the method implemented when the multi-video stream processing program is executed can refer to the various embodiments of the multi-video stream processing method of the present application, and will not be repeated here.

[0081] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0082] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0083] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0084] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0085] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0087] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for processing multiple video streams, characterized in that: The multi-video stream processing method comprises: In the i-th storage cycle, a frame of the first type of image output by a target camera All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1; In the i-th processing cycle, the written data is read and processed by the image signal processor in a first-in-first-out order. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Differing by a small time interval; The first type of image frame output by a target camera Writing all the corresponding data includes multiple write actions, each write action is The corresponding part of the data is written as the data to be written, wherein for each writing action, it includes: If the capacity of the available storage area in the SRAM memory is greater than or equal to the amount of data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the available storage area, wherein the available storage area includes an area where no data is stored and an area where the stored data has been read; If the capacity of the available storage area in the SRAM memory is smaller than the amount of the data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the double data rate synchronous dynamic random access memory DDR.

2. The multi-video stream processing method according to claim 1, characterized in that: In the i-th processing cycle, it starts reading and processing the written The time of the corresponding data At the end of the i-th storage cycle Before.

3. The multi-video stream processing method according to claim 1, characterized in that: Read and process The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than or equal to the duration of a single storage cycle.

4. The multi-video stream processing method according to claim 1, characterized in that: The N frames of the second type of image frames output by the N non-target cameras All corresponding data are written including: The N frames of the second type of image frames output by N non-target cameras All corresponding data are written into DDR.

5. A multi-video stream processing device, characterized in that: The multi-video stream processing device comprises: The writing module is used to write a first-class image frame output by a target camera in the i-th storage cycle. All corresponding data are written and N frames of the second type of image frames output by N non-target cameras All corresponding data are written, where N is a positive integer greater than or equal to 1; The processing module is used to read and process the written data through the image signal processor in the order of first-in first-out in the i-th processing cycle. The corresponding data, in After all corresponding data processing is completed, read and process the N frames that have been written Corresponding data, until N frames All corresponding data processing is completed, among which, in the i-th processing cycle, The time when all corresponding data processing is completed At the end of the i-th storage cycle After that, and and Differing by a small time interval; The first type of image frame output by a target camera Writing all the corresponding data includes multiple write actions, each write action is The corresponding part of the data is written as the data to be written, wherein for each writing action, the writing module is used to: If the capacity of the available storage area in the SRAM memory is greater than or equal to the amount of data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the available storage area, wherein the available storage area includes an area where no data is stored and an area where the stored data has been read; If the capacity of the available storage area in the SRAM memory is smaller than the amount of the data to be written corresponding to the current write action, the data to be written corresponding to the current write action is written into the double data rate synchronous dynamic random access memory DDR.

6. The multi-video stream processing device according to claim 5, characterized in that: In the i-th processing cycle, it starts reading and processing the written The time of the corresponding data At the end of the i-th storage cycle Before.

7. The multi-video stream processing device according to claim 5, characterized in that: Read and process The time required to read and process all the corresponding data is the same as the time required to read and process N frames. The sum of the durations required for all corresponding data is less than or equal to the duration of a single storage cycle.

8. A multi-video stream processing device, characterized in that: The multi-video stream processing device includes a processor, a memory, and a multi-video stream processing program stored in the memory and executable by the processor, wherein when the multi-video stream processing program is executed by the processor, the steps of the multi-video stream processing method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a multi-video stream processing program, wherein when the multi-video stream processing program is executed by a processor, the steps of the multi-video stream processing method according to any one of claims 1 to 4 are implemented.

Images