Video recorder and data processing method

By adopting copy mode and copy-saving mode in the video recorder, the task execution mode is selected according to the amount of image data, realizing the instant release of the public cache pool and the rapid processing of image data, solving the problem of low processing efficiency of the video recorder and improving the processing task efficiency and storage space utilization.

CN118674611BActive Publication Date: 2024-12-03HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411133601.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

When existing video recorders process continuous multi-frame image data, they are inefficient and cannot release the cache area in the public cache pool in time, resulting in an increase in the serial processing time of the video link.

Method used

Using copy mode and copy-saving mode, the master control unit determines the task execution mode based on the reference information of the target task, and the image processing unit pre-copys the image data to the target memory space or apply for cache area multiple times to realize the instant release of the cache area and improve the Processing efficiency.

Benefits of technology

The video recorder realizes rapid processing of continuous multi-frame image data and instant release of cache areas, reduces the serial processing time of the video link, improves the efficiency of processing tasks, and rationally utilizes storage space to avoid excessive Memory consumption.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118674611B_ABST
    Figure CN118674611B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a video recorder and a data processing method, which relate to the field of data processing technology; wherein, a video recorder includes: a main control unit is used to respond to a task execution instruction of a target task, determine the task execution mode of an image processing unit based on reference information of the target task, and sequentially obtain each frame of image data of the target task, and write the frame of image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode; the image processing unit is used to respond to the main control unit determining the task execution mode, and sequentially perform image processing on each frame of image data according to an execution strategy corresponding to the task execution mode to obtain a processing result for each frame of image data; the task execution unit is used to sequentially obtain the processing result of each frame of image data, and execute a corresponding task based on the obtained processing result; it can be seen that this scheme improves the efficiency of the video recorder in processing tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a video recorder and a data processing method. Background Art

[0002] The video recorder can perform a variety of tasks, such as video preview tasks, video encoding and storage tasks, etc. Taking the preview task as an example, the main control unit of the video recorder can write the image data into the cache area applied for from the public cache pool, the image processing unit can read the image data from the cache area, and perform image processing on the read image data, and then the video output unit outputs the processed image data to the preview device to achieve preview.

[0003] However, the cache area is usually used to store one frame of image data, that is, the cache area is a cache area for single-frame image data, and the single-frame image data can be written into the cache area by the main control unit. Exemplarily, the single-frame image data can be YUV data given by the user, that is, the image data that needs to be task processed set by the user through the operation interface.

[0004] Furthermore, the image processing unit reads a frame of image data from the cache area, processes the image, and caches the processed frame of image data in the output cache pool before releasing the cache area. The main control unit then writes the next frame of image data into the cache area, so that the image processing unit continues to read and process the next frame of image data. In this way, when processing multiple consecutive frames of image data, the efficiency of the video recorder processing task is low.

[0005] It can be seen that in the related technology, how to improve the efficiency of video recorder processing tasks is a problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the embodiment of the present application is to provide a video recorder and a data processing method to improve the efficiency of the video recorder processing task. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present application provides a video recorder, comprising: a main control unit, an image processing unit, and a task execution unit;

[0008] The main control unit is used to determine the task execution mode of the image processing unit based on the reference information of the target task in response to the task execution instruction of the target task; and sequentially obtain each frame of image data of the target task, and in response to each acquisition of a frame of image data, write the frame of image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode; the reference information is information used to characterize the data volume of the image data of the target task;

[0009] The image processing unit is used for, in response to the task execution mode determined by the main control unit, performing image processing on each frame of image data written by the main control unit to the public buffer pool in turn according to the execution strategy corresponding to the task execution mode, to obtain a processing result of each frame of image data;

[0010] The task execution unit is used to sequentially obtain the processing result of each frame of image data, and execute the corresponding task based on the obtained processing result;

[0011] Among them, the task execution mode includes: a copy mode or a copy-saving mode; the copy mode is: the image processing unit copies the image data that has been written into the public cache pool to the target memory space in advance, so that the image processing unit can immediately release the cache area occupied by the copied image data; or, the copy-saving mode is: the main control unit applies for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool.

[0012] In a second aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0013] In response to a task execution instruction of a target task, a task execution mode is determined based on reference information of the target task; and, each frame of image data of the target task is sequentially acquired, and in response to each acquisition of a frame of image data, the frame of image data is written into a cache area in a public cache pool according to a write strategy matching the task execution mode; wherein the reference information is information used to characterize the data volume of the image data of the target task;

[0014] According to the execution strategy corresponding to the task execution mode, each frame of image data written into the public buffer pool is processed in turn to obtain a processing result of each frame of image data;

[0015] Obtain the processing results of each frame of image data in turn, and execute corresponding tasks based on the obtained processing results;

[0016] Among them, the task execution mode includes: copy mode or copy-saving mode; the copy mode is: pre-copying the image data written into the public cache pool to the target memory space so that the cache area occupied by the copied image data can be released immediately; or, the copy-saving mode is: applying for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the immediate release of the cache area in the public cache pool.

[0017] Beneficial effects of the embodiments of the present application:

[0018] It can be seen that the image processing unit in the embodiment of the present application does not need to wait for the processing result of each frame of image data to be cached in the output buffer pool before releasing the cache area in the public buffer pool. Specifically, in the copy mode, the image processing unit can pre-copy the image data written into the public buffer pool to the target memory space, so that the cache area occupied by the copied image data can be released in time; while in the copy-saving mode, the main control unit can apply for the cache area in the public buffer pool multiple times to cache the image data, which is equivalent to the image processing unit immediately releasing the cache area in the public buffer pool. In this way, in the face of continuous multiple frames of image data, there is no need to wait for the image processing unit to process the result of any frame of image data, the main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can continuously process each frame of image data, thereby improving the efficiency of the video recorder processing task. In addition, the image processing unit can immediately release the cache area in the public buffer pool, which can reduce the serial processing time of the video link.

[0019] In addition, when the task execution mode is the copy mode, the image processing unit can pre-copy the image data written into the public cache pool to the target memory space, so that the image processing unit can immediately release the cache area occupied by the copied image data, thereby improving the efficiency of the video recorder processing task while reducing the occupancy time of the cache area during the task execution process. In addition, this solution selects different task execution modes through the reference information of the target task. Since the reference information is the data volume of the target task, it can provide a basis for the reasonable and effective use of storage space during the task execution process and avoid excessive memory consumption.

[0020] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0022] FIG1( a ) is a schematic diagram showing the principle of a task execution scheme in the related art;

[0023] FIG1( b ) is a schematic diagram showing the principle of a task execution scheme for a cache area in the related art;

[0024] FIG1( c ) is a schematic diagram showing the principle of the practical application of the task execution scheme in the related art;

[0025] Figure 2 A schematic diagram of the structure of a video recorder provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of the principle of a task execution scheme for the copy mode provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the principle of a task execution scheme for a copy-saving mode provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the principle of the task execution scheme provided in the embodiment of the present application;

[0029] Figure 6 A schematic diagram of the principle of practical application of the task execution solution provided in the embodiment of the present application;

[0030] Figure 7 A flowchart of a data processing method provided in an embodiment of the present application;

[0031] Figure 8 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0033] DVR: Digital Video Recorder. NVR: Network Video Recorder.

[0034] Video output unit: can be called VO (Video Output) or VO unit or video output module, which is used to actively read video / image data and output it to the preview device.

[0035] Video decoding unit: may be called VDEC (Video Decoder) or VDEC unit or video decoding module, used to decode the compressed video stream into raw data and store it into a specified memory, wherein the format of the video stream may be H.264 / H.265 / MJPEG.

[0036] Video capture unit: can be called VCAP (Video Capture) or VCAP unit or video capture module, which is used to collect the image data of the video channel into the specified memory through the corresponding interface, where the specific interface can transmit the data stream in BT.656 / BT.1120 / MIPI format.

[0037] Image processing unit: can be called VP (Video Process) or VP unit or image processing module, which is used to perform scaling, cropping, denoising, deinterlacing and other processing on image data.

[0038] Binding: The data receiver and the data source can establish a binding relationship. After binding, the data source can automatically send data to the data receiver.

[0039] PCIE: Peripheral component interconnect express, a high-speed serial computer expansion bus interface standard. The PCIE bus is a bus based on this standard.

[0040] DDR: Double Data Rate, double data rate, usually refers to double data rate synchronous dynamic random access memory, which can be considered as a type of memory.

[0041] DMA: Direct Memory Access, direct memory access technology, a technology for efficiently realizing data transmission in computer systems.

[0042] Buffer pool: It is composed of a group of memory blocks of equal memory size, which can provide media service memory management function; and each unit has a corresponding buffer pool, for example: the video decoding unit has a decoding input buffer pool and a decoding output buffer pool, the video output unit has a video output buffer pool, etc. This application does not make specific limitations on this.

[0043] Public buffer pool: a type of buffer pool that can be applied for by all business units and users, while other buffer pools have restrictive use. For example, the decoding output buffer pool can only cache image data decoded by the video decoding unit.

[0044] BT.656: A video acquisition and processing standard that defines a digital video signal transmission method based on a parallel interface. This standard is applicable to video signals with standard resolution. ITU-R BT.656 defines a parallel hardware interface for transmitting a 4:2:2 YCbCr digital video stream. BT refers to Broadcasting Service television. The protocol specifies the digital interfaces for high-definition television studio signals (Digital interfaces for studio signals with 1920 × 1080 image formats), mainly from the supported video formats, digital signal encoding parameters, image timing, etc. In addition, BT656 defines a parallel hardware interface for transmitting a 4:2:2 YCbCr digital video stream. YCbCr is a color space, usually used for continuous image processing in movies or in digital photography systems. Y is luminance, while Cb and Cr are the concentration offset components of blue and red.

[0045] BT.1120, the full name of BT1120 protocol is ITU-R BT.1120. ITU is the abbreviation of International Telecommunication Union. R is the abbreviation of Radiocommunications sector.

[0046] MIPI: Mobile Industry Processor Interface, the mobile industry processor interface is an open standard and a specification for mobile application processors initiated by the MIPI Alliance. In addition, BT1120 can be a digital interface for HDTV (High Definition Television) studio signals, which can encode 4:4:4 and 4:4:2 video data into a video data stream with an embedded synchronization timing reference code.

[0047] H.264: It is a new generation of digital video compression format proposed jointly by the International Organization for Standardization (ISO) and the International Telecommunication Union (ITU) after MPEG4.

[0048] H.265: A video coding standard.

[0049] MJPEG: Motion Joint Photographic Experts Group, a motion image frame-by-frame compression technology, a video compression format in which each frame is encoded using JPEG without using inter-frame coding. The compression ratio is usually in the range of 20:1-50:1.

[0050] YUV: "Y" represents brightness, "U" and "V" represent chromaticity, "U" is used to describe the color and saturation of the image, and "V" is used to specify the color of the pixel. YUV type image data has multiple formats. YUV data can also be called YUV image, for example: YUV420 format data, YUV422 format data, etc.

[0051] Also, in order to better understand the present application, before introducing the solution provided by the embodiment of the present application, the video output solution and the solution for switching the screen preview in the related art are introduced, as shown in Figure 1 (a), Figure 1 (b) and Figure 1 (c):

[0052] Figure 1 (a) is a schematic diagram of the principle of the task execution scheme in the related art. The image processing unit can be bound to the subsequent video output unit or the video encoding unit. The video channel of the video output unit is Chn0, and the video channel of the video encoding unit is Chn1. The YUV data can be sent by the main control unit. When the user wants to perform a task on the YUV data, the YUV data can be sent to the image processing unit through the main control unit, thereby finally realizing tasks such as preview / encoding. It should be emphasized that the YUV data is not directly bound to the image processing unit, and the entire video output link can be called a semi-bound link.

[0053] FIG1 (b) is a schematic diagram of the principle of using the cache area of ​​the task execution method in the related art. The public cache pool is a cache pool in the video recorder for caching image data. Ci is a cache area applied for by the main control unit from the public cache pool. The public cache pool management can be considered as the management of the public cache pool by the main control unit, which can specifically include applying for Ci from the public cache pool. The user can cache image data in Ci through the main control unit. The output cache pool of the image processing unit is a cache pool in the video recorder for caching processed image data, which can be used by the image processing unit. Bi is a cache area applied for by the image processing unit from the output cache pool. The output cache pool management can be considered as the management of the output cache pool by the image processing unit, which can specifically include applying for Bi from the public cache pool and releasing the image data in Bi. Then, the main control unit can obtain the image data and cache the image data in Ci, the image processing unit can read the image data in Ci, and the image data is sent to the image processing unit. At this time, Ci will be locked, and the image data in Ci cannot be released. The image processing unit processes and caches the processed image data in Bi, and then the image processing unit can release the image data in Ci; at this time, the image data in Bi is locked, and after the video output unit / video encoding unit completes the task, the image data in Bi can be released through the output buffer pool management. And, the above-mentioned release of the image data in Ci and the release of the image data in Bi is to release Ci and Bi so that Ci and Bi are free areas.

[0054] FIG1 (c) is a practical application of the above-mentioned task execution method in the related art in a video recorder. There may be two integrated circuits in the video recorder, namely, Chip0 and Chip1. Chip0 may be the first integrated circuit for acquiring image data. Video acquisition unit 1, video acquisition unit 2, video decoding unit 1, video decoding unit 2 and image processing unit are all units arranged in Chip0. The image processing unit can process image data of multiple video channels. Chip1 may be the second integrated circuit for processing acquired image data. The image processing unit and the video output unit (not shown in the figure) are units arranged in Chip1. The control unit of Chip1 can receive the image data transmitted by Chip0, and the received image data can be subsequently written into the cache area through the main control unit of Chip1.

[0055] The image processing unit can process the image data of the analog video channel and the digital video channel, that is, it can process the YUV data of A1, the YUV data of A2, the YUV data of D1 and the YUV data of D2, and output the collected image data on the video output unit through the video link of YUV data → image processing unit → video output unit, that is, the video output unit outputs 4 views. A1 and A2 can represent analog channels, and D1 and D2 can represent digital channels.

[0056] Taking the video acquisition unit 1 as an example, the workflow of Chip0 and Chip1 is introduced. The video acquisition unit 1 can acquire the image data of one video channel and send it to the image processing unit of Chip0. The image processing unit can process the image data sent by the video acquisition unit 1 and cache it in RAW BUF (RAW Buffer), that is, the cache area of ​​Chip0, channel PCIE bus, the main control unit of Chip0 can transmit the image data after image processing to the RAW BUF of Chip1, the main control unit of Chip1 can write the received image data to the public buffer pool, the image processing unit of Chip1 can read the YUV data of A1 from the public buffer pool, perform image processing, and then send it to the video output unit to realize the output of YUV data for A1. Obviously, the image data received by the main control unit of Chip1, that is, the image data given by Chip0, is similar to the YUV data directly given by the user, so it is also applicable to this solution.

[0057] It can be seen that the related technology cannot release the cache area in the public cache pool in time, resulting in low efficiency of the recorder processing task; when the load of the image processing unit is large, output jamming may occur for recorders with average hardware performance.

[0058] Based on the problems existing in the related art, the embodiment of the present application provides a video recorder to improve the efficiency of the video recorder processing task. In addition, the "video recorder" mentioned in the present application is not limited to the equipment with the product name "video recorder" in a narrow sense, but refers to the equipment that can receive and process video in a broad sense. As long as it is a video device that can process video data, it is included in the scope of the "video recorder" of the present application.

[0059] The video recorder provided in the embodiment of the present application is introduced below.

[0060] The video recorder provided in the embodiment of the present application can be any electronic device capable of preview display, such as a DVR (Digital Video Recorder), an NVR (Network Video Recorder), etc. DVR is a computer system for image storage and processing, and has the function of long-term recording and control of images. NVR is mostly used in back-end products in the security industry. It can access multiple analog cameras and decode, display, record, store, and transmit the video stream signals transmitted by each analog camera.

[0061] The following is a brief introduction to the application scenarios of the video recorder in the embodiment of the present application:

[0062] The solution provided by the present application can be applied to scenarios where the user sets image data to perform tasks, for example: the user sets multiple frames of image data through the operation interface to perform image preview or image encoding tasks; of course, for a video recorder containing multiple integrated circuits, the situation in which one integrated circuit sends image data to another integrated circuit to perform a task can also be applied to the present solution. In this case, the control unit of the other integrated circuit is the main control unit in the solution of the present application.

[0063] The video recorder provided in the embodiment of the present application includes: a main control unit, an image processing unit and a task execution unit;

[0064] The main control unit is used to determine the task execution mode of the image processing unit based on the reference information of the target task in response to the task execution instruction of the target task; and sequentially obtain each frame of image data of the target task, and in response to each acquisition of a frame of image data, write the frame of image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode; the reference information is information used to characterize the data volume of the image data of the target task;

[0065] The image processing unit is used for, in response to the task execution mode determined by the main control unit, performing image processing on each frame of image data written by the main control unit to the public buffer pool in turn according to the execution strategy corresponding to the task execution mode, to obtain a processing result of each frame of image data;

[0066] The task execution unit is used to sequentially obtain the processing result of each frame of image data, and execute the corresponding task based on the obtained processing result;

[0067] Among them, the task execution mode includes: a copy mode or a copy-saving mode; the copy mode is: the image processing unit copies the image data that has been written into the public cache pool to the target memory space in advance, so that the image processing unit can immediately release the cache area occupied by the copied image data; or, the copy-saving mode is: the main control unit applies for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool.

[0068] It can be seen that the image processing unit in the embodiment of the present application does not need to wait for the processing result of each frame of image data to be cached in the output buffer pool before releasing the cache area in the public buffer pool. Specifically, in the copy mode, the image processing unit can pre-copy the image data written into the public buffer pool to the target memory space, so that the cache area occupied by the copied image data can be released in time; while in the copy-saving mode, the main control unit can apply for the cache area in the public buffer pool multiple times to cache the image data, which is equivalent to the image processing unit immediately releasing the cache area in the public buffer pool. In this way, in the face of continuous multiple frames of image data, there is no need to wait for the image processing unit to process the result of any frame of image data, the main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can continuously process each frame of image data, thereby improving the efficiency of the video recorder processing task. In addition, the image processing unit can immediately release the cache area in the public buffer pool, which can reduce the serial processing time of the video link.

[0069] In addition, when the task execution mode is the copy mode, the image processing unit can pre-copy the image data written into the public cache pool to the target memory space, so that the image processing unit can immediately release the cache area occupied by the copied image data, thereby improving the efficiency of the video recorder processing task while reducing the occupancy time of the cache area during the task execution process. In addition, this solution selects different task execution modes through the reference information of the target task. Since the reference information is the data volume of the target task, it can provide a basis for the reasonable and effective use of storage space during the task execution process and avoid excessive memory consumption.

[0070] It should be noted that the main control unit, image processing unit or task execution unit included in the video recorder described in the embodiment of the present application does not limit the video recorder to include these three hardware parts or these three hardware parts in physical form. The term "unit" and "module" in the embodiment of the present application can be a hardware part, a software program, or a software program and the necessary carrier for the operation of the software program. With the evolution of technology, the steps required to execute these "units" or "modules" or the functions undertaken (for example, a certain unit or a certain module is "used for" the steps described later) may be performed using hardware, software, or a combination of hardware and software. As long as it is hardware, software, or a combination of hardware and software that can execute the steps required to be executed by the "unit", it can be considered as a "unit" or "module" in the embodiment of the present application.

[0071] The video recorder provided in the embodiment of the present application will be described in detail below in conjunction with the accompanying drawings of the specification.

[0072] like Figure 2 As shown, the video recorder provided in the embodiment of the present application includes: a main control unit 210, an image processing unit 220 and a task execution unit 230;

[0073] The main control unit 210 is used to determine the task execution mode of the image processing unit 220 based on the reference information of the target task in response to the task execution instruction of the target task; and sequentially obtain each frame of image data of the target task, and in response to each acquisition of a frame of image data, write the frame of image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit 210 that matches the task execution mode; the reference information is information used to characterize the data volume of the image data of the target task;

[0074] The image processing unit 220 is used for, in response to the task execution mode determined by the main control unit 210, performing image processing on each frame of image data written into the public buffer pool by the main control unit 210 in turn according to the execution strategy corresponding to the task execution mode, to obtain a processing result of each frame of image data;

[0075] The task execution unit 230 is used to sequentially obtain the processing result of each frame of image data, and execute the corresponding task based on the obtained processing result;

[0076] Among them, the task execution mode includes: a copy mode or a copy-saving mode; the copy mode is: the image processing unit 220 pre-copies the image data written into the public cache pool to the target memory space, so that the image processing unit 220 can immediately release the cache area occupied by the copied image data; or, the copy-saving mode is: the main control unit 210 applies for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the image processing unit 220 immediately releasing the cache area in the public cache pool.

[0077] Among them, the target task can be an image preview task or an image encoding task, etc., and the present application does not make specific limitations on this; it can be understood that the main control unit 210 can respond to the task execution instruction of the target task, and determine the task execution mode of the image processing unit 220 based on the reference information of the target task, and each target task corresponds to reference information; exemplarily, the reference information of the image preview task can be the information of the data volume of the image data of the image preview task, and the reference information of the image encoding task can be the information of the data volume of the image data of the image encoding task; and, for the sake of clear layout, the task execution mode of the image processing unit 220 is determined based on the reference information of the target task, which will be introduced in subsequent embodiments, so it will not be repeated here.

[0078] It is understandable that the image data of one frame acquired by the main control unit 210 may be given by the user; when the main control unit 210 is a unit set in an integrated circuit, the acquired image data may be sent by the output of other integrated circuits, and the transmitted image data may be acquired by the video acquisition unit on other integrated circuits from the video channel of an analog camera, or may be acquired and decoded by the video decoding unit on other integrated circuits from the video channel of a network camera, and this application does not make specific restrictions on this. It should be emphasized that the cache area in this application can only cache one frame of image data.

[0079] It can be understood that, for the copy mode and the copy-saving mode, the corresponding write strategies of the main control unit 210 are different. The copy mode can be considered as a mode for the image processing unit 220, while the copy-saving mode can be considered as a mode for the main control unit 210; wherein, the copy-saving mode is a mode in which the main control unit 210 applies for the cache area in the public cache pool multiple times to cache image data. Therefore, the write strategy corresponding to the main control unit 210 matching the copy-saving mode is: whenever the main control unit 210 obtains a frame of image data, it can apply for a cache area for the frame of image data from the public cache pool, and write the frame of image data into the applied cache area. Since each frame of image data is cached in the cache area, the image processing unit 220 can perform image processing on multiple consecutive frames of image data when performing image processing, which is equivalent to the image processing unit 220 immediately releasing the cache area in the public cache pool.

[0080] The above only introduces the write strategy corresponding to the main control unit 210 matching the copy-saving mode. For the sake of clear layout, the write strategy corresponding to the main control unit 210 matching the copy mode will be introduced in other embodiments and will not be repeated here.

[0081] Correspondingly, for the image processing unit 220, the execution strategies corresponding to the copy mode and the copy-saving mode are also different, wherein the copy mode is a mode in which the image data written into the public cache pool is pre-copied to the target memory space by the image processing unit 220, wherein the target memory space is also the memory space in the video recorder, and the present application does not make specific limitations on this; it can be understood that the image processing unit can first copy the image data written into the public cache pool to the target memory space, and then read each frame of the copied image data from the memory space in turn, and perform image processing on each frame of the copied image data, and after the image data in the public cache pool is copied to the target memory space, the image data in the public cache pool can be released, thereby realizing that the image processing unit 220 immediately releases the cache area in the public cache pool, so that the main control unit does not need to wait for the processing result of any frame of image data, and can quickly write the next frame of image data of any frame of image data to the same cache area in the public cache pool. Among them, the target memory space can be applied for by the image processing unit, and it is applied for when the main control unit determines that the task execution mode of the image processing unit is the copy mode; of course, the target memory space can also be applied for by the main control unit, and the embodiment of the present application does not make specific restrictions on this. The present application does not limit who applies for this memory space, but mainly emphasizes that there is a target memory space in the device for storing necessary data. In addition, the image data unit 220 performs image processing for each frame of image data, which may include scaling processing, noise reduction processing, etc., and the present application does not make specific restrictions on this. In addition, the processing result of each frame of image data can be each frame of image data after processing. In addition, the copy mode can copy data based on DMA technology, and the present application does not go into details about this.

[0082] For the sake of clarity, the execution strategy corresponding to the copy-saving mode will be introduced in other embodiments and will not be described here. In one implementation, the image processing unit 220 can cache the processing result of each frame of image data in the output buffer pool, and the subsequent task execution unit 230 can obtain the processing of each frame of image data from the output buffer pool, and the output buffer is the memory space in the video recorder; of course, it can also be sent directly from the image processing unit 220 to the task execution unit 230, and the embodiment of the present application does not specifically limit this.

[0083] It is understandable that when the task execution unit 230 executes a task, it also executes a target task for the processing result of each frame of image data. Different target tasks correspond to different task execution units 230. The target tasks and the task execution unit 230 are briefly introduced below:

[0084] In one implementation, the target task is an image preview task, and the task execution unit is a video output unit;

[0085] The task execution unit executes the corresponding task based on the acquired processing result, which may include step A1:

[0086] Step A1: The video output unit outputs an image to a preview device based on the acquired processing result, so that the preview device performs image preview.

[0087] It is understandable that when the target task is an image preview task, the task execution unit may be a video output unit, and the video output unit may output the acquired processing result to the preview device, thereby enabling the preview device to perform image preview; and, in one implementation, the image processing unit may be pre-bound to the video output unit, and when the target task is an image preview task, the image processing unit may directly send the processing result to the video output unit. Of course, the target task of the present application is not limited to this, and the embodiments of the present application do not specifically limit this.

[0088] It can be seen that when the target task is an image preview task, the task execution unit can be a video output unit, so that image data preview can be realized. For the image preview task, the efficiency of the video recorder in processing the image preview task is improved.

[0089] In another implementation, the target task is an image encoding task, and the task execution unit is a video encoding unit;

[0090] The task execution unit executes the corresponding task based on the acquired processing result, which may include step B1:

[0091] Step B1: The video encoding unit performs image encoding processing based on the acquired processing result to obtain an encoding result.

[0092] It can be understood that when the target task is an image encoding task, the task execution unit can be a video encoding unit, and the video encoding unit can perform image encoding processing on the acquired processing result to obtain an encoding result; and, in one implementation method, the image processing unit can be pre-bound to the video encoding unit, and when the target task is an image encoding task, the video encoding unit can directly send the processing result to the video encoding unit.

[0093] It can be seen that when the target task is an image coding task, the task execution unit can be a video coding unit, so that image data coding can be implemented. For the image coding task, the efficiency of the video recorder in processing the image coding task is improved.

[0094] It can be seen that the image processing unit in the embodiment of the present application does not need to wait for the processing result of each frame of image data to be cached in the output buffer pool before releasing the cache area in the public buffer pool. Specifically, in the copy mode, the image processing unit can pre-copy the image data written into the public buffer pool to the target memory space, so that the cache area occupied by the copied image data can be released in time; while in the copy-saving mode, the main control unit can apply for the cache area in the public buffer pool multiple times to cache the image data, which is equivalent to the image processing unit immediately releasing the cache area in the public buffer pool. In this way, in the face of continuous multiple frames of image data, there is no need to wait for the image processing unit to process the result of any frame of image data, the main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can continuously process each frame of image data, thereby improving the efficiency of the video recorder processing task. In addition, the image processing unit can immediately release the cache area in the public buffer pool, which can reduce the serial processing time of the video link.

[0095] In addition, when the task execution mode is the copy mode, the image processing unit can pre-copy the image data written into the public cache pool to the target memory space, so that the image processing unit can immediately release the cache area occupied by the copied image data, thereby improving the efficiency of the video recorder processing task while reducing the occupancy time of the cache area during the task execution process. In addition, this solution selects different task execution modes through the reference information of the target task. Since the reference information is the data volume of the target task, it can provide a basis for the reasonable and effective use of storage space during the task execution process and avoid excessive memory consumption.

[0096] Optionally, in another embodiment, the write strategy corresponding to the main control unit that matches the copy mode is: a strategy in which each frame of image data shares the same cache area in a public cache pool; accordingly, the execution strategy corresponding to the copy mode is: determining a target memory space required for the image processing, copying each frame of image data written sequentially into the public cache pool to the target memory space for image processing, and in response to any frame of image data having been copied to the target memory space, releasing the cache area of ​​the frame of image data in the public cache pool;

[0097] or,

[0098] The writing strategy corresponding to the main control unit that matches the copy-saving mode is: each frame of image data corresponds to its own cache area in the public cache pool; accordingly, the execution strategy corresponding to the copy-saving mode is: read each frame of image data that has been written to the public cache pool in sequence, and in response to the processing result of any frame of image data having been obtained, release the cache area of ​​the frame of image data in the public cache pool.

[0099] It can be understood that, in the copy mode, each frame of image data written by the main control unit shares the same cache area in the public cache pool, and the cache area can generally only cache one frame of image data. Then, after the frame of image data is released, the main control unit can continue to write the next frame of image data in the cache area. Correspondingly, in the copy mode, the image processing unit can determine the target memory space required for image processing, and the target memory space can be an input cache pool, which is a cache pool for caching copied image data. Each frame of image data written into the public cache pool can be copied to the input cache pool in sequence for image processing. The image processing unit can directly read data from the input cache pool and perform image processing; and, in response to any frame of image data being copied to the input cache pool, the cache area of ​​the frame of image data in the public cache pool can be released. Exemplarily, image data 1 is cached in the cache area in the public cache pool. After the image processing unit copies image data 1 in the cache area to the input cache pool, the image data 1 in the public cache pool can be released. Of course, the memory space may be an input buffer pool, or any memory space of a video recorder, and this embodiment of the present application does not specifically limit this.

[0100] It can be understood that, in the copy-saving mode, each frame of image data written by the main control unit corresponds to a respective cache area in the public cache pool, and the image processing unit can read the corresponding image data from the respective cache area. Since the image processing unit can read the image data from each cache area in the public cache pool, and no longer read the image data from the same cache area, it is no longer necessary to pay attention to the timing when the image data in each cache area is released, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool. Accordingly, in the copy-saving mode, the image processing unit can sequentially read each frame of image data written to the public cache pool, and when the task execution unit obtains the processing result of any frame of image data, the cache area of ​​the frame of image data in the public cache pool is released. In one implementation, when the image processing unit reads each frame of image data written to the public cache pool, and in response to the processing result of any frame of image data being cached in the output cache pool, the image processing unit can release the cache area of ​​the frame of image data in the public cache pool. Exemplarily, image data 2 is cached in a cache area in a public cache pool. The image processing unit performs image processing on image data 2 to obtain a processing result. After caching the processing result in an output cache pool, the image processing unit may release the cache area of ​​image data 2 in the public cache pool.

[0101] It can be seen that the embodiment of the present application can match the copy-saving mode, and the write strategy corresponding to the main control unit can apply for a cache area in the public cache pool for each frame of image data, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool; and the execution strategy corresponding to the copy mode can release the cache area of ​​any frame of image data in the public cache pool after the image data has been copied to the memory space, so as to realize the timely release of the image data. In this way, in the face of multiple consecutive frames of image data, there is no need to wait for the image processing unit to process any frame of image data. The main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can also continuously perform image processing on each frame of image data, thereby improving the efficiency of the video recorder processing task.

[0102] In one implementation, the main control unit writes the frame image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode, which may include step C1:

[0103] Step C1, if the task execution mode is the copy mode, in response to the frame image data being the first frame, writing the frame image data into the cache area corresponding to the target task; in response to the frame image data being a non-first frame, in response to the cache area corresponding to the target task being released, writing the frame image data into the cache area corresponding to the target task; wherein the cache area corresponding to the target task is a cache area applied for and obtained from a public cache pool for storing single-frame image data;

[0104] Correspondingly, the image processing unit sequentially processes each frame of image data written by the main control unit to the public buffer pool according to the execution strategy corresponding to the task execution mode to obtain the processing result of each frame of image data, which may include step D1:

[0105] Step D1, if the task execution mode is the copy mode, in response to the main control unit writing a frame of image data in the cache area of ​​the public cache pool, the frame of image data is read from the cache area where the frame of image data is written, and whenever a frame of image data is read, the read frame of image data is written to the target memory space, and the cache area corresponding to the target task is released; and, image processing is performed on each frame of image data written to the target memory space in turn to obtain a processing result.

[0106] It can be understood that, for step C1, the task execution mode is the copy mode, and each frame image data shares the same cache area in the public cache pool. When the main control unit writes the frame image data to the cache area in the public cache pool, if the frame image data is the first frame, the frame image data can be directly written into the cache area used to store single-frame image data in the public cache pool; then, if the frame image data is not the first frame, it indicates that there is image data in the cache area in the public cache pool. After releasing the image data in the cache area in the public cache pool, the frame image data is written into the cache area of ​​the public cache pool. Exemplarily, image data 2 is not the first frame, and image data 1 exists in the public cache pool. After image data 1 is released, the main control unit can write image data 2 into the cache area of ​​the public cache pool.

[0107] It can be understood that for step D1, the task execution mode is the copy mode, the target memory space can also be the input buffer pool, after the main control unit writes a frame of image data in the buffer area of ​​the public buffer pool, the image processing unit can read the frame of image data from the buffer area where the frame of image data is written, and the image processing unit can write the read frame of image data into the input buffer pool every time after reading a frame of image data, and immediately release the frame of image data in the buffer area in the public buffer pool, and after each frame of image data is written into the input buffer pool, image processing can be performed on each frame of image data written into the input buffer pool in turn, so as to obtain a processing result. Among them, the buffer area corresponding to the target task is the buffer area in the public buffer pool used to cache the image data required by the target task.

[0108] For a better understanding, the scheme of the copy mode is briefly introduced below with reference to the accompanying drawings. Figure 3 As shown:

[0109] The public cache pool management can apply for Ci from the public cache pool. Ci is the cache area applied by the main control unit from the public cache pool. The public cache pool management can be considered as the main control unit / image processing unit managing the public cache pool, which can specifically include that the main control unit can apply for Ci from the public cache pool, the image processing unit can release the image data in Ci, the user can cache the image data in Ci through the main control unit, the main control unit obtains the image data and can cache the image data in Ci, the image processing unit can read the image data in Ci, and the image data is sent to the image processing unit; the image processing unit can cache the image data in the applied cache area, that is, cache it in Ai applied from the input cache pool. Ai is the cache area applied by the image processing unit from the input cache pool. The image processing unit After the image data read from Ci is copied to Ai, the image data in Ci can be released, and the main control unit continues to write the next frame of image data into Ci. The image processing unit can process the image data in Ai and cache the processed image data in the applied Bi. At this time, the image data in Ai can be released based on the input buffer pool management. The input buffer pool management can be considered as the image processing unit managing the input buffer pool. The subsequent video output unit / video encoding unit can read the image data from Bi and perform the corresponding tasks. After completing the tasks, the image data in Bi is released based on the output buffer pool management. The output buffer pool management can be considered as the image processing unit managing the output buffer pool. The input buffer pool and the output buffer pool are both buffer pools corresponding to the image processing unit. The dotted line is the step of releasing the image data.

[0110] It can be seen that in the copy mode, each frame of image data shares the same cache area in the public cache pool. Whenever the image processing unit reads a frame of image data, it can write the read frame of image data into the target memory space and release the cache area corresponding to the target task, thereby realizing the timely release of image data. In this way, when facing multiple consecutive frames of image data, there is no need to wait for the image processing unit to process any frame of image data. The main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can also continuously perform image processing on each frame of image data, thereby improving the efficiency of the video recorder's processing tasks.

[0111] In addition, in the copy mode, the image processing unit can read the image data written into the public buffer pool and write it into the target memory space. At this time, the image processing unit can immediately release the cache area of ​​the public buffer pool, thereby improving the efficiency of the video recorder processing task and reducing the occupancy time of the cache area during the task execution. In addition, compared with the copy-saving mode, the copy mode does not need to apply for a cache area for each frame of image data in the public buffer pool, which can reduce the use of the public buffer pool and improve the management efficiency of the public buffer pool.

[0112] In one implementation, the main control unit writes the frame image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode, which may include step E1:

[0113] Step E1, if the task execution mode is the copy-saving mode, apply for a cache area for caching the frame image data from the public cache pool, and write the frame image data into the applied cache area;

[0114] The image processing unit sequentially processes each frame of image data written into the public buffer pool by the main control unit according to the execution strategy corresponding to the task execution mode to obtain the processing result of each frame of image data, which may include step F1:

[0115] Step F1, if the task execution mode is the copy-saving mode, in response to the main control unit writing a frame of image data in the cache area of ​​the public cache pool, the frame of image data is read from the cache area where the frame of image data is written, image processing is performed on the read frame of image data to obtain a processing result of the frame of image data; and after obtaining the processing result of the frame of image data, the cache area corresponding to the frame of image data in the public cache pool is released.

[0116] It can be understood that for step E1, the task execution mode is a copy-saving mode, and each frame of image data corresponds to a cache area. When the main control unit writes the frame of image data to the cache area in the public cache pool, it can apply for a cache area for caching the frame of image data, and write the frame of image data to the applied cache area. Each cache area can cache one frame of image data, and the public cache pool can include multiple applied cache areas. Exemplarily, the main control unit receives image data 1 and image data 2, and applies for cache areas for image data 1 and image data 2 respectively. Cache area 1 is used to cache image data 1, and cache area 2 is used to cache image data 2.

[0117] It can be understood that for step F1, the task execution mode is the copy-saving mode. After the main control unit writes a frame of image data in the cache area of ​​the public cache pool, the image processing unit can read the frame of image data from the above cache area, and directly perform image processing on the read frame of image data to obtain the processing result of the image data; then, after obtaining the processing result of the frame of image data, when the frame of image data is cached in the output cache pool, the cache area corresponding to the frame of image data in the public cache pool can be released. In addition, in the copy-saving mode, there is no need to apply for memory space from the recorder again, and the image processing unit does not need to access the memory space multiple times, which reduces the DMA hardware consumption caused by using DMA data copying.

[0118] For a better understanding, the scheme of the copy saving mode is briefly introduced below with reference to the accompanying drawings. Figure 4 As shown:

[0119] The public cache pool management can apply for multiple Ci from the public cache pool. Ci is the cache area applied by the main control unit from the public cache pool. The public cache pool management can be considered as the main control unit / image processing unit managing the public cache pool, which can specifically include the main control unit applying for Ci from the public cache pool, and the image processing unit releasing the image data in Ci. Each Ci can cache a single frame of image data. The user can cache image data in Ci through the main control unit. The main control unit obtains image data and can cache the image data in Ci. The image processing unit can read the image data in Ci, and the image data is sent to the image processing unit. Each frame of image data is cached in the requested cache area, that is, cached in Ci. The image processing unit can process the image data. At this time, the image processing unit can release the image data in Ci and cache the processing results in Bi. Bi is the cache area requested from the output cache pool. The subsequent video output unit / video encoding unit can read the image data from Bi and perform the corresponding tasks. After completing the tasks, based on the output cache pool management, the image data in Bi is released; the output cache pool management can be considered as the image processing unit managing the output cache pool. The input cache pool and the output cache pool are both cache pools of the image processing unit. The dotted line is the step of releasing the image data.

[0120] It can be seen that in the case of saving copies, each frame of image data corresponds to a cache area, and the image processing unit does not care whether the cache area is released, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool. In this way, when facing multiple consecutive frames of image data, there is no need to wait for the image processing unit to process the result of any frame of image data. The main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can also continuously perform image processing on each frame of image data, thereby improving the efficiency of the video recorder processing task.

[0121] In addition, compared with the copy mode, the copy-saving mode does not need to apply for memory space again, which reduces storage resource consumption, reduces data bandwidth, and reduces DMA hardware resource consumption caused by using DMA data copying.

[0122] In order to better understand the copy mode and the copy-saving mode, the following is an overall introduction to the two modes in conjunction with the accompanying drawings. Figure 5 As shown:

[0123] The public cache pool management can apply for Ci from the public cache pool, where Ci is the cache area applied by the main control unit from the public cache pool. The public cache pool management can be considered as the main control unit / image processing unit managing the public cache pool, which can specifically include that the main control unit can apply for Ci from the public cache pool, the image processing unit can release the image data in Ci, the user can cache the image data in Ci through the main control unit, the main control unit obtains the image data and caches the image data in Ci, the image processing unit can read the image data in Ci, and the image data is sent to the image processing unit, the image processing unit can apply for Bi from the output cache pool, where Bi is the cache area applied by the image processing unit from the output cache pool, and the image processing unit can also apply for Ai from the input cache pool, where Ai is the cache area applied by the image processing unit from the input cache pool;

[0124] In the copy mode, the image processing unit can cache the image data in the applied cache area, that is, cache it in Ai. After the image processing unit copies the image data read from Ci to Ai, the image data in Ci can be released. The image processing unit can process the image data in Ai and cache the processed image data in the applied Bi. At this time, the image data in Ai can be released based on the input cache pool management. The input cache pool management can be regarded as the image processing unit managing the input cache pool. The subsequent video output unit / video encoding unit can read the image data from Bi and perform the corresponding tasks. After completing the tasks, the image data in Bi is released based on the output cache pool management. The output cache pool management can be regarded as the image processing unit managing the output cache pool. The input cache pool and the output cache pool are both cache pools of the image processing unit.

[0125] In the copy saving mode, the main control unit can apply for Ci, and the number of applied Ci is multiple. Each Ci can cache a single frame of image data. After acquiring the image data, the main control unit can cache the image data in Ci. The image processing unit can read the image data in Ci, and the image data is sent to the image processing unit. This process can also be called transparent transmission; each frame of image data is cached in the applied cache area, that is, cached in Ci. The image processing unit can process the image data. At this time, the image processing unit can release the image data in Ci based on the public cache pool management, and cache the processing results in Bi. The subsequent video output unit / video encoding unit can read the image data from Bi and perform the corresponding task. After completing the task, based on the output cache pool management, the image data in Bi is released. The dotted line is the step of releasing the image data.

[0126] It can be seen that the image processing unit in the embodiment of the present application does not need to wait for the processing result of each frame of image data to be cached in the output cache pool before releasing the cache area in the public cache pool. Specifically, in the copy mode, the image processing unit can pre-copy the image data written into the public cache pool to the target memory space, so that the cache area occupied by the copied image data can be released in time; while in the copy-saving mode, the main control unit can apply for the cache area in the public cache pool multiple times to cache the image data, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool. In this way, in the face of multiple consecutive frames of image data, there is no need to wait for the image processing unit to process the result of any frame of image data. The main control unit can quickly cache the next frame of image data of any frame of image data, and the image processing unit can continuously process each frame of image data, thereby improving the efficiency of the video recorder processing task.

[0127] In one implementation, the reference information includes: one or more of the data volume of a single frame of image data, the total data volume of each frame of image data of the target task, and the data volume of a single frame of image data required to be obtained after being processed by the task execution unit;

[0128] The step of determining the task execution mode of the image processing unit based on the reference information of the target task may include steps G1-G2:

[0129] Step G1, detecting a relationship between the amount of data indicated by the reference information and a predetermined threshold;

[0130] It can be understood that the predetermined threshold is the basis for the main control unit to select the task execution mode. Based on the relationship between the data amount indicated by the reference information and the predetermined threshold, the task execution mode of the image processing unit can be determined. Since the reference information has multiple types, the predetermined threshold also corresponds to the reference information. When the reference information is the data amount of a single frame of image data, the predetermined threshold is also the data amount threshold of the single frame of image data. When the reference information is the total data amount of each frame of image data of the target task, the predetermined threshold can be the total data amount threshold of each frame of image data of the target task. And, when the reference information is the data amount of the single frame of image data required for the task execution unit to process, the predetermined threshold can be the data amount threshold of the single frame of image data required for the task execution unit to process. In addition, the reference information can also include multiple data amounts. Similarly, the predetermined threshold will also include corresponding multiple data amounts.

[0131] Of course, the reference information and predetermined thresholds introduced above are only for exemplary purposes. The reference information may also represent the image size of a single frame of image data, etc. This application does not make any specific limitation on this.

[0132] Exemplarily, the reference information is that the data volume of a single frame of image data is 256 kb, and the predetermined threshold may be that the data volume threshold representing the single frame of image data is 512 kb.

[0133] Step G2, if the data amount indicated by the reference information is not greater than a predetermined threshold, the copy mode is used as the task execution mode of the image processing unit; otherwise, the copy-save mode is used as the task execution mode of the image processing unit.

[0134] It can be understood that when the amount of data indicated by the reference information is less than or equal to a predetermined threshold, the copy mode can be determined as the task execution mode of the image processing unit. Since the amount of data indicated by the reference information is not greater than the predetermined threshold, determining the copy mode as the execution mode of the image processing unit will not excessively increase memory consumption, that is, resource consumption; and, when the amount of data indicated by the reference information is greater than the predetermined threshold, the copy-saving mode can be determined as the execution mode of the image processing unit. Since the amount of data indicated by the reference information is greater than the predetermined threshold, if the copy mode is selected as the execution mode of the image processing unit, the cache area applied for by the image processing unit from the input cache pool needs to cache a large amount of data, which will result in excessive memory consumption. At the same time, the image processing unit copies the image data in the public cache pool to the input cache pool, which will also increase the bandwidth consumption of DDR. At this time, selecting the copy-saving mode can reduce memory consumption and DDR bandwidth consumption.

[0135] Exemplarily, the reference information is that the total data volume of each frame image data of the target task is 352×288×2 bytes, and the predetermined threshold is 300×200×2 bytes. At this time, the copy mode can be determined as the task execution mode of the image processing unit.

[0136] It can be seen that the embodiment of the present application can flexibly select the task execution mode of the image processing unit based on the reference information and the predetermined threshold. When the amount of data indicated by the reference information is greater than the predetermined threshold, the copy-saving mode is determined as the task execution mode of the image processing unit. When the amount of data indicated by the reference information is not greater than the predetermined threshold, the copy mode is determined as the task execution mode of the image processing unit, thereby reducing memory consumption and DDR bandwidth consumption, while improving the efficiency of the recorder processing tasks and reducing the occupancy time of the cache area during the task execution process.

[0137] In one implementation, the video recorder includes a first integrated circuit and a second integrated circuit, the main control unit, the image processing unit and the task execution unit are units arranged in the second integrated circuit, the first integrated circuit and the second integrated circuit are cascaded via a PCIE bus, and the target memory space is a memory space on the second integrated circuit;

[0138] Accordingly, the main control unit sequentially obtains each frame of image data of the target task, including:

[0139] The main control unit sequentially obtains each frame of image data of the target task from the first integrated circuit.

[0140] It can be understood that the first integrated circuit is mainly used to acquire image data. A video acquisition unit, a video decoding unit, etc. can be provided on the first integrated circuit, and the main control unit, the image processing unit and the task execution unit can be provided on the second integrated circuit. The second integrated circuit is mainly used to process the image data and execute the target task. Both the first integrated circuit and the second integrated circuit have corresponding memory space, and the target memory space can be the memory space on the second integrated circuit. Then, the task of previewing the image data can be executed through the first integrated circuit and the second integrated circuit in the preview device and the video recorder.

[0141] It can be understood that each frame of image data acquired by the main control unit of the second integrated circuit can be transmitted by the first integrated circuit. For example, after the video acquisition unit of the first integrated circuit acquires the image data, it can be sent to the second integrated circuit, and the main control unit of the second integrated circuit can acquire the image data. In addition, each frame of image data acquired by the main control unit of the second integrated circuit is not necessarily limited to that acquired from the first integrated circuit, but can also be given by the user, and this application does not make specific limitations on this.

[0142] Furthermore, the first integrated circuit and the second integrated circuit can transmit data and exchange commands through the PCIE bus. For example, the second integrated circuit can issue a command to the first integrated circuit to adjust the size of the image data to be consistent with the window size of the preview device after the first integrated circuit captures the image data. This application does not make any specific limitations on this.

[0143] Of course, the main control unit, image processing unit and task execution unit do not have to be set on an integrated circuit. The above introduction is only an exemplary introduction and does not make any specific limitation to this application.

[0144] In order to better understand this part of the content, the following is a brief introduction with reference to the accompanying drawings. Figure 6 As shown:

[0145] Chip0 RAW BUF can be considered as the memory space of the first integrated circuit. The first integrated circuit can transmit the image data in Chip0 RAW BUF to the second integrated circuit (not shown in the figure). The second integrated circuit can be provided with a main control unit, an image processing unit and a video output unit. The main control unit can cache each frame of image data received in the cache area of ​​the public cache pool, that is, cache areas 1-4. The image processing unit then reads the YUV data of A1, the YUV data of A2, the YUV data of D1 and the YUV data of D2 from the cache areas 1-4 of the public cache pool respectively, processes the above YUV data, and then outputs it to the video output unit of the second integrated circuit, so that the video output unit outputs the image data to the preview device to form a 4-view preview.

[0146] It can be seen that the various units in the embodiments of the present application can be arranged on the second integrated circuit, and the target memory space can be the memory space on the second integrated circuit. In this way, the image processing unit only needs to apply to the memory space of the second integrated circuit, and there is no need to apply to other memory spaces of the video recorder. While improving the efficiency of the video recorder processing tasks, it also takes into account reducing the occupancy time of the cache area during task execution.

[0147] A data processing method provided by an embodiment of the present application is described below in conjunction with the accompanying drawings. In one implementation, the data processing method can be applied to the video recorder in the above embodiment.

[0148] like Figure 7 As shown, a data processing method provided in an embodiment of the present application may include the following steps:

[0149] S701, in response to a task execution instruction of a target task, determining a task execution mode based on reference information of the target task; and sequentially acquiring each frame of image data of the target task, and in response to each acquisition of a frame of image data, writing the frame of image data into a cache area in a public cache pool according to a write strategy matching the task execution mode;

[0150] Wherein, the reference information is information used to characterize the data volume of the image data of the target task;

[0151] S702, performing image processing on each frame of image data written into the public buffer pool in turn according to the execution strategy corresponding to the task execution mode, to obtain a processing result of each frame of image data;

[0152] S703, sequentially obtaining the processing result of each frame of image data, and executing a corresponding task based on the obtained processing result;

[0153] Among them, the task execution mode includes: copy mode or copy-saving mode; the copy mode is: pre-copying the image data written into the public cache pool to the target memory space so that the cache area occupied by the copied image data can be released immediately; or, the copy-saving mode is: applying for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the immediate release of the cache area in the public cache pool.

[0154] Optionally, the writing strategy matching the copy mode is: a strategy in which each frame of image data shares the same cache area in a public cache pool; correspondingly, the execution strategy corresponding to the copy mode is: determining a target memory space required for the image processing, copying each frame of image data written sequentially into the public cache pool to the target memory space for image processing, and in response to any frame of image data having been copied to the target memory space, releasing the cache area of ​​the frame of image data in the public cache pool;

[0155] or,

[0156] The write strategy that matches the copy-saving mode is: each frame of image data corresponds to its own cache area in the public cache pool; accordingly, the execution strategy corresponding to the copy-saving mode is: read each frame of image data that has been written to the public cache pool in sequence, and in response to the processing result of any frame of image data having been obtained, release the cache area of ​​the frame of image data in the public cache pool.

[0157] Optionally, writing the frame image data into a cache area in a public cache pool according to a writing strategy matching the task execution mode includes:

[0158] If the task execution mode is the copy mode, in response to the frame image data being the first frame, the frame image data is written into the cache area corresponding to the target task; in response to the frame image data being a non-first frame, in response to the cache area corresponding to the target task being released, the frame image data is written into the cache area corresponding to the target task; wherein the cache area corresponding to the target task is a cache area applied for and obtained from a public cache pool for storing single-frame image data;

[0159] According to the execution strategy corresponding to the task execution mode, each frame of image data written into the public buffer pool is processed in turn to obtain the processing result of each frame of image data, including:

[0160] If the task execution mode is the copy mode, in response to writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area where the frame of image data is written, and whenever a frame of image data is read, the read frame of image data is written to the target memory space, and the cache area corresponding to the target task is released; and, image processing is performed on each frame of image data written to the memory space in turn to obtain a processing result.

[0161] Optionally, writing the frame image data into a cache area in a public cache pool according to a writing strategy matching the task execution mode includes:

[0162] If the task execution mode is the copy-saving mode, applying for a cache area for caching the frame image data from the public cache pool, and writing the frame image data into the applied cache area;

[0163] According to the execution strategy corresponding to the task execution mode, each frame of image data written into the public buffer pool is processed in turn to obtain the processing result of each frame of image data, including:

[0164] If the task execution mode is the copy-saving mode, in response to writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area in which the frame of image data is written, image processing is performed on the read frame of image data to obtain a processing result of the frame of image data; and after obtaining the processing result of the frame of image data, the cache area corresponding to the frame of image data in the common cache pool is released.

[0165] Optionally, the reference information includes: one or more of the data volume of a single frame of image data, the total data volume of each frame of image data of the target task, and the data volume of the single frame of image data required to be obtained;

[0166] The determining of the task execution mode based on the reference information of the target task includes:

[0167] Detecting a magnitude relationship between the amount of data indicated by the reference information and a predetermined threshold;

[0168] If the amount of data indicated by the reference information is not greater than a predetermined threshold, the copy mode is determined as the task execution mode; otherwise, the copy-save mode is determined as the task execution mode.

[0169] It is understandable that the specific steps of the data processing method have been introduced in the above system embodiment, so they will not be repeated here.

[0170] It can be seen that the embodiment of the present application does not need to wait for the processing result of each frame of image data to be cached in the output buffer pool before releasing the cache area in the public buffer pool. Specifically, in the copy mode, the image data written into the public buffer pool can be copied to the target memory space in advance, so that the cache area occupied by the copied image data can be released in time; and in the copy-saving mode, the cache area in the public buffer pool can be applied for multiple times to cache image data, which is equivalent to the immediate release of the cache area in the public buffer pool. In this way, in the face of multiple consecutive frames of image data, there is no need to wait for the processing result of any frame of image data, and the next frame of image data of any frame of image data can be directly cached quickly, and each frame of image data can be continuously processed, thereby improving the efficiency of the processing task. In addition, the immediate release of the cache area in the public buffer pool can reduce the serial processing time of the video link.

[0171] In addition, when the task execution mode is the copy mode, the image data written into the public cache pool can be copied to the target memory space in advance, so that the cache area occupied by the copied image data can be released immediately, thereby improving the efficiency of processing tasks while reducing the occupation time of the cache area during task execution. In addition, this solution selects different task execution modes through the reference information of the target task. Since the reference information is the data volume of the target task, it can provide a basis for the reasonable and effective use of storage space during task execution and avoid excessive memory consumption.

[0172] Based on the above method embodiment, Figure 8 As shown, an embodiment of the present application provides a data processing device, the device comprising:

[0173] The first writing module 810 is used to determine the task execution mode based on the reference information of the target task in response to the task execution instruction of the target task; and sequentially obtain each frame of image data of the target task, and in response to each acquisition of a frame of image data, write the frame of image data into the cache area in the public cache pool according to the writing strategy matching the task execution mode; wherein the reference information is information used to characterize the data volume of the image data of the target task;

[0174] The second writing module 820 is used to perform image processing on each frame of image data written into the public buffer pool in turn according to the execution strategy corresponding to the task execution mode, so as to obtain a processing result of each frame of image data;

[0175] An execution module 830 is used to sequentially obtain the processing result of each frame of image data, and execute a corresponding task based on the obtained processing result;

[0176] Among them, the task execution mode includes: copy mode or copy-saving mode; the copy mode is: pre-copying the image data written into the public cache pool to the target memory space so that the cache area occupied by the copied image data can be released immediately; or, the copy-saving mode is: applying for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the immediate release of the cache area in the public cache pool.

[0177] Optionally, the writing strategy matching the copy mode is: a strategy in which each frame of image data shares the same cache area in a public cache pool; correspondingly, the execution strategy corresponding to the copy mode is: determining a target memory space required for the image processing, copying each frame of image data written sequentially into the public cache pool to the target memory space for image processing, and in response to any frame of image data having been copied to the target memory space, releasing the cache area of ​​the frame of image data in the public cache pool;

[0178] or,

[0179] The write strategy that matches the copy-saving mode is: each frame of image data corresponds to its own cache area in the public cache pool; accordingly, the execution strategy corresponding to the copy-saving mode is: read each frame of image data that has been written to the public cache pool in sequence, and in response to the processing result of any frame of image data having been obtained, release the cache area of ​​the frame of image data in the public cache pool.

[0180] Optionally, the first writing module is specifically used to:

[0181] If the task execution mode is the copy mode, in response to the frame image data being the first frame, the frame image data is written into the cache area corresponding to the target task; in response to the frame image data being a non-first frame, in response to the cache area corresponding to the target task being released, the frame image data is written into the cache area corresponding to the target task; wherein the cache area corresponding to the target task is a cache area applied for and obtained from a public cache pool for storing single-frame image data;

[0182] The second writing module is specifically used for:

[0183] If the task execution mode is the copy mode, in response to writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area where the frame of image data is written, and whenever a frame of image data is read, the read frame of image data is written to the target memory space, and the cache area corresponding to the target task is released; and, image processing is performed on each frame of image data written to the memory space in turn to obtain a processing result.

[0184] Optionally, the first writing module is specifically used to:

[0185] If the task execution mode is the copy-saving mode, applying for a cache area for caching the frame image data from the public cache pool, and writing the frame image data into the applied cache area;

[0186] The second writing module is specifically used for:

[0187] If the task execution mode is the copy-saving mode, in response to writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area in which the frame of image data is written, image processing is performed on the read frame of image data to obtain a processing result of the frame of image data; and after obtaining the processing result of the frame of image data, the cache area corresponding to the frame of image data in the common cache pool is released.

[0188] Optionally, the reference information includes: one or more of the data volume of a single frame of image data, the total data volume of each frame of image data of the target task, and the data volume of the single frame of image data required to be obtained;

[0189] The determining of the task execution mode based on the reference information of the target task includes:

[0190] Detecting a magnitude relationship between the amount of data indicated by the reference information and a predetermined threshold;

[0191] If the amount of data indicated by the reference information is not greater than a predetermined threshold, the copy mode is determined as the task execution mode; otherwise, the copy-save mode is determined as the task execution mode.

[0192] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0193] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned data processing methods is implemented.

[0194] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the data processing methods in the above embodiments.

[0195] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD), etc.

[0196] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0197] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0198] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A video recorder, characterized in that: include: Main control unit, image processing unit and task execution unit; The main control unit is used to determine the task execution mode of the image processing unit based on the reference information of the target task in response to the task execution instruction of the target task; and sequentially obtain each frame of image data of the target task, and in response to each acquisition of a frame of image data, write the frame of image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode; The reference information is information used to characterize the data volume of the image data of the target task; The image processing unit is used for, in response to the task execution mode determined by the main control unit, sequentially processing each frame of image data written by the main control unit to the public buffer pool according to the execution strategy corresponding to the task execution mode, to obtain a processing result of each frame of image data; The task execution unit is used to sequentially obtain the processing result of each frame of image data, and execute the corresponding task based on the obtained processing result; The task execution mode includes: a copy mode or a copy-saving mode; the copy mode is: the image processing unit copies the image data written into the public cache pool to the target memory space in advance, so that the image processing unit can immediately release the cache area occupied by the copied image data; or, the copy-saving mode is: the main control unit applies for the cache area in the public cache pool for image data caching multiple times, which is equivalent to the image processing unit immediately releasing the cache area in the public cache pool; Wherein, the writing strategy corresponding to the main control unit that matches the copy mode is: a strategy that each frame image data shares the same cache area in the public cache pool; correspondingly, the execution strategy corresponding to the copy mode is: determining the target memory space required for the image processing, copying each frame image data written into the public cache pool in sequence to the target memory space for image processing, and in response to any frame image data having been copied to the target memory space, releasing the cache area of ​​the frame image data in the public cache pool; or, The writing strategy corresponding to the main control unit that matches the copy-saving mode is: each frame of image data corresponds to its own cache area in the public cache pool; accordingly, the execution strategy corresponding to the copy-saving mode is: read each frame of image data that has been written to the public cache pool in sequence, and in response to the processing result of any frame of image data having been obtained, release the cache area of ​​the frame of image data in the public cache pool.

2. The video recorder according to claim 1, characterized in that The main control unit writes the frame image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode, including: If the task execution mode is the copy mode, in response to the frame image data being the first frame, the frame image data is written into the cache area corresponding to the target task; in response to the frame image data being a non-first frame, in response to the cache area corresponding to the target task being released, the frame image data is written into the cache area corresponding to the target task; wherein the cache area corresponding to the target task is a cache area applied for and obtained from a public cache pool for storing single-frame image data; The image processing unit performs image processing on each frame of image data written into the public buffer pool by the main control unit in sequence according to the execution strategy corresponding to the task execution mode, and obtains a processing result of each frame of image data, including: If the task execution mode is the copy mode, in response to the main control unit writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area where the frame of image data is written, and whenever a frame of image data is read, the read frame of image data is written to the target memory space, and the cache area corresponding to the target task is released; and, image processing is performed on each frame of image data written to the target memory space in turn to obtain a processing result.

3. The video recorder according to claim 1, characterized in that The main control unit writes the frame image data into a cache area in a public cache pool according to a write strategy corresponding to the main control unit that matches the task execution mode, including: If the task execution mode is the copy-saving mode, applying for a cache area for caching the frame image data from the public cache pool, and writing the frame image data into the applied cache area; The image processing unit performs image processing on each frame of image data written into the public buffer pool by the main control unit in sequence according to the execution strategy corresponding to the task execution mode, and obtains a processing result of each frame of image data, including: If the task execution mode is the copy-saving mode, in response to the main control unit writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area where the frame of image data is written, image processing is performed on the read frame of image data to obtain a processing result of the frame of image data; and, after obtaining the processing result of the frame of image data, the cache area corresponding to the frame of image data in the common cache pool is released.

4. The video recorder according to any one of claims 1 to 3, characterized in that: The reference information includes: one or more of the data volume of a single frame of image data, the total data volume of each frame of image data of the target task, and the data volume of a single frame of image data required to be obtained after being processed by the task execution unit; The determining the task execution mode of the image processing unit based on the reference information of the target task includes: Detecting a magnitude relationship between the amount of data indicated by the reference information and a predetermined threshold; If the data amount indicated by the reference information is not greater than a predetermined threshold, the copy mode is used as the task execution mode of the image processing unit; otherwise, the copy-save mode is used as the task execution mode of the image processing unit.

5. The video recorder according to any one of claims 1 to 3, characterized in that: The video recorder includes a first integrated circuit and a second integrated circuit, the main control unit, the image processing unit and the task execution unit are units arranged in the second integrated circuit, the first integrated circuit and the second integrated circuit are cascaded via a PCIE bus, and the target memory space is a memory space on the second integrated circuit; The main control unit sequentially obtains each frame of image data of the target task, including: The main control unit sequentially obtains each frame of image data of the target task from the first integrated circuit.

6. A data processing method, characterized in that: The method comprises: In response to a task execution instruction of a target task, a task execution mode is determined based on reference information of the target task; and, each frame of image data of the target task is sequentially acquired, and in response to each acquisition of a frame of image data, the frame of image data is written into a cache area in a public cache pool according to a write strategy matching the task execution mode; wherein the reference information is information used to characterize the data volume of the image data of the target task; According to the execution strategy corresponding to the task execution mode, each frame of image data written into the public buffer pool is processed in turn to obtain a processing result of each frame of image data; Obtain the processing results of each frame of image data in turn, and execute corresponding tasks based on the obtained processing results; The task execution mode includes: a copy mode or a copy-saving mode; the copy mode is: pre-copying the image data written into the public buffer pool to the target memory space so that the buffer area occupied by the copied image data can be released immediately; or the copy-saving mode is: applying for the buffer area in the public buffer pool for image data caching multiple times, which is equivalent to the immediate release of the buffer area in the public buffer pool; Wherein, the writing strategy matching the copy mode is: a strategy in which each frame of image data shares the same cache area in the public cache pool; correspondingly, the execution strategy corresponding to the copy mode is: determining the target memory space required for the image processing, copying each frame of image data written into the public cache pool in sequence to the target memory space for image processing, and in response to any frame of image data having been copied to the target memory space, releasing the cache area of ​​the frame of image data in the public cache pool; or, The write strategy that matches the copy-saving mode is: each frame of image data corresponds to its own cache area in the public cache pool; accordingly, the execution strategy corresponding to the copy-saving mode is: read each frame of image data that has been written to the public cache pool in sequence, and in response to the processing result of any frame of image data having been obtained, release the cache area of ​​the frame of image data in the public cache pool.

7. The method according to claim 6, characterized in that Writing the frame image data into a cache area in a public cache pool according to a writing strategy matching the task execution mode includes: If the task execution mode is the copy mode, in response to the frame image data being the first frame, the frame image data is written into the cache area corresponding to the target task; in response to the frame image data being a non-first frame, in response to the cache area corresponding to the target task being released, the frame image data is written into the cache area corresponding to the target task; wherein the cache area corresponding to the target task is a cache area applied for and obtained from a public cache pool for storing single-frame image data; According to the execution strategy corresponding to the task execution mode, each frame of image data written into the public buffer pool is processed in sequence to obtain the processing result of each frame of image data, including: If the task execution mode is the copy mode, in response to writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area where the frame of image data is written, and whenever a frame of image data is read, the read frame of image data is written to the target memory space, and the cache area corresponding to the target task is released; and, image processing is performed on each frame of image data written to the memory space in turn to obtain a processing result.

8. The method according to claim 6, characterized in that Writing the frame image data into a cache area in a public cache pool according to a writing strategy matching the task execution mode includes: If the task execution mode is the copy-saving mode, applying for a cache area for caching the frame image data from the public cache pool, and writing the frame image data into the applied cache area; According to the execution strategy corresponding to the task execution mode, each frame of image data written into the public buffer pool is processed in sequence to obtain the processing result of each frame of image data, including: If the task execution mode is the copy-saving mode, in response to writing a frame of image data in the cache area of ​​the common cache pool, the frame of image data is read from the cache area in which the frame of image data is written, image processing is performed on the read frame of image data to obtain a processing result of the frame of image data; and after obtaining the processing result of the frame of image data, the cache area corresponding to the frame of image data in the common cache pool is released.

9. The method according to any one of claims 6 to 8, characterized in that: The reference information includes: one or more of the data volume of single-frame image data, the total data volume of each frame image data of the target task, and the data volume of the single-frame image data required to be obtained; The determining of the task execution mode based on the reference information of the target task includes: Detecting a magnitude relationship between the amount of data indicated by the reference information and a predetermined threshold; If the amount of data indicated by the reference information is not greater than a predetermined threshold, the copy mode is determined as the task execution mode; otherwise, the copy-save mode is determined as the task execution mode.