Video data extraction system, video data extraction method and video data processing device

Abstract

The invention provides a video data extraction system and method and a video data processing device. According to the extraction system, a controller inputs to-be-processed data to a hardware extractor through a first control thread, and effective data with anti-competition bytes removed is output to a target module through a second control thread. In this way, the anti-competition bytes of the to-be-processed data are filtered through the hardware extractor and other hardware modules, so that the occupation of the controller in the process of filtering the anti-competition bytes is reduced. And the controller needs to schedule the extraction process of the video data through the thread, and does not need to additionally participate in the video extraction process. Therefore, the video data extraction efficiency is ensured and the dependence on the controller is reduced through a software-hardware cooperation mode, so that the problem that the overall performance of the system is reduced due to the fact that the running load of the controller is increased due to competition-proof byte filtering is solved.

Description

Technical Field

[0001] The present application belongs to the field of video data processing technology, and in particular relates to a video data extraction system, method and video data processing device. Background Art

[0002] During the operation of security back-end products such as surveillance equipment, the collected video data can be encapsulated using a Network Abstraction Layer Unit (NALU) to conform to encoding protocols such as H.264 / AVC or H.265 / HEVC. The encapsulated data is then transmitted to a decoding module. The decoding module can then send the decoded data to a display module, which then displays the image corresponding to the video data.

[0003] In the H.264 encoding protocol, each encapsulated NALU data contains a start code, which is used to indicate the beginning of a NALU data. The specific composition of the start code is pre-agreed, such as 0x000001 or 0x00000001. However, there may be valid data that is the same as the start code in the NALU data. Therefore, during the encoding process, an anti-competition byte is added to the valid data to prevent the start code from conflicting with the valid data, resulting in abnormalities in the decoding stage.

[0004] Before decoding the NALU data, the hardware module can distinguish different NALU data based on the start code, but it also needs to filter the anti-contention bytes from the data to be decoded before decoding. The process of filtering the anti-contention bytes from the data to be decoded is usually completed by the central processing unit (CPU) calling the software algorithm. However, in scenarios where the amount of data to be decoded is large, the operating load of the central processing unit will increase, resulting in a decrease in the overall performance of the system where the central processing unit is located. Summary of the Invention

[0005] The present application provides a video data extraction system, method and video data processing device to solve the problem that the decoding process easily increases the operating load of the central processing unit, resulting in a decrease in the overall performance of the system.

[0006] In a first aspect, the present application provides a video data extraction system, including: a controller and a hardware extractor.

[0007] The controller is configured as:

[0008] When detecting that data to be processed is input into the input buffer, starting a first control thread;

[0009] inputting first data to be processed into a target register in series through the first control thread; the first data to be processed is part of the data to be processed;

[0010] When detecting that the target register is full, controlling the first control thread to enter a dormant state;

[0011] The hardware extractor is configured to:

[0012] When the target register is full, scanning the first data to be processed for a start code;

[0013] If the first data to be processed cached in the first preset address range or the second preset address range of the target register contains the start code, then removing the start code from the target register so that the second data to be processed is cached in the target register to obtain the first target data; the second data to be processed is the subsequent data of the first data to be processed;

[0014] If an anti-contention byte is found in the first target data, the anti-contention byte is discarded to obtain the second target data;

[0015] outputting the second target data to an output buffer, and generating first interrupt information;

[0016] The controller is also configured to:

[0017] detecting the first interrupt information, waking up the first control thread, and starting a second control thread;

[0018] inputting third data to be processed serially into the target register through the first control thread, wherein the third data to be processed is subsequent data of the second data to be processed;

[0019] The second target data is output to the target module through the second control thread.

[0020] In some feasible embodiments, the controller executes the first control thread to serially input the first to-be-processed data into the target register, and is further configured to:

[0021] Obtaining the operating status of the hardware extractor;

[0022] If the running state of the hardware extractor is a busy state, controlling the first control thread to enter a dormant state;

[0023] If the operating state of the hardware extractor is an idle state, the first to-be-processed data is serially input into the target register through the first control thread.

[0024] In some feasible embodiments, the address length of the first preset address range corresponds to 4 bytes, and the address length of the second preset address range corresponds to 3 bytes; the first preset address range or the second preset address range is used to limit the search range of the start code.

[0025] In some feasible embodiments, the hardware extractor performs scanning of the start code in the first to-be-processed data according to a start code definition rule, and is further configured to:

[0026] If the first data to be processed cached in the first preset address range or the second preset address range of the target register does not contain the start code, discarding the first data to be processed located at the first address of the target register and acquiring the supplementary data from the input cache;

[0027] When the complement data is input to the target register, a start code is scanned in the first data to be processed and the complement data.

[0028] In some feasible embodiments, when the first to-be-processed data cached at the preset address of the target register includes the start code, the hardware extractor is further configured to:

[0029] Obtain the number of bytes occupied by the start code;

[0030] The start code is removed from the target register according to the number of bytes.

[0031] In some feasible embodiments, when the first to-be-processed data cached at the preset address of the target register includes the start code, the hardware extractor is further configured to:

[0032] If the anti-contention byte is not found in the first target data, the first target data is output to the output buffer, and the first interrupt information is generated.

[0033] In some feasible embodiments, when the first to-be-processed data cached in the first preset address range or the second preset address range of the target register includes the start code, the hardware extractor is further configured to:

[0034] detecting whether the first target data cached in the first preset address range is the data to be extracted; the second last byte of the data to be extracted conforms to the data format of the anti-contention byte, and the combined byte formed by the last byte of the data to be extracted and other bytes conforms to the anti-contention byte addition rule;

[0035] If the first target data is data to be extracted, the second last byte in the data to be extracted is determined to be an anti-contention byte.

[0036] In some feasible embodiments, the hardware extractor is further configured to:

[0037] When the third data to be processed is completely input into the target register and the target register is not full, the third data to be processed is output to the output buffer, and a second interrupt message is generated; the second interrupt message is used to indicate that the data to be processed corresponding to the start code has been processed.

[0038] In a second aspect, the present application provides a video data processing method, comprising:

[0039] When detecting that data to be processed is input into the input buffer, starting a first control thread;

[0040] inputting first data to be processed into a target register in series through the first control thread; the first data to be processed is part of the data to be processed;

[0041] When it is detected that the target register is full, controlling the first control thread to enter a dormant state; and when the target register is full, scanning a start code in the first data to be processed;

[0042] If the first data to be processed cached in the first preset address range or the second preset address range of the target register contains the start code, then removing the start code from the target register so that the second data to be processed is cached in the target register to obtain the first target data; the second data to be processed is the subsequent data of the first data to be processed;

[0043] If an anti-contention byte is found in the first target data, the anti-contention byte is discarded to obtain the second target data;

[0044] Outputting the second target data to an output buffer, and generating first interrupt information; the first interrupt information is used to wake up the first control thread and start a second control thread;

[0045] inputting third data to be processed serially into the target register through the first control thread, wherein the third data to be processed is subsequent data of the second data to be processed;

[0046] The second target data is output to the target module through the second control thread.

[0047] In a third aspect, the present application provides a video data processing device, comprising: an encoding module, a decoding module, and the video data extraction system described in any one of the first aspects;

[0048] The encoding module is configured to perform encoding on the collected data to obtain data to be processed; and transmit the data to be processed to the video data extraction system;

[0049] The video data extraction system is configured to extract valid data from the video data extraction system and transmit the valid data to the decoding module;

[0050] The decoding module is configured to decode the valid data.

[0051] It can be seen from the above technical content that the present application provides a video data extraction system, method and video data processing device. The extraction system is composed of a controller that inputs the data to be processed into a hardware extractor through a first control thread, and outputs the valid data with the anti-contention bytes removed to a target module through a second control thread. In this way, the anti-contention bytes of the data to be processed are filtered out by a hardware module such as a hardware extractor to reduce the occupancy of the controller during the filtering of the anti-contention bytes. In addition, the controller only needs to schedule the extraction process of the video data through a thread, without the need to participate in the video extraction process additionally. In this way, the efficiency of video data extraction is guaranteed and the dependence on the controller is reduced through the collaboration of software and hardware, which is conducive to alleviating the problem of increased controller operating load due to anti-contention byte filtering, which in turn leads to a decrease in the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 Schematic diagram of the NALU code stream data structure provided in the embodiment of this application;

[0054] Figure 2 A schematic diagram of the execution flow of the controller provided in an embodiment of the present application when storing data in the input buffer;

[0055] Figure 3 A schematic diagram of the execution flow of the hardware extractor provided in an embodiment of the present application when a start code is found;

[0056] Figure 4 A schematic diagram of byte division of data in a target register provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of a process of collaboration between a hardware extractor and a controller through first interrupt information provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the execution flow of the hardware extractor provided in an embodiment of the present application when the start code is not found;

[0059] Figure 7 A schematic diagram of the execution flow of the hardware extractor provided in an embodiment of the present application when the target register is not full;

[0060] Figure 8 This is a diagram of the composition of the video data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0062] In security back-end products such as monitoring equipment, the monitoring equipment needs to encode and encapsulate the collected data, and transmit the encapsulated data to the decoding module. The decoding module decodes the encapsulated data and then sends the decoded data to the display module, which displays the screen corresponding to the data.

[0063] When encoding data, the monitoring device can encode the data based on H.264 / AVC (H.264 standard / Advanced Video Coding specified by the International Telecommunication Union Telecommunication Standardization Sector) and H.265 / HEVC (H.264 standard / High Efficiency Video Coding specified by the International Telecommunication Union Telecommunication Standardization Sector).

[0064] like Figure 1 As shown, a network abstraction unit (NALU) can be used to encode data. Each encoded NALU data is preceded by a start code to distinguish multiple NALU data by the start code. That is, each start code represents the beginning of a NALU. The data format of the start code can be pre-agreed, such as 0x000001, 0x00000001, so that when downstream modules such as decoding modules recognize the same data format, they can distinguish multiple NALU data by the start code and process them one by one.

[0065] However, data identical to the start code may appear in the NALU data, which makes it impossible to distinguish between the NALU data and the start code. Therefore, during encoding, an anti-competition byte is added to the NALU data identical to the start code to distinguish the start code from the NALU data. For example, when encoding is completed, the encoding module can detect whether the encoded data contains data identical to the start code. When data identical to the start code is contained, an anti-competition byte can be inserted before the last byte of the data. For example, if the data identical to the start code is 0x000000, adding the anti-competition byte can result in 0x00000300. This type of data needs to filter out the anti-competition byte before decoding to prevent affecting the accuracy of the data.

[0066] In some embodiments, the process of filtering out anti-contention bytes is a software implementation process, that is, it is completed by the CPU or SoC (System on Chip) by calling a software algorithm. The process of filtering out anti-contention bytes can be called the unpacking process of NALU data. However, calling the software algorithm for filtering will increase the operating load of the CPU or SoC, and in the scenario of multi-channel data transmission, it will cause the operating load of the CPU or SoC to continue to be maintained at a high level, thereby causing the stability of the system where the CPU or SoC is located to decrease.

[0067] In order to solve the above problems, the embodiment of the present application provides a video data extraction system. The video data extraction system includes a controller and a hardware extractor. The controller can be a CPU or a SoC. The hardware extractor can be a hardware circuit for extracting NALU data streams. Figure 2 As shown, the controller is configured as follows during the extraction of NALU data:

[0068] When it is detected that the data to be processed is input into the input buffer, the first control thread is started.

[0069] The first data to be processed is serially input into the target register through the first control thread; the first data to be processed is part of the data to be processed.

[0070] When it is detected that the target register is full, the first control thread is controlled to enter a dormant state.

[0071] In some embodiments, an upstream module, such as an encoding module, may input data to be processed into an input buffer. The data to be processed may be data encoded by the encoding module. The input buffer may be a cache queue corresponding to a hardware extractor. Upon detecting that the encoding module inputs the data to be processed into the input buffer, the controller may initiate a first control thread, which may serially input the first data to be processed into a target register. Upon detecting that the target register is full, the controller may cause the first control thread to enter a dormant state.

[0072] It is understandable that the first control thread can be used to schedule the data to be processed in the input buffer, and the first data to be processed is part of the data to be processed. The target register can be a 32-bit register, or can be selected according to actual needs.

[0073] In this way, the hardware extractor can use the target register as an intermediary to filter out anti-contention bytes from the data being processed. This process of filtering out anti-contention bytes by the hardware extractor is implemented in hardware circuitry, eliminating the need to occupy controller resources and thus reducing the operating pressure on the controller. Furthermore, the first control thread entering a dormant state when no longer needed can also reduce operating pressure on the CPU or SoC.

[0074] In some embodiments, as Figure 3 As shown, the hardware extractor is configured as follows:

[0075] When the target register is full, a start code is scanned in the first data to be processed.

[0076] If the first data to be processed cached in the first preset address range or the second preset address range of the target register contains the start code, the start code is removed from the target register so that the second data to be processed is cached in the target register to obtain the first target data; the second data to be processed is the subsequent data of the first data to be processed.

[0077] If an anti-contention byte is found in the first target data, the anti-contention byte is discarded to obtain the second target data.

[0078] The second target data is output to an output buffer, and first interrupt information is generated.

[0079] In some embodiments, when the target register is full, the hardware extractor can scan the first data to be processed in the target register to see if it contains a start code. This process can be understood as the hardware extractor searching the first data to be processed bit by bit using hardware circuitry. Therefore, no controller resources are required.

[0080] The byte length of the start code is predetermined, and can, for example, occupy 4 bytes or 3 bytes. When the start code occupies 4 bytes, the 32-bit target register is completely occupied by the start code. When the start code occupies 3 bytes, 24 bits of the 32-bit target register are occupied by the start code.

[0081] In some embodiments, the first predetermined address range corresponds to a required address length of 4 bytes.

[0082] In some other embodiments, the second preset address range corresponds to an address length required by 3 bytes. Therefore, the first preset address range or the second preset address range can be used to limit the search range of the start code.

[0083] Since it is necessary to determine and distinguish different NALU data based on the start code, it is necessary to first scan the target register to see if there is a start code. Among them, every 8 bits in the target register are used to form a byte. If the last byte of each group of data stored in the target register is recorded as D0, then Figure 4 As shown, each group of data stored in the target memory is equivalent to Figure 4 Combined with the data format of the start code, the hardware extractor can determine whether D3D2D1D0 conforms to the data format of the start code by searching bit by bit, or determine whether D2D1D0 conforms to the data format of the start code.

[0084] In this way, when the hardware extractor detects the presence of a start code within the first preset address range or the second preset address range, it indicates that it has been determined that the subsequent data is NALU data corresponding to the start code. The start code can be removed from the target register so that the second data to be processed can be stored in the target register. The second data to be processed is the subsequent data of the first data to be processed.

[0085] It should be noted that the target register can be regarded as a first-in-first-out storage queue. For example, when the byte "01" in the start code corresponds to the D0 position, when the start code is removed, the byte "01" will be removed from the target register first. In this way, when the number of bytes in the start code is 3, the byte originally located at D3 will be moved to D0 after the start code is removed. This byte and the subsequent data stored in the target register should be the NALU data corresponding to the start code.

[0086] It is understandable that if the number of bytes of the start code is 4, it indicates that the first data to be processed corresponds to the start code, and then the first target data can be obtained by simply removing all the data in the target register and then storing the second data to be processed into the target register.

[0087] The first target data may be entirely composed of the second data to be processed, or may be composed of the second data to be processed and the first data to be processed that has not been removed from the target register.

[0088] Following the above embodiment, when the hardware extractor removes the start code in the target register and stores the second data to be processed in the target register, it can search for the anti-contention byte in the target register, and then when the anti-contention byte is found, it can discard the anti-contention byte and obtain the second target data. Figure 2 As shown, when the hardware extractor identifies that the byte at D1 is an anti-contention byte, it can directly discard the byte at D1 and output the byte corresponding to D3D2D0 as the second target data to the output buffer. The output buffer can be a buffer used to provide data to downstream modules such as a decoding module.

[0089] In this way, the hardware extractor will not form a data hole by outputting the combination of the remaining bytes as the second target data in the process of filtering out the anti-contention bytes, and can effectively filter out the anti-contention bytes to ensure the validity of the NALU data.

[0090] In addition, when the hardware extractor outputs the second target data to the output cache, it can also generate a first interrupt message to indicate that the data in the current target register has been processed, so as to wake up the first control thread through the first interrupt message and enable the first control thread to continue to input the data in the input cache into the target register.

[0091] In some embodiments, as Figure 5 As shown, the controller is also configured to:

[0092] The first interrupt information is detected, the first control thread is awakened, and a second control thread is started.

[0093] The third data to be processed is serially input into the target register through the first control thread, and the third data to be processed is subsequent data of the second data to be processed.

[0094] The second target data is output to the target module through the second control thread.

[0095] In some embodiments, when the controller detects the first interrupt information, it can wake up the first control thread to serially input the third processed data into the target register through the first control thread, and start the second control thread to output the second target data to the target module through the second control thread.

[0096] In addition, the third data to be processed is subsequent data of the second data to be processed. After the third data to be processed is stored in the target register, the hardware extractor can continue to perform the anti-contention byte search and filtering tasks.

[0097] In this way, the controller wakes up the first control thread by detecting the first terminal information, enabling data scheduling in the input buffer to be connected with data processing by the hardware extractor. Furthermore, the second thread can output the second target data obtained by filtering out the anti-contention bytes in the output buffer to a target module, such as a display module. By generating and detecting the first interrupt information, various processing flows can be connected. Using software processes as the scheduling method and hardware circuits as the processing method, the use of controller resources is reduced through a collaborative approach of software and hardware.

[0098] It should be noted that when the entire extraction process is scheduled by software, it is necessary to determine whether to store data in the target register based on the operating status of the hardware extractor. When the controller serially inputs the first to-be-processed data into the target register through the first control thread, the controller is further configured to:

[0099] Get the running status of the hardware extractor.

[0100] If the running state of the hardware extractor is a busy state, the first control thread is controlled to enter a dormant state.

[0101] If the operating state of the hardware extractor is an idle state, the first to-be-processed data is serially input into the target register through the first control thread.

[0102] In some embodiments, the controller may pre-detect the operating status of the hardware extractor before inputting the data to be processed into the target register. For example, the controller may obtain the storage status of the target register and determine the operating status of the hardware extractor based on the storage status of the target register. That is, if the target register is empty, the controller may determine that the hardware extractor is in an idle state; if the target register is not empty, the controller may determine that the hardware extractor is in a busy state.

[0103] In some embodiments, the controller may also determine the operating status of the hardware extractor by obtaining the status of a specific port of the hardware extractor.

[0104] Thus, when the controller detects that the hardware extractor is busy, it can control the first control thread to enter a dormant state. When the hardware extractor completes the current task and generates an interrupt message, the controller can wake up the first control thread and store the data to be processed in the target register through the first control thread.

[0105] On the contrary, when the controller detects that the operating state of the hardware extractor is idle, the controller may serially input the first to-be-processed data into the target register through the first control thread.

[0106] It is understandable that after the target register is fully populated with data, the hardware extractor can scan the data stored in the target register for a start code. That is, the hardware extractor can determine whether the first data to be processed in the target register contains a start code based on the first preset address range or the second preset address range. In the scenario where the first data to be processed contains a start code, the hardware extractor can execute the steps of discarding the start code and searching for anti-contention bytes. In the scenario where the first data to be processed does not contain a start code, such as Figure 6 As shown, the hardware extractor is also configured to:

[0107] If the first data to be processed cached in the first preset address range or the second preset address range of the target register does not contain the start code, the first data to be processed located at the first address of the target register is discarded, and the complementary data is obtained from the input cache.

[0108] When the complement data is input to the target register, a start code is scanned in the first data to be processed and the complement data.

[0109] In some embodiments, if the hardware extractor does not recognize the start code in the first preset address range or the second preset address range, the first data to be processed located at the first address of the target register may be discarded, which is actually equivalent to discarding 1 bit of data. Moreover, after the 1 bit of data in the target register is discarded, the input buffer will input 1 bit of data into the target register, and this 1 bit of data is the supplementary data. In this way, the hardware extractor can search for the start code in the current data composed of the first data to be processed and the supplementary data again according to the first preset address range or the second preset address range.

[0110] It is understood that if the start code is still not found, the input buffer continues to input the supplementary data to the target register, and the hardware extractor can continue to search for the start code according to the first preset address range or the second preset address range until the hardware extractor finds the start code. In this way, the hardware extractor can ensure that the start code can be efficiently identified.

[0111] In some embodiments, after the hardware extractor recognizes the start code, it needs to remove the start code from the target register so that the NALU data corresponding to the start code in the input buffer can be input into the target register. That is, the hardware extractor is configured to:

[0112] Get the number of bytes occupied by the start code.

[0113] The start code is removed from the target register according to the number of bytes.

[0114] In some embodiments, the number of bytes occupied by the start code is 3, which corresponds to 24 bits in the target register. That is, the hardware extractor can identify the start code corresponding to 3 bytes from the target register based on a bit-by-bit search and remove this start code from the target register. In this way, the remaining 8 bits of data in the target register should actually be part of the NALU data. In addition, the input buffer will continue to serially input 24 bits of data into the target register after the start code is removed. Then the hardware extractor can continue to process the data in the target register.

[0115] In other embodiments, the start code occupies 4 bytes, which corresponds to 32 bits in the target register. That is, the hardware extractor can identify the start code corresponding to the 4 bytes from the target register based on a bit-by-bit search and remove the start code from the target register. In this way, the hardware extractor can continue to process the newly stored 32 bits of data in the target register.

[0116] In this way, the hardware extractor can accurately identify the start code when processing data, distinguish different NALU data based on the start code, and remove the anti-competition byte in the target register based on the hardware circuit. The start code and NALU data are closely connected based on the serial input method, and no data gaps will be formed.

[0117] In some embodiments, not all NALU data contain data identical to the start code. Therefore, the hardware extractor performs the operation of filtering out the anti-contention byte only when the anti-contention byte is found. That is, when the first to-be-processed data cached in the first preset address range or the second preset address range of the target register contains the start code, the hardware extractor is further configured to:

[0118] If the anti-contention byte is not found in the first target data, the first target data is output to the output buffer, and the first interrupt information is generated.

[0119] It is understandable that if the hardware extractor does not find the anti-contention byte in the first target data, it indicates that the first target data is completely valid data in the NALU data. Therefore, the hardware extractor can directly output the first target data to the output buffer. This allows downstream modules such as the decoding module to use the first target data.

[0120] In addition, when the hardware extractor outputs the first target data to the output buffer, it can generate first interrupt information to wake up the first control thread through the first interrupt information, thereby enabling the first control thread to schedule subsequent to-be-processed data from the input buffer to the target register.

[0121] In this way, the hardware extractor can directly output such data to the downstream module under the condition that the data in the target register does not contain anti-contention bytes. Combined with the input cache serial input data method, the data to be processed can be processed sequentially and output to the output cache / downstream module.

[0122] In some embodiments, when the data in the target register contains an anti-contention byte, the hardware extractor needs to first remove the anti-contention byte and output the data without the anti-contention byte to the output buffer. Therefore, the hardware extractor needs to first identify the anti-contention byte. That is, the hardware extractor can search the target register for the anti-contention byte by bit-by-bit. The hardware extractor is configured to:

[0123] It is detected whether the first target data cached in the first preset address range is data to be extracted.

[0124] If the first target data is data to be extracted, the second last byte in the data to be extracted is determined to be an anti-contention byte.

[0125] It is understood that the data to be extracted here refers to the data containing the anti-contention byte, and the first target data is the data stored in the target register after removing the start code. For example, when the start code consists of 4 bytes, the first target data can be the second data to be processed.

[0126] In some embodiments, the anti-contention byte can be searched for in the first target data based on the anti-contention byte addition rule / data format. For example, the anti-contention byte is added before the last byte of the data that matches the start code during the encoding phase. The hardware extractor can then search for the anti-contention byte in the first target data by checking whether the next-to-last byte conforms to the anti-contention byte data format. Furthermore, the hardware extractor may need to verify whether the remaining bytes of the first target data match the start code.

[0127] For example, if the start code is 0x000001 and the anti-contention byte is 03, the second-last byte in the first target data is also 03, and the combination of other bytes is 0x000001, the hardware extractor can determine that the second-last byte in the first target data is the anti-contention byte and therefore discard it.

[0128] For another example, the combination of other bytes in the first target data is 0x000101, which is different from the start code. Even if the second last byte in the first target data is 03, the first target data does not include the anti-contention byte.

[0129] In this way, the hardware extractor can check whether the data in the target register contains the anti-contention byte based on the hardware circuit and the pre-configured start code encoding rules, and then perform subsequent data processing according to the judgment result of whether the anti-contention byte is contained.

[0130] In some embodiments, after the hardware extractor recognizes the start code, it can continue to process the NALU data corresponding to the start code. For example, if the hardware extractor completely processes one NALU data, the last data input to the target register may not fill the target register. At this time, the hardware extractor still needs to process this part of the data. Figure 7 As shown, the hardware extractor is also configured to:

[0131] When the third data to be processed is completely input into the target register and the target register is not full, the third data to be processed is output to the output buffer, and a second interrupt message is generated; the second interrupt message is used to indicate that the data to be processed corresponding to the start code has been processed.

[0132] In some embodiments, the third data to be processed is subsequent data of the second data to be processed. Taking the case where the third data to be processed is the last data in the current NALU data as an example, when the amount of the third data to be processed does not fill the target register, the hardware extractor can directly output the third data to be processed to the output buffer, so that downstream modules such as the decoding module can use the data in the output buffer.

[0133] In some embodiments, even if the third data to be processed is input into the target register together with the start code of the subsequent NALU data, based on the data processing rules of the hardware extractor, the hardware extractor can move the data in the target register bit by bit to the output buffer, thereby ensuring that the data in the preceding NALU data is not lost. And when all these data are moved out to the output buffer, the start code corresponding to the subsequent NALU data in the target register can also be detected, thus forming a close connection of the data processing flow, without forming data holes, which is conducive to improving the efficiency and accuracy of extracting anti-contention bits.

[0134] Some embodiments of the present application provide a video data extraction method, including:

[0135] When detecting that data to be processed is input into the input buffer, starting a first control thread;

[0136] inputting first data to be processed into a target register in series through the first control thread; the first data to be processed is part of the data to be processed;

[0137] When it is detected that the target register is full, controlling the first control thread to enter a dormant state; and when the target register is full, scanning a start code in the first data to be processed;

[0138] If the first data to be processed cached in the first preset address range or the second preset address range of the target register contains the start code, then removing the start code from the target register so that the second data to be processed is cached in the target register to obtain the first target data; the second data to be processed is the subsequent data of the first data to be processed;

[0139] If an anti-contention byte is found in the first target data, the anti-contention byte is discarded to obtain the second target data;

[0140] Outputting the second target data to an output buffer, and generating first interrupt information; the first interrupt information is used to wake up the first control thread and start a second control thread;

[0141] inputting third data to be processed serially into the target register through the first control thread, wherein the third data to be processed is subsequent data of the second data to be processed;

[0142] The second target data is output to the target module through the second control thread.

[0143] Some embodiments of the present application provide a video data processing device, such as Figure 8 As shown, it includes an encoding module, a decoding module and the video data extraction system mentioned in the above embodiment.

[0144] The encoding module is configured to perform encoding on the collected data to obtain data to be processed; and transmit the data to be processed to the video data extraction system;

[0145] The video data extraction system is configured to extract valid data from the video data extraction system and transmit the valid data to the decoding module;

[0146] The decoding module is configured to decode the valid data.

[0147] It can be seen from the above technical content that the present application provides a video data extraction system, method and video data processing device. The extraction system is composed of a controller that inputs the data to be processed into a hardware extractor through a first control thread, and outputs the valid data with the anti-contention bytes removed to a target module through a second control thread. In this way, the anti-contention bytes of the data to be processed are filtered out by a hardware module such as a hardware extractor to reduce the occupancy of the controller during the filtering of the anti-contention bytes. In addition, the controller only needs to schedule the extraction process of the video data through a thread, without the need to participate in the video extraction process additionally. In this way, the efficiency of video data extraction is guaranteed and the dependence on the controller is reduced through the collaboration of software and hardware, which is conducive to alleviating the problem of increased controller operating load due to anti-contention byte filtering, which in turn leads to a decrease in the overall performance of the system.

[0148] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without expending creative work shall fall within the scope of protection of this application.

Claims

1. A video data extraction system, characterized in that: include: Controller,Hardware Extractor; The controller is configured to: When detecting that data to be processed is input into the input buffer, starting a first control thread; inputting first data to be processed into a target register in series through the first control thread; the first data to be processed is part of the data to be processed; When detecting that the target register is full, controlling the first control thread to enter a dormant state; The hardware extractor is configured to: When the target register is full, scanning the first data to be processed for a start code; If the first data to be processed cached in the first preset address range or the second preset address range of the target register contains the start code, then removing the start code from the target register so that the second data to be processed is cached in the target register to obtain the first target data; the second data to be processed is the subsequent data of the first data to be processed; If an anti-contention byte is found in the first target data, the anti-contention byte is discarded to obtain the second target data; outputting the second target data to an output buffer, and generating first interrupt information; The controller is further configured to: detecting the first interrupt information, waking up the first control thread, and starting a second control thread; inputting third data to be processed serially into the target register through the first control thread, wherein the third data to be processed is subsequent data of the second data to be processed; The second target data is output to the target module through the second control thread.

2. The system according to claim 1, wherein: The controller executes the first control thread to serially input the first to-be-processed data into the target register, and is further configured to: Obtaining the operating status of the hardware extractor; If the running state of the hardware extractor is a busy state, controlling the first control thread to enter a dormant state; If the operating state of the hardware extractor is an idle state, the first to-be-processed data is serially input into the target register through the first control thread.

3. The system according to claim 1, wherein: The address length of the first preset address range corresponds to 4 bytes, and the address length of the second preset address range corresponds to 3 bytes; the first preset address range or the second preset address range is used to limit the search range of the start code.

4. The system according to claim 3, characterized in that The hardware extractor performs scanning of a start code in the first to-be-processed data according to a start code definition rule, and is further configured to: If the first data to be processed cached in the first preset address range or the second preset address range of the target register does not contain the start code, discarding the first data to be processed located at the first address of the target register and acquiring the supplementary data from the input cache; When the complement data is input to the target register, a start code is scanned in the first data to be processed and the complement data.

5. The system according to claim 1, wherein: When the first to-be-processed data cached at the first preset address or the second preset address of the target register includes the start code, the hardware extractor is further configured to: Obtain the number of bytes occupied by the start code; The start code is removed from the target register according to the number of bytes.

6. The system according to claim 1, wherein: When the first to-be-processed data cached at the first preset address or the second preset address of the target register includes the start code, the hardware extractor is further configured to: If the anti-contention byte is not found in the first target data, the first target data is output to the output buffer, and the first interrupt information is generated.

7. The system according to claim 1, wherein: When the first to-be-processed data cached at the first preset address or the second preset address of the target register includes the start code, the hardware extractor is further configured to: detecting whether the first target data cached in the first preset address range is the data to be extracted; the second last byte of the data to be extracted conforms to the data format of the anti-contention byte, and the combined byte formed by the last byte of the data to be extracted and other bytes conforms to the anti-contention byte addition rule; If the first target data is data to be extracted, the second last byte in the data to be extracted is determined to be an anti-contention byte.

8. The system according to claim 1, wherein: The hardware extractor is also configured to: When the third data to be processed is completely input into the target register and the target register is not full, outputting the third data to be processed to the output buffer and generating second interrupt information; The second interrupt information is used to indicate that the to-be-processed data corresponding to the start code has been processed.

9. A video data extraction method, characterized in that: include: When detecting that data to be processed is input into the input buffer, starting a first control thread; inputting the first to-be-processed data serially into the target register through the first control thread; The first data to be processed is part of the data to be processed; When detecting that the target register is full, controlling the first control thread to enter a dormant state; and when the target register is full, scanning a start code in the first data to be processed; If the first data to be processed cached at the first preset address or the second preset address of the target register contains the start code, then removing the start code from the target register so that the second data to be processed is cached in the target register to obtain the first target data; the second data to be processed is the subsequent data of the first data to be processed; If an anti-contention byte is found in the first target data, the anti-contention byte is discarded to obtain the second target data; outputting the second target data to an output buffer, and generating first interrupt information; The first interrupt information is used to wake up the first control thread and start the second control thread; inputting third data to be processed serially into the target register through the first control thread, wherein the third data to be processed is subsequent data of the second data to be processed; The second target data is output to the target module through the second control thread.

10. A video data processing device, characterized in that: include: An encoding module, a decoding module, and a video data extraction system according to any one of claims 1 to 8; The encoding module is configured to perform encoding on the collected data to obtain data to be processed; and transmit the data to be processed to the video data extraction system; The video data extraction system is configured to extract valid data from the video data extraction system and transmit the valid data to the decoding module; The decoding module is configured to decode the valid data.

Images