Improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hardware decoding

Abstract

The present invention relates to an improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding. The method reads the original image data and obtains the original size of each frame; performs feature extraction on the original image data, analyzes the image format based on the image features, and determines whether the image format information is supported by DVPP. If so, the decoding information is calculated based on the original image size, and decoding output memory space is adaptively created. The original image data in the memory is copied to DVPP for decoding according to the set decoding format; if not, the OpenCV library is used on the CPU side to perform a soft decoding algorithm to ensure that formats not supported by DVPP can be successfully decoded, avoiding DVPP decoding failures that cause program exceptions. The industry's common graphics coding formats are all supported by soft decoding. This method solves the problem of DVPP failing to decode images in some formats, joins a rich ecosystem of soft decoding, supports image decoding of various coding formats, and does not cause decoding task failures and program exit due to DVPP decoding failures.

Description

Technical Field

[0001] The present invention belongs to the field of DVPP decoding technology, and in particular relates to an improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding. Background Art

[0002] Existing Ascend DVPP hardware decoding technology is indeed many times faster than software decoding in terms of decoding performance. However, DVPP hardware decoding has its own limitations and its ecosystem is not as rich as software decoding. Therefore, decoding may fail in some cases, such as when the format is not supported or the vertical orientation of the image is not 1.

[0003] The specific method of traditional DVPP hard decoding is as follows:

[0004] Read original image data: Get the original size, image format, and original image data of each frame.

[0005] DVPP decoding: Manually set the decoding output format to create decoding output memory space, copy the original image data in the memory to DVPP for decoding according to the set decoding format. DVPP decoding supports limited formats, and the detailed constraints are as follows:

[0006] Hardware constraints:

[0007] Supports up to 4 Huffman tables, including 2 DC (direct current) tables and 2 AC (alternating current) tables;

[0008] Supports up to 3 quantization tables;

[0009] Only supports 8-bit sampling accuracy;

[0010] Only supports decoding of sequentially encoded pictures;

[0011] Only supports JPEG format decoding based on DCT (Discrete Cosine Transform);

[0012] Only supports decoding of images with an SOS (Start of Scan) mark;

[0013] Decoding successful: After decoding is successful, the decoded data is copied back to the memory to complete the decoding;

[0014] Decoding failure: If decoding fails, the program will exit abnormally.

[0015] Software constraints:

[0016] Supports image decoding of 3 SOS signs;

[0017] Supports decoding of abnormal images with insufficient MCU (Minimum Coded Unit) data.

[0018] Therefore, how to solve the problem of DVPP failing to decode some image formats, add a rich ecosystem of soft decoding, support image decoding of various encoding formats, and exit the program due to decoding task failure caused by DVPP decoding failure, are technical issues that need to be solved urgently. Summary of the Invention

[0019] The present invention aims to provide an improved dual-software and dual-hardware automatic error correction decoding method based on Ascend DVPP hard decoding. This method addresses the issue of DVPP failing to decode certain image formats, supports image decoding in various encoding formats, and enables program exit due to decoding task failure caused by DVPP decoding failure. This method incorporates a rich ecosystem of soft decoding, supports image decoding in various encoding formats, and achieves high performance of DVPP decoding.

[0020] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0021] The improved dual-software and hardware automatic error correction decoding method based on Ascend DVPP hardware decoding includes the following steps:

[0022] S1: Read the original image data and obtain the original image information of the original size of each frame;

[0023] S2: Extract features from the original image data, analyze the image format based on the extracted image features, read the image format information, and determine whether the image format information supports DVPP. If so, execute step S3; if not, execute step S4.

[0024] S3: Calculates decoding information based on the original image size and adaptively creates decoding output memory space. Due to the limited formats supported by DVPP decoding, the original image data in the memory is copied to DVPP for decoding according to the set decoding format;

[0025] S4: Use the OpenCV library as a soft decoding algorithm on the CPU side to ensure that formats not supported by DVPP can be successfully decoded, avoiding DVPP decoding failures that may cause program exceptions. The industry's common graphics coding formats are all supported by soft decoding.

[0026] S5: Complete decoding using the improved dual-software and hardware automatic error correction decoding method based on the Ascend DVPP hardware decoding described above.

[0027] Preferably, the specific process of reading the original image data in step S1 and obtaining the original size, image format, and original image information of each frame of image is as follows:

[0028] S11: Preset an image processing library in the system, and read the original image data based on an interface of the image processing library;

[0029] S12: Importing an image module, opening the original image data using a preset path, and obtaining the original image information including the original size, original image format, and color mode of the image, and obtaining the RGB information of each pixel in the original image through a nested loop.

[0030] Preferably, the specific process of extracting features from the original image data in step S2, analyzing the image format based on the extracted image features, and reading the image format information is as follows:

[0031] S21: Divide the original image into MxM feature extraction areas;

[0032] S22: For each pixel point in each feature extraction area, obtain pixel points in the neighborhood of the pixel point, compare the pixel point with the pixel values of a specified number of pixels in the neighborhood, and mark the pixel point in the feature extraction area whose pixel value is greater than the pixel value of the central pixel as 1, otherwise mark it as 0;

[0033] S23: Calculate the histogram of each feature extraction area and perform normalization on most histograms;

[0034] S24: Connecting the obtained histograms of the same level of each feature extraction area to form a feature vector of the original image, matching the feature vector of the original image with the feature vectors of various image formats, and obtaining the original image format according to the matching result.

[0035] Preferably, the specific process of step S3 is as follows:

[0036] S31: Calculate the total number of pixels in the original image by multiplying resolution by frame rate. Create the decoded output memory space based on the number of bits of storage space required per pixel. For example, a BMP image with an 800x600 resolution has 480,000 pixels, so one pixel is 24-bit true color, 800x600x24 / 8 = 1,440,000 bytes. This is the storage space required for one frame, or a static image. Multiplying this by the number of frames gives the total storage space. The overall calculation formula is: Storage Space = Resolution * Frame Rate * Bytes per Pixel.

[0037] S32: Convert the original image into YUV format and perform discrete cosine transform on each of the MxM regions divided into the original image;

[0038] S33: Obtain color component data of each region of the original image, read the color component data stream until the read color component data stream matches a leaf node in the Huffman tree, and use the Huffman tree to find the weight corresponding to the leaf node;

[0039] S34: Continue to read the bit data of the original image until the read component code is consistent with a leaf node of the Huffman tree of the component method and find the weight corresponding to the leaf node, and repeat the process of finding the weight until the preset end condition is met.

[0040] Preferably, in step S4, the specific process of using the OpenCV library to perform the soft decoding algorithm on the CPU side is:

[0041] S41: extracting the original image in the original image format obtained in step S24, and obtaining the specified key parameters in the original image format;

[0042] S42: Obtain pixel data of the original image, store the pixel data in a specified array according to specified key parameters, convert the original image into a grayscale image, and release resources.

[0043] The beneficial effects of the present invention include:

[0044] The improved soft-hard dual automatic error correction decoding method based on Ascend DVPP hard decoding provided by this invention reads the original image data, obtains the original size of each image frame, performs feature extraction on the original image data, analyzes the image format based on image features, and determines whether the image format information is supported by DVPP. If so, the decoding information is calculated based on the original image size, and decoding output memory space is adaptively created. The original image data in the memory is copied to DVPP for decoding according to the set decoding format. If not, the CPU uses the OpenCV library to perform a soft decoding algorithm to ensure that formats not supported by DVPP can be successfully decoded, avoiding DVPP decoding failures that cause program exceptions. The software decoding of common graphics encoding formats in the industry supports decoding. This method solves the problem of DVPP failing to decode some image formats, joins the rich soft decoding ecosystem, supports image decoding of various encoding formats, and prevents the decoding task from failing and causing program exit due to DVPP decoding failure.

[0045] First, the image format parsing code parses the image format and automatically determines whether DVPP or CPU decoding should be used to complete the task. The key code in the parsing part is the conclusion analysis results of multiple important experimental tests during the research and development process, which can quickly determine whether DVPP decoding is possible.

[0046] Secondly, the dual decoding strategy of DVPP hard decoding and CPU soft decoding uses CPU soft decoding to uniformly implement tasks that DVPP cannot decode. This strategy is innovative in the industry. It automatically corrects errors that prevent DVPP from completing decoding and decodes normally, solving the problem of DVPP failing to decode images in some formats. It has the high performance of DVPP decoding and adds a rich ecosystem of soft decoding to support image decoding in various encoding formats. It will not cause decoding task failure and program exit due to DVPP decoding failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the traditional soft and hard dual automatic error correction decoding method.

[0048] Figure 2 This is a flowchart of the improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding of the present invention. DETAILED DESCRIPTION

[0049] The following is combined with Figure 1~Figure 2 The present invention is described in further detail:

[0050] Example 1

[0051] See attached Figure 2 As shown in the figure, the improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding includes the following steps:

[0052] S1: Read the original image data and obtain the original image information of the original size of each frame;

[0053] S2: Extract features from the original image data, analyze the image format based on the extracted image features, read the image format information, and determine whether the image format information supports DVPP. If so, execute step S3; if not, execute step S4.

[0054] S3: Calculates decoding information based on the original image size and adaptively creates decoding output memory space. Due to the limited formats supported by DVPP decoding, the original image data in the memory is copied to DVPP for decoding according to the set decoding format;

[0055] S4: Use the OpenCV library as a soft decoding algorithm on the CPU side to ensure that formats not supported by DVPP can be successfully decoded, avoiding DVPP decoding failures that may cause program exceptions. The industry's common graphics coding formats are all supported by soft decoding.

[0056] S5: Complete decoding using the improved dual-software and hardware automatic error correction decoding method based on the Ascend DVPP hardware decoding described above.

[0057] See also Figure 1As shown in the figure, in the traditional DVPP hardware decoding process, raw image data is read: the original size, image format, and raw image data of each frame are obtained. DVPP decoding: the decoding output format is manually set to create decoding output memory space, and the raw image data in memory is copied to DVPP for decoding according to the set decoding format. DVPP decoding supports limited formats, and detailed constraints include hardware and software constraints. After successful decoding, the decoded data is copied back to memory to complete the decoding. Decoding failure: If the decoding fails, the program will exit abnormally.

[0058] The hardware constraints are as follows:

[0059] Supports up to 4 Huffman tables, including 2 DC (direct current) tables and 2 AC (alternating current) tables.

[0060] Supports up to 3 quantization tables.

[0061] Only supports 8-bit sampling accuracy.

[0062] Only supports decoding of sequentially encoded pictures.

[0063] Only supports JPEG format decoding based on DCT (Discrete Cosine Transform) transformation,

[0064] Only supports decoding of images with one SOS (Start of Scan) mark.

[0065] The software constraints are as follows:

[0066] Supports image decoding of 3 SOS signs.

[0067] Supports decoding of abnormal images with insufficient MCU (Minimum Coded Unit) data.

[0068] The present invention improves the traditional DVPP hard decoding process by reading the original image data, obtaining the original size of each frame of the image, extracting features of the original image data, analyzing the image format based on the image features, and determining whether the image format information is supported by DVPP. If so, the decoding information is calculated according to the original image size, and a decoding output memory space is adaptively created. The original image data in the memory is copied to DVPP for decoding according to the set decoding format. If not, the OpenCV library is used on the CPU side to perform a soft decoding algorithm to ensure that formats not supported by DVPP can be successfully decoded, avoiding program exceptions caused by DVPP decoding failure. The industry's general graphics coding format soft decoding all supports the decoding process, solving the problem of DVPP failing to decode images in some formats. The present invention adds a rich soft decoding ecosystem, supports image decoding of various coding formats, and will not cause decoding task failure and program exit due to DVPP decoding failure.

[0069] Part of the code for the above process is as follows:

[0070] Data-Info data- info,

[0071] if(filevec-.empty()){

[0072] ACLLITE –LOG- ERROR("Failed to deal all empty path");

[0073] return ACLLITE- ERROR;

[0074] }

[0075] if(frameCnt- == fileVec-.size()){

[0076] carDetectDataMsg->isLastFrame =1;

[0077] return ACLLITE-OK;

[0078] std::string picFile =fileVec-[frameCnt];

[0079] / / cout<<"file name:"< <picFile<<endl;

[0080] / / 01 Read raw data

[0081] AclLiteError ret =ReadData(carDetectDataMsg->imageFrame, picFile);

[0082] / / 02 Analyze image format

[0083] AclLiteError ret = AnalyzeData(carDetectDataMsg->imageFrame, datainfo);

[0084] / / 03 Determine whether the image format should be DVPP decoding or CPU soft decoding

[0085] switch(data-info.decode-type)

[0086] {

[0087] case TYPE DVPP:

[0088] / * code * /

[0089] / / 04 DVPP hard decoding

[0090] AclLiteError ret=ProcessData DVPp(carDetectDataMsg,picFile);

[0091] break;

[0092] case TYPE-CPU:

[0093] / * code *

[0094] / / 05 CPU soft decoding

[0095] AclLiteError ret= ProcessDataCPU(carDetectDataMsg,picFile):

[0096] break;

[0097] default:

[0098] break;

[0099] Example 2

[0100] On the basis of Example 1, the specific process of reading the original image data in step S1 and obtaining the original size, image format, and original image information of each frame of image is as follows:

[0101] S11: Preset an image processing library in the system, and read the original image data based on an interface of the image processing library;

[0102] S12: Importing an image module, opening the original image data using a preset path, and obtaining the original image information including the original size, original image format, and color mode of the image, and obtaining the RGB information of each pixel in the original image through a nested loop.

[0103] In this embodiment, the specific process of extracting features from the original image data in step S2, analyzing the image format based on the extracted image features, and reading the image format information is as follows:

[0104] S21: Divide the original image into MxM feature extraction areas;

[0105] S22: For each pixel point in each feature extraction area, obtain pixel points in the neighborhood of the pixel point, compare the pixel point with the pixel values of a specified number of pixels in the neighborhood, and mark the pixel point in the feature extraction area whose pixel value is greater than the pixel value of the central pixel as 1, otherwise mark it as 0;

[0106] S23: Calculate the histogram of each feature extraction area and perform normalization on most histograms;

[0107] S24: Connecting the obtained histograms of the same level of each feature extraction area to form a feature vector of the original image, matching the feature vector of the original image with the feature vectors of various image formats, and obtaining the original image format according to the matching result.

[0108] Example 3

[0109] Based on Example 1 or Example 2, the specific process of step S3 is as follows:

[0110] S31: Calculate the total number of pixels in the original image by multiplying resolution by frame rate. Create the decoded output memory space based on the number of bits of storage space required per pixel. For example, a BMP image with a resolution of 800x600 has 480,000 pixels, one of which is 24-bit true color. 800x600x24 / 8 = 1,440,000 bytes. This is the storage space required for one frame, or a static image. Multiplying this by the number of frames gives the total storage space. The overall calculation formula is: Storage Space = Resolution * Frame Rate * Bytes per Pixel.

[0111] S32: Convert the original image into YUV format and perform discrete cosine transform on each of the MxM regions divided into the original image;

[0112] S33: Obtain color component data of each region of the original image, read the color component data stream until the read color component data stream matches a leaf node in the Huffman tree, and use the Huffman tree to find the weight corresponding to the leaf node;

[0113] S34: Continue to read the bit data of the original image until the read component code is consistent with a leaf node of the Huffman tree of the component method and find the weight corresponding to the leaf node, and repeat the process of finding the weight until the preset end condition is met.

[0114] In this embodiment, the specific process of using the OpenCV library to perform the soft decoding algorithm on the CPU side in step S4 is as follows:

[0115] S41: extracting the original image in the original image format obtained in step S24, and obtaining the specified key parameters in the original image format;

[0116] S42: Obtain pixel data of the original image, store the pixel data in a specified array according to specified key parameters, convert the original image into a grayscale image, and release resources.

[0117] In summary, the improved dual-software and hardware automatic error correction decoding method based on Ascend DVPP hard decoding provided by the present invention reads the original image data and obtains the original size of each frame of the image; performs feature extraction on the original image data, analyzes the image format based on the image features, and determines whether the image format information is supported by DVPP. If so, the decoding information is calculated based on the original image size, and decoding output memory space is adaptively created. The original image data in the memory is copied to DVPP for decoding according to the set decoding format; otherwise, the OpenCV library is used on the CPU side to perform a soft decoding algorithm to ensure that formats not supported by DVPP can be successfully decoded, avoiding DVPP decoding failures that cause program exceptions. The industry's common graphics coding formats are all supported by soft decoding. This solves the problem of DVPP failing to decode images in some formats, joins the rich ecosystem of soft decoding, supports image decoding of various coding formats, and will not cause decoding task failures and program exit due to DVPP decoding failures.

[0118] The image format parsing code analyzes the image format and automatically determines whether DVPP or CPU decoding should be used to complete the task. The key code in the parsing section is the conclusion analysis results of multiple important experimental tests during the R&D process, which can quickly determine whether DVPP decoding is possible. A dual decoding strategy of DVPP hard decoding and CPU soft decoding is implemented. Tasks that cannot be decoded by DVPP are uniformly implemented using CPU soft decoding. This strategy is innovative in the industry and automatically corrects errors that prevent DVPP from completing decoding and decodes normally, solving the problem of DVPP failing to decode some image formats. It has the high performance of DVPP decoding and adds a rich ecosystem of soft decoding to support image decoding of various encoding formats. It will not cause decoding task failure and program exit due to DVPP decoding failure.

Claims

1. The improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hardware decoding is characterized by: The following steps are involved: S1: Read the original image data and obtain the original image information of the original size of each frame; S2: extracting image features from the original image data, analyzing the image format based on the extracted image features, reading the image format information, and determining whether the image format information supports DVPP. If so, proceed to step S3; otherwise, proceed to step S4. S3: Calculates decoding information based on the original image size and adaptively creates decoding output memory space. Due to the limited formats supported by DVPP decoding, the original image data in the memory is copied to DVPP for decoding according to the set decoding format; S4: Use the OpenCV library as a soft decoding algorithm on the CPU side to ensure that formats not supported by DVPP can be successfully decoded, avoiding DVPP decoding failures that may cause program exceptions. The industry's common graphics coding formats are all supported by soft decoding. S5: Decoding is completed using the improved dual-software and hardware automatic error correction decoding method based on the Ascend DVPP hardware decoding described above. In step S2, the specific process of extracting features from the original image data, analyzing the image format based on the extracted image features, and reading the image format information is as follows: S21: Divide the original image into M×M feature extraction areas; S22: For each pixel point in each feature extraction area, obtain pixel points in the neighborhood of the pixel point, compare the pixel point with the pixel values of a specified number of pixels in the neighborhood, and mark the pixel point in the feature extraction area whose pixel value is greater than the pixel value of the central pixel as 1, otherwise mark it as 0; S23: Calculate the histogram of each feature extraction area and perform normalization on most histograms; S24: Connecting the obtained histograms of the same level of each feature extraction area to form a feature vector of the original image, matching the feature vector of the original image with the feature vectors of various image formats, and obtaining the original image format according to the matching result.

2. The improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding according to claim 1 is characterized in that: The specific process of reading the original image data in step S1 and obtaining the original size, image format, and original image information of each frame is as follows: S11: Preset an image processing library in the system, and read the original image data based on an interface of the image processing library; S12: Importing an image module, opening the original image data using a preset path, and obtaining the original image information including the original size, original image format, and color mode of the image, and obtaining the RGB information of each pixel in the original image through a nested loop.

3. The improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding according to claim 1 is characterized in that: The specific process of step S3 is as follows: S31: Calculate the total number of pixels of the original image by resolution*frame, and create decoding output memory space based on the number of bits of storage space used for each pixel; S32: converting the original image into a YUV format and performing discrete cosine transform on each of the M×M regions divided into the original image; S33: Obtain color component data of each region of the original image, read the color component data stream until the read color component data stream matches a leaf node in the Huffman tree, and use the Huffman tree to find the weight corresponding to the leaf node; S34: Continue to read the bit data of the original image until the read component code is consistent with a leaf node of the Huffman tree of the component method and find the weight corresponding to the leaf node, and repeat the process of finding the weight until the preset end condition is met.

4. The improved soft and hard dual automatic error correction decoding method based on Ascend DVPP hard decoding according to claim 1 is characterized in that: The specific process of using the OpenCV library to perform soft decoding algorithm on the CPU side in step S4 is as follows: S41: extracting the original image in the original image format obtained in step S24, and obtaining the specified key parameters in the original image format; S42: Obtain pixel data of the original image, store the pixel data in a specified array according to specified key parameters, convert the original image into a grayscale image, and release resources.

Images