A multi-channel video parallel soft decoding method

Abstract

The application provides a multi-channel video parallel soft decoding method, comprising the following steps: configuring the compiling attribute and parameter of ffmpeg and transplanting to HiSilicon platform; according to the obtained decoding parameter, configuring the context environment of the ffmpeg decoding library, initializing the soft decoding environment; using the decoding environment corresponding to the decoding parameter to perform parallel decoding on the ffmpeg decoding library; according to the original video data and the data packet thread, using the ffmpeg decoding library to perform decoding; obtaining the decoded image data. The method directly obtains the parameter frame information and data frame information in the original frame from the ffmpeg decoding library, can effectively filter the error frame, realizes the decoding function in the case that the protocol frame does not completely conform to the decoding protocol or the error frame is relatively large; the soft decoding method can return the decoded image frame with the parameter frame, makes the data of the image more clear, effectively reduces the error rate; in the case of multi-channel video data transmission, the parallel decoding method can effectively reduce the decoding delay and guarantee the completion of the decoding function.

Description

Technical Field

[0001] This disclosure generally relates to the field of video decoding technology, and specifically to a multi-channel video parallel software decoding method. Background Technology

[0002] The Hi3559AV100 is a professional 8K Ultra HD Mobile Camera SOC that provides broadcast-quality digital video recording in 8K30 / 4K120 format. It supports multiple sensor inputs, H.265 encoded output or film-grade RAW data output, and integrates high-performance ISP processing. It also adopts advanced low-power technology and low-power architecture design to provide users with excellent image processing capabilities.

[0003] The Hi3559AV100 supports industry-leading multi-channel 4K sensor input, multi-channel ISP image processing, HDR10 high dynamic range technology standard, and multi-channel panoramic hardware stitching. When recording 8K30 / 4K120 video, the Hi3559AV100 offers hardened 6-DoF digital image stabilization, reducing reliance on mechanical gimbals.

[0004] However, the Hi3559AV100 falls under the category of hardware decoding. When the protocol frame does not fully conform to the decoding protocol, or when there are many erroneous frames, the decoding efficiency is relatively low or decoding is impossible. Furthermore, the hardware decoding module cannot return decoded image frames with parameter frames. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a multi-channel video parallel soft decoding method.

[0006] This application provides a multi-channel video parallel software decoding method, including:

[0007] Configure ffmpeg's compilation properties and parameters, port the ffmpeg decoding library to the HiSilicon platform, and obtain the decoding parameters;

[0008] Set up the initial decoding environment: Configure the context environment of the ffmpeg decoding library according to the decoding parameters and initialize the software decoding environment; obtain the decoding environment corresponding to the decoding parameters;

[0009] Parallel decoding of the ffmpeg decoding library is performed using a decoding environment corresponding to the decoding parameters; including:

[0010] Obtain raw video data;

[0011] Create a thread to retrieve data packets;

[0012] Based on the original video data and the data packet thread, the ffmpeg decoding library is used to perform decoding to obtain decoded image data;

[0013] The decoded image data is acquired and written to a file for local storage.

[0014] According to the technical solution provided in the embodiments of this application, the specific steps for porting the ffmpeg decoding library to the HiSilicon platform include:

[0015] Configure ffmpeg compilation properties, including: configuring parameters based on platform type, CPU type, codec properties, format conversion properties, and cross-compilation properties;

[0016] Modify the ffmpeg decoding library source code: add parameter frame length and parameter array, and determine the parameter frame size according to application layer requirements;

[0017] Cross-compile the ffmpeg decoding library source code to enable the decode_nal_sei_prefix function in the frame header file to obtain frame parameter data;

[0018] Executing the configure command will generate multiple different decoding libraries in subfolders of the configuration folder, resulting in multiple decoding files.

[0019] Copy the multiple decoding files to the subfolder path of the decoding board.

[0020] According to the technical solution provided in the embodiments of this application, the specific steps for obtaining frame parameter data include:

[0021] Obtain the data size parameter from the decode_nal_sei_prefix function;

[0022] Assign the data size parameter to the parameter frame length;

[0023] When the data size parameter is less than or equal to the frame size parameter:

[0024] Copy the data in the context parameter array to the parameter array, with the copy length equal to the data size parameter.

[0025] According to the technical solution provided in the embodiments of this application, the steps for setting up the initial decoding environment include:

[0026] Use the library function av_packet_alloc to dynamically allocate data packets, with the number of packets allocated equal to the number of video channels;

[0027] Use the library function av_cpu_count to check the number of CPUs supported by the decoding board;

[0028] When the number of CPUs is greater than or equal to the number of channels:

[0029] The array storing each channel of image data is initialized by setting the array to 0.

[0030] Use the library function avcodec_find_decoder to find the decoder required for the current decoding and obtain the decoder object;

[0031] The ID of the decoder object is initialized.

[0032] Configure the context environment based on the decoder object;

[0033] Set the number of parallel decoding threads for the context environment object based on the number of channels;

[0034] Sets the type of parallel thread in the context environment object;

[0035] Open the decoder object based on the decoder object and the context environment object;

[0036] Use the library function av_frame_alloc to dynamically allocate image frame buffers, with the number of buffers allocated equal to the number of channels;

[0037] Decoding threads are created using system functions, and the number of threads is equal to the number of channels.

[0038] According to the technical solution provided in the embodiments of this application, the steps for parallel decoding of the ffmpeg decoding library include:

[0039] S5.1: Use the system function fopen to open the raw video data and distinguish each image file according to the channel number;

[0040] S5.2: Determine if the file has reached its end; if so, proceed to S5.10; otherwise, proceed to the next step.

[0041] S5.3: Use the system function fread to read the data to be decoded from the original video data. The read data is stored in the array that stores the image data of each channel. The maximum number of bytes read each time is obtained, and the number of array bytes actually read is obtained.

[0042] S5.4: When the number of bytes in the array is 0, the starting address of the array storing the image data of each channel is saved into the pointer array;

[0043] S5.5: Determine if the number of bytes in the array is zero. If it is zero, jump to S5.2; otherwise, proceed to the next step.

[0044] S5.6: Use the library function av_parser_parse2 to obtain the complete frame from the pointer array, and the complete frame is stored in the data packet;

[0045] S5.7: When the number of bytes in the complete frame is not zero, move the pointer array forward by a number equal to the number of bytes in the complete frame; decrease the number of bytes in the array by a number equal to the number of bytes in the complete frame;

[0046] S5.8: Determine if the size of the data packet is zero. If yes, proceed to S5.10; otherwise, proceed to the next step.

[0047] S5.9: Perform the decoding process;

[0048] S5.10: End parallel decoding.

[0049] According to the technical solution provided in the embodiments of this application, the decoding process includes:

[0050] S6.1: Using the context environment object and the data packet as parameters, use the library function avcode_send_packet to send the data packet to the specified decoder, obtain the result value, and assign the result value to the number of bytes of the complete frame;

[0051] S6.2: Determine whether the result value is greater than or equal to zero. If yes, proceed to the next step; otherwise, proceed to S6.19.

[0052] S6.3: Determine whether the number of bytes in the complete frame is greater than or equal to zero. If yes, proceed to the next step; otherwise, proceed to S6.7.

[0053] S6.4: Using the context environment object and the frame object as parameters, use the library function avcodec_receive_frame to obtain the decoded image frame, and store the function return value in the result value;

[0054] S6.5: Determine whether the result value is an error code. If yes, proceed to S6.19; otherwise, proceed to the next step.

[0055] S6.6: Determine whether the result value is less than zero. If so, proceed to S6.19; otherwise, jump to S6.3.

[0056] S6.7: Determine if the width and height of the image saved in the history are consistent with the currently decoded image frame. If they are consistent, proceed to S6.9; otherwise, proceed to the next step.

[0057] S6.8: Obtain the anomaly flag of the image and set the anomaly flag to 1;

[0058] S6.9: Determine if the anomaly flag is 1. If yes, proceed to the next step; otherwise, proceed to S6.12.

[0059] S6.10: Determine whether the context environment object of the current channel conversion format is empty. If yes, proceed to S6.12; otherwise, proceed to the next step.

[0060] S6.11: Destroy the memory resources of the context object;

[0061] S6.12: Save the current image's width and height in the history;

[0062] S6.13: Using the target image's image format, width, height, and the converted image array as parameters, use the library function avpicture_fill to fill the converted image array;

[0063] S6.14: Use the library function sws_getContext to decode the current image and the context object that needs to be converted to a different format;

[0064] S6.15: Clear the abnormality markers in the image;

[0065] S6.16: Use the library function sws_scale to convert the size and format of the decoded image into the target image. The converted image is stored in the image array, and the conversion result is stored in the converted image result.

[0066] S6.17: Determine whether the value of the converted image result is consistent with the height of the target image. If they are consistent, proceed to the next step; otherwise, proceed to S6.19.

[0067] S6.18: Conversion successful;

[0068] S6.19: End the decoding process.

[0069] According to the technical solution provided in the embodiments of this application, the step of writing the decoded image data into a file and storing it locally includes:

[0070] Obtain the data within the image array, where the data size is the product of the width and height;

[0071] Obtain the parameter data stored in the context environment object;

[0072] The image writing and saving operation is performed using the first array address of the image array, the row data size, the image frame width, the height, and the file name to be written as parameters, including:

[0073] Use the system function fopen to open a file, grant write permissions, and obtain the file operation object;

[0074] Use the library function fprint to write a string to a file object with a specified first line, width after a newline, height after a newline, and a fixed line size;

[0075] The image frame array data is iterated in a loop. The total number of iterations is equal to the height of the line break. The current number of iterations is i. The system function fwrite is used to write to the file. The data written each time is the first array address of the image array, offset by i plus the size of the line data. Each data is written in 1-byte units, and the size of the line break is the size of the write.

[0076] Close the image file after writing;

[0077] Write the parameter data into a new save file and name the new save file with the specified file name;

[0078] After you finish writing, close the new saved file.

[0079] The beneficial effects of this application are as follows:

[0080] By configuring ffmpeg's compilation attributes and parameters, the ffmpeg decoding library is ported to the HiSilicon platform; decoding parameters are obtained; based on the decoding parameters, the ffmpeg decoding library is configured with a context environment and a software decoding environment is initialized; a decoding environment corresponding to the decoding parameters is obtained; the ffmpeg decoding library is used to perform parallel decoding using the decoding environment corresponding to the decoding parameters; based on the original video data and the data packet thread, the ffmpeg decoding library is used to decode; decoded image data is obtained; finally, the decoded image data is acquired and written to a file for local storage.

[0081] The process of obtaining frame parameters can return the decoded image frame with parameter frames to the ffmpeg decoding library, and then directly obtain the parameter frame information and data frame information in the original frame from the ffmpeg decoding library. It can effectively filter out erroneous frames generated during the decoding process; it can dynamically filter non-protocol erroneous frames when the protocol frame does not fully conform to the decoding protocol or when there are many erroneous frames, thus realizing the decoding function; it makes the image data clearer and effectively reduces the error rate; the method of initial decoding environment, parallel decoding process and storage can effectively reduce decoding latency and ensure the completion of decoding function in the case of multi-channel video data transmission. Attached Figure Description

[0082] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0083] Figure 1 A flowchart illustrating a multi-channel video parallel software decoding method provided in this application;

[0084] Figure 2 A flowchart illustrating the process of porting the ffmpeg decoding library to the HiSilicon platform;

[0085] Figure 3 A flowchart illustrating the process of obtaining frame parameter data;

[0086] Figure 4 A flowchart illustrating the process of setting up the initial decoding environment;

[0087] Figure 5 This is a flowchart illustrating the parallel decoding process;

[0088] Figure 6 This is a flowchart illustrating the decoding process. Detailed Implementation

[0089] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0090] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0091] Please refer to Figure 1 This is a schematic diagram of a multi-channel video parallel software decoding method provided in this embodiment, including:

[0092] S1: Configure ffmpeg's compilation properties and parameters, port the ffmpeg decoding library to the HiSilicon platform; obtain the decoding parameters;

[0093] S2: Set the initial decoding environment: Configure the context environment of the ffmpeg decoding library according to the decoding parameters, and initialize the software decoding environment; obtain the decoding environment corresponding to the decoding parameters;

[0094] S3: Perform parallel decoding on the ffmpeg decoding library using a decoding environment corresponding to the decoding parameters; including:

[0095] Obtain raw video data;

[0096] Create a thread to retrieve data packets;

[0097] Based on the original video data and the data packet thread, the ffmpeg decoding library is used to perform decoding to obtain decoded image data;

[0098] S4: Obtain the decoded image data and write the decoded image data to a file for local storage.

[0099] In some implementations, the HiSilicon Hi3559AV100 chip is used, which leverages its support for multiple sensor inputs, H.265 encoded output or film-grade RAW data output, and integrated high-performance ISP processing to enhance image processing capabilities, provide hardened 6-DoF digital image stabilization, and reduce reliance on mechanical gimbals.

[0100] Specifically, the ffmpeg decoding library was ported to the Hi3559AV100, and after configuring the context environment of the ffmpeg decoding library, the video files were decoded in parallel using the ffmpeg decoding library. This effectively filters out error frames, dynamically filtering non-protocol error frames when protocol frames do not fully conform to the decoding protocol or when there are many error frames, thus achieving the decoding function. The software decoding method can return decoded image frames with parameter frames, making the image data clearer and effectively reducing the error rate. In the case of multi-channel video data transmission, the parallel decoding method effectively reduces decoding latency and ensures the completion of the decoding function.

[0101] Further, refer to Figure 2 The specific steps for porting the ffmpeg decoding library to the HiSilicon platform include:

[0102] Configure ffmpeg compilation properties, including: configuring parameters based on platform type, CPU type, codec properties, format conversion properties, and cross-compilation properties;

[0103] Modify the ffmpeg decoding library source code: add parameter frame length and parameter array, and determine the parameter frame size according to application layer requirements;

[0104] Cross-compile the ffmpeg decoding library source code to enable the decode_nal_sei_prefix function in the frame header file to obtain frame parameter data;

[0105] Executing the configure command will generate multiple different decoding libraries in subfolders of the configuration folder, resulting in multiple decoding files.

[0106] Copy the multiple decoding files to the subfolder path of the decoding board.

[0107] Specifically, the method of directly obtaining parameter frame information and data frame information from the original frame from the ffmpeg decoding library can effectively filter out erroneous frames. It can dynamically filter out non-protocol erroneous frames and realize the decoding function when the protocol frame does not fully conform to the decoding protocol or when there are many erroneous frames.

[0108] Specifically, the configure command is used to automatically generate compilation properties, and the generated decoding libraries libavcodec, libavformat, libavutil, and libswscale will be located in the lib subfolder of the configuration folder.

[0109] In some implementations, the dynamic library files are copied to the / usr / lib path of the decoding board.

[0110] Further, refer to Figure 3 The specific steps for obtaining frame parameter data include:

[0111] Obtain the data size parameter from the decode_nal_sei_prefix function;

[0112] Assign the data size parameter to the parameter frame length;

[0113] When the data size parameter is less than or equal to the frame size parameter:

[0114] Copy the data in the context parameter array to the parameter array, with the copy length equal to the data size parameter.

[0115] In some implementations, the array index i of the parameter array is the index of the context parameter array divided by 8.

[0116] Further, refer to Figure 4 The steps for setting up the initial decoding environment include:

[0117] Use the library function av_packet_alloc to dynamically allocate data packets, with the number of packets allocated equal to the number of video channels;

[0118] Use the library function av_cpu_count to check the number of CPUs supported by the decoding board;

[0119] When the number of CPUs is greater than or equal to the number of channels:

[0120] The array storing each channel of image data is initialized by setting the array to 0.

[0121] Use the library function avcodec_find_decoder to find the decoder required for the current decoding and obtain the decoder object;

[0122] The ID of the decoder object is initialized.

[0123] Configure the context environment based on the decoder object;

[0124] Set the number of parallel decoding threads for the context environment object based on the number of channels;

[0125] Sets the type of parallel thread in the context environment object;

[0126] Open the decoder object based on the decoder object and the context environment object;

[0127] Use the library function av_frame_alloc to dynamically allocate image frame buffers, with the number of buffers allocated equal to the number of channels;

[0128] Decoding threads are created using system functions, and the number of threads is equal to the number of channels.

[0129] Specifically, initializing the array storing each video stream's data prevents the occurrence of erroneous data not being reread each time. Setting up the initial decoding environment enables the ffmpeg decoding library to adapt to multi-channel video decoding processes, facilitating parallel decoding.

[0130] Further, refer to Figure 5 The steps for parallel decoding of the ffmpeg decoding library include:

[0131] S5.1: Use the system function fopen to open the raw video data and distinguish each image file according to the channel number;

[0132] S5.2: Determine if the file has reached its end; if so, proceed to S5.10; otherwise, proceed to the next step.

[0133] S5.3: Use the system function fread to read the data to be decoded from the original video data. The read data is stored in the array that stores the image data of each channel. The maximum number of bytes read each time is obtained, and the number of array bytes actually read is obtained.

[0134] S5.4: When the number of bytes in the array is 0, the starting address of the array storing the image data of each channel is saved into the pointer array;

[0135] S5.5: Determine if the number of bytes in the array is zero. If it is zero, jump to S5.2; otherwise, proceed to the next step.

[0136] S5.6: Use the library function av_parser_parse2 to obtain the complete frame from the pointer array, and the complete frame is stored in the data packet;

[0137] S5.7: When the number of bytes in the complete frame is not zero, move the pointer array forward by a number equal to the number of bytes in the complete frame; decrease the number of bytes in the array by a number equal to the number of bytes in the complete frame;

[0138] S5.8: Determine if the size of the data packet is zero. If yes, proceed to S5.10; otherwise, proceed to the next step.

[0139] S5.9: Perform the decoding process;

[0140] S5.10: End parallel decoding.

[0141] Specifically, each image file is distinguished according to its channel number. The image file is read from multiple channels simultaneously to read the data to be decoded, and then the image data is stored in a pointer array. The complete frames stored in the pointer array are shifted forward and reduced. This preprocessing of the image files facilitates the parallel decoding of the image files in multiple channels.

[0142] Further, refer to Figure 6 The decoding process includes:

[0143] S6.1: Using the context environment object and the data packet as parameters, use the library function avcode_send_packet to send the data packet to the specified decoder, obtain the result value, and assign the result value to the number of bytes of the complete frame;

[0144] S6.2: Determine whether the result value is greater than or equal to zero. If yes, proceed to the next step; otherwise, proceed to S6.19.

[0145] S6.3: Determine whether the number of bytes in the complete frame is greater than or equal to zero. If yes, proceed to the next step; otherwise, proceed to S6.7.

[0146] S6.4: Using the context environment object and the frame object as parameters, use the library function avcodec_receive_frame to obtain the decoded image frame, and store the function return value in the result value;

[0147] S6.5: Determine whether the result value is an error code. If yes, proceed to S6.19; otherwise, proceed to the next step.

[0148] S6.6: Determine whether the result value is less than zero. If so, proceed to S6.19; otherwise, jump to S6.3.

[0149] S6.7: Determine if the width and height of the image saved in the history are consistent with the currently decoded image frame. If they are consistent, proceed to S6.9; otherwise, proceed to the next step.

[0150] S6.8: Obtain the anomaly flag of the image and set the anomaly flag to 1;

[0151] S6.9: Determine if the anomaly flag is 1. If yes, proceed to the next step; otherwise, proceed to S6.12.

[0152] S6.10: Determine whether the context environment object of the current channel conversion format is empty. If yes, proceed to S6.12; otherwise, proceed to the next step.

[0153] S6.11: Destroy the memory resources of the context object;

[0154] S6.12: Save the current image's width and height in the history;

[0155] S6.13: Using the target image's image format, width, height, and the converted image array as parameters, use the library function avpicture_fill to fill the converted image array;

[0156] S6.14: Use the library function sws_getContext to decode the current image and the context object that needs to be converted to a different format;

[0157] S6.15: Clear the abnormality markers in the image;

[0158] S6.16: Use the library function sws_scale to convert the size and format of the decoded image into the target image. The converted image is stored in the image array, and the conversion result is stored in the converted image result.

[0159] S6.17: Determine whether the value of the converted image result is consistent with the height of the target image. If they are consistent, proceed to the next step; otherwise, proceed to S6.19.

[0160] S6.18: Conversion successful;

[0161] S6.19: End the decoding process.

[0162] In some implementations, the specific value of the current channel is obtained from data packet parsing.

[0163] Specifically, the data packet is sent to the designated decoder using library functions. When the result value and the complete frame are both greater than or equal to zero, and the image width and height saved in the history are consistent with the currently decoded image frame, the decoded image frame is obtained using the library functions with the context environment object and frame object as parameters; the image anomaly flag is set to 1; multiple library functions are used to fill the converted image array, decode the context environment object that needs to be converted for the current image, and clear the anomaly flag. Multiple video data streams undergo the same processing in different channels, enabling parallel decoding, effectively reducing decoding latency, and improving image generation efficiency.

[0164] Furthermore, the step of writing the decoded image data to a file and storing it locally includes:

[0165] Obtain the data within the image array, where the data size is the product of the width and height;

[0166] Obtain the parameter data stored in the context environment object;

[0167] The image writing and saving operation is performed using the first array address of the image array, the row data size, the image frame width, the height, and the file name to be written as parameters, including:

[0168] Use the system function fopen to open a file, grant write permissions, and obtain the file operation object;

[0169] Use the library function fprint to write a string to a file object with a specified first line, width after a newline, height after a newline, and a fixed line size;

[0170] The image frame array data is iterated in a loop. The total number of iterations is equal to the height of the line break. The current number of iterations is i. The system function fwrite is used to write to the file. The data written each time is the first array address of the image array, offset by i plus the size of the line data. Each data is written in 1-byte units, and the size of the line break is the size of the write.

[0171] Close the image file after writing;

[0172] Write the parameter data into a new save file and name the new save file with the specified file name;

[0173] After you finish writing, close the new saved file.

[0174] In some implementations, the library function `fprint` is used to write a string with the specified first line being P5 to a file object. After a newline, a fixed line size of 255 is written.

[0175] Specifically, after decoding the image data using ffmpeg, the data is written to a file and stored locally, generating a decoded video file for easy display and observation of the decoding effect.

[0176] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A multi-channel video parallel software decoding method, characterized in that, include: Configure ffmpeg's compilation properties and parameters, port the ffmpeg decoding library to the HiSilicon platform, and obtain the decoding parameters; Set up the initial decoding environment: Configure the context environment of the ffmpeg decoding library according to the decoding parameters and initialize the software decoding environment; obtain the decoding environment corresponding to the decoding parameters; Parallel decoding of the ffmpeg decoding library is performed using a decoding environment corresponding to the decoding parameters; including: Obtain raw video data; Create a thread to retrieve data packets; Based on the original video data and the data packet thread, the ffmpeg decoding library is used to perform decoding to obtain decoded image data; The decoded image data is acquired and written to a file for local storage; The steps to set up the initial decoding environment include: Use the library function av_packet_alloc to dynamically allocate data packets, with the number of packets allocated equal to the number of video channels; Use the library function av_cpu_count to check the number of CPUs supported by the decoding board; When the number of CPUs is greater than or equal to the number of channels: The array storing each channel of image data is initialized by setting the array to 0. Use the library function avcodec_find_decoder to find the decoder required for the current decoding and obtain the decoder object; The ID of the decoder object is initialized. Configure the context environment based on the decoder object; Set the number of parallel decoding threads for the context environment object based on the number of channels; Sets the type of parallel thread in the context environment object; Open the decoder object based on the decoder object and the context environment object; Use the library function av_frame_alloc to dynamically allocate image frame buffers, with the number of buffers allocated equal to the number of channels; Decoding threads are created using system functions, and the number of threads is equal to the number of channels.

2. The multi-channel video parallel soft decoding method according to claim 1, characterized in that, The specific steps for porting the ffmpeg decoding library to the HiSilicon platform include: Configure ffmpeg compilation properties, including: configuring parameters based on platform type, CPU type, codec properties, format conversion properties, and cross-compilation properties; Modify the ffmpeg decoding library source code: add parameter frame length and parameter array, and determine the parameter frame size according to application layer requirements; Cross-compile the ffmpeg decoding library source code to enable the decode_nal_sei_prefix function in the frame header file to obtain frame parameter data; Executing the configure command will generate multiple different decoding libraries in subfolders of the configuration folder, resulting in multiple decoding files. Copy the multiple decoding files to the subfolder path of the decoding board.

3. The multi-channel video parallel soft decoding method according to claim 2, characterized in that, The specific steps for obtaining frame parameter data include: Obtain the data size parameter from the decode_nal_sei_prefix function; Assign the data size parameter to the parameter frame length; When the data size parameter is less than or equal to the frame size parameter: Copy the data in the context parameter array to the parameter array, with the copy length equal to the data size parameter.

4. The multi-channel video parallel soft decoding method according to claim 1, characterized in that, The steps for parallel decoding of the ffmpeg decoding library include: S5.1: Use the system function fopen to open the raw video data and distinguish each image file according to the channel number; S5.2: Determine if the file has reached the end; if so, proceed to S5.10; otherwise, proceed to the next step. S5.3: Use the system function fread to read the data to be decoded from the raw video data. The read data is stored in the array that stores the image data of each channel. The maximum number of bytes read each time is obtained, along with the actual number of array bytes read. S5.4: When the number of bytes in the array is 0, the starting address of the array storing the image data of each channel is saved into the pointer array; S5.5: Determine if the number of bytes in the array is zero. If it is zero, jump to S5.2; otherwise, proceed to the next step. S5.6: Use the library function av_parser_parse2 to obtain the complete frame from the pointer array, and the complete frame is stored in the data packet; S5.7: When the number of bytes in the complete frame is not zero, move the pointer array forward by a number equal to the number of bytes in the complete frame; decrease the number of bytes in the array by a number equal to the number of bytes in the complete frame; S5.8: Determine if the size of the data packet is zero. If yes, proceed to S5.10; otherwise, proceed to the next step. S5.9: Perform the decoding process; S5.10: End parallel decoding.

5. The multi-channel video parallel soft decoding method according to claim 4, characterized in that, The decoding process includes: S6.1: Using the context environment object and the data packet as parameters, use the library function avcode_send_packet to send the data packet to the specified decoder, obtain the result value, and assign the result value to the number of bytes of the complete frame; S6.2: Determine whether the result value is greater than or equal to zero. If yes, proceed to the next step; otherwise, proceed to S6.

19. S6.3: Determine whether the number of bytes in the complete frame is greater than or equal to zero. If yes, proceed to the next step; otherwise, proceed to S6.

7. S6.4: Using the context object and frame object as parameters, use the library function avcodec_receive_frame to obtain the decoded image frame, and store the function return value in the result value; S6.5: Determine whether the result value is an error code. If yes, proceed to S6.19; otherwise, proceed to the next step. S6.6: Determine whether the result value is less than zero. If so, proceed to S6.19; otherwise, jump to S6.

3. S6.7: Determine if the width and height of the image saved in the history are consistent with the currently decoded image frame. If they are consistent, proceed to S6.9; otherwise, proceed to the next step. S6.8: Obtain the anomaly flag of the image and set the anomaly flag to 1; S6.9: Determine if the anomaly flag is 1. If yes, proceed to the next step; otherwise, proceed to S6.

12. S6.10: Determine whether the context environment object of the current channel conversion format is empty. If yes, proceed to S6.12; otherwise, proceed to the next step. S6.11: Destroy the memory resources of the context object; S6.12: Save the current image's width and height in the history; S6.13: Using the target image's image format, width, height, and the converted image array as parameters, the library function avpicture_fill is used to fill the converted image array; S14: Use the library function sws_getContext to decode the current image's format conversion context object; S6.15: Clear the abnormality markers in the image; S6.16: Use the library function sws_scale to convert the size and format of the decoded image into the target image. The converted image is stored in the image array, and the conversion result is stored in the converted image result. S6.17: Determine whether the value of the converted image result is consistent with the height of the target image. If they are consistent, proceed to the next step; otherwise, proceed to S6.

19. S6.18: Conversion successful; S6.19: End the decoding process.

6. The multi-channel video parallel soft decoding method according to claim 5, characterized in that, The step of writing the decoded image data to a file and storing it locally includes: Obtain the data within the image array, where the data size is the product of the width and height; Obtain the parameter data stored in the context environment object; The image writing and saving operation is performed using the first array address of the image array, the row data size, the image frame width, the height, and the file name to be written as parameters, including: Use the system function fopen to open a file, grant write permissions, and obtain the file operation object; Use the library function fprint to write a string to a file object with a specified first line, width after a newline, height after a newline, and a fixed line size; The image frame array data is iterated in a loop. The total number of iterations is equal to the height of the line break. The current number of iterations is i. The system function fwrite is used to write to the file. The data written each time is the first array address of the image array, offset by i plus the size of the line data. Each data is written in 1-byte units, and the size of the line break is the size of the write. Close the image file after writing; Write the parameter data into a new save file and name the new save file with the specified file name; After you finish writing, close the new saved file.

Images