Media file transmission method, encapsulation method, and decapsulation method

Abstract

The application provides a media file transmission method, a packaging method and an unpackaging method. The method relates to the technical field of file transmission in the multimedia field, and comprises the following steps: acquiring a media resource description file, wherein the media resource description file comprises description files of a plurality of media files; selecting a description file of a target media file from the description files of the plurality of media files; the target media file comprises extension information of a target image in the target media file; sending a resource request based on the description file of the target media file, wherein the resource request is used for requesting the target media file; receiving the target media file; and presenting the extension information or presenting a control used for a user to perform an operation on the extension information in response to a selection operation performed on a target region of the target image. The media file transmission method provided by the application can improve the viewing experience.

Description

Technical Field

[0001] This application relates to the field of file transfer technology in the multimedia field, and more specifically, to media file transfer methods, encapsulation methods, and decapsulation methods. Background Technology

[0002] For text information within images, there are technologies that can recognize text in images in real time. However, this technology relies on the device itself, and the recognition of text in images is limited by the device, lacking portability. Similarly, existing technologies for displaying product information within images, such as product links as QR codes, also lack portability across application software. Furthermore, displaying QR codes within video images can lead to a poor viewing experience for users. Summary of the Invention

[0003] This application provides a media file transmission method, a packaging method, and a depackaging method, which can improve the viewing experience for users.

[0004] On the one hand, this application provides a media file transmission method, including:

[0005] Obtain the media resource description file, which includes descriptions for multiple media files;

[0006] Select the description file of the target media file from the description files of the multiple media files; the target media file includes extended information of the target image in the target media file, which is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file;

[0007] A resource request is sent based on the description file of the target media file, and the resource request is used to request the target media file;

[0008] Receive the target media file;

[0009] In response to a selection operation performed by the user on a target area of ​​the target image, the extended information is presented based on the extended information or a control for the user to perform an operation on the extended information is presented.

[0010] On the other hand, this application provides a media file transfer method, including:

[0011] Send a media resource description file to the client, which includes descriptions of multiple media files;

[0012] Receive a resource request sent by the client, which requests a target media file from the multiple media files; the target media file includes extended information of the target image in the target media file, which describes the text attributes or resource attributes of the target image, or, when the target media file is a video file, describes the resource metadata track or resource entity group of the target media file;

[0013] Send the target media file to the client.

[0014] On the other hand, this application provides a media file encapsulation method, including:

[0015] Obtain the bitstream obtained by encoding the target image or a video including the target image;

[0016] The stream is encapsulated to obtain a target media file that includes extended information about the target image. This extended information is used to describe the text attributes or resource attributes of the target image, or, when the target media file is a video file, to describe the resource metadata tracks or resource entity groups of the target media file.

[0017] On the other hand, this application provides a method for decapsulating media files, including:

[0018] Obtain the target media file;

[0019] Based on the decapsulation of the target media file, the extended information of the target image is obtained, or the bitstream of the target image and the extended information of the target image are obtained. The extended information is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file.

[0020] On the other hand, this application provides a client, including:

[0021] The acquisition unit is used to acquire a media resource description file, which includes description files for multiple media files.

[0022] The selection unit is used to select the description file of a target media file from the description files of the multiple media files; the target media file includes extended information of the target image in the target media file, the extended information is used to describe the text attributes or resource attributes of the target image, or the extended information is used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file;

[0023] The sending unit is configured to send a resource request based on the description file of the target media file, the resource request being used to request the target media file;

[0024] The receiving unit is used to receive the target media file;

[0025] A response unit is used to respond to a selection operation performed by a user on a target area of ​​the target image, and to present the extended information or a control for the user to perform an operation on the extended information based on the extended information.

[0026] On the other hand, this application provides a server, including:

[0027] The sending unit is used to send a media resource description file to the client, which includes description files for multiple media files;

[0028] The receiving unit is configured to receive a resource request sent by the client, the resource request being used to request a target media file from the plurality of media files; the target media file includes extended information of a target image in the target media file, the extended information being used to describe the text attributes or resource attributes of the target image, or the extended information being used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file;

[0029] The sending unit is also used to send the target media file to the client.

[0030] On the other hand, this application provides a server, including:

[0031] The acquisition unit is used to acquire the bitstream obtained by encoding the target image or a video including the target image.

[0032] The encapsulation unit is used to encapsulate the bitstream to obtain a target media file including extended information of the target image. The extended information is used to describe the text attributes or resource attributes of the target image, or, when the target media file is a video file, to describe the resource metadata tracks or resource entity groups of the target media file.

[0033] On the other hand, this application provides a client, including:

[0034] The acquisition unit acquires the target media file;

[0035] The decapsulation unit is used to obtain extended information of the target image or obtain the bitstream of the target image and extended information of the target image based on the decapsulation of the target media file. The extended information is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file.

[0036] On the other hand, this application provides a client, including:

[0037] Processor, adapted to implement computer instructions; and,

[0038] A computer-readable storage medium storing computer instructions adapted for loading by a processor and performing the methods of the first or fourth aspect described above.

[0039] On the other hand, this application provides a server, including:

[0040] Processor, adapted to implement computer instructions; and,

[0041] A computer-readable storage medium storing computer instructions adapted for loading by a processor and performing the methods of the second or third aspect described above.

[0042] On the other hand, embodiments of this application provide a computer-readable storage medium storing computer instructions, which, when read and executed by a processor of a computer device, cause the computer device to perform the media file transmission method provided in the first or fourth aspect described above.

[0043] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the media file method provided in the first or fourth aspect described above.

[0044] Based on the above technical solutions, this application introduces extended information for the target image in the target media file. Based on this, after receiving the target media file, the client can respond to the selection operation performed by the user on the target area of ​​the target image in the media file, and present the extended information or a control for the user to perform operations on the extended information. In other words, the extended information included in the target image can be viewed and portable, and can support more diverse image and video application forms, thereby improving the user experience. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic block diagram of a video encoding and decoding system provided in an embodiment of this application.

[0047] Figure 2 This is a schematic block diagram of a video encoder provided in an embodiment of this application.

[0048] Figure 3 This is a schematic block diagram of the decoding framework provided in the embodiments of this application.

[0049] Figure 4 This is a schematic flowchart of the media file transmission method provided in the embodiments of this application.

[0050] Figure 5 This is another illustrative flowchart of the media file transmission method provided in the embodiments of this application.

[0051] Figure 6 This is a schematic flowchart of the media file encapsulation method provided in the embodiments of this application.

[0052] Figure 7 This is a schematic flowchart of the media file decapsulation method provided in the embodiments of this application.

[0053] Figure 8 This is a schematic block diagram of the client provided in the embodiments of this application.

[0054] Figure 9 This is a schematic block diagram of the server provided in the embodiments of this application.

[0055] Figure 10 This is another schematic block diagram of the server provided in the embodiments of this application.

[0056] Figure 11 This is another schematic block diagram of the client provided in the embodiments of this application.

[0057] Figure 12 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0059] The solutions provided in this application may involve artificial intelligence technology.

[0060] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0061] It should be understood that artificial intelligence (AI) technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0062] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0063] This application's embodiments may relate to Computer Vision (CV) technology within artificial intelligence. Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision that uses cameras and computers to replace human eyes in recognizing, tracking, and measuring targets, and further performs image processing to transform the computer-processed images into those more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision researches related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0064] The solutions provided in this application may also relate to the field of vehicle technology.

[0065] For example, the client involved in this application can be a user terminal. Optionally, the user terminal includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0066] The solutions provided in this application may also involve encoding / decoding standards or technologies.

[0067] The solutions provided in this application can be applied to the field of digital video coding technology, such as image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, and real-time video encoding and decoding. Alternatively, the solutions provided in this application can be combined with audio video coding standards (AVS), second-generation AVS standards (AVS2), or third-generation AVS standards (AVS3). Specifically, this includes, but is not limited to, the H.264 / Audio Video Coding (AVC) standard, the H.265 / High Efficiency Video Coding (HEVC) standard, and the H.266 / Versatile Video Coding (VVC) standard. Alternatively, the solutions provided in this application can be incorporated into other proprietary or industry standards. For example, they may specifically include ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), and also include Scalable Video Codec (SVC) and Multi-View Video Codec (MVC) extensions.

[0068] Furthermore, the solution provided in this application can be used for lossy compression of images, or for lossless compression of images. This lossless compression can be visually lossless compression or mathematically lossless compression.

[0069] To facilitate understanding, let's first combine... Figure 1 The video encoding and decoding system involved in the embodiments of this application will be described.

[0070] Figure 1 This is a schematic block diagram of a video encoding / decoding system 100 according to an embodiment of this application. It should be noted that... Figure 1 This is merely an example; the video encoding and decoding system in this application includes, but is not limited to, [other systems]. Figure 1 As shown. Figure 1 As shown, the video encoding / decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device encodes (can be understood as compressing) video data to generate a bitstream and transmits the bitstream to the decoding device. The decoding device decodes the bitstream generated by the encoding device to obtain the decoded video data.

[0071] Encoding device 110 can be understood as a device with video encoding capabilities, and decoding device 120 can be understood as a device with video decoding capabilities. That is, in this embodiment, encoding device 110 and decoding device 120 include a wider range of devices, such as smartphones, desktop computers, mobile computing devices, laptops (e.g., tablet computers), tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, etc. In some embodiments, encoding device 110 can transmit encoded video data (such as a bitstream) to decoding device 120 via channel 130. Channel 130 may include one or more media and / or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120.

[0072] In one example, channel 130 includes one or more communication media that enable encoding device 110 to transmit encoded video data directly to decoding device 120 in real time. In this example, encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum; optionally, the communication media may also include wired communication media, such as one or more physical transmission lines. In another example, channel 130 includes a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various local access data storage media, such as optical discs, DVDs, flash memory, etc. In this example, decoding device 120 can retrieve the encoded video data from the storage medium. In another example, channel 130 may include a storage server that can store the video data encoded by encoding device 110. In this example, decoding device 120 can download the stored encoded video data from the storage server. Optionally, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0073] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0074] In some embodiments, in addition to the video encoder 112 and the output interface 113, the encoding device 110 may also include a video source 111. The video source 111 may include at least one of a video acquisition device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate the video data.

[0075] Video encoder 112 encodes video data from video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains the encoding information of the pictures or picture sequences in the form of a bitstream. The encoding information may include encoded image data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. The SPS may contain parameters applied to one or more sequences. The PPS may contain parameters applied to one or more pictures. A syntax structure refers to a set of zero or more syntax elements arranged in a specified order within the bitstream.

[0076] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data can also be stored on a storage medium or a storage server for subsequent retrieval by the decoding device 120.

[0077] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122. In some embodiments, in addition to the input interface 121 and the video decoder 122, the decoding device 120 may also include a display device 123. The input interface 121 includes a receiver and / or a modem. The input interface 121 can receive encoded video data through channel 130. The video decoder 122 is used to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123.

[0078] Display device 123 displays the decoded video data. Display device 123 may be integrated with decoding device 120 or external to decoding device 120. Display device 123 may include various display devices, such as liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other types of display devices.

[0079] It should be noted that, Figure 1 This is merely one example of the present application, and the technical solutions of the embodiments of the present application are not limited to this. Figure 1 For example, the technology of this application can also be applied to one-sided video encoding or one-sided video decoding.

[0080] The video coding framework involved in the embodiments of this application is described below.

[0081] Figure 2 This is a schematic block diagram of the video encoder 200 provided in an embodiment of this application. It should be understood that the video encoder 200 can be used for lossy compression of images or lossless compression of images. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0082] The video encoder 200 can be applied to image data in luminance-chrominance (YCbCr, YUV) format.

[0083] For example, the video encoder 200 reads video data and divides each frame into several coding tree units (CTUs). In some examples, CTUs may be called "tree blocks," "largest coding unit" (LCU), or "coding treeblock" (CTB). Each CTU can be associated with a pixel block of equal size within the image. Each pixel can correspond to one luminance (luma) sample and two chrominance (chroma) samples. Therefore, each CTU can be associated with one luminance sample block and two chrominance sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU can be further divided into several coding units (CUs) for encoding. CUs can be rectangular or square blocks. The CU can be further divided into prediction units (PU) and transform units (TU), thus separating encoding, prediction, and transformation for more flexible processing. In one example, the CTU is divided into CUs using a quadtree structure, and the CUs are further divided into TUs and PUs using a quadtree structure.

[0084] The video encoder 200 supports various PU sizes. Assuming a specific CU size of 2N×2N, the video encoder and decoder can support PU sizes of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. The video encoder and decoder can also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0085] like Figure 2 As shown, the video encoder 200 may include:

[0086] The video encoder 200 includes a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filtering unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may contain more, fewer, or different functional components.

[0087] The prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Because there is strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video encoding and decoding techniques to eliminate spatial redundancy between adjacent pixels. Because there is strong similarity between adjacent frames in a video, inter-frame prediction is used in video encoding and decoding techniques to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0088] Inter-frame prediction unit 211 can be used for inter-frame prediction. Inter-frame prediction can refer to image information from different frames. Inter-frame prediction uses motion information to find reference blocks from reference frames and generates prediction blocks based on the reference blocks to eliminate temporal redundancy. The frames used for inter-frame prediction can be P-frames and / or B-frames. P-frames refer to forward prediction frames, and B-frames refer to bidirectional prediction frames. Motion information includes a list of reference frames, the reference frame index, and motion vectors. Motion vectors can be integer-pixel or fractional-pixel. If the motion vector is fractional-pixel, interpolation filtering needs to be used in the reference frame to create the required fractional-pixel blocks. Here, the integer-pixel or fractional-pixel blocks in the reference frame found based on the motion vectors are called reference blocks. Some techniques directly use the reference blocks as prediction blocks, while others process the reference blocks to generate prediction blocks. Processing the reference blocks to generate prediction blocks can also be understood as using the reference blocks as prediction blocks and then processing them to generate new prediction blocks. The most commonly used inter-frame prediction methods include geometric partitioning mode (GPM) in the VVC video codec standard and angular weighted prediction (AWP) in the AVS3 video codec standard. These two intra-frame prediction modes share common principles.

[0089] The intra-prediction unit 212 refers only to information from the same frame image to predict pixel information within the current code image block, thereby eliminating spatial redundancy. The frame used for intra-prediction can be an I-frame. HEVC uses Planar, DC, and 33 angle modes for a total of 35 prediction modes. VVC uses Planar, DC, and 65 angle modes for a total of 67 prediction modes. AVS3 uses DC, Plane, Bilinear, and 63 angle modes for a total of 66 prediction modes. In some embodiments, the intra-prediction unit 212 can be implemented using intra-block copying and intra-string copying techniques.

[0090] The residual unit 220 can generate a residual block of the CU based on the pixel block of the CU and the prediction block of the PU of the CU. For example, the residual unit 220 can generate a residual block of the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel block of the CU and the corresponding sample in the prediction block of the PU of the CU.

[0091] Transform / quantization unit 230 quantizes the transform coefficients. Transform / quantization unit 230 quantizes the transform coefficients associated with the TU of the CU based on the quantization parameter (QP) value associated with the CU. Video encoder 200 can adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU. Inverse transform / quantization unit 240 applies inverse quantization and inverse transform to the quantized transform coefficients to reconstruct the residual block from the quantized transform coefficients.

[0092] The reconstruction unit 250 can add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by the prediction unit 210 to produce a reconstructed image block associated with the TU. By reconstructing the sampled blocks of each TU of the CU in this way, the video encoder 200 can reconstruct the pixel blocks of the CU.

[0093] The loop filtering unit 260 can perform deblocking filtering operations to reduce the block effect of pixel blocks associated with the CU. In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive compensation / adaptive loop filtering (SAO / ALF) unit, wherein the deblocking filtering unit is used to remove the block effect and the SAO / ALF unit is used to remove the ringing effect.

[0094] The decoded image buffer 270 can store reconstructed pixel blocks. The inter-frame prediction unit 211 can use a reference image containing the reconstructed pixel blocks to perform inter-frame prediction on PUs of other images. In addition, the intra-frame prediction unit 212 can use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra-frame prediction on other PUs in the same image as the CU.

[0095] Entropy coding unit 280 can receive quantized transform coefficients from transform / quantization unit 230. Entropy coding unit 280 can perform one or more entropy coding operations on the quantized transform coefficients to produce entropy-coded data.

[0096] Figure 3 This is a schematic block diagram of the decoding framework 300 provided in the embodiments of this application.

[0097] like Figure 3 As shown, the video decoder 300 includes:

[0098] The video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filtering unit 350, and a decoded image buffer 360. It should be noted that the video decoder 300 may contain more, fewer, or different functional components.

[0099] The video decoder 300 can receive a bitstream. The entropy decoding unit 310 can parse the bitstream to extract syntax elements from it. As part of parsing the bitstream, the entropy decoding unit 310 can parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the dequantization / transform unit 330, the reconstruction unit 340, and the loop filtering unit 350 can decode the video data based on the syntax elements extracted from the bitstream, i.e., generate decoded video data.

[0100] The prediction unit 320 includes an intra-frame prediction unit 321 and an inter-frame prediction unit 322.

[0101] Intra-prediction unit 321 can perform intra-prediction to generate prediction blocks for the PU. Intra-prediction unit 321 can use an intra-prediction mode to generate prediction blocks for the PU based on pixel blocks of spatially adjacent PUs. Intra-prediction unit 321 can also determine the intra-prediction mode of the PU based on one or more syntax elements parsed from the bitstream.

[0102] Inter-frame prediction unit 322 can construct a first reference image list (list 0) and a second reference image list (list 1) based on the syntax elements parsed from the bitstream. Furthermore, if the PU uses inter-frame prediction coding, the entropy decoding unit 310 can parse the motion information of the PU. Inter-frame prediction unit 322 can determine one or more reference blocks of the PU based on the motion information of the PU. Inter-frame prediction unit 322 can generate prediction blocks for the PU based on one or more reference blocks of the PU.

[0103] The dequantization / transform unit 330 reversibly quantizes (i.e., dequantizes) the transform coefficients associated with the TU. The dequantization / transform unit 330 can use the QP value associated with the CU of the TU to determine the degree of quantization. After dequantizing the transform coefficients, the dequantization / transform unit 330 can apply one or more inverse transforms to the dequantized transform coefficients to produce a residual block associated with the TU.

[0104] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add the sample of the residual block to the corresponding sample of the prediction block to reconstruct the pixel block of the CU, thereby obtaining the reconstructed image block.

[0105] The loop filter unit 350 can perform deblocking filtering operations to reduce the block effect of pixel blocks associated with the CU.

[0106] The video decoder 300 can store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 can use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0107] The basic process of video encoding and decoding is as follows:

[0108] At the encoding end, a frame of image is divided into blocks. For the current block, prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. Residual unit 220 can calculate a residual block based on the prediction block and the original block of the current block, i.e., the difference between the prediction block and the original block of the current block. This residual block can also be called residual information. This residual block is transformed and quantized by transform / quantization unit 230 to remove information that is not sensitive to the human eye, thereby eliminating visual redundancy. Optionally, the residual block before transformation and quantization by transform / quantization unit 230 can be called a temporal residual block, and the temporal residual block after transformation and quantization by transform / quantization unit 230 can be called a frequency residual block or a frequency domain residual block. Entropy coding unit 280 receives the quantized change coefficients output by change quantization unit 230 and can perform entropy coding on the quantized change coefficients to output a bitstream. For example, entropy coding unit 280 can eliminate character redundancy based on the target context model and the probability information of the binary bitstream. Optionally, in this application, the current block can be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block can also be referred to as the predicted image block or the image prediction block, and the reconstructed image block can also be referred to as the reconstruction block or the image reconstruction block.

[0109] At the decoding end, the entropy decoding unit 310 can parse the bitstream to obtain the prediction information and quantization coefficient matrix of the current block. The prediction unit 320 uses intra-frame prediction or inter-frame prediction to generate the prediction block of the current block based on the prediction information. The dequantization / transform unit 330 uses the quantization coefficient matrix obtained from the bitstream to perform dequantization and inverse transform on the quantization coefficient matrix to obtain the residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain the reconstructed block. The reconstructed blocks form the reconstructed image. The loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or based on the blocks to obtain the decoded image. The encoding end also needs similar operations to the decoding end to obtain the decoded image. This decoded image can also be called the reconstructed image, which can be used as a reference frame for inter-frame prediction in subsequent frames.

[0110] It should be noted that the block partitioning information determined at the encoding end, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, are carried in the bitstream when necessary. The decoding end determines the same block partitioning information, prediction, transform, quantization, entropy coding, and loop filtering mode information or parameter information as the encoding end by parsing the bitstream and analyzing existing information, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0111] It should also be understood that the above is the basic process of a video codec under a block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or process may be optimized. This application is applicable to the basic process of a video codec under this block-based hybrid coding framework, but is not limited to this framework and process.

[0112] The above text combined Figures 1 to 3 The encoding / decoding systems, encoders, and decoders applicable to the embodiments of this application are described. From the perspective of signal acquisition methods, video signals can be divided into two types: those captured by a camera and those generated by a computer. Due to differences in statistical characteristics, their corresponding compression encoding methods may also differ, but they will all perform one or more of the following operations and processes on the input raw video signal:

[0113] 1) Block Partition Structure: The input image is divided into several non-overlapping processing units based on its size. Each processing unit undergoes a similar compression operation. This processing unit is called a CTU or LCU. Further subdivisions can be made below the CTU to obtain one or more basic coding units, called CUs. Each CU is the most basic element in a coding process. The following describes the various coding methods that can be used for each CU.

[0114] 2) Predictive Coding: This includes intra-frame prediction and inter-frame prediction. The original video signal is predicted using a selected reconstructed video signal to obtain the residual video signal. The encoder needs to determine the most suitable predictive coding mode from among many possible modes for the current CU and inform the decoder.

[0115] a. Intra-frame prediction: The predicted signal comes from a region within the same image that has already been encoded and reconstructed.

[0116] b. Inter-frame prediction: The predicted signal comes from other encoded images (called reference images) that are different from the current image.

[0117] 3) Transform & Quantization: The residual video signal undergoes transform operations such as DFT and DCT to convert the signal to the transform domain, resulting in transform coefficients. In the transform domain, the signal undergoes further lossy quantization, losing some information to make the quantized signal more suitable for compression. Some video coding standards may offer more than one transform option; therefore, the encoder needs to select one transform for the current encoding CU and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually leads to greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually leads to less distortion and a higher bitrate.

[0118] 4) Entropy Coding or Statistical Coding: The quantized transform domain signal is statistically compressed and encoded based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected mode and motion vectors, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless coding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) or Content Adaptive Binary Arithmetic Coding (CABAC).

[0119] 5) Loop Filtering: After the encoded image undergoes inverse quantization, inverse transform, and prediction compensation operations (the reverse of operations 2-4 above), a reconstructed decoded image is obtained. Compared to the original image, the reconstructed image differs in some information due to the influence of quantization, resulting in distortion. Filtering the reconstructed image, such as deblocking, SAO, or ALF filters, can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent encoded images to predict future signals, the above filtering operations are also called loop filtering, or filtering operations within the coding loop.

[0120] During the decoding process, for each CU, after obtaining the compressed bitstream, the decoder first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transform to obtain the residual signal. On the other hand, based on the known coding mode information, the prediction signal corresponding to that CU can be obtained. The two are then added together to obtain the reconstructed signal. Finally, the reconstructed value of the decoded image needs to undergo a loop filtering operation to generate the final output signal.

[0121] In video media transmission systems, the encoded video stream needs to be encapsulated to store it in a more easily parsed structure on the computing device. The information added during the video file encapsulation stage typically improves the efficiency of storage, association, and transmission of the video file on the server or intermediate transmission nodes, as well as the decoding efficiency at the terminal. At the user terminal, decapsulation and decoding are required for the final video content to be displayed. To facilitate understanding of the solution provided in this application, the media file transmission process is described below.

[0122] The content creation device encodes and encapsulates the acquired images or videos to form image or video encapsulation files. These encapsulation files can be a single media file or multiple media files. Furthermore, the content creation device records the metadata of the image or video encapsulation file using Media Presentation Description (MPD) information according to the file format requirements of the media files. For example, it may use a Media Presentation Description (MPD) file to record the metadata of the image or video encapsulation file. Metadata here refers to all information related to the presentation of the image or video, including descriptive information about the media content, descriptive information about the window, and signaling information related to the presentation of the media content. The content creation device then sends the MPD file to the content consumption device, enabling the content consumption device to request the image or video encapsulation file (i.e., the media file) based on the relevant descriptive information in the MDP file. Specifically, the image or video encapsulation file can be sent from the content creation device to the content consumption device via a transmission mechanism. For example, the transmission mechanism could be Dynamic Adaptive Streaming over HTTP (DASH) or Smart Media Transport (SMT).

[0123] Content production devices encapsulate compressed images or videos into a series of small media files based on the Hypertext Transfer Protocol (HTTP). Each media file contains a configurable amount of time, typically short, but each file has multiple bitrate versions, allowing for more precise network-adaptive downloading. Content consumption devices adaptively select and download the highest bitrate version the network can support, based on current network conditions. This ensures media quality while avoiding playback stuttering or rebuffering caused by excessively high bitrates. Based on this, dynamic and seamless adaptation to real-time network conditions is achieved, providing high-quality playback content with fewer stutters and significantly improving the user experience. In other words, bitrate switching is done on a per-media-file basis. When network bandwidth is good, the content consumption device can request the media file with the corresponding higher bitrate for the specified time; conversely, when bandwidth is poor, the device downloads the media file with the corresponding lower bitrate. Because media files of different qualities are time-aligned, the transitions between them are smooth and natural. Optionally, media files can also be called media segments.

[0124] Media presentation descriptions (MPDs) provide a precise description of media files. An MPD file can be an Extensible Markup Language (XML) file that comprehensively describes all information about the media file, including various audio and video parameters, file duration, bitrate, resolution, and corresponding Uniform Resource Locator (URL). Content consuming devices first download and parse the MPD file to obtain the media file best suited to their performance and bandwidth. An MPD file can contain one or more adaptation sets. For example, one adaptation set might contain multiple video media with the same video content at different bitrates, while another might contain multiple video media with the same audio content at different bitrates. An adaptation set can contain multiple representations. A representation can include a combination of one or more media contents; for example, a video file at a certain resolution can be considered a representation.

[0125] The content consumption device sends a request to the server based on the MPD file's URL to retrieve the MPD file. The device first parses the MPD file to obtain its content information, including video resolution, video content type, segmentation, frame rate, bitrate, and the URLs of various media files. By analyzing this content information, the device selects a suitable media file based on the current network conditions and the client's buffer size. Then, it sends a request to the content production device to download the corresponding media file based on the media URL and streams it. Upon receiving the media file, the content consumption device decapsulates it to obtain the raw bitstream, which is then sent to the decoder for playback.

[0126] Figure 4 This is a schematic flowchart of a media file transfer method 400 provided in an embodiment of this application. The executing entity of this method 400 can be a client. For example, the client can be a desktop computer, mobile computing device, laptop (e.g., tablet computer), set-top box, smartphone or other handheld device, television, camera, display device, digital media player, video game console, in-vehicle computer, or similar devices. Furthermore, the client can be... Figure 1 The encoding device 110 shown or Figure 2 The video encoder 200 shown.

[0127] like Figure 4 As shown, the media file transfer method 400 may include:

[0128] S410, Obtain a media resource description file, which includes description files for multiple media files;

[0129] S420, Select the description file of the target media file from the description files of the multiple media files; The target media file includes extended information of the target image in the target media file, which is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file.

[0130] S430, a resource request is sent based on the description file of the target media file, the resource request being used to request the target media file;

[0131] S440, receive the target media file;

[0132] S450, in response to a selection operation performed on a target area of ​​the target image, presents the extended information or presents controls for the user to perform operations on the extended information.

[0133] For example, after obtaining the media resource description file sent by the server, the client selects a target media file containing extended information from the descriptions of multiple media files included in the media resource description file; then requests the target media file from the server, and upon receiving the target media file sent by the server, in response to the user's selection operation on a target region in the target image of the target media file, presents the extended information or presents controls for performing operations on the extended information. Optionally, the extended information may be metadata related to extended information added by the server to certain image frames of the image or video during the file encapsulation process according to application requirements, based on the encoded image or video bitstream.

[0134] In other words, on the one hand, the server can generate multiple media files based on local images or videos; on the other hand, the server can generate a media resource description file including additional extended information based on local images or videos. For example, the media resource description file is an MPD file. For example, the media resource description file includes the extended information and media configuration information. The media configuration information is configuration information used to obtain media files. For example, the media configuration information may include video resolution, video content type, segmentation, frame rate, bitrate, and the URL address of each media file. The server sends the media resource description file to the client. After receiving the media resource description file, the client selects the target media file from the multiple media files based on the description information of the multiple media files included in the media resource description file, and sends a resource request to the server to request the target media file. After receiving the resource request from the client, the server sends the media file to the client. Since the target media file in this application includes extended information, on the client side, in response to the user's selection operation on the target area of ​​the target image, the extended information or a control for the user to perform operations on the extended information can be presented.

[0135] Based on the above technical solutions, this application introduces extended information for the target image in the target media file. Based on this, after receiving the target media file, the client can respond to the selection operation performed by the user on the target area of ​​the target image in the media file, and present the extended information or a control for the user to perform operations on the extended information. In other words, the extended information included in the target image can be viewed and portable, and can support more diverse image and video application forms, thereby improving the user experience.

[0136] It should be noted that the extended information involved in this application includes information on the target image and / or information obtained by extending the information on the target image. That is to say, the extended information includes information displayed on the target image and information obtained by extending the information displayed on the target image. For example, the information obtained by extending the information on the target image includes, but is not limited to, information such as the location, type, quantity, and identification of the information on the target image. This application does not limit the specific type of the extended information.

[0137] For example, the information on the target image includes text information contained within the target image and / or resource information or resource links contained within the target image that can be further used for acquisition. Of course, the information on the target image can also be other types of information, and this application does not specifically limit it.

[0138] For example, if the extended information includes text information displayed on the target image, the client responds to a selection operation performed on the target area of ​​the target image and presents the text information; if the extended information includes resource links displayed on the target image, the client responds to a selection operation performed on the target area of ​​the target image and presents the resource link or the page or information indicated by the resource link.

[0139] For example, if the extended information includes text information displayed on the target image, the client, in response to a selection operation performed on the target area of ​​the target image, presents a control for the user to perform a copy operation on the text information; if the extended information includes resource links displayed on the target image, the client, in response to a selection operation performed on the target area of ​​the target image, presents a control for the user to perform a jump operation on the resource link, or presents a control for the user to perform a copy operation on the resource link.

[0140] Furthermore, it should be understood that the media files involved in this application can be used in the storage, encapsulation, and decapsulation processes of images or videos.

[0141] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute describing text information contained in the target image and the location of the text information; wherein, the image item text attribute includes at least one of the following:

[0142] Text string count, used to indicate the number of text strings contained in the target image;

[0143] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0144] Text information, used to indicate the text contained in the target image.

[0145] For example, the image item text attribute can be carried within the ISOBMFF data box; that is, the ISOBMFF data box can be extended using the image item text attribute to obtain an extended ISOBMFF data box. Optionally, the extended ISOBMFF data box can be defined in the following format:

[0146] Data box type: 'itip';

[0147] Attribute type: Project description attribute;

[0148] Contained in: ItemPropertyContainerBox;

[0149] Mandatory (per item): No;

[0150] Quantity (per item): 0 or 1.

[0151] For example, the image item text attribute can be implemented as the following syntax element:

[0152]

[0153] Here, `num_text_strings` indicates the number of text strings contained in the target image. `region_vertex_x` and `region_vertex_y` indicate the coordinates of the bottom-left vertex of the region containing the text string in the target image, in pixels, for example. `region_width` and `region_height` indicate the width and height of the region containing the text string in the target image, in pixels, for example. `text_info` can be a null-terminated string; `text_info` indicates the text contained in the target image.

[0154] In some embodiments, the target media file is an image file, and the extended information is an image item associated resource attribute describing information about resources associated with the target image; wherein the image item associated resource attribute includes at least one of the following:

[0155] Associated resource count, used to indicate the number of resources associated with the target image;

[0156] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0157] Associated area information, used to indicate the location information of this specific area;

[0158] Associated resource locators are Uniform Resource Locators (URLs) used to indicate the resources associated with the target image.

[0159] For example, the image item associated resource attribute can be carried in the ISOBMFF data box; that is, the ISOBMFF data box can be extended using the image item associated resource attribute to obtain an extended ISOBMFF data box. Optionally, the extended ISOBMFF data box can be defined in the following format:

[0160] Data box type: 'iarp';

[0161] Attribute type: Project description attribute;

[0162] Contained in: ItemPropertyContainerBox;

[0163] Mandatory (per item): No;

[0164] Quantity (per item): 0 or 1.

[0165] For example, the image item text attribute can be implemented as the following syntax element:

[0166]

[0167]

[0168] Here, `num_associated_resources` indicates the number of resources associated with the target image. A value of 1 for `associated_region_flag` indicates that a specific region in the target image is associated with a corresponding resource. A value of 0 for `associated_region_flag` indicates that the entire region of the target image is associated with a corresponding resource. `region_vertex_x` and `region_vertex_y` indicate the coordinates of the lower-left vertex of the specific region, in pixels. `region_width` and `region_height` indicate the width and height of the specific region, in pixels. `associated_resource_url` can be a null-terminated string; `associated_resource_url` indicates the URL of the resource associated with the target image.

[0169] In some embodiments, the target media file is a video file, and the extended information is an image-associated resource metadata track describing information about resources associated with the video track of the target media file; wherein the image-associated resource metadata track includes at least one of the following:

[0170] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0171] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0172] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0173] Associated area information, used to indicate the location information of this specific area;

[0174] Associated resource locator, which is a Uniform Resource Locator URL that indicates the resource associated with the video track or the images in the video track.

[0175] For example, the image-associated resource metadata track can be carried in the description information of the video track. That is to say, the description information of the video track can be extended using the image-associated resource metadata track to obtain an extended video track.

[0176] For example, the image-associated resource metadata track can be implemented as the following syntax elements:

[0177]

[0178]

[0179] Here, `num_associated_resources` indicates the number of resources associated with the video track or the images within it. `resource_id` indicates the identifier of the resource associated with the video track or the images within it. `associated_resource_url` can be a null-terminated string; it indicates the URL of the resource associated with the video track or the images within it. A value of 1 for `associated_region_flag` indicates that a specific region within the images of the video track is associated with the corresponding resource. A value of 0 for `associated_region_flag` indicates that the entire region within the images of the video track is associated with the corresponding resource. `region_vertex_x` and `region_vertex_y` indicate the coordinates of the lower-left vertex of the specific region, in pixels. `region_width` and `region_height` indicate the width and height of the specific region, in pixels.

[0180] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0181] In this embodiment, the image-associated resource metadata track is used to describe information about the resources associated with the video track. The sample entry of this associated resource metadata track can be used to indicate the number and URL of all resources associated with the video track. The current video sample of the associated resource metadata track may include the number of resources associated with the current video sample and the identifier of the resources associated with the current video sample. Furthermore, the current video sample of the associated resource metadata track may also include the region where the resources associated with the current video sample are located. Based on this, different numbers of resources or different resources can be associated with different video samples in the target video file, further improving the user experience.

[0182] In some embodiments, the target media file is a video file, and the extended information is an image-associated resource entity group that describes information about resources associated with the entity group of the target media file; wherein the image-associated resource entity group includes at least one of the following:

[0183] Image associated resource box, used to indicate the resources associated with this entity group;

[0184] Associated resource count, used to indicate the number of resources associated with this entity group;

[0185] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0186] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0187] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0188] For example, the image-associated resource entity group can be carried in the description information of the entity group. That is to say, the description information of the entity group can be extended using the image-associated resource entity group to obtain an extended entity group.

[0189] For example, the image-associated resource entity group can be implemented as the following syntax element:

[0190]

[0191]

[0192] The `ImageAssociatedResourceBox` is used to associate resources that include this entity group (e.g., multiple images of the target image). `num_associated_resources` indicates the number of resources associated with the entity group. `associated_resource_url` can be a null-terminated string; it indicates the URL information of the associated resources. `num_entities_in_group` indicates the number of entities mapped to this entity group. `entity_id` indicates the identifier of the entity mapped to this entity group; for example, `entity_id` could be the value of `item_ID` for a specific project or `track_ID` for a specific track.

[0193] In some embodiments, the target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following:

[0194] The extension type indicates whether the extended information is text information or resource information.

[0195] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0196] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0197] In this embodiment, by introducing an extended information application data box, the client can determine whether the target media file supports the application of the extended information or whether the target media file includes the extended information before selecting the target media file. This helps the client select a suitable media file and thus improves the user experience.

[0198] For example, the extended information application data box can be an ISOBMFF data box. Optionally, the extended ISOBMFF data box can be defined in the following format:

[0199] Data box type: 'ieap';

[0200] Included in: MetaBox;

[0201] Mandatory (per item): No;

[0202] Quantity (per item): 0 or 1.

[0203] For example, the extended information application data box can be implemented as the following syntax element:

[0204]

[0205]

[0206] The `extension_type` parameter indicates the type of extended information. A value of 0 indicates text information, and a value of 1 indicates associated resources. Other values ​​are reserved. A value of 1 for `image_info_flag` indicates that the image items in the target media file contain extended information. A value of 0 indicates that the image items in the target media file do not contain extended information. A value of 1 for `video_frame_info_flag` indicates that the video track in the target media file contains extended information. A value of 0 for `video_frame_info_flag` indicates that the video track in the file does not contain extended information. Optionally, when `extension_type` is 0, the value of `video_frame_info_flag` is also 0.

[0207] In some embodiments, the description file of the target media file includes an application descriptor of the target media file; wherein the application descriptor of the target media file includes at least one of the following:

[0208] The extension type indicates whether the extended information is text information or resource information.

[0209] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0210] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0211] For example, the description file of the target media file includes an application descriptor for the target media file, which means that the description file of the target media file supports displaying the extended information on the image in the target media file or displaying controls for the user to perform operations on the extended information. Alternatively, the description file of the target media file includes an application descriptor for the target media file, which means that the target media file includes the extended information.

[0212] For example, the application descriptor of the target media file may exist at the representation level or the adaptation set level, indicating whether the target media file supports rendering the extended information in the target image or rendering controls for the user to perform operations on the extended information.

[0213] For example, the attributes and semantics of the application descriptor of the target media file can be implemented as shown in Table 1:

[0214] Table 1

[0215]

[0216]

[0217] As shown in Table 1, for this target media file, it can be determined whether the media file includes extended information by defining a descriptor. When the application descriptor is included, the type of extended information included in the media file and whether the extended information is included in the image project or in the video track can be determined by defining the attributes of the application descriptor.

[0218] Of course, this application only uses the application descriptor of the target media file as an example. In other alternative embodiments, the media resource description file may also include the application descriptor of each of some or all of the media files in the plurality of media files described by the media resource description file. This application does not specifically limit this. In addition, the application descriptor may also be referred to as the application descriptor element.

[0219] In some embodiments, S420 may include:

[0220] Determine whether the client supports displaying information in an image or displaying controls for users to perform actions on the information;

[0221] If the client supports displaying information in an image or displaying controls for users to perform operations on the information, then the description file including the application descriptor among the multiple media files is selected as the description file for the target media file.

[0222] For example, when a client determines that it supports presenting information in an image or presenting controls for user interaction with the information, it selects the media file containing the application descriptor as the description file for the target media file. Furthermore, when a client determines that it does not support presenting information in an image or presenting controls for user interaction with the information, it may select the media file excluding the application descriptor as the description file for the target media file.

[0223] In this embodiment, introducing a descriptor simplifies the process of the client selecting the target media file, which helps to improve the user experience.

[0224] The solution of this application will be described below with reference to specific embodiments.

[0225] Example 1:

[0226] The server adds the text information contained within the image to the image file after encoding. Additionally, based on the local image file, the server generates a media resource description file including extra extended information and sends this file to the client. This media resource description information includes descriptions for multiple media files.

[0227] After receiving the media resource description file, the client requests the required media files based on the description files of multiple media files included in the media resource description information.

[0228] For example, if client A does not support applications with extended information, it can select a description file that does not contain an application descriptor from the description files of the multiple media files, and then request and consume the corresponding media file from the server.

[0229] For example, if client B supports applications with extended information, it can select the description file containing the application descriptor from among the multiple media file description files, and then request and consume the corresponding media file from the server; for example, the attributes of the application descriptor included in the selected description file are as follows:

[0230] {ImageExtensionApplication@extension_type=0; ImageExtensionApplication@image_info_flag=1; ImageExtensionApplication@video_frame_info_flag=0}.

[0231] Correspondingly, after receiving the image file F1, during the decapsulation process, the client can obtain the text information in the image file F1 and the location information of the region where the text information is located through ImageTextInfoProperty, namely {num_text_strings=1;region:(10,10)width=20;height=5;string=“hello”}; furthermore, when the user selects the corresponding region while viewing the image file F1, the client combines the extended information to provide the user with further information feedback and presentation, such as providing a copy operation for the string “hello”, etc.

[0232] Example 2:

[0233] The server adds the text information contained within the image to the image file after encoding. Additionally, based on the local image file, the server generates a media resource description file including extra extended information and sends this file to the client. This media resource description information includes descriptions for multiple media files.

[0234] After receiving the media resource description file, the client requests the required media files based on the description files of multiple media files included in the media resource description information.

[0235] For example, if client A does not support applications with extended information, it can select a description file that does not contain an application descriptor from the description files of the multiple media files, and then request and consume the corresponding media file from the server.

[0236] For example, if client B supports applications with extended information, it can select the description file containing the application descriptor from among the multiple media file description files, and then request and consume the corresponding media file from the server; for example, the attributes of the application descriptor included in the selected description file are as follows:

[0237] {ImageExtensionApplication@extension_type=1; ImageExtensionApplication@image_info_flag=1; ImageExtensionApplication@video_frame_info_flag=0}.

[0238] Correspondingly, after receiving the image file F2, during the decompression process, the client can obtain the resources associated with the image file F2 through ImageAssociatedResourceProperty. This means obtaining the number of associated resources and their links, for example, {num_associated_resources = 2; associated_region_flag = 0; string1 = "xx.xxx.com"; string2 = "xx.xxx.com"}. Furthermore, when a user clicks on an image while viewing it, the client combines this extended information to provide further feedback and presentation, such as redirecting to a browsing page or product page.

[0239] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.

[0240] It should also be understood that, in the various method embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. Specifically, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0241] The above text combined Figure 4 This application describes in detail the media file transfer method provided in its embodiments from the client's perspective. The following section, in conjunction with... Figure 5 The media file transmission method provided in the embodiments of this application will be described from the perspective of the server.

[0242] Figure 5 This is a schematic flowchart of a media file transfer method 500 provided in an embodiment of this application. The executing entity of this method 500 can be a server. For example, the server can be a desktop computer, mobile computing device, laptop (e.g., tablet computer), tablet computer, set-top box, smartphone or other handheld device, television, camera, display device, digital media player, video game console, in-vehicle computer, or the like. Furthermore, the server could be... Figure 1 The decoding device 120 shown Figure 3 The video decoder 300 shown.

[0243] S510, sends a media resource description file to the client, which includes description files for multiple media files;

[0244] S520, receive a resource request sent by the client, the resource request is used to request a target media file among the multiple media files; the target media file includes extended information of the target image in the target media file, the extended information is used to describe the text attributes or resource attributes of the target image, or the extended information is used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file;

[0245] S530 sends the target media file to the client.

[0246] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute describing text information contained in the target image and the location of the text information; wherein, the image item text attribute includes at least one of the following:

[0247] Text string count, used to indicate the number of text strings contained in the target image;

[0248] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0249] Text information, used to indicate the text contained in the target image.

[0250] In some embodiments, the target media file is an image file, and the extended information is an image item associated resource attribute describing information about resources associated with the target image; wherein the image item associated resource attribute includes at least one of the following:

[0251] Associated resource count, used to indicate the number of resources associated with the target image;

[0252] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0253] Associated area information, used to indicate the location information of this specific area;

[0254] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0255] In some embodiments, the target media file is a video file, and the extended information is an image-associated resource metadata track describing information about resources associated with the video track of the target media file; wherein the image-associated resource metadata track includes at least one of the following:

[0256] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0257] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0258] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0259] Associated area information, used to indicate the location information of this specific area;

[0260] Associated resource locator, which is a Uniform Resource Locator URL that indicates the resource associated with the video track or the images in the video track.

[0261] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0262] In some embodiments, the target media file is a video file, and the extended information is an image-associated resource entity group that describes information about resources associated with the entity group of the target media file; wherein the image-associated resource entity group includes at least one of the following:

[0263] Image associated resource box, used to indicate the resources associated with this entity group;

[0264] Associated resource count, used to indicate the number of resources associated with this entity group;

[0265] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0266] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0267] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0268] In some embodiments, the target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following:

[0269] The extension type indicates whether the extended information is text information or resource information.

[0270] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0271] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0272] In some embodiments, the description file of the target media file includes an application descriptor of the target media file; wherein the application descriptor of the target media file includes at least one of the following:

[0273] The extension type indicates whether the extended information is text information or resource information.

[0274] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0275] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0276] It should be noted that the steps and terms in method 500 provided in this application can be referred to the relevant steps or terms in method 400. To avoid repetition, they will not be repeated here.

[0277] The following is combined Figure 6 and Figure 7 The media file encapsulation method and media file decapsulation method provided in this application are described.

[0278] Figure 6 This is a schematic flowchart of the media file encapsulation method 540 provided in the embodiments of this application.

[0279] like Figure 6 As shown, the method 540 includes:

[0280] S541, Obtain the bitstream obtained by encoding the target image or a video including the target image;

[0281] S542, the bitstream is encapsulated to obtain a target media file including extended information of the target image. The extended information is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file.

[0282] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information, or the extended information is an image item associated resource attribute used to describe the information of the resources associated with the target image.

[0283] The text attribute of the image item includes at least one of the following:

[0284] Text string count, used to indicate the number of text strings contained in the target image;

[0285] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0286] Text information, used to indicate the text contained in the target image;

[0287] The associated resource attributes of the image project include at least one of the following:

[0288] Associated resource count, used to indicate the number of resources associated with the target image;

[0289] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0290] Associated area information, used to indicate the location information of this specific area;

[0291] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0292] In some embodiments, the target media file is a video file, and the extended information is either an image associated resource metadata track describing information about resources associated with the video track of the target media file, or an image associated resource entity group describing information about resources associated with the entity group of the target media file.

[0293] The image-associated resource metadata track includes at least one of the following:

[0294] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0295] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0296] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0297] Associated area information, used to indicate the location information of this specific area;

[0298] Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the images in the video track;

[0299] The image-associated resource entity group includes at least one of the following:

[0300] Image associated resource box, used to indicate the resources associated with this entity group;

[0301] Associated resource count, used to indicate the number of resources associated with this entity group;

[0302] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0303] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0304] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0305] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0306] In some embodiments, the target media file may also include an extended information application data box;

[0307] The extended information application data box includes at least one of the following:

[0308] The extension type indicates whether the extended information is text information or resource information.

[0309] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0310] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0311] Figure 7 This is a schematic flowchart of the media file decapsulation method 550 provided in the embodiments of this application.

[0312] like Figure 7 As shown, the method 550 may include:

[0313] S551, Obtain the target media file;

[0314] S551, based on the decapsulation of the target media file to obtain the extended information of the target image or to obtain the bitstream of the target image and the extended information of the target image, the extended information is used to describe the text attributes or resource attributes of the target image, or the extended information is used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file.

[0315] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information, or the extended information is an image item associated resource attribute used to describe the information of the resources associated with the target image.

[0316] The text attribute of the image item includes at least one of the following:

[0317] Text string count, used to indicate the number of text strings contained in the target image;

[0318] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0319] Text information, used to indicate the text contained in the target image;

[0320] The associated resource attributes of the image project include at least one of the following:

[0321] Associated resource count, used to indicate the number of resources associated with the target image;

[0322] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0323] Associated area information, used to indicate the location information of this specific area;

[0324] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0325] In some embodiments, the target media file is a video file, and the extended information is either an image associated resource metadata track describing information about resources associated with the video track of the target media file, or an image associated resource entity group describing information about resources associated with the entity group of the target media file.

[0326] The image-associated resource metadata track includes at least one of the following:

[0327] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0328] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0329] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0330] Associated area information, used to indicate the location information of this specific area;

[0331] Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the images in the video track;

[0332] The image-associated resource entity group includes at least one of the following:

[0333] Image associated resource box, used to indicate the resources associated with this entity group;

[0334] Associated resource count, used to indicate the number of resources associated with this entity group;

[0335] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0336] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0337] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0338] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0339] In some embodiments, the target media file may also include an extended information application data box;

[0340] The extended information application data box includes at least one of the following:

[0341] The extension type indicates whether the extended information is text information or resource information.

[0342] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0343] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0344] It should be understood that the terminology in the media file encapsulation method 540 and the media file decapsulation method 550 provided in this application can be referred to the relevant terminology in method 400 or method 500. To avoid repetition, it will not be repeated here.

[0345] The above text combined Figures 4 to 7 The method embodiments of this application have been described in detail below, in conjunction with... Figures 8 to 12 The following describes in detail the device embodiments of this application.

[0346] Figure 8 This is a schematic flowchart of the client 600 provided in the embodiments of this application.

[0347] like Figure 8 As shown, the client 600 may include:

[0348] The acquisition unit 610 is used to acquire a media resource description file, which includes description files for multiple media files;

[0349] Selection unit 620 is used to select a description file of a target media file from the description files of the plurality of media files; the target media file includes extended information of a target image in the target media file, the extended information being used to describe the text attributes or resource attributes of the target image, or the extended information being used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file;

[0350] Sending unit 630 is configured to send a resource request based on the description file of the target media file, the resource request being used to request the target media file;

[0351] The receiving unit 640 is used to receive the target media file;

[0352] The response unit 650 is configured to, in response to a selection operation performed on a target area of ​​the target image, present the extended information or present a control for the user to perform an operation on the extended information.

[0353] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information, or the extended information is an image item associated resource attribute used to describe the information of the resources associated with the target image.

[0354] The text attribute of the image item includes at least one of the following:

[0355] Text string count, used to indicate the number of text strings contained in the target image;

[0356] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0357] Text information, used to indicate the text contained in the target image;

[0358] The associated resource attributes of the image project include at least one of the following:

[0359] Associated resource count, used to indicate the number of resources associated with the target image;

[0360] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0361] Associated area information, used to indicate the location information of this specific area;

[0362] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0363] In some embodiments, the target media file is a video file, and the extended information is either an image associated resource metadata track describing information about resources associated with the video track of the target media file, or an image associated resource entity group describing information about resources associated with the entity group of the target media file.

[0364] The image-associated resource metadata track includes at least one of the following:

[0365] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0366] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0367] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0368] Associated area information, used to indicate the location information of this specific area;

[0369] Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the images in the video track;

[0370] The image-associated resource entity group includes at least one of the following:

[0371] Image associated resource box, used to indicate the resources associated with this entity group;

[0372] Associated resource count, used to indicate the number of resources associated with this entity group;

[0373] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0374] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0375] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0376] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0377] In some embodiments, the target media file may also include an extended information application data box;

[0378] The extended information application data box includes at least one of the following:

[0379] The extension type indicates whether the extended information is text information or resource information.

[0380] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0381] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0382] In some embodiments, the description file of the target media file includes an application descriptor of the target media file; wherein the application descriptor of the target media file includes at least one of the following:

[0383] The extension type indicates whether the extended information is text information or resource information.

[0384] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0385] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0386] In some embodiments, the selection unit 620 is specifically used for:

[0387] Determine whether the client supports displaying information in an image or displaying controls for users to perform actions on the information;

[0388] If the client supports displaying information in an image or displaying controls for users to perform operations on the information, then the description file including the application descriptor among the multiple media files is selected as the description file for the target media file.

[0389] Figure 9 This is a schematic flowchart of the server 700 provided in the embodiments of this application.

[0390] like Figure 9 As shown, the server 700 may include:

[0391] The sending unit 710 is used to send a media resource description file to the client, the media resource description file including description files of multiple media files;

[0392] The receiving unit 720 is configured to receive a resource request sent by the client, the resource request being used to request a target media file among the multiple media files; the target media file includes extended information of a target image in the target media file, the extended information being used to describe the text attributes or resource attributes of the target image, or the extended information being used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file;

[0393] The sending unit 710 is also used to send the target media file to the client.

[0394] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information, or the extended information is an image item associated resource attribute used to describe the information of the resources associated with the target image.

[0395] The text attribute of the image item includes at least one of the following:

[0396] Text string count, used to indicate the number of text strings contained in the target image;

[0397] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0398] Text information, used to indicate the text contained in the target image;

[0399] The associated resource attributes of the image project include at least one of the following:

[0400] Associated resource count, used to indicate the number of resources associated with the target image;

[0401] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0402] Associated area information, used to indicate the location information of this specific area;

[0403] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0404] In some embodiments, the target media file is a video file, and the extended information is either an image associated resource metadata track describing information about resources associated with the video track of the target media file, or an image associated resource entity group describing information about resources associated with the entity group of the target media file.

[0405] The image-associated resource metadata track includes at least one of the following:

[0406] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0407] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0408] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0409] Associated area information, used to indicate the location information of this specific area;

[0410] Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the images in the video track;

[0411] The image-associated resource entity group includes at least one of the following:

[0412] Image associated resource box, used to indicate the resources associated with this entity group;

[0413] Associated resource count, used to indicate the number of resources associated with this entity group;

[0414] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0415] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0416] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0417] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0418] In some embodiments, the target media file may also include an extended information application data box;

[0419] The extended information application data box includes at least one of the following:

[0420] The extension type indicates whether the extended information is text information or resource information.

[0421] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0422] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0423] In some embodiments, the description file of the target media file includes an application descriptor of the target media file; wherein the application descriptor of the target media file includes at least one of the following:

[0424] The extension type indicates whether the extended information is text information or resource information.

[0425] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0426] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0427] Figure 10 This is a schematic block diagram of the server 730 provided in the embodiments of this application.

[0428] like Figure 10 As shown, the server 730 may include:

[0429] The acquisition unit 731 is used to acquire a bitstream obtained by encoding a target image or a video including the target image.

[0430] The encapsulation unit 732 is used to encapsulate the bitstream to obtain a target media file including extended information of the target image. The extended information is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file.

[0431] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information, or the extended information is an image item associated resource attribute used to describe the information of the resources associated with the target image.

[0432] The text attribute of the image item includes at least one of the following:

[0433] Text string count, used to indicate the number of text strings contained in the target image;

[0434] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0435] Text information, used to indicate the text contained in the target image;

[0436] The associated resource attributes of the image project include at least one of the following:

[0437] Associated resource count, used to indicate the number of resources associated with the target image;

[0438] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0439] Associated area information, used to indicate the location information of this specific area;

[0440] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0441] In some embodiments, the target media file is a video file, and the extended information is either an image associated resource metadata track describing information about resources associated with the video track of the target media file, or an image associated resource entity group describing information about resources associated with the entity group of the target media file.

[0442] The image-associated resource metadata track includes at least one of the following:

[0443] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0444] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0445] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0446] Associated area information, used to indicate the location information of this specific area;

[0447] Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the images in the video track;

[0448] The image-associated resource entity group includes at least one of the following:

[0449] Image associated resource box, used to indicate the resources associated with this entity group;

[0450] Associated resource count, used to indicate the number of resources associated with this entity group;

[0451] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0452] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0453] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0454] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0455] In some embodiments, the target media file may also include an extended information application data box;

[0456] The extended information application data box includes at least one of the following:

[0457] The extension type indicates whether the extended information is text information or resource information.

[0458] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0459] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0460] Figure 11 This is a schematic block diagram of the client 740 provided in the embodiments of this application.

[0461] like Figure 11 As shown, the client 740 may include:

[0462] Acquisition unit 741 acquires the target media file;

[0463] The decapsulation unit 742 is used to obtain extended information of the target image or obtain the bitstream of the target image and extended information of the target image based on the decapsulation of the target media file. The extended information is used to describe the text attributes or resource attributes of the target image, or when the target media file is a video file, the extended information is used to describe the resource metadata track or resource entity group of the target media file.

[0464] In some embodiments, the target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information, or the extended information is an image item associated resource attribute used to describe the information of the resources associated with the target image.

[0465] The text attribute of the image item includes at least one of the following:

[0466] Text string count, used to indicate the number of text strings contained in the target image;

[0467] Text string location information, used to indicate the location of the region containing the text string in the target image;

[0468] Text information, used to indicate the text contained in the target image;

[0469] The associated resource attributes of the image project include at least one of the following:

[0470] Associated resource count, used to indicate the number of resources associated with the target image;

[0471] Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource;

[0472] Associated area information, used to indicate the location information of this specific area;

[0473] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with the target image.

[0474] In some embodiments, the target media file is a video file, and the extended information is either an image associated resource metadata track describing information about resources associated with the video track of the target media file, or an image associated resource entity group describing information about resources associated with the entity group of the target media file.

[0475] The image-associated resource metadata track includes at least one of the following:

[0476] Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track;

[0477] Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track;

[0478] Associated region identifier, used to indicate that a specific region of an image in the video track is associated with a corresponding resource, or used to indicate that the entire region of an image video frame in the video track is associated with a corresponding resource;

[0479] Associated area information, used to indicate the location information of this specific area;

[0480] Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the images in the video track;

[0481] The image-associated resource entity group includes at least one of the following:

[0482] Image associated resource box, used to indicate the resources associated with this entity group;

[0483] Associated resource count, used to indicate the number of resources associated with this entity group;

[0484] The associated resource locator is the Uniform Resource Locator URL that indicates the resource associated with this entity group.

[0485] The number of entities in the entity group indicates the number of entities mapped to that entity group;

[0486] Entity identifier, used to indicate the identifier of the entity mapped to this entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

[0487] In some embodiments, the sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

[0488] In some embodiments, the target media file may also include an extended information application data box;

[0489] The extended information application data box includes at least one of the following:

[0490] The extension type indicates whether the extended information is text information or resource information.

[0491] Image information identifier, used to indicate whether the image items of the target media file include this extended information;

[0492] Video information identifier, used to indicate whether the video track of the target media file includes this extended information.

[0493] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details are omitted here. Specifically, Figure 8 The client 600 shown can correspond to the corresponding subject in the method 400 of the embodiments of this application, that is, the foregoing and other operations and / or functions of each unit in the client 600 are respectively for implementing the corresponding process in the method 400. Figure 9 The server 700 shown can correspond to a corresponding entity in the method 500 of the embodiments of this application, and the foregoing and other operations and / or functions of each unit in the server 700 are respectively for implementing the corresponding processes in the method 500. Similarly, Figure 10The server 730 shown may correspond to the corresponding subject in the method 540 of the embodiments of this application, and the foregoing and other operations and / or functions of each unit in the server 730 are respectively for implementing the corresponding process in the method 540. Figure 11 The client 740 shown can correspond to the corresponding subject in the method 550 of the embodiments of this application, that is, the foregoing and other operations and / or functions of each unit in the client 740 are respectively for implementing the corresponding process in the method 550. Exemplarily, the client 600 and the client 740 can be the same client or different clients, and the server 700 and the server 730 can be the same server or different clients. This application does not make specific limitations in this regard.

[0494] It should also be understood that the various units in the client 600 (or client 740) or server 700 (or server 730) involved in the embodiments of this application can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This can achieve the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the client 600 (or client 740) or server 700 (or server 730) may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by multiple units working together. According to another embodiment of this application, the client 600 (or client 740) or server 700 (or server 730) involved in the embodiments of this application can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), and implementing the media file transfer method, media file encapsulation method, or media file decapsulation method provided in the embodiments of this application. The computer program can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device through the computer-readable storage medium, and run therein to implement the corresponding methods of the embodiments of this application.

[0495] In other words, the units mentioned above can be implemented in hardware, in software instructions, or in a combination of hardware and software. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software in the decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads the information in the memory and completes the steps in the above method embodiments in conjunction with its hardware.

[0496] Figure 12 This is a schematic structural diagram of the electronic device 800 provided in the embodiments of this application.

[0497] like Figure 12 As shown, the electronic device 800 includes at least a processor 810 and a computer-readable storage medium 820. The processor 810 and the computer-readable storage medium 820 can be connected via a bus or other means. The computer-readable storage medium 820 stores a computer program 821, which includes computer instructions. The processor 810 executes the computer instructions stored in the computer-readable storage medium 820. The processor 810 is the computing and control core of the electronic device 800, and is adapted to implement one or more computer instructions, specifically to load and execute one or more computer instructions to achieve a corresponding method flow or function.

[0498] As an example, processor 810 may also be referred to as a central processing unit (CPU). Processor 810 may include, but is not limited to: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0499] As an example, the computer-readable storage medium 820 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer-readable storage medium located remotely from the aforementioned processor 810. Specifically, the computer-readable storage medium 820 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0500] like Figure 12 As shown, the electronic device 800 may also include a transceiver 830.

[0501] The processor 810 can control the transceiver 830 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 830 may include a transmitter and a receiver. The transceiver 830 may further include antennas, and the number of antennas may be one or more.

[0502] It should be understood that the various components in the communication device 800 are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0503] In one implementation, the electronic device 800 may be a client, encoding end, encoder, or encoding framework as described in the embodiments of this application; the computer-readable storage medium 820 stores first computer instructions; the processor 810 loads and executes the first computer instructions stored in the computer-readable storage medium 820 to implement the corresponding steps in the media file transfer method 400 or the media file decapsulation method 550 provided in the embodiments of this application; in other words, the first computer instructions in the computer-readable storage medium 820 are loaded and executed by the processor 810 to implement the corresponding steps, which will not be described again here to avoid repetition.

[0504] In one implementation, the electronic device 800 may be a server, decoding terminal, decoder, or decoding framework as described in the embodiments of this application; the computer-readable storage medium 820 stores second computer instructions; the processor 810 loads and executes the second computer instructions stored in the computer-readable storage medium 820 to implement the corresponding steps in the media file transfer method 500 or media file encapsulation method 540 provided in the embodiments of this application; in other words, the second computer instructions in the computer-readable storage medium 820 are loaded and executed by the processor 810 to implement the corresponding steps, which will not be described again here to avoid repetition.

[0505] According to another aspect of this application, embodiments of this application also provide a computer-readable storage medium (Memory), which is a memory device in an electronic device 800 for storing programs and data. For example, a computer-readable storage medium 820. It is understood that the computer-readable storage medium 820 here may include both the built-in storage medium in the electronic device 800 and extended storage media supported by the electronic device 800. The computer-readable storage medium provides storage space that stores the operating system of the electronic device 800. Furthermore, the storage space also stores one or more computer instructions suitable for loading and execution by a processor 810, which may be one or more computer programs 821 (including program code).

[0506] According to another aspect of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. For example, computer program 821. In this case, electronic device 800 may be a computer, and processor 810 reads the computer instructions from computer-readable storage medium 820, executes the computer instructions, causing the computer to perform the media file transfer method, media file encapsulation method, or media file decapsulation method provided in the various optional embodiments described above.

[0507] In other words, when implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes of the embodiments of this application are run or the functions of the embodiments of this application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0508] Those skilled in the art will recognize that the units and process steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0509] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for transferring media files, characterized in that, The method is applied to a client, and the method includes: Obtain a media resource description file, which includes description files for multiple media files; Selecting a description file for a target media file from among the description files of the plurality of media files includes: determining whether the client supports presenting information in an image or presenting controls for users to perform operations on the information; if the client supports presenting information in an image or presenting controls for users to perform operations on the information, then selecting the description file from the plurality of media files that includes the application descriptor and determining it as the description file for the target media file; the target media file includes extended information of the target image in the target media file; the extended information is used to describe the text attributes or resource attributes of the target image, or the extended information is used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file; the extended information includes at least one of text information, resource information, and resource links, and at least one of the location, type, quantity, and identifier of information on the target image; A resource request is sent based on the description file of the target media file, and the resource request is used to request the target media file; Receive the target media file; In response to a selection operation performed on a target region of the target image, the extended information is presented or a control for the user to perform an operation on the extended information is presented, wherein the presentation includes: a control for displaying text information, resource links, a copy operation, or a jump operation; The description file of the target media file includes an application descriptor for the target media file. This application descriptor can exist at the presentation level or the adaptation level, and is used to indicate whether the target media file supports presenting the extended information in the target image or presenting controls for user operations on the extended information. The application descriptor of the target media file includes at least one of the following: an extension type, indicating whether the extended information is text information or resource information; an image information identifier, indicating whether the image items of the target media file include the extended information; and a video information identifier, indicating whether the video track of the target media file includes the extended information. The target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following: an extension type, used to indicate that the type of the extended information is text information or resource information; an image information identifier, used to indicate whether the image items of the target media file include the extended information; and a video information identifier, used to indicate whether the video track of the target media file includes the extended information.

2. The method according to claim 1, characterized in that, The target media file is an image file, and the extended information is either an image item text attribute used to describe the text information contained in the target image and the location of the text information, or an image item associated resource attribute used to describe the information of the resources associated with the target image. The image item text attribute includes at least one of the following: Text string count, used to indicate the number of text strings contained in the target image; Text string location information, used to indicate the location information of the area where the text string contained in the target image is located; Text information, used to indicate the text contained in the target image; The image project associated resource attributes include at least one of the following: Associated resource count, used to indicate the number of resources associated with the target image; Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource; Associated region information, used to indicate the location information of the specific region; The associated resource locator is a Uniform Resource Locator URL that indicates the resource associated with the target image.

3. The method according to claim 1, characterized in that, The target media file is a video file, and the extended information is either an image associated resource metadata track used to describe the resources associated with the video track of the target media file, or an image associated resource entity group used to describe the resources associated with the entity group of the target media file. The image-associated resource metadata track includes at least one of the following: Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track; Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track; Associated region identifier, used to indicate that a specific region in the image of the video track is associated with a corresponding resource, or used to indicate that the entire region of the image video frame in the video track is associated with a corresponding resource; Associated region information, used to indicate the location information of the specific region; Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the image in the video track; The image-associated resource entity group includes at least one of the following: Image-associated resource box, used to indicate the resources associated with the entity group; Associated resource count, used to indicate the number of resources associated with the entity group; Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the entity group; The number of entities in the entity group indicates the number of entities mapped to the entity group; An entity identifier is used to indicate the identifier of the entity mapped to the entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

4. The method according to claim 3, characterized in that, The sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

5. A method for transferring media files, characterized in that, The method is applicable to servers, and the method includes: Sending a media resource description file to a client, the media resource description file including description files for multiple media files, so that the client selects a description file for a target media file from the description files of the multiple media files, includes: the client determining whether the client supports presenting information in an image or presenting controls for the user to perform operations on the information; if the client supports presenting information in an image or presenting controls for the user to perform operations on the information, then selecting the description file of the multiple media files that includes the application descriptor and determining it as the description file of the target media file; The system receives a resource request sent by the client based on the description file of the target media file. The resource request is used to request a target media file from the plurality of media files. The target media file includes extended information of a target image in the target media file. The extended information is used to describe the text attributes or resource attributes of the target image, or, when the target media file is a video file, the extended information describes the resource metadata track or resource entity group of the target media file. The extended information includes at least one of text information, resource information, and resource links, as well as at least one of the location, type, quantity, and identifier of information on the target image. Send the target media file to the client so that the client, in response to a selection operation performed on a target area of ​​the target image, presents the extended information or presents controls for the user to perform operations on the extended information, wherein the presentation includes: controls for displaying text information, resource links, copy operations, or jump operations; The description file of the target media file includes an application descriptor for the target media file. This application descriptor can exist at the presentation level or the adaptation level, and is used to indicate whether the target media file supports presenting the extended information in the target image or presenting controls for user operations on the extended information. The application descriptor of the target media file includes at least one of the following: an extension type, indicating whether the extended information is text information or resource information; an image information identifier, indicating whether the image items of the target media file include the extended information; and a video information identifier, indicating whether the video track of the target media file includes the extended information. The target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following: an extension type, used to indicate that the type of the extended information is text information or resource information; an image information identifier, used to indicate whether the image items of the target media file include the extended information; and a video information identifier, used to indicate whether the video track of the target media file includes the extended information.

6. A media file encapsulation method, characterized in that, The method is applied to a server, and the method includes: Obtain the bitstream obtained by encoding the target image or a video including the target image; The bitstream is encapsulated to obtain a target media file including extended information of the target image. The extended information is used to describe the text attributes or resource attributes of the target image, or, when the target media file is a video file, to describe the resource metadata track or resource entity group of the target media file. The extended information includes at least one of text information, resource information, and resource links, as well as at least one of the location, type, quantity, and identifier of information on the target image. The description file of the target media file includes an application descriptor for the target media file. This application descriptor can exist at the presentation level or the adaptation level, and is used to indicate whether the target media file supports presenting the extended information in the target image or presenting controls for user operations on the extended information. The application descriptor of the target media file includes at least one of the following: an extension type, indicating whether the extended information is text information or resource information; an image information identifier, indicating whether the image items of the target media file include the extended information; and a video information identifier, indicating whether the video track of the target media file includes the extended information. The target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following: an extension type, used to indicate that the type of the extended information is text information or resource information; an image information identifier, used to indicate whether the image items of the target media file include the extended information; and a video information identifier, used to indicate whether the video track of the target media file includes the extended information.

7. The method according to claim 6, characterized in that, The target media file is an image file, and the extended information is an image item text attribute used to describe the text information contained in the target image and the location of the text information; The image item text attribute includes at least one of the following: Text string count, used to indicate the number of text strings contained in the target image; Text string location information, used to indicate the location information of the area where the text string contained in the target image is located; Text information, used to indicate the text contained in the target image.

8. The method according to claim 6, characterized in that, The target media file is an image file, and the extended information is an image item associated resource attribute used to describe the resources associated with the target image. The image item associated resource attribute includes at least one of the following: Associated resource count, used to indicate the number of resources associated with the target image; Associated region identifier, used to indicate that a specific region in the target image is associated with a corresponding resource or that the entire region of the target image is associated with a corresponding resource; Associated region information, used to indicate the location information of the specific region; The associated resource locator is a Uniform Resource Locator URL that indicates the resource associated with the target image.

9. The method according to claim 6, characterized in that, The target media file is a video file, and the extended information is an image-associated resource metadata track used to describe the resources associated with the video track of the target media file; The image-associated resource metadata track includes at least one of the following: Associated resource quantity, used to indicate the number of resources associated with the video track or the images in the video track; Associated resource identifier, used to indicate the identifier of the resource associated with the video track or the image in the video track; Associated region identifier, used to indicate that a specific region in the image of the video track is associated with a corresponding resource, or used to indicate that the entire region of the image video frame in the video track is associated with a corresponding resource; Associated region information, used to indicate the location information of the specific region; Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the video track or the image in the video track.

10. The method according to claim 6, characterized in that, The target media file is a video file, and the extended information is an image associated resource entity group used to describe the information of the resources associated with the entity group of the target media file; The image-associated resource entity group includes at least one of the following: Image-associated resource box, used to indicate the resources associated with the entity group; Associated resource count, used to indicate the number of resources associated with the entity group; Associated resource locator, used to indicate the Uniform Resource Locator URL of the resource associated with the entity group; The number of entities in the entity group indicates the number of entities mapped to the entity group; An entity identifier is used to indicate the identifier of the entity mapped to the entity group. The value of the entity identifier is either the identifier of the project or the identifier of the track.

11. The method according to claim 10, characterized in that, The sample entry of the associated resource metadata track includes the number of associated resources and the associated resource locator; the current video sample of the associated resource metadata track includes the number of resources associated with the current video sample, the identifier of the resources associated with the current video sample, and the region corresponding to the resources associated with the current video sample.

12. A method for decapsulating media files, characterized in that, The method is applied to a client, and the method includes: Obtain the target media file; Based on the decapsulation of the target media file to obtain extended information of the target image, or obtaining the bitstream of the target image and the extended information of the target image, the extended information is used to describe the text attributes or resource attributes of the target image, or the extended information is used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file; the extended information includes at least one of text information, resource information, and resource links, as well as at least one of the location, type, quantity, and identifier of information on the target image; The description file of the target media file includes an application descriptor for the target media file. This application descriptor can exist at the presentation level or the adaptation level, and is used to indicate whether the target media file supports presenting the extended information in the target image or presenting controls for user operations on the extended information. The application descriptor of the target media file includes at least one of the following: an extension type, indicating whether the extended information is text information or resource information; an image information identifier, indicating whether the image items of the target media file include the extended information; and a video information identifier, indicating whether the video track of the target media file includes the extended information. The target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following: an extension type, used to indicate that the type of the extended information is text information or resource information; an image information identifier, used to indicate whether the image items of the target media file include the extended information; and a video information identifier, used to indicate whether the video track of the target media file includes the extended information.

13. A client application, characterized in that, include: The acquisition unit is used to acquire a media resource description file, wherein the media resource description file includes description files for multiple media files; The selection unit is configured to select a description file for a target media file from the description files of the plurality of media files, including: determining whether the client supports presenting information in an image or presenting controls for the user to perform operations on the information; if the client supports presenting information in an image or presenting controls for the user to perform operations on the information, then selecting the description file of the plurality of media files that includes the application descriptor and determining it as the description file of the target media file; the target media file includes extended information of the target image in the target media file; the extended information is used to describe the text attributes or resource attributes of the target image, or the extended information is used to describe the resource metadata track or resource entity group of the target media file when the target media file is a video file; the extended information includes at least one of text information, resource information, and resource links, and at least one of the location, type, quantity, and identifier of information on the target image; The sending unit is configured to send a resource request based on the description file of the target media file, wherein the resource request is used to request the target media file; A receiving unit is configured to receive the target media file; A response unit is configured to, in response to a selection operation performed on a target area of ​​the target image, present the extended information or present controls for a user to perform an operation on the extended information, wherein the presentation includes: controls for displaying text information, resource links, copy operations, or jump operations; The description file of the target media file includes an application descriptor for the target media file. This application descriptor can exist at the presentation level or the adaptation level, and is used to indicate whether the target media file supports presenting the extended information in the target image or presenting controls for user operations on the extended information. The application descriptor of the target media file includes at least one of the following: an extension type, indicating whether the extended information is text information or resource information; an image information identifier, indicating whether the image items of the target media file include the extended information; and a video information identifier, indicating whether the video track of the target media file includes the extended information. The target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following: an extension type, used to indicate that the type of the extended information is text information or resource information; an image information identifier, used to indicate whether the image items of the target media file include the extended information; and a video information identifier, used to indicate whether the video track of the target media file includes the extended information.

14. A server, characterized in that, include: A sending unit is configured to send a media resource description file to a client, the media resource description file including description files for multiple media files, so that the client selects a description file for a target media file from the description files of the multiple media files, including: the client determining whether the client supports presenting information in an image or presenting controls for the user to perform operations on the information; if the client supports presenting information in an image or presenting controls for the user to perform operations on the information, then selecting the description file including the application descriptor from the multiple media files and determining it as the description file for the target media file; A receiving unit is configured to receive a resource request sent by the client based on a description file of the target media file. The resource request is used to request a target media file from the plurality of media files. The target media file includes extended information of a target image within the target media file. The extended information is used to describe the text attributes or resource attributes of the target image, or, when the target media file is a video file, the extended information describes a resource metadata track or resource entity group of the target media file. The extended information includes at least one of text information, resource information, and resource links, as well as at least one of the location, type, quantity, and identifier of information on the target image. The sending unit is further configured to send the target media file to the client, so that the client, in response to a selection operation performed on a target area of ​​the target image, presents the extended information or presents controls for the user to perform operations on the extended information, wherein the presentation includes: controls for displaying text information, resource links, copy operations, or jump operations; The description file of the target media file includes an application descriptor for the target media file. This application descriptor can exist at the presentation level or the adaptation level, and is used to indicate whether the target media file supports presenting the extended information in the target image or presenting controls for user operations on the extended information. The application descriptor of the target media file includes at least one of the following: an extension type, indicating whether the extended information is text information or resource information; an image information identifier, indicating whether the image items of the target media file include the extended information; and a video information identifier, indicating whether the video track of the target media file includes the extended information. The target media file further includes an extended information application data box; wherein the extended information application data box includes at least one of the following: an extension type, used to indicate that the type of the extended information is text information or resource information; an image information identifier, used to indicate whether the image items of the target media file include the extended information; and a video information identifier, used to indicate whether the video track of the target media file includes the extended information.

15. An electronic device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the method as claimed in any one of claims 1 to 4, the method as claimed in claim 5, the method as claimed in any one of claims 6 to 11, or the method as claimed in claim 12.

16. A computer-readable storage medium, characterized in that, Used to store computer programs that cause a computer to perform the method as claimed in any one of claims 1 to 4, the method as claimed in claim 5, the method as claimed in any one of claims 6 to 11, or the method as claimed in claim 12.

Images