Streaming media transmission methods and related products

Abstract

This application belongs to the field of audio and video technology, specifically relating to a streaming media transmission method. The method includes: parsing streaming media transmission signaling to obtain a sorting relationship descriptor, the sorting relationship descriptor representing sorting information within a specified range; selecting a target streaming media object to be received from the specified range according to the sorting relationship descriptor; and sending a streaming media transmission request for the target streaming media object to a data source. This application can ensure the stability and reliability of streaming media data transmission and improve the transmission efficiency of streaming media data.

Description

Technical Field

[0001] This application belongs to the field of audio and video technology, specifically relating to a streaming media transmission method, streaming media transmission device, computer-readable medium, electronic device, and computer program product. Background Technology

[0002] Streaming media refers to a technology and process that compresses a series of media data, transmits it in segments over the internet, and streams audio and video in real time for viewing. This technology allows data packets to be sent like a flowing stream. The real-time segmented transmission characteristic of streaming media means that users can obtain media data in real time without downloading the complete media file. However, it also places high demands on the user's device performance and network conditions. Once device performance degrades or network conditions are not ideal, media data transmission can easily be blocked. Therefore, streaming media transmission generally suffers from problems such as low transmission efficiency and poor stability. Summary of the Invention

[0003] The purpose of this application is to provide a streaming media transmission method, streaming media transmission device, computer-readable medium, electronic device, and computer program product, which at least to some extent overcomes the technical problems of low transmission efficiency and poor transmission stability in related technologies.

[0004] According to one aspect of the embodiments of this application, a streaming media transmission method is provided, the method comprising: parsing streaming media transmission signaling to obtain a sorting relationship descriptor, the sorting relationship descriptor being used to represent sorting information within a specified range; selecting a target streaming media object to be received from the specified range according to the sorting relationship descriptor; and sending a streaming media transmission request for the target streaming media object to a data source.

[0005] According to one aspect of the embodiments of this application, a streaming media transmission method is provided, the method comprising: sending streaming media transmission signaling to a data receiver, the streaming media transmission signaling carrying a sorting relationship descriptor, the sorting relationship descriptor being used to represent sorting information within a specified range; and returning a target streaming media object to the data receiver according to a streaming media transmission request sent by the data receiver, the target streaming media object being an object selected from the specified range according to the sorting relationship descriptor.

[0006] According to one aspect of the embodiments of this application, a streaming media transmission apparatus is provided, the apparatus comprising:

[0007] The parsing module is configured to parse streaming media transmission signaling to obtain a sorting relationship descriptor, which is used to represent sorting information within a specified range; The selected module is configured to select a target streaming media object to be received from the specified range based on the sorting relationship descriptor; The sending module is configured to send a streaming media transmission request for the target streaming media object to the data source.

[0008] According to one aspect of the embodiments of this application, a streaming media transmission apparatus is provided, the apparatus comprising: The signaling sending module is configured to send streaming media transmission signaling to the data receiver. The streaming media transmission signaling carries a sorting relationship descriptor, which is used to represent sorting information within a specified range. The object sending module is configured to return a target streaming media object to the data receiver based on the streaming media transmission request sent by the data receiver. The target streaming media object is an object selected from the specified range according to the sorting relationship descriptor.

[0009] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the streaming media transmission method as described above.

[0010] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing a computer program; wherein the processor is configured to perform the streaming media transmission method as described above by executing the computer program.

[0011] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the streaming media transmission method as described above.

[0012] In the technical solution provided in the embodiments of this application, by dividing the streaming media object into multiple different sorting ranges and sorting the streaming media object according to different sorting criteria within a specific sorting range, the flexibility and diversity of streaming media object sorting can be improved, thereby providing a variety of options for the network transmission of streaming media objects. Therefore, the stability and reliability of streaming media data transmission can be guaranteed, and the transmission efficiency of streaming media data can be improved.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0014] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown.

[0015] Figure 2 This illustrates the placement of video encoding and decoding devices in a streaming environment.

[0016] Figure 3 A basic flowchart of a video encoder is shown.

[0017] Figure 4 A flowchart illustrating the steps of a streaming media transmission method performed by the decoding side in one embodiment of this application is shown.

[0018] Figure 5 A flowchart illustrating the steps of a streaming media transmission method based on combined sorting in one embodiment of this application is shown.

[0019] Figure 6 A flowchart illustrating the steps of a streaming media transmission method performed by the encoding side in one embodiment of this application is shown.

[0020] Figure 7 A structural block diagram of the streaming media transmission device on the decoding side provided in an embodiment of this application is shown.

[0021] Figure 8 A structural block diagram of an encoding-side streaming media transmission apparatus according to one embodiment of this application is shown.

[0022] Figure 9 A computer system architecture block diagram suitable for implementing the embodiments of this application is shown. Detailed Implementation

[0023] In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present application. However, those skilled in the art will recognize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc. may be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the present application.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0026] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0027] In the specific implementation of this application, user-related data such as streaming media resources are involved. When the various embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0028] The relevant terms or abbreviations used in the embodiments of this application are explained as follows.

[0029] DASH: Dynamic Adaptive Streaming over HTTP. Dynamic adaptive streaming over HTTP is an adaptive bitrate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP web servers.

[0030] MPD: Media Presentation Description, a media presentation description signaling in DASH used to describe information about media segments.

[0031] Representation: In DASH, a combination of one or more media components, such as a video file of a certain resolution, can be considered a representation.

[0032] Adaptation Sets: In DASH, an Adaptation Set is a collection of one or more video streams. An Adaptation Set can contain multiple Representations.

[0033] Multi-view / multi-point video: refers to video with depth information captured from multiple angles using multiple camera arrays. Also called free-viewpoint / free-viewpoint video, it is an immersive media offering a six-degrees-of-freedom experience.

[0034] Point cloud: A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud has at least three-dimensional positional information, and depending on the application scenario, may also have color, material, or other information. Typically, each point in a point cloud has the same number of additional attributes.

[0035] V3C (Visual Volumetric Video-Based Coding Media) refers to immersive media that captures visual content in three-dimensional space and provides a 3DoF+ or 6DoF viewing experience. It uses traditional video encoding and includes volumetric video type tracks in the file encapsulation, including multi-view videos and video-coded point clouds.

[0036] DoF: Degree of Freedom. In this application, it refers to the degree of freedom that users have to move and interact with content while watching immersive media.

[0037] 3DoF: refers to three degrees of freedom, which means the user's head rotates around the x, y, and z axes.

[0038] 3DoF+: In addition to the three degrees of freedom, the user also has finite degrees of freedom to move along the x, y, and z axes.

[0039] 6DoF: In addition to the three degrees of freedom, users also have the freedom to move freely along the x, y, and z axes.

[0040] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown.

[0041] like Figure 1 As shown, system architecture 100 includes multiple terminal devices that can communicate with each other via, for example, a network 150. For instance, system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected via network 150. Figure 1 In one embodiment, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0042] For example, the first terminal device 110 can encode video data (e.g., a video image stream captured by the terminal device 110) to transmit it to the second terminal device 120 via the network 150. The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device 120 can receive the encoded video data from the network 150, decode the encoded video data to recover the video data, and display video images based on the recovered video data.

[0043] In one embodiment of this application, system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded video data, such as during a video conference. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 may encode video data (e.g., a video image stream captured by the terminal device) for transmission over network 150 to the other terminal device. Each of the third terminal device 130 and the fourth terminal device 140 may also receive encoded video data transmitted by the other terminal device, decode the encoded video data to recover the video data, and display the video images on an accessible display device based on the recovered video data.

[0044] exist Figure 1 In the embodiments disclosed herein, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited to these. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 refers to any number of networks that transmit encoded video data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140, including, for example, wired and / or wireless communication networks. Communication network 150 may exchange data in circuit-switched and / or packet-switched channels. This network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of network 150 may be irrelevant to the operation of this application.

[0045] The server in this application embodiment can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0046] In one embodiment of this application, Figure 2The illustration shows the placement of video encoding and decoding devices in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital television (television), storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0047] The streaming system may include an acquisition subsystem 213, which may include a video source 201 such as a digital camera, which creates an uncompressed video image stream 202. In an embodiment, the video image stream 202 includes samples captured by a digital camera. The video image stream 202 is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data 204 (or encoded video bitstream 204). The video image stream 202 may be processed by an electronic device 220, which includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter as described in more detail below. The encoded video data 204 (or encoded video bitstream 204) is depicted as a thin line to emphasize the lower data volume of the encoded video data 204 (or encoded video bitstream 204), which may be stored on a streaming server 205 for future use. One or more streaming client subsystems, such as Figure 2 Client subsystems 206 and 208 can access streaming server 205 to retrieve copies 207 and 209 of encoded video data 204. Client subsystem 206 may include, for example, a video decoding device 210 in electronic device 230. Video decoding device 210 decodes the incoming copy 207 of the encoded video data and produces an output video picture stream 211 that can be displayed on display 212 (e.g., a screen) or another presentation device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T TH.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0048] It should be noted that electronic devices 220 and 230 may include other components not shown in the figures. For example, electronic device 220 may include a video decoding device, and electronic device 230 may also include a video encoding device.

[0049] In one embodiment of this application, taking the international video coding standards HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding), as well as the Chinese national video coding standard AVS, as examples, after an input video frame image, the video frame image is divided into several non-overlapping processing units according to a block size. Each processing unit will perform a similar compression operation. This processing unit is called a CTU (Coding Tree Unit) or LCU (Largest Coding Unit). The CTU can be further subdivided into one or more basic coding units CU, which are the most basic elements in a coding process. The following introduces some concepts when encoding CU: Predictive coding includes intra-frame prediction and inter-frame prediction. The original video signal is predicted from a selected reconstructed video signal to obtain a residual video signal. The encoder needs to determine which predictive coding mode to choose for the current CU and inform the decoder. Intra-frame prediction refers to predicting a signal from a region within the same image that has already been encoded and reconstructed; inter-frame prediction refers to predicting a signal from another encoded image (called a reference image) that is different from the current image.

[0050] Transform and Quantization: After the residual video signal undergoes transformation operations such as DFT (Discrete Fourier Transform) and DCT (Discrete Cosine Transform), the signal is transformed into the transform domain, and these are called transform coefficients. The transform coefficients are then subjected to lossy quantization, losing some information to make the quantized signal more suitable for compression. In some video coding standards, there may be more than one transform method to choose from; therefore, the encoder needs to select one transform method for the current CU and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually results in greater distortion and a lower bit rate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually results in less distortion and a higher bit rate.

[0051] Entropy coding, or statistical coding, involves statistically compressing the quantized transform-domain signal based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected coding mode and motion vector data, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless coding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) and Content-Adaptive Binary Arithmetic Coding (CABAC).

[0052] Context-Based Adaptive Binary Arithmetic Coding (CABAC) primarily involves three steps: binarization, context modeling, and binary arithmetic coding. After binarizing the input syntax elements, the binary data can be encoded using either a regular coding mode or a bypass coding mode. The bypass coding mode does not require assigning a specific probability model to each binary bit; the input binary bit bin values ​​are directly encoded using a simple bypass encoder, thus accelerating the overall encoding and decoding speed. Generally, different syntax elements are not completely independent, and the same syntax elements themselves possess a certain degree of memory. Therefore, according to conditional entropy theory, using other encoded syntax elements for conditional coding can further improve coding performance compared to independent coding or memoryless coding. This encoded symbol information used as conditions is called the context. In the regular coding mode, the binary bits of the syntax elements sequentially enter the context modeler. The encoder assigns an appropriate probability model to each input binary bit based on the values ​​of previously encoded syntax elements or binary bits; this process is called context modeling. The context model corresponding to a syntax element can be located using ctxIdxInc (context index increment) and ctxIdxStart (context index start). After the bin value and the assigned probability model are fed into the binary arithmetic encoder for encoding, the context model needs to be updated based on the bin value, which is the adaptive process in encoding.

[0053] Loop Filtering: The transformed and quantized signal undergoes inverse quantization, inverse transform, and prediction compensation to obtain a reconstructed image. Due to the effects of quantization, the reconstructed image differs from the original image in some aspects, resulting in distortion. Therefore, filtering operations such as deblocking filters (DB), Sample Adaptive Offset (SAO), or Adaptive Loop Filter (ALF) can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images serve as a reference for subsequent coded images in predicting future image signals, the aforementioned filtering operations are also called loop filtering, i.e., filtering operations within the coding loop.

[0054] In one embodiment of this application, Figure 3 A basic flowchart of a video encoder is shown, illustrating the process using intra-frame prediction as an example. The original image signal... With predicted image signal Perform the difference operation to obtain the residual signal. , the residual signal After transformation and quantization, quantization coefficients are obtained. These coefficients are then used to obtain the encoded bitstream through entropy encoding, and to obtain the reconstructed residual signal through inverse quantization and inverse transform. Predicting image signals With reconstructed residual signal Image signals generated by superposition Image signal On one hand, the signal is input to the intra-frame mode decision module and the intra-frame prediction module for intra-frame prediction processing; on the other hand, the reconstructed image signal is output through loop filtering. Reconstructing image signals It can be used as a reference image for the next frame for motion estimation and motion compensation prediction. Then, based on the results of the motion compensation prediction... Intra-frame prediction results Obtain the predicted image signal for the next frame. And continue repeating the above process until the encoding is complete.

[0055] Based on the above encoding process, at the decoding end, for each CU, after acquiring the compressed bitstream (i.e., bitstream), entropy decoding is performed to obtain various mode information and quantization coefficients. Then, the quantization coefficients undergo inverse quantization and inverse transform processing to obtain the residual signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to the CU can be obtained. Then, the residual signal and the prediction signal are added together to obtain the reconstructed signal. The reconstructed signal is then subjected to loop filtering and other operations to generate the final output signal.

[0056] In video transmission technology, streaming technology is commonly used to handle the transmission of media resources between servers and clients. Common media streaming technologies include DASH (Dynamic Adaptive Streaming over HTTP), HLS (HTTP Live Streaming), and SMT (Smart Media Transport).

[0057] Taking DASH as an example, DASH is an adaptive bitrate streaming technology that enables high-quality streaming media to be delivered over the internet via traditional HTTP web servers. DASH breaks down content into a series of small, HTTP-based file segments, each containing a short, playable segment, while the total content length can be several hours (e.g., movies or live sports broadcasts). The content is prepared into multiple bitrate alternatives, providing various bitrate versions to choose from. When media content is played by a DASH client, the client automatically selects which alternative to download and play based on current network conditions. The client will choose the highest bitrate segment that can be downloaded immediately for playback, thus avoiding playback stuttering or rebuffering events. As a result, DASH clients can seamlessly adapt to constantly changing network conditions and provide a high-quality playback experience with fewer stutters and rebuffering occurrences.

[0058] DASH utilizes existing HTTP web server infrastructure. It allows devices such as internet TVs, set-top boxes, desktop computers, smartphones, and tablets to consume multimedia content (such as video, television, and radio) delivered over the internet, and can adapt to changing internet reception conditions.

[0059] Emerging media resource playback scenarios place higher demands on the network transmission of streaming media data. This application provides a method for indicating information such as the quality and priority of streaming media data. The method proposed in this application allows for the flexible definition and indication of quality, priority, and other related information of streaming media data within a specific range, enabling clients to request corresponding streaming media data according to their needs.

[0060] The following detailed description, in conjunction with specific embodiments, outlines the technical solutions provided in this application, including the streaming media transmission method, streaming media transmission device, computer-readable medium, electronic device, and computer program product. The various technical solutions of the embodiments of this application can be applied to the server-side, player-side, and intermediate nodes of a streaming media transmission system.

[0061] In one embodiment of this application, a streaming media transmission method applied to a decoding end, a data receiving end, or a terminal device includes: parsing streaming media transmission signaling to obtain a sorting relationship descriptor, the sorting relationship descriptor being used to represent sorting information within a specified range; selecting a target streaming media object to be received from the specified range according to the sorting relationship descriptor; and sending a streaming media transmission request for the target streaming media object to the data source.

[0062] In one embodiment of this application, the sorting relationship descriptor includes a sorting identifier field, and the specified range includes multiple streaming media objects having the same sorting identifier field.

[0063] Figure 4 The flowchart illustrates the steps of a streaming media transmission method executed by the decoding side in one embodiment of this application. This method can be applied to a user's terminal device or to an intermediate node for data communication between a server and a terminal device.

[0064] like Figure 4 As shown, the streaming media transmission method may include the following steps S410 to S440.

[0065] In step S410, the streaming media transmission signaling is parsed to obtain sorting relationship descriptors corresponding to different streaming media objects. The sorting relationship descriptors include a sorting identifier field for representing different sorting ranges and a sorting information field for serving as the sorting basis.

[0066] Streaming media transmission signaling is a message used to coordinate the communication process transmitted between the data source and the data receiver. For example, it can be DASH signaling based on the DASH protocol, or it can be SMT signaling.

[0067] The ranking relationship descriptor is used to define the quality ranking, priority ranking, or other ranking information between different streaming media objects. For example, in DASH signaling, the ranking relationship descriptor can be defined as the element RankingRelationship, and in SMT signaling, it can be defined as the element Ranking_set_descriptor.

[0068] This application defines a sorting identifier field in the sorting relationship descriptor to represent different sorting ranges and a sorting information field to serve as the sorting basis. By dividing streaming media objects into multiple different sorting ranges, streaming media objects can be sorted according to different sorting criteria within a specific sorting range.

[0069] In step S420, streaming media objects with the same sorting identifier field are grouped into a sorting relationship set.

[0070] The streaming media transmission signaling carries multiple sorting relationship descriptors for streaming media objects. By comparing these descriptors, the sorting relationship between the various streaming media objects can be determined. This sorting relationship includes two levels: sorting range and sorting position. The sorting range is determined based on the sorting identifier field in the sorting relationship descriptor. When any two streaming media objects have the same sorting identifier field, it means that the two streaming media objects belong to the same sorting range, and streaming media objects within the same sorting range can be grouped into a sorting relationship set. For example, if the sorting identifier fields of streaming media objects A and B are both 1, and the sorting identifier fields of streaming media objects C and D are both 2, then streaming media objects A and B can be grouped into one sorting relationship set, and streaming media objects C and D can be grouped into another sorting relationship set.

[0071] In one embodiment of this application, a streaming media object may have a sorting relationship descriptor, thereby determining a corresponding sorting identifier field for the streaming media object, and then combining the streaming media object into a specified sorting relationship set.

[0072] In one embodiment of this application, a streaming media object may also have multiple different sorting relation descriptors. Therefore, when a streaming media object has sorting identifier fields with different values ​​in different sorting relation descriptors, the streaming media object can be combined into multiple different sorting relation sets.

[0073] In step S430, the streaming media objects are sorted according to the sorting information field in the sorting relationship set to obtain a streaming media object sequence.

[0074] If two or more streaming media objects are distributed in the same sorting relationship set, the multiple streaming media objects distributed in the sorting relationship set can be sorted according to the value of the sorting information field to obtain a sequence of streaming media objects arranged in a specified order.

[0075] In one embodiment of this application, streaming media objects can be sorted in ascending order of the sorting information field value, or they can be sorted in descending order of the sorting information field value.

[0076] In one embodiment of this application, the sorting information field may include at least one of quality sorting information, priority sorting information, or undefined sorting information. The quality sorting information is used to indicate the sorting level of streaming media objects according to streaming media quality, the priority sorting information is used to indicate the sorting level of streaming media objects according to selected priority, and the undefined sorting information is used to indicate the sorting level of streaming media objects according to specified rules.

[0077] In one embodiment of this application, the sorting information field may include one of the following: quality sorting information, priority sorting information, or undefined sorting information; it may also include two or more types of specified information.

[0078] In one embodiment of this application, within the same set of sorting relationships, streaming media objects can be sorted according to the same type of specified information, or they can be sorted comprehensively according to multiple types of specified information.

[0079] In step S440, a target streaming media object to be received is selected from the streaming media object sequence, and a streaming media transmission request for the target streaming media object is sent to the data source.

[0080] In one embodiment of this application, the data receiver can select objects from the streaming media object sequence that meet its data needs as target streaming media objects to be received. For example, when the data receiver's device performance is good and the network condition is good, it can preferentially select streaming media objects with higher quality ranking or higher priority ranking as target streaming media objects to be received; when the data receiver's device performance is poor or the network condition is poor, it can select streaming media objects with lower quality ranking or lower priority ranking as target streaming media objects to be received.

[0081] After selecting the target streaming media object, the data receiver can send a streaming media transmission request for the target streaming media object to the data source. When the data source receives the streaming media transmission request, it can transmit the corresponding target streaming media object to the data receiver according to the data receiver's needs.

[0082] In the streaming media transmission method provided in this application embodiment, by dividing the streaming media object into multiple different sorting ranges and sorting the streaming media object according to different sorting criteria within a specific sorting range, the flexibility and diversity of streaming media object sorting can be improved, thereby providing a variety of options for the network transmission of streaming media objects. Therefore, the stability and reliability of streaming media data transmission can be guaranteed, and the transmission efficiency of streaming media data can be improved.

[0083] In one embodiment of this application, the sorting relationship descriptor further includes a combination determination field for indicating whether streaming media objects need to be sorted in combination, and a combination identifier field for indicating the range of combination sorting. The combination determination field can be used to determine whether streaming media objects need to be sorted in combination, and the combination identifier field can be used to determine the range of combination sorting for the streaming media objects.

[0084] Figure 5 A flowchart illustrating the steps of a streaming media transmission method based on combined sorting in one embodiment of this application is shown. Figure 5 As shown, the streaming media transmission method may include the following steps S510 to S520. In step S510, the streaming media transmission signaling is parsed to obtain sorting relationship descriptors corresponding to different streaming media objects. The sorting relationship descriptor includes a sorting identifier field for representing different sorting ranges, a sorting information field for serving as the sorting basis, a combination determination field for indicating whether streaming media objects are combined for sorting, and a combination identifier field for representing the range of combined sorting.

[0085] Streaming media transmission signaling is a message used to coordinate the communication process transmitted between the data source and the data receiver. For example, it can be DASH signaling based on the DASH protocol, or it can be SMT signaling.

[0086] The ranking relationship descriptor is used to define the quality ranking, priority ranking, or other ranking information between different streaming media objects. For example, in DASH signaling, the ranking relationship descriptor can be defined as the element RankingRelationship, and in SMT signaling, it can be defined as the element Ranking_set_descriptor.

[0087] This application defines a sorting identifier field in the sorting relationship descriptor to represent different sorting ranges and a sorting information field to serve as the sorting basis. By dividing streaming media objects into multiple different sorting ranges, streaming media objects can be sorted according to different sorting criteria within a specific sorting range.

[0088] In step S520, streaming media objects with the same sorting identifier field are grouped into a sorting relationship set.

[0089] The streaming media transmission signaling carries multiple sorting relationship descriptors for streaming media objects. By comparing these descriptors, the sorting relationship between the various streaming media objects can be determined. This sorting relationship includes two levels: sorting range and sorting position. The sorting range is determined based on the sorting identifier field in the sorting relationship descriptor. When any two streaming media objects have the same sorting identifier field, it means that the two streaming media objects belong to the same sorting range, and streaming media objects within the same sorting range can be grouped into a sorting relationship set. For example, if the sorting identifier fields of streaming media objects A and B are both 1, and the sorting identifier fields of streaming media objects C and D are both 2, then streaming media objects A and B can be grouped into one sorting relationship set, and streaming media objects C and D can be grouped into another sorting relationship set.

[0090] In one embodiment of this application, a streaming media object may have a sorting relationship descriptor, thereby determining a corresponding sorting identifier field for the streaming media object, and then combining the streaming media object into a specified sorting relationship set.

[0091] In one embodiment of this application, a streaming media object may also have multiple different sorting relation descriptors. Therefore, when a streaming media object has sorting identifier fields with different values ​​in different sorting relation descriptors, the streaming media object can be combined into multiple different sorting relation sets.

[0092] In step S530, it is determined whether to perform combined sorting on the streaming media objects based on the combination determination field.

[0093] In one embodiment of this application, the value of the combination determination field can be used to determine whether streaming media objects need to be combined and sorted. For example, when the combination determination field of a streaming media object is a preset target field value, it means that the streaming media object needs to be combined with at least one other streaming media object to form a sorting-bound object combination, and the object combination will be sorted as a whole; conversely, when the combination determination field of a streaming media object is not a target field value, it means that the streaming media object does not need to be combined with other streaming media objects to form a sorting-bound object combination, but participates in the object sorting independently.

[0094] In step S540, if the value of the combination determination field is the target field value, then streaming media objects with the same combination identifier field are grouped into an object group with a sorting binding relationship.

[0095] For multiple streaming media objects whose combination determination field value is the target field value, they can be combined and matched based on the combination identifier field. Streaming media objects with the same combination identifier field can form an object combination with a sorting binding relationship. For example, if the combination identifier field of streaming media objects E and F both have a value of 100, and the combination identifier field of streaming media objects G and H both have a value of 101, then streaming media objects E and F can be combined into one object combination with a sorting binding relationship, and streaming media objects G and H can be combined into another object combination with a sorting binding relationship.

[0096] In one embodiment of this application, when the value of the combination determination field of a streaming media object is not the target field value, its corresponding combination identifier field can be omitted, that is, the combination identifier field is not shown in the sorting relationship descriptor. When the value of the combination determination field of a streaming media object is the target field value, the corresponding combination identifier field must be shown in the sorting relationship descriptor.

[0097] In step S550, the streaming media objects or combinations of objects are sorted according to the sorting information field in the sorting relationship set to obtain a streaming media object sequence composed of streaming media objects or combinations of objects.

[0098] In one embodiment of this application, all elements in the sorting relationship set are independent streaming media objects. The multiple streaming media objects distributed in the sorting relationship set can be sorted according to the value of the sorting information field of each streaming media object to obtain a sequence of streaming media objects arranged in a specified order.

[0099] In one embodiment of this application, all elements in the sorting relationship set are object combinations composed of streaming media objects. Based on the value of the sorting information field of each object combination, the multiple object combinations distributed in the sorting relationship set can be sorted to obtain a streaming media object sequence arranged in a specified order.

[0100] In one embodiment of this application, the elements in the sorting relationship set include both independent streaming media objects and object combinations composed of streaming media objects. Based on the values ​​of the sorting information fields of each streaming media object and object combination, at least one streaming media object and at least one object combination distributed in the sorting relationship set can be sorted to obtain a sequence of streaming media objects arranged in a specified order.

[0101] In one embodiment of this application, when two streaming media objects form an object combination, the two streaming media objects can share the same sorting information field.

[0102] In one embodiment of this application, streaming media objects and / or object combinations can be sorted in ascending order of the sorting information field value, or they can be sorted in descending order of the sorting information field value.

[0103] In one embodiment of this application, the sorting information field may include at least one of quality sorting information, priority sorting information, or undefined sorting information. The quality sorting information is used to indicate the sorting level of streaming media objects according to streaming media quality, the priority sorting information is used to indicate the sorting level of streaming media objects according to selected priority, and the undefined sorting information is used to indicate the sorting level of streaming media objects according to specified rules.

[0104] In one embodiment of this application, the sorting information field may include one of the following: quality sorting information, priority sorting information, or undefined sorting information; it may also include two or more types of specified information.

[0105] In one embodiment of this application, within the same set of sorting relationships, each streaming media object and / or object combination can be sorted according to the same type of specified information, or the various streaming media objects and / or object combinations can be comprehensively sorted according to multiple types of specified information.

[0106] In step S560, a target streaming media object to be received is selected from the streaming media object sequence, and a streaming media transmission request for the target streaming media object is sent to the data source.

[0107] In one embodiment of this application, the data receiver can select objects from the streaming media object sequence that meet its data needs as target streaming media objects to be received. The target streaming media object can be a single streaming media object or all streaming media objects in a combination of objects. For example, when the data receiver's device performance is good and the network condition is good, it can preferentially select streaming media objects with higher quality ranking or higher priority ranking as target streaming media objects to be received; when the data receiver's device performance is poor or the network condition is poor, it can select streaming media objects with lower quality ranking or lower priority ranking as target streaming media objects to be received.

[0108] After selecting the target streaming media object, the data receiver can send a streaming media transmission request for the target streaming media object to the data source. When the data source receives the streaming media transmission request, it can transmit the corresponding target streaming media object to the data receiver according to the data receiver's needs.

[0109] In the streaming media transmission method provided in this application embodiment, by grouping streaming media objects with sorting binding relationships into object groups, the object groups can be sorted as a whole when sorting objects, thus reducing the computational load of object sorting and improving the efficiency of object sorting. When transmitting data to a target streaming media object, the object groups can also be transmitted as a whole, thus further improving the transmission efficiency of streaming media data.

[0110] In one embodiment of this application, the sorting relationship descriptor further includes a range quantity field, which represents the number of sorting ranges to which a streaming media object belongs. For example, in SMT signaling, the range quantity field can be defined as the element `ranking_set_num`, representing the number of sorting ranges to which a streaming media object belongs. By defining the range quantity field in the sorting relationship descriptor, the decoding side can easily traverse the sorting ranges to which the streaming media object belongs, avoiding sorting errors and omissions, and also avoiding full detection. Therefore, the sorting reliability and sorting efficiency of streaming media objects can be further improved.

[0111] In one embodiment of this application, a streaming media object may include at least one of a stream file representation, an adaptive subset, or a preselection. A stream file representation is a file object composed of one or more streaming media resources; an adaptive subset is a file object composed of one or more stream file representations; and a preselection is a file object obtained by combining a stream file representation or an adaptive subset according to preset rules. This embodiment of the application is applicable to streaming media object transmission in various dimensions, possessing strong universality and adaptability to different versions of hardware devices and different types of streaming media transmission protocols.

[0112] In one embodiment of this application, the streaming media resource is a point cloud media resource; the streaming media object includes a geometric component and an attribute component, wherein the geometric component is an adaptive subset for representing the geometric components of the point cloud, and the attribute component is an adaptive subset for representing the attribute components of the point cloud.

[0113] A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud has at least three-dimensional position information and, depending on the application scenario, may also have color, material, or other information. Typically, each point in a point cloud has the same number of additional attributes. Point cloud media can be further divided into video-based point cloud compression (VPCC) and geometry-based point cloud compression (GPCC) based on encoding methods. In the file encapsulation of point cloud media, the three-dimensional position information is usually called the geometry component, and the attribute information is called the attribute component. A point cloud file has only one geometry component but can have one or more attribute components.

[0114] Point clouds can flexibly and conveniently represent the spatial structure and surface attributes of three-dimensional objects or scenes, and therefore have a wide range of applications. Their main applications can be categorized into two main types: 1) Machine-perceived point clouds, such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and disaster relief robots. 2) Human-perceived point clouds, such as digital cultural heritage, free-viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0115] Point clouds are primarily acquired through the following methods: computer generation, 3D laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual 3D objects and scenes. 3D scanning can obtain point clouds of static real-world 3D objects or scenes, acquiring millions of point clouds per second. 3D photography can obtain point clouds of dynamic real-world 3D objects or scenes, acquiring tens of millions of point clouds per second. Furthermore, in the medical field, point clouds of biological tissues and organs can be obtained from MRI, CT, and electromagnetic positioning information. These technologies have reduced the cost and time required for point cloud data acquisition and improved data accuracy. This transformation in point cloud data acquisition methods has made the acquisition of massive amounts of point cloud data possible. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, publication, sharing, and standardization of point cloud data have become crucial for point cloud applications.

[0116] After encoding point cloud media, the encoded data stream needs to be encapsulated and transmitted to the user. Correspondingly, on the point cloud media player, the point cloud file needs to be decapsulated first, then decoded, and finally the decoded data stream is presented.

[0117] In one embodiment of this application, the streaming media resource can be an immersive media resource, such as multi-view video, video-coded point clouds, etc. Correspondingly, the streaming media object can include volumetric media with different degrees of freedom, such as immersive media containing volumetric video type tracks with degrees of freedom of 3DoF, 3DoF+, or 6DoF.

[0118] In one embodiment of this application, a streaming media transmission method applied to an encoding end, a data sending end, or a server includes: sending streaming media transmission signaling to a data receiver, the streaming media transmission signaling carrying a sorting relationship descriptor, the sorting relationship descriptor being used to represent sorting information within a specified range; and returning a target streaming media object to the data receiver according to a streaming media transmission request sent by the data receiver, the target streaming media object being an object selected from the specified range according to the sorting relationship descriptor.

[0119] Figure 6 This diagram illustrates a flowchart of a streaming media transmission method executed by the encoding side in one embodiment of this application. This method can be applied to a server or to an intermediate node for data communication between a server and a terminal device. Figure 6 As shown, the streaming media transmission method may include the following steps S610 to S650.

[0120] In step S610, the sorting range of the streaming media object and the sorting criteria used to sort the streaming media object are obtained.

[0121] In one embodiment of this application, multiple selectable streaming media objects are classified to obtain multiple different sorting ranges; within each sorting range, sorting criteria for sorting the streaming media objects are obtained.

[0122] The sorting relationship between various streaming media objects includes two aspects: sorting range and sorting position. Streaming media objects belonging to the same sorting range can be sorted according to the sorting criteria, while streaming media objects belonging to different sorting ranges cannot be sorted together.

[0123] In one embodiment of this application, a streaming media object belongs to a single sorting range, or a streaming media object may belong to multiple different sorting ranges simultaneously.

[0124] In one embodiment of this application, multiple streaming media objects belonging to the same sorting range may have a single type of sorting criteria, or they may have multiple types of sorting criteria.

[0125] In step S620, the sorting identifier field of the sorting relationship descriptor is assigned a value according to the sorting range in which the streaming media object is located.

[0126] When two streaming media objects belong to the same sorting range, they can be configured with the same sorting identifier field. When two streaming media objects belong to two different sorting ranges, they can be configured with different sorting identifier fields. For example, if streaming media objects A and B belong to one sorting range, and streaming media objects C and D belong to another sorting range, then the sorting identifier fields of streaming media objects A and B can both be assigned a value of 1, while the sorting identifier fields of streaming media objects C and D can both be assigned a value of 2.

[0127] When a streaming media object belongs to two or more sorting ranges at the same time, multiple different sorting identifier field values ​​can be configured for it.

[0128] In step S630, the sorting information field of the sorting relationship descriptor is assigned a value according to the sorting criteria of the streaming media object.

[0129] In one embodiment of this application, the sorting information field may include at least one of quality sorting information, priority sorting information, or undefined sorting information. The quality sorting information is used to indicate the sorting level of streaming media objects according to streaming media quality, the priority sorting information is used to indicate the sorting level of streaming media objects according to selected priority, and the undefined sorting information is used to indicate the sorting level of streaming media objects according to specified rules.

[0130] In one embodiment of this application, the sorting information field may include one of the following: quality sorting information, priority sorting information, or undefined sorting information; it may also include two or more types of specified information.

[0131] In one embodiment of this application, the number of sorting ranges to which a streaming media object belongs can be counted; then, the range number field of the sorting relationship descriptor is assigned a value based on the range number. The range number field is used to represent the number of sorting ranges to which a streaming media object belongs. For example, in SMT signaling, the range number field can be defined as the element `ranking_set_num`, representing the number of sorting ranges to which a streaming media object belongs. By defining the range number field in the sorting relationship descriptor, the decoding side can easily traverse the sorting ranges to which the streaming media object belongs, avoiding sorting errors and omissions, and also avoiding full detection. Therefore, the sorting reliability and sorting efficiency of streaming media objects can be further improved.

[0132] In one embodiment of this application, multiple selectable streaming media objects are compared to determine whether there is a sorting binding relationship between the streaming media objects; if there is a sorting binding relationship between at least two streaming media objects, the combination identifier field of the sorting relationship descriptor is assigned a value according to the sorting binding relationship, and the combination determination field of the sorting relationship descriptor is assigned a value according to the target field value.

[0133] In one embodiment of this application, streaming media objects with order binding relationships can have their combination identifier fields assigned the same value, and their order relationship descriptor combination determination fields can be assigned values ​​according to the target field values. By assigning values ​​to the combination identifier field and the combination determination field, the decoding side can efficiently determine whether streaming media objects can be combined. If streaming media objects with order binding relationships can be grouped into object combinations, then when sorting objects, the object combinations can be sorted as a whole, thus reducing the computational load of object sorting and improving the efficiency of object sorting. When transmitting data to the target streaming media object, the object combinations can also be transmitted as a whole, thus further improving the transmission efficiency of streaming media data.

[0134] In step S640, streaming media transmission signaling carrying multiple sorting relation descriptors is sent to the data receiver.

[0135] Streaming media transmission signaling is a message used to coordinate the communication process transmitted between the data source and the data receiver. For example, it can be DASH signaling based on the DASH protocol, or it can be SMT signaling.

[0136] The ranking relationship descriptor is used to define the quality ranking, priority ranking, or other ranking information between different streaming media objects. For example, in DASH signaling, the ranking relationship descriptor can be defined as the element RankingRelationship, and in SMT signaling, it can be defined as the element Ranking_set_descriptor.

[0137] This application defines a sorting identifier field in the sorting relationship descriptor to represent different sorting ranges and a sorting information field to serve as the sorting basis. By dividing streaming media objects into multiple different sorting ranges, streaming media objects can be sorted according to different sorting criteria within a specific sorting range.

[0138] In step S650, in response to the streaming media transmission request sent by the data receiver corresponding to the streaming media transmission signaling, a target streaming media object matching the streaming media transmission request is returned to the data receiver.

[0139] In one embodiment of this application, the data receiver can select objects from the streaming media object sequence that meet its data needs as target streaming media objects to be received. The target streaming media object can be a single streaming media object or all streaming media objects in a combination of objects. For example, when the data receiver's device performance is good and the network condition is good, it can preferentially select streaming media objects with higher quality ranking or higher priority ranking as target streaming media objects to be received; when the data receiver's device performance is poor or the network condition is poor, it can select streaming media objects with lower quality ranking or lower priority ranking as target streaming media objects to be received.

[0140] After selecting the target streaming media object, the data receiver can send a streaming media transmission request for the target streaming media object to the data source. When the data source receives the streaming media transmission request, it can transmit the corresponding target streaming media object to the data receiver according to the data receiver's needs.

[0141] In one application scenario of this application embodiment, the server organizes the media file into a video stream and generates a corresponding signaling file based on the media file's structure and the quality and priority information it contains. The quality and priority information in the file indicates the quality and priority of the video stream within a specific range. The server sends the signaling file to the client. The client, based on its own needs, network conditions, or decoding capabilities, and in conjunction with the quality and priority information in the signaling file, requests a suitable video stream and decodes and consumes it.

[0142] This application embodiment can add several descriptive fields at the system layer, including field extensions at the signaling message level, to support the implementation steps of this application embodiment. Below, using the extension of existing DASH signaling as an example, a method for indicating the quality, priority, and other related sorting information of video streams is defined.

[0143] A SupplementalProperty element with the @schemeIdUri attribute value of "urn:avs:ims:2020:rk-relation" represents a sorting relation descriptor. This descriptor is used to define quality or priority sorting information between specific representations or specific adaptation sets. This descriptor can describe media resources at the representation, adaptation set, or preselection level. One or more sorting relation descriptors can exist in an MPD signaling. The syntax and semantics of this descriptor are shown in Table 1 below.

[0144] Table 1. Syntax and Semantics of Order Relation Descriptors

[0145] In some embodiments of this application, the RankingRelationship@qualityRanking field in the above descriptor can be replaced by the @qualityRanking field. Similarly, the RankingRelationship@priorityRanking field in the above descriptor can be replaced by the @selectionPriority field.

[0146] In some embodiments of this application, the ordering relation descriptor can also be extended based on existing fields of streaming media transmission signaling to achieve flexible quality and priority ordering of media resources such as Representation, Adaptation Set, and Preselection within a specific range. For example, Table 2 below shows the syntax and semantics of extending the quality ordering equivalent descriptor in one embodiment of this application, Table 3 shows the syntax and semantics of extending the priority selection information descriptor in one embodiment of this application, and Table 4 shows the syntax and semantics of extending the replaceable information descriptor in one embodiment of this application.

[0147] Table 2. Syntax and Semantics of Quality Ordering Equivalent Descriptor Extensions

[0148] Table 3. Syntax and semantics of priority selection information descriptor extension

[0149] Table 4 Syntax and Semantics of Replaceable Information Descriptor Extensions

[0150] The following uses SMT signaling extension as an example to illustrate the signaling extension of the ranking association descriptor in the embodiments of this application. In SMT signaling, the ranking association descriptor can be defined as a ranking set descriptor (Ranking_set_descriptor). The ranking set descriptor (Ranking_set_descriptor) is used to indicate the ranking set to which the current media resource belongs; a media resource can belong to one or more ranking sets. Table 5 shows the syntax information of the ranking set descriptor (Ranking_set_descriptor).

[0151] Table 5 Syntax information for sorted set descriptors

[0152] The individual elements are defined as follows.

[0153] descriptor_tag: Identifier used to indicate the type of descriptor.

[0154] descriptor_length: The length of the identifier in bytes.

[0155] ranking_set_num: Indicates the number of ranking sets to which the current media resource belongs.

[0156] `ranking_set_id`: The ranking set identifier. Media resources with the same `ranking_set_id` collectively constitute the range of quality ranking, priority ranking, and undefined ranking. Media resources with different `ranking_set_id` cannot be compared based on their respective `quality_ranking`, `priority_ranking`, or `undefined_ranking`.

[0157] combine_ranking_flag: A value of 1 for this field indicates that all resource sets with the same combine_ranking_id are considered as a whole for quality ranking, priority ranking, or undefined ranking.

[0158] combine_ranking_id: All resource sets carrying the same combine_ranking_id are treated as a whole for quality ranking, priority ranking, or undefined ranking.

[0159] quality_ranking: Specifies the quality ranking information for the corresponding resource or resource set. The smaller the value of this field, the higher the quality level.

[0160] priority_ranking: Specifies the priority ranking information for the corresponding resource or resource set. The smaller the value of this field, the higher the priority.

[0161] undefined_ranking: When different resources or resource sets are classified according to special rules (such as algorithm type) and cannot be sorted by conventional quality level or priority, this field specifies the classification method under the special rules. Resources or resource sets with the same value in this field are resources or resource sets of the same level under this classification method.

[0162] The following uses point cloud media resources as an example to illustrate the process of streaming media transmission of point cloud media resources in an application scenario according to an embodiment of this application.

[0163] First, the server organizes the media file into a video stream based on its structure and included quality information, and generates corresponding signaling files. Based on the quality information in the files, it indicates the quality of the video stream within a specific range. Assume point cloud file F1 contains geometric components and an attribute component—color. Each geometric component and attribute component has two different quality versions, and these different quality versions combine to constitute the overall quality information for point cloud consumption. Based on this, the DASH signaling is generated as follows:

Adaptation Set1 (Geometric Components):

[0164] By combining signaling messages, the client can determine that for point cloud file F1, the server contains two geometric components (Representation1, Representation2) of different qualities and two attribute components (Representation3, Representation4) of different qualities. Furthermore, the four media streams—Representation1+Representation3, Representation1+Representation4, Representation2+Representation3, and Representation2+Representation4—are combined and ranked by quality.

[0165] Finally, the client requests and decodes the appropriate video stream based on its own needs, network conditions, or decoding capabilities, combined with the quality information in the signaling file.

[0166] To meet application requirements, this application proposes a method for indicating the quality and priority information of video streams, particularly immersive media (point clouds, multi-view videos, panoramic videos, etc.). The method proposed in this application allows for the flexible definition and indication of the quality and priority information of video streams within a specific range, enabling clients to request the appropriate video stream according to their needs.

[0167] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0168] The following describes an apparatus embodiment of this application, which can be used to execute the streaming media transmission method in the above embodiments of this application.

[0169] In one embodiment of this application, the streaming media transmission device applied to a decoding end, a data receiving end, or a terminal device includes: The parsing module is configured to parse streaming media transmission signaling to obtain a sorting relationship descriptor, which is used to represent sorting information within a specified range; The selected module is configured to select a target streaming media object to be received from the specified range based on the sorting relationship descriptor; The sending module is configured to send a streaming media transmission request for the target streaming media object to the data source.

[0170] Figure 7 A structural block diagram of a streaming media transmission device on the decoding side provided in an embodiment of this application is shown. Figure 7 As shown, the streaming media transmission device 700 on the decoding side mainly includes: The signaling parsing module 710 is configured to parse streaming media transmission signaling to obtain sorting relationship descriptors corresponding to different streaming media objects. The sorting relationship descriptors include a sorting identifier field for representing different sorting ranges and a sorting information field for serving as the sorting basis. The object collection module 720 is configured to form a sorting relationship set of streaming media objects that have the same sorting identifier field; The object sorting module 730 is configured to sort the streaming media objects in the sorting relationship set according to the sorting information field to obtain a streaming media object sequence; The request sending module 740 is configured to select a target streaming media object to be received from the streaming media object sequence and send a streaming media transmission request for the target streaming media object to the data source.

[0171] In one embodiment of this application, based on the above technical solution, the sorting relationship descriptor further includes a combination determination field for indicating whether streaming media objects are sorted in combination and a combination identifier field for indicating the range of combination sorting; the device further includes: The combination sorting module 750 is configured to determine whether to perform combination sorting on the streaming media objects based on the combination determination field. The object combination module 760 is configured to combine streaming media objects with the same combination identifier field into an object combination with a sorting binding relationship if the value of the combination determination field is the target field value.

[0172] In one embodiment of this application, based on the above technical solution, the object sorting module 730 can be further configured to: sort the streaming media object or object combination according to the sorting information field to obtain a streaming media object sequence composed of the streaming media object or object combination.

[0173] In one embodiment of this application, based on the above technical solution, the sorting relationship descriptor further includes a range quantity field, which is used to represent the number of sorting ranges to which the streaming media object belongs.

[0174] In one embodiment of this application, based on the above technical solution, the sorting information field includes at least one of quality sorting information, priority sorting information, or undefined sorting information. The quality sorting information is used to represent the sorting level of streaming media objects according to streaming media quality, the priority sorting information is used to represent the sorting level of streaming media objects according to selected priority, and the undefined sorting information is used to represent the sorting level of streaming media objects according to specified rules.

[0175] In one embodiment of this application, based on the above technical solutions, the streaming media object includes at least one of a bitstream file, an adaptive subset, or a preselection set. The bitstream file is a file object composed of one or more streaming media resources. The adaptive subset is a file object composed of one or more bitstream files. The preselection set is a file object obtained by combining the bitstream file or the adaptive subset according to preset rules.

[0176] In one embodiment of this application, based on the above technical solution, the streaming media resource is a point cloud media resource; the streaming media object includes a geometric component and an attribute component, wherein the geometric component is an adaptive subset for representing the geometric components of the point cloud, and the attribute component is an adaptive subset for representing the attribute components of the point cloud.

[0177] In one embodiment of this application, the streaming media transmission device applied to the encoding end, data sending end, or server includes: The signaling sending module is configured to send streaming media transmission signaling to the data receiver. The streaming media transmission signaling carries a sorting relationship descriptor, which is used to represent sorting information within a specified range. The object sending module is configured to return a target streaming media object to the data receiver based on the streaming media transmission request sent by the data receiver. The target streaming media object is an object selected from the specified range according to the sorting relationship descriptor.

[0178] Figure 8 A structural block diagram of a streaming media transmission apparatus on the encoding side according to one embodiment of this application is shown. Figure 8 As shown, the streaming media transmission device 800 on the encoding side mainly includes: The acquisition module 810 is configured to acquire the sorting range of the streaming media object and the sorting criteria used to sort the streaming media object; The range assignment module 820 is configured to assign a value to the sort identifier field of the sorting relationship descriptor according to the sorting range in which the streaming media object is located; According to the assignment module 830, it is configured to assign a value to the sorting information field of the sorting relationship descriptor based on the sorting criteria of the streaming media object; The signaling sending module 840 is configured to send streaming media transmission signaling carrying multiple sorting relation descriptors to the data receiver; The object return module 850 is configured to return a target streaming media object matching the streaming media transmission request to the data receiver in response to a streaming media transmission request sent by the data receiver corresponding to the streaming media transmission signaling.

[0179] In one embodiment of this application, based on the above technical solution, the acquisition module 810 may include: The classification submodule 811 is configured to classify multiple selectable streaming media objects to obtain multiple different sorting ranges; The acquisition submodule 812 is configured to acquire the sorting criteria for sorting the streaming media objects within the sorting range.

[0180] In one embodiment of this application, based on the above technical solution, the streaming media transmission device 800 on the encoding side further includes: The quantity statistics module 860 is configured to count the number of ranges within the sorting range of the streaming media object; The quantity assignment module 870 is configured to assign a value to the range quantity field of the sorting relation descriptor based on the range quantity.

[0181] In one embodiment of this application, based on the above technical solution, the streaming media transmission device 800 on the encoding side further includes: The binding comparison module 880 is configured to compare multiple selectable streaming media objects to determine whether there is a sorting binding relationship between the streaming media objects; The binding assignment module 890 is configured to, if there is a sorting binding relationship between at least two streaming media objects, assign a value to the combination identifier field of the sorting relationship descriptor according to the sorting binding relationship, and assign a value to the combination determination field of the sorting relationship descriptor according to the target field value.

[0182] The specific details of the streaming media transmission devices provided in the various embodiments of this application have been described in detail in the corresponding method embodiments, and will not be repeated here.

[0183] Figure 9 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0184] It should be noted that, Figure 9The computer system 900 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0185] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which performs various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for system operation. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output interface 905 (I / O interface) is also connected to the bus 904.

[0186] The following components are connected to the input / output interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a local area network card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0187] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit 901, it performs various functions defined in the system of this application.

[0188] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0189] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0190] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0191] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0192] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0193] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A streaming media transmission method, characterized in that, include: Parse the streaming media transmission signaling to obtain a sorting relationship descriptor, which is used to represent sorting information within a specified range; The target streaming media object to be received is selected from the specified range according to the sorting relationship descriptor; Send a streaming media transmission request for the target streaming media object to the data source.

2. The method according to claim 1, characterized in that, The sorting relationship descriptor includes a sorting identifier field, and the specified range includes multiple streaming media objects that have the same sorting identifier field.

3. The method according to claim 2, characterized in that, The sorting relationship descriptor also includes a sorting information field for use as the sorting basis; the step of selecting the target streaming media object to be received from the specified range according to the sorting relationship descriptor includes: Multiple streaming media objects with the same sorting identifier field are grouped into a sorting relationship set; In the sorting relationship set, the multiple streaming media objects are sorted according to the sorting information field to obtain a streaming media object sequence; Select a target streaming media object to be received from the sequence of streaming media objects, and send a streaming media transmission request for the target streaming media object to the data source.

4. The method according to claim 3, characterized in that, The sorting relationship descriptor further includes a combination determination field for indicating whether streaming media objects are to be sorted in a combined manner, and a combination identifier field for indicating the range of combined sorting; before grouping streaming media objects with the same sorting identifier field into a sorting relationship set, the method further includes: The decision on whether to perform combined sorting of streaming media objects is determined based on the combined judgment field. If the value of the combination determination field is the target field value, then streaming media objects with the same combination identifier field will be grouped into an object group with a sorting binding relationship.

5. The method according to claim 4, characterized in that, Sort the streaming media objects according to the sorting information field, including: The streaming media objects or combinations of objects are sorted according to the sorting information field to obtain a sequence of streaming media objects composed of the streaming media objects or combinations of objects.

6. The method according to any one of claims 1 to 5, characterized in that, The sorting relationship descriptor also includes a range quantity field, which is used to indicate the number of sorting ranges to which the streaming media object belongs.

7. The method according to any one of claims 1 to 5, characterized in that, The sorting information field includes at least one of quality sorting information, priority sorting information, or undefined sorting information. The quality sorting information is used to indicate the sorting level of streaming media objects according to streaming media quality. The priority sorting information is used to indicate the sorting level of streaming media objects according to selected priority. The undefined sorting information is used to indicate the sorting level of streaming media objects according to specified rules.

8. The method according to any one of claims 1 to 5, characterized in that, The streaming media object includes at least one of a bitstream file, an adaptive subset, or a preselection set. The bitstream file is a file object composed of one or more streaming media resources. The adaptive subset is a file object composed of one or more bitstream files. The preselection set is a file object obtained by combining the bitstream file or the adaptive subset according to preset rules.

9. The method according to claim 8, characterized in that, The streaming media resource is a point cloud media resource; the streaming media object includes a geometric component and an attribute component, wherein the geometric component is an adaptive subset for representing the geometric components of the point cloud, and the attribute component is an adaptive subset for representing the attribute components of the point cloud.

10. A streaming media transmission method, characterized in that, include: Send streaming media transmission signaling to the data receiver, wherein the streaming media transmission signaling carries a sorting relationship descriptor, the sorting relationship descriptor being used to represent sorting information within a specified range; Based on the streaming media transmission request sent by the data receiver, a target streaming media object is returned to the data receiver, wherein the target streaming media object is an object selected from the specified range according to the sorting relationship descriptor.

11. The method according to claim 10, characterized in that, Before sending streaming media transmission signaling to the data receiver, the method further includes: Obtain the sorting range of the streaming media object and the sorting criteria used to sort the streaming media object; Assign a value to the sort identifier field of the sorting relationship descriptor based on the sorting range in which the streaming media object is located; The sorting information field of the sorting relationship descriptor is assigned a value according to the sorting criteria of the streaming media object.

12. The streaming media transmission method according to claim 11, characterized in that, Obtaining the sorting range of the streaming media object and the sorting criteria used to sort the streaming media object, including: Multiple streaming media objects can be categorized to obtain multiple different sorting ranges; Within the sorting range, obtain the sorting criteria used to sort the streaming media objects.

13. The streaming media transmission method according to claim 12, characterized in that, Before sending the streaming media transmission signaling carrying the sorting relation descriptor to the data receiver, the method further includes: Count the number of sorting ranges containing the streaming media object; The range quantity field of the sorting relation descriptor is assigned a value based on the range quantity.

14. The streaming media transmission method according to any one of claims 10 to 13, characterized in that, Before sending the streaming media transmission signaling carrying the sorting relation descriptor to the data receiver, the method further includes: Compare multiple available streaming media objects to determine if there are any order binding relationships between them; If there is a sorting binding relationship between at least two streaming media objects, then the combination identifier field of the sorting relationship descriptor is assigned a value according to the sorting binding relationship, and the combination determination field of the sorting relationship descriptor is assigned a value according to the target field value.

15. A streaming media transmission device, characterized in that, include: The parsing module is configured to parse streaming media transmission signaling to obtain a sorting relationship descriptor, which is used to represent sorting information within a specified range; The selected module is configured to select a target streaming media object to be received from the specified range based on the sorting relationship descriptor; The sending module is configured to send a streaming media transmission request for the target streaming media object to the data source.

16. A streaming media transmission device, characterized in that, include: The signaling sending module is configured to send streaming media transmission signaling to the data receiver. The streaming media transmission signaling carries a sorting relationship descriptor, which is used to represent sorting information within a specified range. The object sending module is configured to return a target streaming media object to the data receiver based on the streaming media transmission request sent by the data receiver. The target streaming media object is an object selected from the specified range according to the sorting relationship descriptor.

17. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program that, when executed by a processor, implements the streaming media transmission method according to any one of claims 1 to 14.

18. An electronic device, characterized in that, include: processor; as well as Memory, used to store computer programs; The processor is configured to cause the electronic device to perform the streaming media transmission method according to any one of claims 1 to 14 by executing the computer program.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the streaming media transmission method according to any one of claims 1 to 14.

Images