Image frame transmission method, image frame encoding device, image frame decoding device and medium
By employing a balanced packet distribution strategy and adding auxiliary information, the problem of data packet disorder in large-scale data transmission for AR devices was solved, achieving stable, low-latency image transmission and playback effects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GEER TECH CO LTD
- Filing Date
- 2024-10-31
- Publication Date
- 2026-05-01
AI Technical Summary
When AR devices receive large amounts of data, the data packets are prone to becoming out of order, resulting in poor playback quality.
A balanced packet splitting strategy is adopted to process the encoded data into packets, and auxiliary fragment numbers and image frame numbers are added to the data packets to ensure the orderliness of data packet transmission.
It achieves stable, low-latency transmission of large image data packets, improves the image processing effect of AR devices, avoids data packet out-of-order issues, and ensures the stability of playback effects.
Smart Images
Figure CN121967709A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of augmented reality technology, and in particular to image frame transmission methods, image frame encoding devices, image frame decoding devices, and media. Background Technology
[0002] The AR (Augmented Reality) modular approach refers to separating different components of an AR device. These separated components are connected wirelessly or via wired connections, providing a more flexible way to use the AR device. The AR components, such as the puck component, after being separated from the main body, need to process image data in real time and transmit the generated virtual information to the display device of the main AR device.
[0003] During data transmission, when the data volume is large, the encoded data needs to be packetized. Auxiliary information is then added to the header of the packetized image frames to facilitate unpacking by the AR device. In related transmission methods, image data is typically transmitted based on a fixed packet size. However, when transmitting large amounts of data, the same video frame is split into different data packets, wasting bandwidth and causing the AR device to receive out-of-order data packets. This can lead to delays and dropped frames in the content played by the AR device.
[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention
[0005] The main purpose of this application is to provide an image frame transmission method, an image frame encoding device, an image frame decoding device, and a medium, in order to solve the technical problem that data packets received by AR devices are prone to being out of order, resulting in poor playback effects of AR devices.
[0006] To achieve the above objectives, this application proposes an image frame transmission method applied to the encoding side, the image frame transmission method comprising:
[0007] Obtain the encoded data of the image frame to be transmitted, and determine the data length of the encoded data;
[0008] If the data length is greater than the preset length, the encoded data is divided into packets based on the balanced packet division strategy to generate at least two sub-encoded data.
[0009] The sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted are used as the data segment of the data packet;
[0010] The data packet is sent to the decoding side.
[0011] In one embodiment, the data segment is ordered as follows: the sub-coded data, the image frame number of the image frame to be transmitted, and the fragment number corresponding to the sub-coded data; or
[0012] The data segment is ordered as follows: the image frame number of the image frame to be transmitted, the segment number corresponding to the sub-encoded data, and the sub-encoded data.
[0013] In one embodiment, the step of dividing the encoded data into packets based on a balanced packet division strategy to generate at least two sub-encoded data if the data length is greater than a preset length includes:
[0014] If the data length is greater than the preset length, determine the quotient of the data length and the preset length, and set the rounded-up value of the quotient as the number of packets of the encoded data;
[0015] Based on the number of packets and the data length, the packet size is determined, and the encoded data is subjected to balanced packet splitting processing according to the number of packets and the packet size to generate at least two sub-encoded data.
[0016] In one embodiment, the step of acquiring the encoded data of the image frame to be transmitted and determining the data length of the encoded data includes:
[0017] If an image frame transmission instruction is received, the data to be encoded is determined, and the data to be encoded is converted to a different color space to obtain the first data.
[0018] The first data is encoded based on a preset encoding rule to obtain the encoded data of the image frame to be transmitted.
[0019] Determine the data length of the encoded data of the image frame to be transmitted.
[0020] In one embodiment, before the step of using the sub-coded data, the fragment number corresponding to the sub-coded data, and the image frame number of the image frame to be transmitted as the data segment of the data packet, the method further includes...
[0021] Determine the sub-data length of the sub-encoded data after packet processing, and the difference between the preset length and the auxiliary information length;
[0022] If the length of the sub-data is less than the difference, the step of using the sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted as the data segment of the data packet is executed.
[0023] In one embodiment, after the steps of acquiring the encoded data of the image frame to be transmitted and determining the data length of the encoded data, the method further includes:
[0024] If the data length is less than or equal to the preset length, the image frame number of the image frame to be transmitted and the encoded data are used as the data segment of the data packet.
[0025] Furthermore, to achieve the above objectives, this application also proposes an image frame transmission method applied to the decoding side, the image frame transmission method comprising:
[0026] Receive data packets sent by the encoding side;
[0027] Based on the data segment of the data packet, the encoded data is parsed out, and the corresponding fragment number and image frame number of the encoded data are obtained.
[0028] Based on the encoded data, the corresponding fragment number, and the image frame number, the data packet is rearranged to obtain the target image frame data.
[0029] In addition, to achieve the above objectives, this application also proposes an image frame encoding device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image frame transmission method described above.
[0030] In addition, to achieve the above objectives, this application also proposes an image frame decoding device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image frame transmission method described above.
[0031] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the image frame transmission method described above.
[0032] One or more technical solutions proposed in this application have at least the following technical effects:
[0033] After obtaining the encoded data of the image frame to be transmitted, if the data length of the encoded data is greater than the preset length, the encoded data is processed by equalization packetization. At the same time, auxiliary packetization information such as the fragment number of the encoded data and the image frame number of the image frame to be transmitted are added to the data segment position of the data packet after equalization packetization. Finally, the data packet is sent to the decoding side. Based on this, by effectively packetizing and auxiliary packetization processing of the image frame to be transmitted, the data packet transmission will not be out of order, realizing stable and low-latency transmission of large image data packets and improving the image processing effect of AR devices. Attached Figure Description
[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is a flowchart illustrating the first embodiment of the image frame transmission method of this application.
[0037] Figure 2 This is a schematic diagram of the structure of the data segment of the image encoded data in the image frame transmission method of this application;
[0038] Figure 3 A simplified flowchart illustrating the image frame transmission method provided in the first embodiment of this application;
[0039] Figure 4 A flowchart illustrating the second embodiment of the image frame transmission method of this application.
[0040] Figure 5 This is a schematic diagram of data transmission between the encoding and decoding sides in the image frame transmission method of this application embodiment;
[0041] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the image frame transmission method in the embodiments of this application.
[0042] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0043] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0044] The main solution of this application embodiment is: to obtain the encoded data of the image frame to be transmitted and to determine the data length of the encoded data;
[0045] If the data length is greater than the preset length, the encoded data is divided into packets based on the balanced packet division strategy to generate at least two sub-encoded data.
[0046] The sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted are used as the data segment of the data packet;
[0047] The data packet is sent to the decoding side.
[0048] Specifically, in this embodiment, if the length of the encoded data is greater than the preset length, it indicates that the image frame to be transmitted needs to be packetized. In this case, after packetizing the data using a balanced packetization strategy, auxiliary packetization information such as the fragment number of the sub-encoded data and the image frame number of the image frame to be transmitted are added to the data segment. Thus, when transmitting a large amount of image data, the packetization strategy can achieve stable and low-latency transmission of data packets. At the same time, the auxiliary packetization information in the data segment ensures that the AR device can efficiently assemble the data packets after receiving them, thereby improving the image processing effect of the AR device.
[0049] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0050] This application provides an image frame transmission method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image frame transmission method of this application.
[0051] In this embodiment, the image frame transmission method is applied to the encoding side, which is the PUCK of the augmented reality device, wherein the PUCK and the augmented reality device are separately configured. Based on this, the image frame transmission method includes steps S10 to S30:
[0052] Step S10: Obtain the encoded data of the image frame to be transmitted, and determine the data length of the encoded data.
[0053] It should be noted that encoded data refers to the actual encoded content of the image frame to be transmitted, and data length refers to the number of bytes occupied by the encoded data. The encoded data of the image frame to be transmitted refers to the data that has undergone encoding processing in the PUCK encoder. Specifically, during the process of sending data to the augmented reality device, the data to be sent needs to be encoded and compressed, and the encoded data is the result of this encoding and compression process.
[0054] When the augmented reality device transmits data to the PUCK in the split configuration, the number of bytes transmitted is usually less than the maximum number of bytes specified by the transmission protocol to ensure stable data transmission. Therefore, in this embodiment, after obtaining the encoded data of the image frame to be transmitted, the data length of the encoded data can be directly read to determine whether it is necessary to perform packet processing on the encoded data of the image frame to be transmitted based on the data length.
[0055] As an optional implementation, during the process of acquiring the encoded data of the image frame to be transmitted, if a transmission instruction for the image frame is received, the data to be encoded is first determined, and the data to be encoded is converted to a different color space to obtain first data. The data to be encoded is usually in RGB (Red Green Blue) format, which needs to be converted to YUV (Luminance Chrominance) format for data compression encoding. After obtaining the first data, it is also necessary to encode the first data according to a preset encoding rule, such as encoding it according to the H.265 or H.264 encoding rule, to obtain the encoded data of the image frame to be transmitted according to the H.265 or H.264 encoding standard. Finally, the data length of the encoded data of the image frame to be transmitted is determined.
[0056] For example, after the PUCK of the enhanced display device receives the raw data (RGB format data) to be encoded, the data is sent to its internal encoder for encoding. The encoder performs C2D conversion on the received RGB data, converting the RGB data into YUV data for subsequent compression processing. Then, the YUV data is encoded using the H.265 or H.264 encoding standard to obtain H.265 or H.264 data.
[0057] It should be noted that the encoding format of the data to be encoded and the compressed format of the encoded data shown in the above examples are for illustrative purposes only and are not intended to limit this application.
[0058] Step S20: If the data length is greater than the preset length, the encoded data is divided into packets based on the balanced packet division strategy to generate at least two sub-encoded data.
[0059] In this embodiment, the length of the encoded data output after encoding and compression is analyzed and judged. If the length of the data is greater than the preset length, for example, the transmission protocol of PUCK and augmented reality devices is UDP (User Datagram Protocol), which has a preset length of 65507 bytes. If the length of the data (encoded data + auxiliary information) is greater than the preset length, then the encoded data needs to be processed by balanced packet splitting to generate at least two sub-encoded data.
[0060] Balanced packet splitting strategy refers to the balanced distribution of encoded data. When splitting data, the number of packets and their sizes must first be determined, and then packet splitting is performed based on these factors. Specifically, as an optional implementation of balanced packet splitting strategy, if the data length is greater than the preset length, the quotient of the data length and the preset length can be determined. The rounded-up value of the quotient is then set as the number of packets for the encoded data. Finally, the packet size is determined using the number of packets and the data length, and balanced packet splitting is performed based on these factors. For example, if the current data length is 66,000 bytes and the preset length is 65,507 bytes, the quotient is rounded up, resulting in a packet size of 2. The encoded data needs to be split into two sub-encoded data packets, each with a data packet length of 33,000 bytes.
[0061] In another optional implementation, a preset packet size corresponding to the balanced packet splitting rule can be determined, and a preset number of packets can be determined based on the preset packet size and the data length. Then, based on the preset packet size and the preset number of packets, the encoded data is subjected to balanced packet splitting processing. For example, if the current data length is 66,000 bytes and the preset packet size is 40,000 bytes, it can be split into two data packets of 40,000 bytes and 26,000 bytes respectively.
[0062] Understandably, the conventional packet splitting method involves encapsulating data exceeding a preset byte size into another packet before sending it. For example, 66,000 bytes of data might be split into two packets: one 65,507 bytes and the other 493 bytes, and then sent sequentially. However, because the first packet's size is much smaller than the second, even with sequential packet sending, the second packet might arrive first, resulting in out-of-order delivery. The receiving end then needs time to adjust for this, leading to delays. By using balanced packet splitting rules, out-of-order delivery can be effectively reduced. By balancing packet sizes, out-of-order delivery is minimized, enabling stable, low-latency transmission of large image data packets.
[0063] Step S30: The sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted are used as the data segment of the data packet.
[0064] In this embodiment, the data segment of the data packet includes the valid data to be transmitted, namely the sub-coded data, as well as the image frame number of the image frame to be transmitted, which facilitates the decoding end, namely the augmented reality device, to perform auxiliary packet identification of the sub-coded data, and auxiliary packet information such as the fragment number corresponding to the sub-coded data.
[0065] Therefore, as an optional implementation, the data sequence of the data segment is as follows: sub-coded data, the image frame number of the image frame to be transmitted, and the fragment number corresponding to the sub-coded data. That is, auxiliary packet information is added to the end of the valid data of the data segment. Optionally, the auxiliary packet information may also include timestamps, residual information, and other auxiliary packet information.
[0066] For example, the data segment structure of a data packet after packetization is as follows: Figure 2 As shown, 1 represents the encoded data output by the encoder or the sub-encoded data after packetization, which is in H265 / H264 format. 2-5 are auxiliary packetization information. 2 is the current image frame number, represented by frameId. 3 represents the slice number SliceId corresponding to the data frame. 4 represents the timestamp Time. 5 is other auxiliary information, which can be empty.
[0067] Optionally, auxiliary packet information such as image frame number and segment number can be added before the sub-coded data, i.e. before the valid data.
[0068] It should be noted that when auxiliary packet information is added before the valid data in a data segment, the encoder's encoding rules on the encoding side are fixed. Therefore, when new auxiliary packet information, such as timestamps or residual information, needs to be added, the position of the sub-encoded data in the data segment must be adjusted. During this adjustment process, the sub-encoded data needs to be copied to another memory location before processing. This method increases data packet transmission time, leading to delays in data packet transmission.
[0069] Therefore, in actual processing, auxiliary packet information is usually added to the end of the sub-encoded data, that is, the end of the effective data. So when additional auxiliary packet information needs to be added, the data can be added directly to the end of the data segment. By adding auxiliary packet information to the end of the data segment of the image encoded data packet, there is no need to copy the contents of the data segment to another memory. Thus, when transmitting a large amount of data, stable packet segmentation and stable low-latency transmission of large image data packets can be achieved through auxiliary packet information.
[0070] Optionally, if the data length is less than or equal to the preset length, it means that packet processing is not required at present. In this case, the image frame number of the image frame to be transmitted and the encoded data can be directly used as the data segment of the data packet, and then the data packet can be sent to the augmented reality device.
[0071] Step S40: Send the data packet to the decoding side.
[0072] In this embodiment, after the data processing is completed, the packetized data packets can be sent to the decoding side in the order of packetization.
[0073] For example, to aid in understanding the implementation process corresponding to steps S10-S40, the data transmission protocol is UDP. Please refer to... Figure 3 After obtaining the image frame to be transmitted, if the data length n of the encoded data plus auxiliary information of the image frame to be transmitted is greater than the preset 65507 bytes, then balanced packet splitting is performed. The number of balanced packets is [n / 65507] (i.e., the floor value of the quotient), and the size of the balanced packets is [n / (n / 65507)]. After the packet splitting process, the image frame number FrameId of the image frame to be transmitted and the slice number ScliceId corresponding to the encoded data in the image frame number are added to the end of the data segment. Finally, the packetized data packet is sent to the augmented reality device. Optionally, if n is less than or equal to the preset length, the image frame number FrameId can be directly added to the end of the data segment, and the data packet can be sent directly.
[0074] This embodiment provides an image frame transmission method that encodes, compresses, and packets the data to be transmitted. During the packetization process, the encoded data is evenly packetized to effectively avoid situations where data packets arrive before the data packets arrive during packet transmission, which could cause delays or stuttering in the output of the augmented reality device. This reduces out-of-order packet transmission and achieves low-latency transmission. At the same time, auxiliary packetization information is added to the end of the data segment of the data packet to achieve stable packetization and reassembly, thereby improving the playback effect of the augmented reality device.
[0075] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 After step S20, the image frame transmission method further includes steps S50 to S60:
[0076] Step S50: Determine the sub-data length of the sub-encoded data after packet processing, and the difference between the preset length and the auxiliary information length;
[0077] Step S60: If the length of the sub-data is less than the difference, perform the step of using the sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted as the data segment of the data packet.
[0078] It should be noted that, since auxiliary packet information such as fragment numbers and image frame numbers needs to be added to the data segment, and the length of the data after adding the sub-encoded data to this auxiliary packet information cannot exceed the preset length, as an optional implementation, if the transmission protocol is UDP, the preset length can be 65507 minus the length of the auxiliary packet information. Here, the auxiliary information length refers to the data length occupied by the auxiliary packet information.
[0079] In this embodiment, if the length of the sub-data is less than the difference between the preset length and the length of the auxiliary information, the segment number corresponding to the sub-coded data and the image frame number of the image frame to be transmitted can be directly added to the end of the data segment.
[0080] This embodiment provides an image frame transmission method. By comparing the difference between the sub-data length and the preset length and the auxiliary information length, when the sub-data length is greater than the difference, auxiliary packet information is added to the end of the data segment without moving the original data in the data segment. This achieves stable and low-latency transmission of data packets when transmitting a large amount of image data, ensuring that data packets are not out of order and improving the image processing effect of virtual reality devices.
[0081] Based on the first and second embodiments of this application, in the third embodiment, the image frame transmission method is applied to the decoding side, which can be an augmented reality device. Accordingly, if the augmented reality device receives a data packet sent by the encoding side, it needs to parse the encoded data, the corresponding fragment number, and the image frame number from the data packet's data segment. Then, based on the encoded data, the corresponding fragment number, and the image frame number, the data packet is reordered to obtain the target image frame data. The reordered data packet does not contain the auxiliary packet information; that is, the auxiliary packet information at the end of the data segment needs to be deleted during packet reordering. Finally, the reordered data packet is decoded, and the decoded image data is output.
[0082] Specifically, in large image data transmission scenarios, after receiving data packets, the augmented reality device first performs out-of-order packet rearrangement and packet reassembly. During this process, auxiliary packet information is also removed. The reassembled data packets contain only normal data, ensuring that the reassembled packets do not contain auxiliary packet information. After packet reassembly, the processed data packets are sent to the decoder. The decoder decodes the reassembled data packets for subsequent output display. Finally, the decoded image data is output to the display terminal.
[0083] For example, to help understand the data processing and interaction implementation flow of the image frame transmission method obtained by combining the first and second embodiments described above, please refer to... Figure 5The encoding side is the PUCK, a separate component of the virtual reality device, while the decoding side is the GLASS glasses, the main body of the virtual display device. In this process, the PUCK encodes and performs balanced packet processing on the image data. The data segments of the packetized data contain auxiliary packet information. The packetized data is then transmitted to the glasses via wireless (e.g., WIFI) or wired (e.g., USB). The glasses then perform packet assembly based on the auxiliary packet information, decode the assembled data, and output the decoded data for display.
[0084] In this embodiment, the data segments are processed based on the auxiliary packetization information at the end of the data segments of the received data packets, thereby effectively reordering the packetized data packets and avoiding delays or stuttering in the images output by the virtual reality device, thus improving the data quality.
[0085] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the image frame transmission method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.
[0086] This application provides an image frame transmission device, which can be the encoding side or the decoding side of image frame processing. The image frame transmission device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image frame transmission method described in the first or third embodiment above.
[0087] The following is for reference. Figure 6 It shows a schematic diagram of the structure of an image frame transmission device suitable for implementing the embodiments of this application. Figure 6 The image frame transmission device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0088] like Figure 6As shown, the image frame transmission device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the image frame transmission device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the image frame transmission device to communicate wirelessly or wiredly with other devices to exchange data. Although image frame transmission devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0089] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0090] The image frame transmission device provided in this application, employing the image frame transmission method described in the above embodiments, can solve the technical problem that data packets received by AR devices are prone to being out of order, resulting in poor playback effects on AR devices. Compared with the prior art, the beneficial effects of the image frame transmission device provided in this application are the same as those of the image frame transmission method provided in the above embodiments, and other technical features in this image frame transmission device are the same as those disclosed in the previous embodiment method, and will not be repeated here.
[0091] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0092] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0093] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the image frame transmission method described in the above embodiments.
[0094] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.
[0095] The aforementioned computer-readable storage medium may be included in the image frame transmission device; or it may exist independently and not be assembled into the image frame transmission device.
[0096] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the image frame transmission device, cause the encoding side of the image frame transmission device to:
[0097] Obtain the encoded data of the image frame to be transmitted, and determine the data length of the encoded data;
[0098] If the data length is greater than the preset length, the encoded data is divided into packets based on the balanced packet division strategy to generate at least two sub-encoded data.
[0099] The sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted are used as the data segment of the data packet;
[0100] The data packet is sent to the decoding side.
[0101] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0103] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0104] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image frame transmission method. This solves the technical problem that data packets received by AR devices are easily out of order, resulting in poor playback performance. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image frame transmission method provided in the above embodiments, and will not be repeated here.
[0105] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. An image frame transmission method, characterized in that, Applied to the encoding side, the image frame transmission method includes: Obtain the encoded data of the image frame to be transmitted, and determine the data length of the encoded data; If the data length is greater than the preset length, the encoded data is divided into packets based on the balanced packet division strategy to generate at least two sub-encoded data. The sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted are used as the data segment of the data packet; The data packet is sent to the decoding side.
2. The image frame transmission method as described in claim 1, characterized in that, The data segment is ordered as follows: the sub-coded data, the image frame number of the image frame to be transmitted, and the segment number corresponding to the sub-coded data; or The data segment is ordered as follows: the image frame number of the image frame to be transmitted, the segment number corresponding to the sub-encoded data, and the sub-encoded data.
3. The image frame transmission method as described in claim 1, characterized in that, If the data length is greater than a preset length, the step of splitting the encoded data into packets based on a balanced packet splitting strategy to generate at least two sub-encoded data includes: If the data length is greater than the preset length, determine the quotient of the data length and the preset length, and set the rounded-up value of the quotient as the number of packets of the encoded data; Based on the number of packets and the data length, the packet size is determined, and the encoded data is subjected to balanced packet splitting processing according to the number of packets and the packet size to generate at least two sub-encoded data.
4. The image frame transmission method as described in claim 1, characterized in that, The steps of acquiring the encoded data of the image frame to be transmitted and determining the data length of the encoded data include: If an image frame transmission instruction is received, the data to be encoded is determined, and the data to be encoded is converted to a different color space to obtain the first data. The first data is encoded based on a preset encoding rule to obtain the encoded data of the image frame to be transmitted. Determine the data length of the encoded data of the image frame to be transmitted.
5. The image frame transmission method as described in claim 1, characterized in that, Before the step of using the sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted as the data segment of the data packet, the method further includes... Determine the sub-data length of the sub-encoded data after packet processing, and the difference between the preset length and the auxiliary information length; If the length of the sub-data is less than the difference, the step of using the sub-encoded data, the fragment number corresponding to the sub-encoded data, and the image frame number of the image frame to be transmitted as the data segment of the data packet is executed.
6. The image frame transmission method as described in claim 1, characterized in that, After the steps of acquiring the encoded data of the image frame to be transmitted and determining the data length of the encoded data, the method further includes: If the data length is less than or equal to the preset length, the image frame number of the image frame to be transmitted and the encoded data are used as the data segment of the data packet.
7. An image frame transmission method, characterized in that, Applied to the decoding side, the image frame transmission method includes: Receive data packets sent by the encoding side; Based on the data segment of the data packet, the encoded data is parsed out, and the corresponding fragment number and image frame number of the encoded data are obtained. Based on the encoded data, the corresponding fragment number, and the image frame number, the data packet is rearranged to obtain the target image frame data.
8. An image frame encoding device, characterized in that, The image frame encoding device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image frame transmission method as described in any one of claims 1 to 6.
9. An image frame decoding device, characterized in that, The image frame decoding device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image frame transmission method as described in claim 7.
10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the image frame transmission method as described in any one of claims 1 to 7.