Max plugin assisted video data transmission method and system

Abstract

The application provides a Max plug-in assisted video data transmission method and system, relates to the technical field of video transmission optimization, and comprises the following steps: Max plug-in is used to analyze and monitor a video stream, video content information containing a plurality of semantic areas and associated position identifiers and importance parameters is obtained, a plurality of encoding code streams are generated by using different encoding strategies according to the importance parameters, a composite code stream is obtained through bit domain non-orthogonal multiplexing, transmission resources and encoding parameters are dynamically adjusted in combination with real-time network states and the importance parameters during transmission, video data is recovered through sequential interference cancellation decoding at a receiving end, and finally, a Max plug-in drives a digital twin scene to realize differential three-dimensional rendering and presentation according to the recovered data and the associated position identifiers and the importance parameters, so that efficient transmission of monitoring video data and differential three-dimensional rendering and presentation of the digital twin scene can be realized.

Description

Technical Field

[0001] This application relates to the field of video transmission optimization technology, and in particular to a Max plugin-assisted video data transmission method and system. Background Technology

[0002] Video data transmission is a crucial link connecting physical scenes and digital applications, enabling the long-distance transmission and sharing of real-time video information. With the widespread adoption of technologies such as digital twins and smart surveillance, the demand for video data in smart cities, industrial production, and emergency command continues to grow. Efficient transmission methods can ensure the real-time nature and integrity of video data, providing support for subsequent scene rendering and decision analysis, and thus have a very broad application prospect.

[0003] In current digital twin scenarios, video data transmission often employs direct copying and streaming or transmission after compression using a uniform standard. Some technologies adjust the encoding format based on fixed parameters or adapt to the network using traditional adaptive transmission protocols, but most do not differentiate the video content itself. These methods generally follow a holistic transmission approach, encoding and transmitting the video stream as a complete data block without dynamically optimizing based on content importance and real-time network conditions.

[0004] Existing technologies employ relatively simplistic encoding strategies, making it difficult to allocate resources based on the importance of different content within the video. This results in insufficient protection for critical information transmission or excessive bandwidth consumption for non-critical information. Furthermore, the lack of flexible adaptation to network fluctuations during transmission easily leads to issues such as stuttering and image quality degradation, consequently affecting the rendering accuracy and real-time performance of digital twin scenes. These shortcomings prevent video transmission efficiency from fully matching actual needs in terms of scene presentation. Therefore, existing technologies face the technical challenge of achieving coordinated optimization between video data transmission adaptability and digital twin presentation quality. Summary of the Invention

[0005] The purpose of this application is to provide a Max plugin-assisted video data transmission method and system to solve the problem that it is difficult to coordinate the optimization of video data transmission adaptability and digital twin presentation quality in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a Max plugin-assisted video data transmission method, comprising:

[0007] The input surveillance video stream is parsed using the Max plugin to obtain video content information, which includes multiple semantic regions divided according to the video frame, as well as location identifiers and importance parameters associated with the semantic regions.

[0008] Based on the importance parameters in the video content information, different encoding strategies are used to encode the video data corresponding to each semantic region in order to generate multiple encoded bitstreams;

[0009] The multi-coded bitstreams are non-orthogonally multiplexed in the bit domain to generate a composite bitstream for transmission;

[0010] During the transmission of the composite bitstream, the transmission resources and encoding parameters used for transmission are dynamically adjusted according to the real-time network status and the importance parameters in the video content information.

[0011] At the receiving end, sequential interference cancellation decoding is performed on the received composite bitstream to recover the video data corresponding to each semantic region, thus obtaining the recovered video data;

[0012] Based on the restored video data, as well as the location identifier and importance parameter associated with the restored video data, the Max plugin drives the digital twin scene to perform differentiated 3D rendering and presentation.

[0013] Optionally, the step of driving the digital twin scene to perform differentiated 3D rendering and presentation based on the restored video data, the location identifier associated with the restored video data, and the importance parameter, by the Max plugin, includes:

[0014] Based on the location identifier, the recovered video data of each semantic region is used as a dynamic texture and bound to the corresponding 3D model surface in the digital twin scene;

[0015] The corresponding rendering level is determined based on the importance parameters of each semantic region;

[0016] Based on the rendering level, adjust the texture mapping quality, lighting computation complexity, and display priority of the corresponding 3D model surface in the digital twin scene;

[0017] The 3D graphics engine, driven by the Max plugin, renders the digital twin scene and outputs the display screen based on the adjusted texture mapping quality, lighting calculation complexity, and display priority.

[0018] Optionally, the step of encoding the video data corresponding to each semantic region using different encoding strategies based on the importance parameters in the video content information to generate multiple encoded bitstreams includes:

[0019] Based on the importance parameter of each semantic region in the video content information, an encoding level is assigned to the semantic region;

[0020] Based on the encoding level, a set of preset encoding parameters are selected for each semantic region;

[0021] Extract pixel data for each semantic region in each video frame;

[0022] Based on the encoding parameters selected for each semantic region, the pixel data of each semantic region is compressed and encoded to generate an independent encoded bitstream;

[0023] All independent encoded bitstreams are aggregated to form the multi-encoded bitstream.

[0024] Optionally, after performing sequence interference cancellation decoding on the received composite bitstream to recover the video data corresponding to each semantic region and obtaining the recovered video data, the method further includes:

[0025] The quality of the recovered video data for each semantic region is evaluated to obtain the corresponding quality evaluation value;

[0026] The quality evaluation value of the video data in each semantic region is compared with a preset quality threshold. Based on the comparison result and the importance parameter of the corresponding semantic region, the video data is enhanced.

[0027] The enhancement process includes:

[0028] If the importance parameter of a certain semantic region is higher than a preset importance threshold and the quality evaluation value is lower than the quality threshold, then a request is made to retransmit the encoded bitstream corresponding to the semantic region.

[0029] If the importance parameter of a certain semantic region is lower than or equal to a preset importance threshold, and the quality evaluation value is lower than the quality threshold, then data compensation is performed on the semantic region based on the video data of adjacent semantic regions in the same video frame.

[0030] Optionally, the step of non-orthogonally multiplexing the multiplexed bitstreams in the bit domain to generate a composite bitstream for transmission includes:

[0031] For each of the multiple encoded bitstreams, the superposition coefficient is determined based on the importance parameter of the semantic region corresponding to the encoded bitstream;

[0032] In the bit field, the bit sequence of each encoded bit stream is multiplied by the superposition coefficient corresponding to the encoded bit stream to obtain the corresponding weighted bit sequence;

[0033] The weighted bit sequences corresponding to all encoded bit streams are superimposed and merged to form a composite bit sequence;

[0034] The composite bit sequence is modulated to generate a composite code stream for transmission.

[0035] Optionally, the step of performing sequence interference cancellation decoding on the received composite bitstream to recover the video data corresponding to each semantic region, thereby obtaining the recovered video data, includes:

[0036] The received composite bit stream is demodulated to obtain a composite bit sequence;

[0037] Based on the order of the superposition coefficients corresponding to each semantic region, multiple rounds of sequential decoding and interference cancellation operations are performed on the composite bit sequence to sequentially decode the bit sequence of the encoded code stream corresponding to each semantic region.

[0038] Based on the decoded bit sequences, video decoding is performed to recover the video data corresponding to each semantic region, which is then used as the recovered video data.

[0039] Optionally, dynamically adjusting the transmission resources and encoding parameters used for transmission based on the real-time network status and the importance parameters in the video content information includes:

[0040] The real-time network status is periodically acquired, including available bandwidth and channel bit error rate;

[0041] If the available bandwidth decreases, the encoding parameters are adjusted to increase the compression ratio; if the available bandwidth increases, the encoding parameters are adjusted to decrease the compression ratio.

[0042] If the channel bit error rate increases, the transmission resources allocated to the corresponding coded bitstream are adjusted according to the importance parameters of each semantic region, including: increasing the proportion of transmission resources for coded bitstreams corresponding to semantic regions with importance parameters higher than a first threshold, and correspondingly reducing the proportion of transmission resources for coded bitstreams corresponding to semantic regions with importance parameters lower than a second threshold.

[0043] Secondly, this application provides a Max plugin-assisted video data transmission system, comprising:

[0044] The acquisition module is used to parse the input monitoring video stream through the Max plugin and acquire video content information. The video content information includes multiple semantic regions divided according to the video frame, as well as location identifiers and importance parameters associated with the semantic regions.

[0045] The encoding module is used to encode the video data corresponding to each semantic region according to the importance parameters in the video content information using different encoding strategies, so as to generate multiple encoded bitstreams.

[0046] The processing module is used to perform non-orthogonal multiplexing of the multi-channel coded bitstream in the bit domain to generate a composite bitstream for transmission;

[0047] An adjustment module is used to dynamically adjust the transmission resources and encoding parameters used for transmission based on the real-time network status and the importance parameters in the video content information during the transmission of the composite bitstream.

[0048] The decoding module is used at the receiving end to perform sequence interference cancellation decoding on the received composite bitstream in order to recover the video data corresponding to each semantic region and obtain the recovered video data.

[0049] The rendering module is used to drive the digital twin scene to perform differentiated 3D rendering and presentation based on the restored video data, the location identifier associated with the restored video data, and the importance parameters, driven by the Max plugin.

[0050] Thirdly, this application provides an electronic device, comprising:

[0051] Memory, used to store computer programs;

[0052] A processor, configured to implement the steps of the Max plug-in assisted video data transmission method as described in the first aspect above when executing the computer program.

[0053] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the Max plug-in assisted video data transmission method described in the first aspect above.

[0054] The Max plugin-assisted video data transmission method provided in this application parses the monitoring video stream using the Max plugin to obtain semantic regions, location identifiers, and importance parameters, enabling accurate extraction of key video information and laying the foundation for subsequent differentiated processing. It generates multiple bitstreams using different encoding strategies based on importance parameters, ensuring the encoding quality of key content and rationally allocating encoding resources. Non-orthogonal multiplexing of bit fields generates composite bitstreams, effectively improving transmission efficiency and reducing redundancy. Dynamic adjustment of transmission resources and encoding parameters during transmission adapts to network fluctuations, ensuring priority and stable transmission of key content. Sequential interference elimination decoding at the receiving end accurately recovers video data from each semantic region, ensuring data integrity. The Max plugin drives differentiated digital twin rendering, achieving precise matching of video data with the scene, enhancing the relevance and intuitiveness of the presentation.

[0055] Furthermore, based on location identifiers, the restored video data of each semantic region is dynamically bound to the surface of the corresponding 3D model in the digital twin scene as textures. The rendering level is determined according to the importance parameters of each semantic region. Then, the texture mapping quality, lighting computation complexity, and display priority of the corresponding 3D model surface are adjusted according to the rendering level. Finally, the Max plugin drives the 3D graphics engine to render the digital twin scene according to the adjusted parameters and output the display screen. This achieves precise binding between restored video data and the digital twin 3D model. Through differentiated rendering settings based on importance parameters, the display effect of key semantic regions is highlighted, enhancing the layering and practicality of scene rendering, ensuring the relevance and intuitiveness of the output screen, and making the digital twin scene presentation more closely aligned with actual application needs. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a Max plugin-assisted video data transmission method provided in an embodiment of this application;

[0058] Figure 2 A flowchart illustrating another Max plugin-assisted video data transmission method provided in this application embodiment;

[0059] Figure 3 This is a schematic diagram of the structure of a Max plugin-assisted video data transmission system provided in an embodiment of this application. Detailed Implementation

[0060] In current video data transmission and digital twin scene presentation, traditional methods mostly adopt a unified encoding and overall transmission mode, without differentiating processing based on the importance of the video content. Furthermore, the lack of flexible adaptation to real-time network fluctuations during transmission often leads to unstable transmission of critical information and excessive bandwidth consumption of non-critical information, thus affecting the accuracy and real-time performance of digital twin scene rendering and failing to meet the dual requirements of transmission efficiency and presentation quality in practical applications.

[0061] To address the aforementioned shortcomings, this application proposes a Max plugin-assisted video data transmission method. The core of this method is to leverage the Max plugin to mine the semantic features and importance differences of video content, generate an adapted transmission bitstream through differentiated encoding, and dynamically adjust the transmission strategy based on real-time network conditions. Ultimately, targeted digital twin scene rendering is achieved at the receiving end. This method, through precise differentiation of video content and dynamic allocation of transmission resources, ensures stable and prioritized transmission of critical information while achieving rational utilization of bandwidth resources. It also enhances the layering and adaptability of the digital twin scene presentation, effectively solving the problem of difficulty in co-optimizing transmission adaptability and digital twin presentation quality in existing technologies, thus providing support for the efficient application of related scenarios.

[0062] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] The core of this application is to provide a Max plugin-assisted video data transmission method, the flowchart of one specific implementation of which is shown below. Figure 1 As shown, the method includes:

[0064] S101. Parse the input monitoring video stream using the Max plugin to obtain video content information.

[0065] The video content information includes multiple semantic regions divided according to the video frame, as well as location identifiers and importance parameters associated with the semantic regions.

[0066] Semantic regions are independent areas divided from video frames based on their content features. Each region corresponds to a specific type of monitoring content, such as pedestrians, equipment, or open areas. Location identifiers are unique identifiers assigned to each semantic region, used to establish a correspondence between the region and the spatial coordinates of the digital twin scene. Importance parameters are numerical values ​​that measure the importance of a semantic region, reflecting the value of the monitoring information in that region to practical applications.

[0067] In this embodiment, the Max plugin first establishes a connection channel with the monitoring video stream, reading the stream data frame by frame according to the time sequence of the video frames. Simultaneously, basic preprocessing is performed on the read frame data to remove image noise and correct frame synchronization deviations, ensuring the accuracy of subsequent analysis. For each preprocessed video frame, an AI image recognition algorithm, combined with image content features such as color distribution, object outlines, and motion trajectories, is used to divide the video frame into multiple independent semantic regions, such as pedestrian activity areas, equipment operation areas, and open background areas. Each region corresponds to a type of image content with clear monitoring significance.

[0068] Subsequently, a unique location identifier is assigned to each semantic region. This identifier consists of a frame number and a region number. Simultaneously, through a pre-defined mapping rule, the location identifier is associated with three-dimensional spatial coordinates in a pre-constructed digital twin scene, ensuring that the semantic region accurately corresponds to a specific location within the digital twin scene. In the importance parameter calculation stage, three core dimensions are comprehensively considered: first, the type of monitoring event within the semantic region, such as abnormal intrusion or equipment malfunction, which correspond to regions with higher weight; second, the intensity of motion of objects within the region, with more intensely moving regions requiring greater attention; and third, the size proportion of the region in the monitoring image, with larger regions having a more significant impact on the overall monitoring effect. An importance parameter in the 0-1 range is calculated using a weighted summation method, with parameters closer to 1 indicating greater region importance.

[0069] Finally, for each video frame, a one-to-one association and binding between semantic region, location identifier and importance parameter is established to form the video content information of a single frame; then, the association results of all video frames are integrated in time sequence to construct continuous and complete video content information data, providing accurate data support for subsequent encoding, transmission and rendering steps.

[0070] S102. Based on the importance parameters in the video content information, different encoding strategies are used to encode the video data corresponding to each semantic region to generate a multi-channel encoded bitstream.

[0071] S102 specifically includes:

[0072] S1021. Assign a coding level to the semantic region according to the importance parameter of each semantic region in the video content information.

[0073] The encoding level is a hierarchy of encoding priorities based on the semantic region importance parameter. It is used to match and adapt the encoding configuration to ensure that regions of different importance receive the corresponding encoding resources.

[0074] S1022. Based on the encoding level, select a set of preset encoding parameters for each semantic region.

[0075] Among them, encoding parameters are the core settings that control the video compression encoding process, including compression ratio, resolution, frame rate, etc., which directly determine the image quality and data size of the encoded data.

[0076] S1023. Extract the pixel data of each semantic region in each video frame.

[0077] S1024. Based on the encoding parameters selected for each semantic region, the pixel data of each semantic region is compressed and encoded to generate an independent encoded bitstream.

[0078] S1025. Summarize all independent encoded bitstreams to form the multi-channel encoded bitstream.

[0079] Among them, the multi-channel encoded bitstream is an independent encoded data set formed after each semantic region has been encoded. Each semantic region corresponds to one bitstream, which facilitates subsequent targeted transmission.

[0080] In this embodiment of the application, step S102 first assigns a corresponding coding level based on the importance parameter of each semantic region, and then selects preset coding parameters according to the coding level, wherein a higher level corresponds to a lower compression ratio. Then, the pixel data of each semantic region in each video frame is extracted, and the pixel data is compressed and encoded according to the selected coding parameters to generate an independent coding stream. Finally, all independent streams are aggregated to form a multi-channel coding stream, thereby realizing differentiated coding processing for semantic regions of different importance.

[0081] As an example, let's take a factory monitoring scenario:

[0082] First, coding levels are assigned to each semantic region in step S1021. A fixed threshold is set based on the importance parameter P (0-1 interval) to classify the levels, using the following determination rules:

[0083] When P ≥ 0.8, the coding level is Level 1;

[0084] When 0.5 ≤ P < 0.8, the coding level is level two;

[0085] When P < 0.5, the coding level is level three.

[0086] Where P is the importance parameter of the semantic region, ranging from 0 to 1, with a larger value indicating a more important region. For example, in factory monitoring, the importance parameter of the equipment operation area containing key production equipment is calculated to be 0.92, meeting the condition of P≥0.8, and is assigned as Level 1; the importance parameter of the pedestrian passage area for daily personnel passage is 0.65, meeting the condition of 0.5≤P<0.8, and is assigned as Level 2; the importance parameter of the open area in the corner is 0.3, meeting the condition of P<0.5, and is assigned as Level 3.

[0087] Secondly, in step S1022, preset encoding parameters are selected based on the encoding level. The preset encoding parameters include compression ratio, resolution, and frame rate. In this example, the preset parameter configuration is shown in Table 1 below:

[0088] Table 1. Correspondence between encoding levels and encoding parameter configurations

[0089]

[0090] As shown in Table 1, the encoding parameters selected for the equipment operating area (Level 1) are 2:1 compression ratio, 1080P resolution, and 30fps frame rate; the encoding parameters selected for the pedestrian passage area (Level 2) are 5:1 compression ratio, 720P resolution, and 24fps frame rate; and the encoding parameters selected for the corner open area (Level 3) are 10:1 compression ratio, 480P resolution, and 15fps frame rate.

[0091] Next, pixel data is extracted in step S1023. Using region masking technology after image segmentation, all pixel information within each semantic region is accurately extracted, ensuring that redundant pixels outside the region are not included. For example, for the semantic region mask of the device operating area, only the relevant pixel data of the device body, operating indicator lights, etc., within this region are extracted, excluding surrounding irrelevant background pixels.

[0092] Then, compression encoding is performed in step S1024 using the mainstream H.265 encoding algorithm. The pixel data of each semantic region is compressed according to the corresponding encoding parameters to generate an independent encoded bitstream. During the encoding process, the algorithm controls the data compression intensity based on the compression ratio parameter. For example, when the first-level region is compressed at a 2:1 ratio, only a small amount of redundant pixel information is removed; when the third-level region is compressed at a 10:1 ratio, non-critical details are appropriately simplified to reduce the data volume. Finally, an independent encoded bitstream is generated for each semantic region.

[0093] Finally, in step S1025, all independent bitstreams are aggregated. The independent encoded bitstreams corresponding to all semantic regions, such as the device operating area, pedestrian passage area, and open corner area, are aligned by timestamp and integrated to form a complete multi-channel encoded bitstream. The above example is only one example of this application. In practical applications, the threshold division rules and encoding parameter configurations can be adjusted according to the needs of the monitoring scenario. This application does not limit this.

[0094] In another specific implementation, the coding level can be divided using a dynamic threshold strategy, by calculating the mean of the importance parameters of all semantic regions. and standard deviation ,according to" (Level 1) (Level 2) The "(Level 3)" rule classifies encoding levels to better suit the characteristics of video content in different monitoring scenarios. Encoding parameters can also be dynamically adjusted based on real-time encoding resource usage. For example, when encoding server resources are scarce, the compression ratio of the Level 2 region can be appropriately increased, such as from 5:1 to 6:1, while ensuring that the encoding quality of the Level 1 region remains unchanged, to further reduce resource consumption.

[0095] Through the above steps, this application achieves differentiated encoding of semantic regions, which not only ensures the encoding quality of key monitoring information, but also effectively controls the amount of encoded data for non-key information, improves overall encoding efficiency, and provides support for the optimized allocation of resources in subsequent transmission stages.

[0096] S103. The multi-channel encoded bitstream is non-orthogonally multiplexed in the bit domain to generate a composite bitstream for transmission.

[0097] S103 specifically includes:

[0098] S1031. For each of the multi-channel encoded bitstreams, determine the superposition coefficient based on the importance parameter of the semantic region corresponding to the encoded bitstream.

[0099] The superposition coefficient is a weighting coefficient determined based on the semantic region importance parameter. It is used to adjust the weight of the corresponding coded bitstream in the composite sequence. The higher the importance of the bitstream, the larger the superposition coefficient, which can ensure the anti-interference of key information during transmission.

[0100] S1032. In the bit field, multiply the bit sequence of each encoded bit stream by the superposition coefficient corresponding to the encoded bit stream to obtain the corresponding weighted bit sequence.

[0101] The weighted bit sequence is the sequence obtained by multiplying the bit sequence of the encoded bit stream by the corresponding superposition coefficient, and it carries the bit stream information and weight identifier.

[0102] S1033. Superimpose and merge the weighted bit sequences corresponding to all encoded bit streams to form a composite bit sequence.

[0103] Among them, the composite bit sequence is a unified sequence obtained by superimposing and merging all weighted bit sequences, realizing the bit field fusion of multiple code streams.

[0104] S1034. Modulate the composite bit sequence to generate a composite code stream for transmission.

[0105] Among them, composite bit streams are transmittable data formed by modulating composite bit sequences, which facilitates efficient transmission in networks.

[0106] In this embodiment, step S103 is fundamentally about achieving non-orthogonal multiplexing of the bit domain of multiple encoded bitstreams, balancing transmission efficiency with the priority of critical information. The specific process is as follows: First, the importance parameters of the semantic regions corresponding to each bitstream are combined, and the superposition coefficient is determined through normalization calculation to ensure that the bitstream with higher importance receives greater weight, thereby guaranteeing its transmission priority. Then, bit domain processing is performed, multiplying the bit sequence of each encoded bitstream bitstream bit by bit with the corresponding superposition coefficient to generate a weighted bit sequence carrying bitstream information and weight identifiers, highlighting the signal proportion of critical bitstreams. Subsequently, the weighted bit sequences of all streams are superimposed and merged at corresponding positions to form a composite bit sequence integrating information from multiple bitstreams, enabling a single sequence to carry multiple data streams. Finally, modulation techniques adapted to network transmission are used to convert the composite bit sequence into a signal, generating a composite bitstream that can be transmitted stably. This ultimately completes the non-orthogonal multiplexing of the bit domain of multiple encoded bitstreams, reducing transmission channel occupancy and providing an adaptive data foundation for subsequent dynamic transmission adjustments.

[0107] As an example, taking a factory monitoring scenario, this example continues the semantic regions mentioned earlier: the equipment operating area corresponds to the important new parameter P=0.92, the pedestrian passage area corresponds to the important new parameter P=0.65, and the open area in the corner corresponds to the important new parameter P=0.3.

[0108] First, in step S1031, the superposition coefficient is determined for the importance parameter of the semantic region corresponding to each of the multi-channel coded bitstreams. A normalization calculation method based on the importance parameter is adopted to ensure that the sum of the superposition coefficients of all bitstreams is 1, as shown in equation (1):

[0109] (1)

[0110] in, Let be the superposition coefficient of the i-th coded bitstream. Let n be the importance parameter of the semantic region corresponding to the i-th bitstream, and n be the total number of encoded bitstreams. This is the sum of the importance parameters for all semantic regions.

[0111] Based on the example data: the important new parameter P=0.92 for the equipment operating area, the important new parameter P=0.65 for the pedestrian passage area, and the important new parameter P=0.3 for the corner open area, the superposition coefficient of each coded bitstream corresponding to the equipment operating area, pedestrian passage area, and corner open area is calculated based on equation (1): First, calculate the sum of the importance parameters: Next, calculate the superposition coefficient of each bitstream separately, as follows:

[0112] Route 1, Equipment Operating Area: ;

[0113] Route 2, Pedestrian Crossing Area: ;

[0114] Route 3, open area at the corner: ;

[0115] This calculation allows the code streams in the operating areas of more important devices to obtain the maximum superposition coefficient, thus ensuring their transmission priority.

[0116] Next, a weighted bit sequence is generated in step S1032. The bit sequence of each coded bitstream is extracted. Taking a simplified example, assume the bit sequences of a frame from the three bitstreams are: Stream 1 [1, 0, 1, 0], Stream 2 [0, 1, 0, 1], and Stream 3 [1, 1, 0, 0]. Each bit value in each sequence is multiplied by its corresponding superposition coefficient to obtain the weighted bit sequence.

[0117] The first weighted sequence: [1×0.492, 0×0.492, 1×0.492, 0×0.492] = [0.492, 0, 0.492, 0];

[0118] The second weighted sequence: [0×0.348, 1×0.348, 0×0.348, 1×0.348] = [0, 0.348, 0, 0.348];

[0119] The third weighted sequence: [1×0.160, 1×0.160, 0×0.160, 0×0.160] = [0.160, 0.160, 0, 0].

[0120] Next, a composite bit sequence is generated by superimposing and merging the bits in step S1033. The bit values ​​at corresponding positions of the three weighted bit sequences are added together to obtain the composite bit sequence [0.652, 0.508, 0.492, 0.348].

[0121] Finally, a composite bitstream is generated through modulation in step S1034. Using the well-known QPSK modulation technique, the composite bit sequence is converted into an analog signal suitable for network transmission, thus obtaining the composite bitstream for transmission. The above example is merely one example of this application; in practical applications, the calculation method of the superposition coefficient, modulation technique, etc., can be adjusted according to requirements, and this application does not limit this.

[0122] In another specific implementation, the determination of the superposition coefficient can introduce the bitstream length weight, and the adjusted calculation formula is shown in the following formula (2):

[0123] (2)

[0124] in, This is the weighting coefficient, with a value between 0 and 1. This is the normalized length value of the i-th bitstream. This method can balance the importance and length of the bitstream, avoid interference from short bitstreams due to excessively small coefficients, and further improve the flexibility of multiplexing.

[0125] This application achieves efficient bit-domain multiplexing of multiple encoded bitstreams through the above steps, ensures the transmission priority of key bitstreams through superposition coefficient allocation, reduces transmission channel occupancy, improves transmission efficiency and anti-interference capability, and lays the foundation for subsequent dynamic transmission adjustment.

[0126] S104. During the transmission of the composite bitstream, the transmission resources and encoding parameters used for transmission are dynamically adjusted according to the real-time network status and the importance parameters in the video content information.

[0127] S104 specifically includes:

[0128] S1041. Periodically obtain real-time network status.

[0129] The real-time network status includes available bandwidth and channel bit error rate.

[0130] S1042. If the available bandwidth decreases, the encoding parameters are adjusted to increase the compression ratio; if the available bandwidth increases, the encoding parameters are adjusted to decrease the compression ratio.

[0131] S1043. If the channel bit error rate increases, the transmission resources allocated to the corresponding coded bitstream are adjusted according to the importance parameters of each semantic region, including: increasing the proportion of transmission resources for the coded bitstreams corresponding to semantic regions with importance parameters higher than the first threshold, and correspondingly reducing the proportion of transmission resources for the coded bitstreams corresponding to semantic regions with importance parameters lower than the second threshold.

[0132] The first threshold and the second threshold are preset importance parameter criteria used to classify the importance levels of semantic regions. The first threshold is higher than the second threshold, serving as the core basis for increasing and decreasing the proportion of transmission resources, respectively. The proportion of transmission resources is the share of a single coded bitstream allocated to the total transmission resources, directly affecting the stability and rate of bitstream transmission.

[0133] In this embodiment, step S104 first obtains the real-time network status, including available bandwidth and channel bit error rate, at a fixed period. Then, it adjusts the compression ratio of the coding parameters according to the rise and fall of available bandwidth. If the bandwidth decreases, the compression ratio is increased; if the bandwidth increases, the compression ratio is decreased. Finally, if the channel bit error rate increases, based on the importance parameters of each semantic region and the preset first and second thresholds, the transmission resource ratio of the coding stream corresponding to the region with higher importance than the first threshold is increased, and the transmission resource ratio of the coding stream corresponding to the region with lower importance than the second threshold is decreased, so as to realize the dynamic adaptation of transmission resources and coding parameters with network status.

[0134] As an example, taking a factory monitoring scenario, this example continues the semantic regions mentioned earlier: the equipment operating area corresponds to an importance parameter P=0.92, the pedestrian passage area corresponds to an importance parameter P=0.65, and the open area in the corner corresponds to an importance parameter P=0.3.

[0135] First, real-time network status is obtained through step S1041. A network status monitoring tool is used to collect network data from the transmission link at 500ms intervals, simultaneously acquiring available bandwidth and channel bit error rate (BER). Available bandwidth is measured in Mbps, and BER is measured in %; the collected data is then fed back to the adjustment and control module in real time. For example, the initial network status is 10Mbps available bandwidth and 1% BER; after a period of time, network fluctuations occur, and it is detected that available bandwidth drops to 5Mbps and the BER rises to 5%.

[0136] Secondly, the encoding parameters are adjusted in step S1042. The compression ratio is adjusted based on the bandwidth change trend, following the principle of "increasing the compression ratio when bandwidth decreases to reduce data volume, and decreasing the compression ratio when bandwidth increases to improve image quality." The preset initial encoding parameters are known as follows: the device operating area belongs to level one, corresponding to a compression ratio of 2:1; the pedestrian passage area belongs to level two, corresponding to a compression ratio of 5:1; and the open area in the corner belongs to level three, corresponding to a compression ratio of 10:1.

[0137] When the available bandwidth is detected to drop from 10Mbps to 5Mbps, which is half the bandwidth, the encoding parameters are adjusted as follows: the compression ratio of the device operating area is increased to 3:1 to ensure moderate compression and preserve details; the compression ratio of the pedestrian passage area is increased to 7:1 to balance image quality and data volume; and the compression ratio of the empty corner area is increased to 15:1 to minimize redundancy and ensure that the total encoded data volume is adapted to the reduced bandwidth.

[0138] Next, the transmission resource allocation is adjusted in step S1043. The first threshold is preset to 0.7 and the second threshold to 0.5. The initial transmission resource allocation is as follows: 40% for the equipment operating area, 35% for the pedestrian passage area, and 25% for the open area in the corner.

[0139] When the channel bit error rate (BER) is detected to rise from 1% to 5%, the increased BER leads to decreased transmission stability. Resource allocation is adjusted according to the following rules: Since the critical new parameter P=0.92 in the equipment operating area is greater than the first threshold of 0.7, its transmission resource allocation is increased to 50% to improve anti-interference capabilities; since the critical parameter P=0.3 in the corner open area is less than the second threshold of 0.5, its allocation is reduced to 15% to free up redundant resources; since the critical new parameter P=0.65 in the pedestrian passage area is between the first threshold of 0.7 and the second threshold of 0.5, its allocation remains unchanged at 35%. The total allocation after adjustment remains 100%, thus tilting resources towards critical code streams.

[0140] The above example is only one example of this application. In practical applications, the collection cycle, threshold setting, compression ratio adjustment range, etc. can be adjusted according to needs. This application does not limit these aspects.

[0141] In another specific implementation, a network latency parameter can be introduced to supplement the real-time network status. When the latency increases beyond a preset threshold, in addition to adjusting the compression ratio, the frame rate of non-critical areas (P < 0.5) is also reduced to further reduce data transmission latency. At the same time, the first and second thresholds are dynamically adjusted. When the network fluctuates drastically, the first threshold is reduced to 0.6 and the second threshold is increased to 0.55 to expand the resource protection range of critical areas and improve transmission stability.

[0142] Through the above steps, this application achieves dynamic adaptation between transmission configuration and network status, prioritizes the transmission quality of critical semantic regions, avoids problems such as stuttering and image quality degradation caused by network fluctuations, optimizes the utilization efficiency of transmission resources, and improves overall transmission reliability.

[0143] S105. At the receiving end, sequential interference cancellation decoding is performed on the received composite bitstream to recover the video data corresponding to each semantic region, thereby obtaining the recovered video data.

[0144] Sequential interference cancellation decoding is a decoding method that decodes and eliminates interference from decoded signals in a specific order. Its core is to prioritize decoding high-weight bitstreams and then eliminate the influence of decoded signals on subsequent decoding through interference cancellation, thereby ensuring the decoding accuracy of critical bitstreams.

[0145] S105 specifically includes:

[0146] S1051. Demodulate the received composite code stream to obtain a composite bit sequence.

[0147] S1052. Based on the order of the superposition coefficients corresponding to each semantic region, perform multiple rounds of sequential decoding and interference cancellation operations on the composite bit sequence to sequentially decode the bit sequence of the encoded code stream corresponding to each semantic region.

[0148] S1053. Based on the decoded bit sequences, perform video decoding to recover the video data corresponding to each semantic region, which is then used as the recovered video data.

[0149] In this embodiment of the application, step S105 first demodulates the received composite bit stream to obtain a composite bit sequence, then sequentially decodes the original bit sequence of each coded bit stream in descending order of the superposition coefficients corresponding to each semantic region through multiple rounds of sequential decoding and interference cancellation operations, and finally performs video decoding on each original bit sequence to recover the video data corresponding to each semantic region, thereby obtaining the recovered video data.

[0150] As an example, taking a factory monitoring scenario, continuing from the previous example: the semantic regions and overlay coefficients corresponding to the three bitstreams are respectively the equipment operating area. Pedestrian walkway area , corresponding to the open area in the corner The composite bit sequence is [0.652, 0.508, 0.492, 0.348]:

[0151] First, demodulation is performed in step S1051. Using QPSK demodulation technology matched to the transmitter, the received composite bit stream is converted into a composite bit sequence in digital form, that is, the previously superimposed and merged [0.652, 0.508, 0.492, 0.348] is restored, laying the foundation for subsequent decoding.

[0152] Secondly, sequential decoding and interference cancellation are performed in step S1052. The components are sorted from largest to smallest by superposition coefficient. Decoding and interference cancellation are completed in three rounds:

[0153] First round of decoding: For the device operating area with the largest superposition coefficient, the maximum likelihood decoding algorithm is used to decode the original bit sequence [1, 0, 1, 0] corresponding to the device operating area from the composite bit sequence [0.652, 0.508, 0.492, 0.348].

[0154] Specifically, we first determine that the current sequence to be decoded is a composite bit sequence [0.652, 0.508, 0.492, 0.348]. The decoding target is the device operating area with the largest superposition coefficient. We need to find the original bit sequence (4-bit binary) of the device operating area and make its weighted sequence best match the sequence to be decoded.

[0155] Since the original bit sequence is 4 bits, there are 16 possible combinations of the original bit sequence, such as [0, 0, 0, 0], [0, 0, 0, 1]...[1, 1, 1, 1]. However, considering the semantics of the monitoring scenario, the device operating area is a critical area, and the bit sequence needs to reflect effective information. Therefore, the all-zero sequence [0, 0, 0, 0] which has no image information, and the all-one sequence [1, 1, 1, 1] which has no actual monitoring value, are removed. The sequences with "effective bit ratio ≥ 50% and differentiated bit distribution" are retained. Effective bits, i.e., bit 1, reflect the effective information in the image. Differentiated distribution means that non-continuous 0 / 1 matches the image characteristics of the device operating area. Single meaningless sequences, such as [0, 0, 1, 1], which have continuous distribution and low information, can also be removed as appropriate.

[0156] After screening, only three core candidate combinations were retained: [1, 0, 1, 0], [1, 0, 0, 1], and [0, 1, 1, 0]. Their weighted sequences were then calculated.

[0157] Candidate 1: [1, 0, 1, 0], its weighted sequence is [0.492, 0, 0.492, 0];

[0158] Candidate 2: [1, 0, 0, 1], its weighted sequence is [0.492, 0, 0, 0.492];

[0159] Candidate 3: [0, 1, 1, 0], whose weighted sequence is [0, 0.492, 0.492, 0].

[0160] Subsequently, similarity determination is performed. The sum of squared errors (SSE) is used to quantify the similarity between the composite bit sequence and the candidate weighted sequence. The smaller the SSE, the higher the matching degree. The calculation formula is shown in equation (3):

[0161] (3)

[0162] in, It is the k-th bit value of the sequence to be decoded. It is the k-th value in the candidate weighted sequence, and m is the sequence length, where m=4.

[0163] Substituting the three candidate weighted bit sequences and the composite bit sequence into equation (3) respectively, the sum of squared errors is calculated. The SSE of candidate 1 with the composite bit sequence is 0.4046, the SSE of candidate 2 with the composite bit sequence is 0.5466, and the SSE of candidate 3 with the composite bit sequence is 0.54626. At this time, the SSE corresponding to candidate 1 is the smallest, so [1, 0, 1, 0] is determined to be the original bit sequence of the device operating area.

[0164] The second round of decoding: For the pedestrian passage area with the second-highest superposition coefficient, first reconstruct the weighted bit sequence of the device operating area [1×0.492, 0×0.492, 1×0.492, 0×0.492]=[0.492, 0, 0.492, 0] by "original bit sequence × superposition coefficient"; then subtract this reconstructed sequence from the current composite bit sequence to calculate the remaining bit sequence as [0.652-0.492, 0.508-0, 0.492-0.492, 0.348-0]=[0.160, 0.508, 0, 0.348]; finally, decode the original bit sequence of the pedestrian passage area [0, 1, 0, 1] from the remaining bit sequence. The decoding process has been shown in the first round of decoding and will not be repeated here.

[0165] The third round of decoding: For the corner open area with the smallest superposition coefficient, reconstruct the weighted bit sequence of the decoded equipment operation area and pedestrian passage area, and sum it to get [0.492+0, 0+0.348, 0.492+0, 0+0.348]=[0.492, 0.348, 0.492, 0.348]; Subtract this summed sequence from the composite bit sequence to get the remaining bit sequence [0.652-0.492, 0.508-0.348, 0.492-0.492, 0.348-0.348]=[0.160, 0.160, 0, 0]; Decode the original bit sequence of the corner open area [1, 1, 0, 0] from this sequence. The decoding process has been shown in the first round of decoding and will not be repeated here.

[0166] Finally, video decoding is performed in step S1053. Using an H.265 decoding algorithm matched to the transmitting end, the three original bit sequences are decoded separately to reconstruct the video data corresponding to the device operating area, pedestrian passage area, and open corner area, thus obtaining the recovered video data. The above example is merely one example of this application. In practical applications, the decoding algorithm and interference cancellation calculation accuracy can be adjusted according to requirements, and this application does not limit these adjustments.

[0167] In another specific implementation, when the difference in the superposition coefficients of multiple bitstreams is small, such as less than 0.05, a bitstream length weight can be introduced during sorting, and the bitstreams can be sorted according to the comprehensive score of "superposition coefficient × normalized bitstream length value" to avoid decoding interference caused by close coefficients; at the same time, an error correction factor is added in the interference elimination stage to reduce the deviation between the reconstructed weighted sequence and the original sequence, and further improve the decoding accuracy.

[0168] Through the above steps, this application achieves accurate recovery of video data in each semantic region, prioritizes the integrity and decoding quality of data in key regions, effectively reduces decoding interference caused by multiplexing of multiple streams, improves the reliability of recovered video data, and provides high-quality data support for subsequent digital twin scene rendering.

[0169] Following S105, it also includes:

[0170] The quality of the recovered video data for each semantic region is assessed to obtain a corresponding quality evaluation value. The quality evaluation value of the video data for each semantic region is compared with a preset quality threshold. Based on the comparison result and the importance parameter of the corresponding semantic region, the video data is enhanced. The enhancement process includes: if the importance parameter of a certain semantic region is higher than the preset importance threshold and the quality evaluation value is lower than the quality threshold, then a request is made to retransmit the encoded bitstream corresponding to the semantic region; if the importance parameter of a certain semantic region is lower than or equal to the preset importance threshold and the quality evaluation value is lower than the quality threshold, then data compensation is performed on the semantic region based on the video data of adjacent semantic regions in the same video frame.

[0171] The quality evaluation value is a quantitative indicator that measures the quality of video data recovered from semantic regions. It reflects the clarity, integrity, and distortion-free nature of the video image; a higher value indicates better quality. The preset quality threshold is a baseline value used to determine whether the video data meets the usage requirements, and is set according to the image quality requirements of the actual application scenario. The preset importance threshold is a reference standard for classifying the importance of semantic regions, used to distinguish whether data quality needs to be prioritized.

[0172] In this embodiment, the quality evaluation value is first obtained by performing a quality assessment on the recovered video data of each semantic region. Then, the quality evaluation value is compared with a preset quality threshold. Based on the relationship between the semantic region importance parameter and the preset importance threshold, differentiated enhancement processing, retransmission or data compensation are carried out on the data that does not meet the quality standard.

[0173] As an example, continuing with the factory monitoring scenario, and maintaining the semantic regions mentioned earlier: the equipment operation area corresponds to an importance parameter P=0.92, the pedestrian passage area corresponds to an importance parameter P=0.65, and the open area in the corner corresponds to an importance parameter P=0.3.

[0174] First, a quality assessment is conducted. The Peak Signal-to-Noise Ratio (PSNR) algorithm, commonly used in video quality assessment, is employed, combined with simplified calculations based on the characteristics of the surveillance video, to obtain a quality evaluation value for each area. The value ranges from 0 to 100, with higher values ​​indicating better image quality. For example, the assessment results show: a quality evaluation value of 75 for the equipment operating area, 82 for the pedestrian walkway area, and 70 for the open area in the corner.

[0175] Secondly, preset thresholds are set. Based on the image quality requirements of the factory monitoring scenario, the preset quality threshold is 80, that is, an evaluation value ≥ 80 is considered to meet the usage requirements. The preset importance threshold is 0.7, which is consistent with the first threshold in step S104 to ensure that the importance judgment standard is consistent.

[0176] Next, enhancement processing is performed based on the comparison results and importance parameters:

[0177] For the device operating area, its importance parameter P=0.92 > the preset importance threshold of 0.7, and its quality evaluation value 75 < the preset quality threshold of 80, meeting the conditions of high importance and low quality. At this time, a retransmission request is sent to the sending end, explicitly specifying the encoded bitstream corresponding to this semantic region. After receiving the request, the sending end retransmits the bitstream, and the receiving end recovers high-quality video data through the decoding process in step S105, ensuring that the image quality of this critical area, the device operating area, meets the standards.

[0178] For pedestrian walkways, if the quality evaluation value is 82 or higher than the preset quality threshold of 80, no enhancement processing is required, and the original restored video data can be directly retained, regardless of the importance parameter.

[0179] For the empty corner area, its importance parameter P=0.3≤preset importance threshold 0.7, and its quality evaluation value 70<preset quality threshold 80, meeting the conditions of low importance and low quality. Data compensation is used for optimization. Video data from adjacent pedestrian walkways in the same video frame is extracted. This video data has acceptable image quality and includes environmental features such as walls and ground. A pixel interpolation algorithm is used to adapt the environmental texture and brightness parameters of the pedestrian walkway area to the empty corner area, supplementing the details missing due to substandard quality in this area, resulting in a more natural transition in the overall image.

[0180] The above example is only one example of this application. In practical applications, the quality assessment algorithm, the specific value of the preset threshold, or the implementation method of data compensation can be adjusted according to the needs of the monitoring scenario. This application does not limit these aspects.

[0181] In another specific implementation, the retransmission request can be set with an upper limit on the number of retries. If the quality is still not up to standard after multiple retransmissions, it will switch to data compensation based on model texture to avoid consuming too many transmission resources due to continuous retransmission.

[0182] S106. Based on the restored video data, and the location identifier and importance parameter associated with the restored video data, the Max plugin drives the digital twin scene to perform differentiated 3D rendering and presentation.

[0183] S106 specifically includes:

[0184] S1061. Based on the location identifier, the recovered video data of each semantic region is used as a dynamic texture and bound to the corresponding three-dimensional model surface in the digital twin scene.

[0185] Among them, dynamic textures use the recovered video data of semantic regions as dynamically updatable texture resources to fit the surface of 3D models and reproduce the dynamic changes of real scenes.

[0186] S1062. Determine the corresponding rendering level based on the importance parameters of each semantic region.

[0187] S1063. Based on the rendering level, adjust the texture mapping quality, lighting calculation complexity, and display priority of the corresponding 3D model surface in the digital twin scene.

[0188] Texture mapping quality refers to the accuracy of the fit and clarity of the dynamic texture onto the surface of the 3D model, directly affecting the visual effect of the model's surface. Lighting computational complexity refers to the precision of the calculations for light reflection and refraction during rendering; higher complexity results in more realistic lighting effects. Display priority refers to the order in which models corresponding to different semantic regions are drawn during 3D rendering; higher priority means earlier rendering and more prominent visual presentation.

[0189] S1064. The 3D graphics engine, driven by the Max plugin, renders the digital twin scene based on the adjusted texture mapping quality, lighting calculation complexity, and display priority, and outputs the display screen.

[0190] In the embodiments of this application, such as Figure 2 As shown, in step S106, the restored video data of each semantic region is first bound to the surface of the 3D model corresponding to the digital twin scene as a dynamic texture through the location identifier. Then, the rendering level is determined according to the importance parameter. The texture mapping quality, lighting calculation complexity and display priority are adjusted according to the rendering level. Finally, the 3D graphics engine is driven by the Max plugin to complete the rendering and output the image.

[0191] As an example, taking a factory monitoring scenario, continuing with the semantic regions mentioned earlier: the equipment operating area corresponds to an importance parameter P=0.92, the pedestrian passage area corresponds to an importance parameter P=0.65, and the open area in the corner corresponds to an importance parameter P=0.3.

[0192] First, dynamic textures are bound via step S1061. Utilizing the spatial coordinate mapping relationship between location identifiers and the 3D model of the digital twin scene, the restored video data for each area is encapsulated as dynamic texture resources: the restored video data for the equipment operating area is bound to the 3D model surface of the production equipment in the scene; the restored video data for the pedestrian passage area is bound to the 3D model surface of the ground corresponding to the passage; and the restored video data for the open area in the corner is bound to the 3D model surface of the corner and surrounding environment, ensuring precise matching between dynamic video changes and model positions.

[0193] Secondly, the rendering level is determined in step S1062. The rendering level is divided using a threshold rule consistent with the encoding level to ensure a unified standard for determining importance.

[0194] When P ≥ 0.8, the rendering level is level one, the highest level;

[0195] When 0.5 ≤ P < 0.8, the rendering level is level 2, which is medium level.

[0196] When P < 0.5, the rendering level is level three, the basic level.

[0197] Based on the example data, the equipment operating area (P=0.92) is classified as Level 1, the pedestrian passage area (P=0.65) as Level 2, and the open area in the corner (P=0.3) as Level 3.

[0198] Next, the rendering parameters are adjusted in step S1063. These parameters include texture mapping quality, lighting computation complexity, and display priority. The parameter configurations for different rendering levels in this example are shown in Table 2 below.

[0199] Table 2: Correspondence between Rendering Levels and Rendering Parameter Configurations

[0200]

[0201] As shown in Table 2, the equipment operating area uses high-precision texture mapping, high-complexity lighting calculation, and the highest display priority; the pedestrian passage area uses medium-precision texture mapping, medium-complexity lighting calculation, and medium display priority; and the open area in the corner uses basic-precision texture mapping, low-complexity lighting calculation, and the lowest display priority.

[0202] Finally, step S1064 drives the rendering and output of the image. The Max plugin calls its built-in 3D graphics engine, such as Direct3D, to render the digital twin scene according to the adjusted parameters: first, it draws the device model of the device operating area, whose surface dynamic texture clearly restores the details of device operation, and the lighting effects realistically present the device outline and operating status; then, it draws the ground model of the pedestrian passage area, with smooth textures adapted to the shape of the passage, and lighting effects clearly distinguishing people from the ground; finally, it draws the environment model of the empty area in the corner, retaining the core environment textures to meet the requirements of scene integrity. In the final output image, the key device operating areas are visually prominent and the details are clear, while non-critical areas balance integrity and resource efficiency. The above example is only one example of this application. In actual applications, the rendering level threshold, parameter configuration, etc. can be adjusted according to the scene requirements, and this application does not limit this.

[0203] In another specific implementation, the division of rendering levels can introduce a real-time rendering resource utilization parameter. When rendering resources are detected to be scarce, such as when the CPU utilization exceeds 80%, the lighting calculation complexity of the secondary region is automatically reduced to low complexity, and the texture mapping quality of the tertiary region is further simplified to ensure smooth and lag-free scene rendering. When rendering resources are sufficient, the texture mapping quality of the secondary region can be upgraded to high precision to improve the overall visual effect of the scene.

[0204] Through the above steps, this application achieves differentiated 3D rendering of digital twin scenes, highlights the visual presentation and detail restoration of key monitoring areas, rationally allocates rendering resources, and balances scene presentation quality and rendering efficiency, making digital twin scenes more in line with actual monitoring and decision-making needs.

[0205] Figure 3 This is a schematic diagram illustrating a specific implementation of a Max plugin-assisted video data transmission system provided in this application. (Refer to...) Figure 3 The system may include:

[0206] The acquisition module 31 is used to parse the input monitoring video stream through the Max plugin and acquire video content information. The video content information includes multiple semantic regions divided according to the video frame, as well as location identifiers and importance parameters associated with the semantic regions.

[0207] The encoding module 32 is used to encode the video data corresponding to each semantic region according to the importance parameters in the video content information using different encoding strategies to generate multiple encoded bitstreams.

[0208] Processing module 33 is used to perform non-orthogonal multiplexing of the multi-channel coded bitstream in the bit domain to generate a composite bitstream for transmission;

[0209] The adjustment module 34 is used to dynamically adjust the transmission resources and encoding parameters used for transmission based on the real-time network status and the importance parameters in the video content information during the transmission of the composite bitstream.

[0210] Decoding module 35 is used at the receiving end to perform sequence interference cancellation decoding on the received composite bitstream in order to recover the video data corresponding to each semantic region and obtain the recovered video data;

[0211] The rendering module 36 is used to drive the digital twin scene to perform differentiated 3D rendering and presentation based on the restored video data, the location identifier and the importance parameter associated with the restored video data, driven by the Max plugin.

[0212] The Max plugin-assisted video data transmission system of this application is used to implement the aforementioned Max plugin-assisted video data transmission method. Therefore, the specific implementation of the Max plugin-assisted video data transmission system can be found in the embodiment section of the Max plugin-assisted video data transmission method above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0213] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the Max plug-in assisted video data transmission methods described above.

[0214] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the Max plug-in assisted video data transmission methods described above.

[0215] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0216] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the Max plugin-assisted video data transmission method described above.

[0217] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0218] The foregoing has provided a detailed description of a Max plugin-assisted video data transmission method and system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for Max plugin assisted video data transmission, the method comprising: The method comprises the following steps: parsing an input monitoring video stream by a Max plugin to obtain video content information, the video content information comprising a plurality of semantic regions divided according to video pictures, and position identifiers and importance parameters associated with the semantic regions; encoding video data corresponding to each semantic region by using different encoding strategies according to the importance parameters in the video content information to generate a plurality of encoding code streams; non-orthogonal multiplexing the plurality of encoding code streams in a bit domain to generate a composite code stream for transmission; during transmission of the composite code stream, dynamically adjusting transmission resources and encoding parameters for transmission according to real-time network status and the importance parameters in the video content information; at a receiving end, performing sequential interference cancellation decoding on the received composite code stream to recover video data corresponding to each semantic region to obtain recovered video data; based on the recovered video data and the position identifiers and the importance parameters associated with the recovered video data, driving a digital twin scene by the Max plugin to perform differential three-dimensional rendering and presentation; the non-orthogonal multiplexing of the plurality of encoding code streams in the bit domain to generate the composite code stream for transmission comprises: for each encoding code stream in the plurality of encoding code streams, determining an overlay coefficient according to the importance parameter of the semantic region corresponding to the encoding code stream; in the bit domain, multiplying a code stream bit sequence of each encoding code stream by the overlay coefficient corresponding to the encoding code stream to obtain a corresponding weighted bit sequence; adding bit values at corresponding positions of the weighted bit sequences corresponding to all encoding code streams to form a composite bit sequence; modulating the composite bit sequence to generate the composite code stream for transmission; the sequential interference cancellation decoding performed on the received composite code stream to recover video data corresponding to each semantic region to obtain recovered video data comprises: demodulating the received composite code stream to obtain a composite bit sequence; based on the size order of the overlay coefficients corresponding to each semantic region, performing a plurality of rounds of sequential decoding and interference cancellation operations on the composite bit sequence to sequentially decode bit sequences of the encoding code streams corresponding to each semantic region; performing video decoding according to the decoded bit sequences to recover video data corresponding to each semantic region as recovered video data.

2. The method of claim 1, wherein, the differential three-dimensional rendering and presentation of the digital twin scene by the Max plugin based on the recovered video data and the position identifiers and the importance parameters associated with the recovered video data comprises: binding the recovered video data of each semantic region as a dynamic texture to a corresponding three-dimensional model surface in the digital twin scene according to the position identifiers; determining a corresponding rendering level according to the importance parameter of each semantic region; adjusting texture mapping quality, lighting calculation complexity and display priority of the corresponding three-dimensional model surface in the digital twin scene according to the rendering level. The Max plug-in drives a three-dimensional graphics engine, renders the digital twin scene according to the adjusted texture mapping quality, illumination calculation complexity and display priority, and outputs a display picture.

3. The method of claim 1, wherein, The video data corresponding to each semantic region is encoded by using different encoding strategies according to the importance parameter in the video content information to generate a multi-path encoding code stream, including: According to the importance parameter of each semantic region in the video content information, an encoding level is assigned to the semantic region; According to the encoding level, a set of preset encoding parameters is selected for each semantic region; The pixel data of each semantic region in each video frame is extracted respectively; According to the encoding parameters selected for each semantic region, the pixel data of each semantic region is compressed and encoded to generate an independent encoding code stream; All independent encoding code streams are aggregated to form the multi-path encoding code stream.

4. The method of claim 1, wherein, After performing sequential interference cancellation decoding on the received composite code stream to recover the video data corresponding to each semantic region and obtaining the recovered video data, the method further includes: Quality evaluation is performed on the recovered video data of each semantic region to obtain a corresponding quality evaluation value; The quality evaluation value of the video data of each semantic region is compared with a preset quality threshold, and according to the comparison result and the importance parameter of the corresponding semantic region, the video data is subjected to enhancement processing; The enhancement processing includes: If the importance parameter of a certain semantic region is higher than a preset importance threshold and the quality evaluation value is lower than the quality threshold, the encoding code stream corresponding to the semantic region is requested to be retransmitted; If the importance parameter of a certain semantic region is lower than or equal to a preset importance threshold and the quality evaluation value is lower than the quality threshold, data compensation is performed on the semantic region based on the video data of adjacent semantic regions in the same video frame.

5. The method of claim 1, wherein, The transmission resource and the encoding parameter used for transmission are dynamically adjusted according to the real-time network state and the importance parameter in the video content information, including: The real-time network state is periodically acquired, including available bandwidth and channel error rate; If the available bandwidth decreases, the encoding parameter is adjusted to increase the compression ratio; if the available bandwidth increases, the encoding parameter is adjusted to decrease the compression ratio; If the channel error rate increases, the transmission resource allocated to the corresponding encoding code stream is adjusted according to the importance parameter of each semantic region, including: increasing the transmission resource proportion for the encoding code stream corresponding to the semantic region whose importance parameter is higher than a first threshold, and correspondingly decreasing the transmission resource proportion for the encoding code stream corresponding to the semantic region whose importance parameter is lower than a second threshold.

6. A Max plug-in assisted video data transmission system, characterized by, The method includes: An acquisition module is configured to parse an input monitoring video stream by a Max plug-in, and acquire video content information, the video content information including a plurality of semantic regions divided according to a video picture, and a position identifier and an importance parameter associated with the semantic regions; An encoding module is configured to encode video data corresponding to each semantic region by using different encoding strategies according to the importance parameter in the video content information to generate a multi-path encoding code stream; An acquisition module is configured to parse an input monitoring video stream by a Max plug-in, and acquire video content information, the video content information including a plurality of semantic regions divided according to a video picture, and a position identifier and an importance parameter associated with the semantic regions; The processing module is configured to non-orthogonally multiplex the multiple encoded code streams in a bit domain to generate a composite code stream for transmission. The adjusting module is configured to dynamically adjust transmission resources and encoding parameters for transmission according to real-time network states and importance parameters in the video content information during transmission of the composite code stream. The decoding module is configured to perform sequential interference cancellation decoding on the received composite code stream at a receiving end to recover video data corresponding to each semantic region to obtain recovered video data. The rendering module is configured to drive the digital twin scene to perform differential three-dimensional rendering and presentation based on the recovered video data, the position identifier associated with the recovered video data, and the importance parameters by the Max plugin. The processing module is configured to non-orthogonally multiplex the multiple encoded code streams in a bit domain to generate a composite code stream for transmission. For each of the multiple encoded code streams, an overlay coefficient is determined according to an importance parameter of a semantic region corresponding to the encoded code stream. In the bit domain, a code bit sequence of each of the multiple encoded code streams is multiplied by the overlay coefficient corresponding to the encoded code stream to obtain a corresponding weighted bit sequence. Bit values at corresponding positions of the weighted bit sequences corresponding to all the encoded code streams are added to form a composite bit sequence. The composite bit sequence is modulated to generate the composite code stream for transmission. The decoding module is configured to perform sequential interference cancellation decoding on the received composite code stream at a receiving end to recover video data corresponding to each semantic region to obtain recovered video data. The received composite code stream is demodulated to obtain a composite bit sequence. Based on a size order of the overlay coefficients corresponding to the semantic regions, the composite bit sequence is sequentially decoded and subjected to interference cancellation operation in multiple rounds to sequentially decode bit sequences of the encoded code streams corresponding to the semantic regions. Based on the decoded bit sequences, video decoding is performed to recover video data corresponding to each semantic region as the recovered video data.

7. An electronic device, comprising: The memory is configured to store a computer program. The processor is configured to implement steps of the Max plugin assisted video data transmission method according to any one of claims 1 to 5 when the computer program is executed. The computer readable storage medium stores a computer program, and the computer program is executable by the processor to implement the Max plugin assisted video data transmission method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, ​

Images