A wearable badge-style audio and video device that enables 4G network transmission

Abstract

The present application relates to the field of smart devices, and provides a wearable badge-style audio and video device that realizes 4G network transmission. The device first collects data through a microphone and camera built into the device, then compresses the collected data, and then adds FEC information to the compressed data to obtain anti-packet loss compressed data, and then transmits the anti-packet loss compressed data through the 4G network. During the data transmission process, the RTT and packet loss rate are monitored in real time, and the transmission rate is dynamically adjusted based on the monitoring results. In this way, the wearable badge-style audio and video device can better realize efficient and reliable data transmission under the 4G network.

Description

Technical Field

[0001] The present application relates to the field of smart devices, and more specifically, to a wearable badge-style audio and video device that implements 4G network transmission. Background Art

[0002] Wearable badge-style audio and video equipment is a small, portable device designed to be similar to a traditional badge, which can be conveniently hung on clothes or fixed to a specific position of work clothes. In some industries, such as security, logistics, and healthcare, workers need to keep communication open at all times and be able to interact with audio and video on the move. With the development of mobile communication technology, 4G networks have become the main transmission medium for many applications, especially in scenarios that require high bandwidth and low latency, such as real-time audio and video communications.

[0003] The transmission rate directly affects the efficiency and quality of data transmission during the communication process. The appropriate transmission rate can maximize the use of network bandwidth, reduce latency and packet loss, ensure smooth audio and video transmission, and improve user experience. However, many traditional devices use fixed transmission rate settings and cannot be dynamically adjusted according to actual network conditions. This static configuration is prone to low transmission efficiency, data loss, and reduced user experience when encountering network fluctuations or congestion. Due to the inability to respond to network changes in real time, traditional devices may continue to use inappropriate transmission rates in high latency or packet loss environments, resulting in bandwidth waste and unnecessary retransmissions. In addition, the lack of an intelligent adjustment mechanism makes it difficult for devices to optimize resource utilization and maintain stable performance under different network conditions, further affecting the smoothness and reliability of communication.

[0004] Therefore, an optimized wearable badge-type audio and video device that realizes 4G network transmission is desired, which can improve the response speed and accuracy of the device by intelligently adjusting the transmission rate. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a wearable badge-style audio and video device that implements 4G network transmission.

[0006] According to one aspect of the present application, a wearable badge-type audio and video device for realizing 4G network transmission is provided, which includes: an audio and video data acquisition module, which is used to collect audio data or video data using a built-in microphone and / or camera of the audio and video device; a data compression module, which is used to compress the audio data or the video data to obtain compressed data; an information adding module, which is used to add FEC information to the compressed data to obtain anti-packet loss compressed data; a data sending module, which is used to send the anti-packet loss compressed data through a 4G network; a sending rate adjustment module, which is used to monitor RTT and packet loss rate in real time, and dynamically adjust the sending rate based on the monitoring results of the RTT and the packet loss rate;

[0007] Among them, the sending rate adjustment module includes: an RTT packet loss rate data acquisition unit, which is used to obtain the time queue of RTT and the time queue of packet loss rate; a time series implicit coding unit, which is used to take the time queue of RTT as the main variable and the time queue of packet loss rate as the covariate, and perform multi-scale time series coding on the main variable and the covariate to obtain the time series implicit features of the sending rate main variable and the time series implicit features of the sending rate covariate; a dynamic interaction unit, which is used to perform dynamic memory feature attention interaction on the time series implicit features of the sending rate main variable and the time series implicit features of the sending rate covariate to obtain the time series fusion features of the sending rate main variable-covariate, and obtain optimization instructions based on the time series fusion features of the sending rate main variable-covariate, and the optimization instructions include the recommended value of the sending rate at the next time point.

[0008] Compared with the prior art, the wearable badge-style audio and video device that realizes 4G network transmission provided by the present application first collects data through the microphone and camera built into the device, then compresses the collected data, and then adds FEC information to the compressed data to obtain anti-packet loss compressed data, and then transmits the anti-packet loss compressed data through the 4G network. During the data transmission process, the RTT and packet loss rate are monitored in real time, and the sending rate is dynamically adjusted based on the monitoring results. In this way, the wearable badge-style audio and video device can better realize efficient and reliable data transmission under the 4G network. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0010] Figure 1The present invention is a block diagram of a wearable badge-type audio and video device that implements 4G network transmission according to an embodiment of the present application.

[0011] Figure 2 The present invention is a block diagram of a sending rate adjustment module in a wearable badge-type audio and video device that implements 4G network transmission according to an embodiment of the present application.

[0012] Figure 3 The present invention is a block diagram of a dynamic interaction unit in a wearable badge-type audio and video device that implements 4G network transmission according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0014] Wearable badge-style audio and video equipment is a small portable device designed similar to a traditional badge and can be easily fixed on clothing. It is suitable for industries such as security, logistics and medical care to achieve audio and video communication anytime and anywhere. With the development of 4G networks, these devices play an important role in high-bandwidth, low-latency real-time communications.

[0015] In communication, the transmission rate is related to the efficiency and quality of data transmission. The appropriate transmission rate can optimize network bandwidth utilization, reduce delays and packet loss to ensure smooth transmission and improve user experience. However, traditional devices mostly use fixed transmission rate settings, which cannot be adjusted dynamically according to actual network conditions. When the network fluctuates or is congested, it is easy to have problems such as low transmission efficiency, data loss, and poor user experience. In addition, due to the lack of an intelligent adjustment mechanism, it is difficult to optimize resource utilization and maintain stable performance under different network conditions, affecting the smoothness and reliability of communication.

[0016] Based on this, the present application provides an optimized wearable badge-style audio and video device that implements 4G network transmission. Figure 1 FIG. 1 is a block diagram of a wearable badge-type audio and video device that implements 4G network transmission according to an embodiment of the present application. Figure 1As shown, in the wearable badge-type audio and video device 100 that realizes 4G network transmission, it includes: an audio and video data acquisition module 110, which is used to use the built-in microphone and / or camera of the audio and video device to collect audio data or video data; a data compression module 120, which is used to compress the audio data or the video data to obtain compressed data; an information adding module 130, which is used to add FEC information to the compressed data to obtain anti-packet loss compressed data; a data sending module 140, which is used to send the anti-packet loss compressed data through the 4G network; a sending rate adjustment module 150, which is used to monitor RTT and packet loss rate in real time, and dynamically adjust the sending rate based on the monitoring results of the RTT and the packet loss rate.

[0017] In the embodiment of the present application, the audio and video data acquisition module 110 is used to collect audio data or video data using the built-in microphone and / or camera of the audio and video device. It should be understood that the audio data or video data collected by the audio and video device is the most original data source, and subsequent processing is centered around how to effectively transmit these data.

[0018] In an embodiment of the present application, the data compression module 120 is used to compress the audio data or the video data to obtain compressed data. It should be understood that although the 4G network provides a certain bandwidth, audio and video data are usually large-capacity data. For example, the amount of data per second of uncompressed high-definition video data may reach several megabytes or even more. Through data compression, the amount of data can be significantly reduced, so that these data can better adapt to the bandwidth limitations of the 4G network. And during network transmission, a smaller amount of data means a shorter time required for transmission. Compressed data can transmit more content in the same time, thereby improving transmission efficiency. And on the device side, compressed data can also reduce the requirements for device storage capacity. The storage resources of wearable badge-type devices are usually limited. By compressing audio and video data, more data can be stored in the device, which is convenient for subsequent processing and transmission.

[0019] In the wearable badge-type audio and video equipment that realizes 4G network transmission, compressing audio data or video data to obtain compressed data is a key link to ensure efficient operation of the equipment and effective data transmission. The following is a detailed description of its implementation process:

[0020] First of all, choosing a suitable compression algorithm is the basis for achieving effective data compression. For audio data, common compression algorithms such as MP3 and AAC play an important role. The MP3 algorithm is based on the human ear auditory perception model. It decomposes the audio signal into sub-bands of different frequencies, and then quantizes and encodes each sub-band based on the psychoacoustic model. In this process, the high-frequency parts that are difficult for the human ear to detect and the masked signal components will be removed, thereby significantly reducing the amount of data. The AAC algorithm also utilizes the auditory characteristics of the human ear and further improves the compression efficiency through more advanced encoding technology. Under the same sound quality conditions, it can achieve a higher compression ratio than MP3, which is extremely advantageous for the limited storage and transmission resources of wearable badge-type devices.

[0021] In terms of video data compression, standards such as H.264 and H.265 are widely used. The H.264 algorithm makes full use of the temporal and spatial redundancy between video frames for efficient compression. Intra-frame prediction technology uses the correlation between adjacent pixels to predict the current pixel value, effectively reducing the spatial redundancy within the frame; while inter-frame prediction searches for similar areas in the reference frame to accurately predict the pixel value of the current frame, thereby significantly reducing temporal redundancy. In the transform coding stage, the image or residual data is converted to the frequency domain, so that the energy is concentrated on a few low-frequency coefficients, and then combined with quantization and entropy coding technology to further reduce the amount of data. As an upgraded version of H.264, H.265 has made further progress in compression efficiency. It can reduce the amount of data by about half while maintaining the same video quality. It is particularly suitable for wearable badge-type equipment scenarios that require high-definition video transmission and have limited bandwidth, such as high-definition video transmission in security monitoring.

[0022] In addition to selecting a compression algorithm, setting appropriate compression parameters is also crucial. The adjustment of the compression ratio requires comprehensive consideration of the device's storage capacity, network bandwidth, and requirements for audio and video quality. In wearable badge-type devices with limited storage capacity, if the requirements for audio and video quality are not extreme, appropriately increasing the compression ratio can significantly save storage space, allowing the device to store more audio and video data. For example, in the logistics industry, workers use badge-type devices to record the process of loading and unloading goods, and do not require high picture details. At this time, increasing the compression ratio can not only meet basic video recording needs, but also ensure that the device has enough storage space to cope with long-term work. However, if in scenarios such as video conferencing where high image and sound quality are required, the compression ratio needs to be adjusted carefully to avoid excessive compression that causes information loss and affects communication effectiveness.

[0023] For video data, resolution adjustment is also an effective compression method. Reducing the resolution can directly reduce the amount of data, but it will sacrifice the clarity of the picture. In some specific scenarios, such as when the wearable badge device is in an area with poor network signal, reducing the video resolution can ensure smooth video playback. For example, in logistics warehouse monitoring in remote areas, when the 4G network signal is unstable, reducing the HD video resolution to SD can ensure real-time transmission of the monitoring screen, allowing monitoring personnel to grasp the warehouse dynamics in a timely manner.

[0024] Frame rate adjustment should not be ignored either. Reducing the video frame rate can effectively reduce the amount of data transmission. This method has obvious advantages when real-time requirements are not high and network conditions are limited. For example, in some long-term monitoring scenarios with relatively low requirements for video fluency, reducing the frame rate from 30fps to 15fps or even lower can not only reduce data transmission pressure, but also meet monitoring needs to a certain extent, while extending the battery life of the device.

[0025] In order to further improve the efficiency of data compression, the audio and video equipment can be equipped with special hardware modules to accelerate the compression process, such as hardware codecs. These hardware modules use dedicated circuits to implement efficient compression and decompression algorithms, which greatly improves the processing speed and reduces the occupation of the device's CPU resources. Taking the video codec hardware unit integrated in the mobile processor as an example, it can quickly process video data compression and decompression tasks in formats such as H.264 and H.265, ensuring that the performance and response speed of the device are not affected when collecting and transmitting audio and video data. This enables the wearable badge-type device to use device resources more efficiently when performing data compression tasks, while ensuring the smoothness of the device when performing other functions (such as data collection, network transmission control, etc.).

[0026] Data compression works closely with other functions throughout the operation of the device. From a storage perspective, compressed data can store more audio and video data in the limited storage capacity of the device, which is convenient for subsequent processing and transmission. However, when implementing data compression, a trade-off must be made between data reduction and audio and video quality. Different application scenarios and user needs have different requirements for audio and video quality, so it is necessary to choose the most appropriate compression strategy. In video conferencing scenarios, clear and smooth pictures and sounds are essential for effective communication. At this time, the compression ratio should be appropriately reduced to ensure the clarity and smoothness of the video; while in scenarios such as logistics warehouse monitoring, the focus is on the status of goods and the activities of personnel, and the requirements for picture details are relatively low. Under the premise of not affecting the identification of key information, a higher compression ratio can be used to save resources. At the same time, by continuously optimizing the compression algorithm and parameter settings, the compression efficiency can be maximized while ensuring a certain quality.

[0027] In summary, the wearable badge-type audio and video device achieves efficient compression of audio and video data through a series of measures such as carefully selecting compression algorithms, reasonably setting compression parameters, relying on hardware acceleration support, and weighing compression quality, laying a solid foundation for the stable operation of the device under the 4G network and the reliable transmission of data.

[0028] In the embodiment of the present application, the information adding module 130 is used to add FEC information to the compressed data to obtain anti-packet loss compressed data. It should be explained that FEC, that is, forward error correction code, is a coding technology used for error correction during data transmission. At the sending end, redundant information is generated and added to the data by performing a specific encoding algorithm on the original data. The generated redundant information often has a specific mathematical relationship with the original data. At the receiving end, using these redundant information and a predefined decoding algorithm, data errors within a certain range can be detected and corrected without the need for the sending end to resend the data. Accordingly, considering that in a 4G network environment, especially in a mobile state, the network signal may be interfered by various factors, such as building obstruction, electromagnetic interference, etc., resulting in packet loss during data transmission of audio and video equipment. When the transmitted audio and video data is lost, the audio and video quality of the receiving end will be seriously affected, such as the video screen being stuck, the audio being interrupted, etc. In order to cope with the problem of packet loss in network transmission, additional protection can be provided by adding FEC information to the compressed data to enhance the anti-packet loss capability of the data in network transmission.

[0029] Adding FEC information to compressed data to obtain anti-packet loss compressed data is a key measure to ensure the reliability of data transmission in 4G networks. The following is a detailed description of its implementation process:

[0030] In this wearable badge-style audio and video device, the first thing to do is to select an adaptive FEC encoding algorithm. Common ones include Reed-Solomon codes, which have the ability to correct multiple random and burst errors and are suitable for data types such as audio and video that have strong continuity and high real-time requirements. For example, during video transmission, if part of the frame data is lost, the Reed-Solomon code can be restored through redundant information to ensure the continuity of video playback. Considering the limited device resources, algorithms such as low-density parity check codes with low computational complexity and efficient encoding and decoding are also good choices. They can reduce the demand for device computing power and storage capacity while ensuring error correction performance, ensuring the efficiency and stability of the device when performing the operation of adding FEC information.

[0031] The next step is to generate redundant information based on the compressed data. The compressed audio and video data needs to be grouped according to specific rules. For example, video data can be grouped by frame or consecutive frames, and audio data can be divided according to the number of sampling points or time segments. Taking Reed-Solomon code as an example, every several frames of video data are set as a data block, and the encoding algorithm is used to calculate and generate redundant bytes for each data block. These redundant bytes and the original data block together constitute an anti-packet loss compressed data block with added FEC information. In this process, through reasonable grouping and accurate calculation of redundant information, the receiving end has enough information to recover data when packet loss occurs, which effectively enhances the data anti-packet loss capability.

[0032] Then the redundant information is combined with the compressed data. To this end, a reasonable data structure should be designed to store and transmit the anti-packet loss compressed data after adding FEC information. Generally, an identification field is added to the data header to indicate the type and length of the subsequent FEC information, followed by the original compressed data and redundant check data. This structure facilitates the separation of encoding combination at the sender and decoding at the receiver. When the device sends data over the 4G network, the complete anti-packet loss compressed data will be transmitted.

[0033] During the operation of the entire device, FEC information is added to work in conjunction with other functions. The 4G network environment is complex and changeable, and factors such as signal strength fluctuations and network congestion often lead to unstable packet loss rates. Adding FEC information enables the device to adapt to this change. When the network is in good condition, FEC improves reliability as an additional guarantee; when the network deteriorates, the receiving end uses FEC to correct errors in a timely manner, reducing the impact of packet loss on audio and video quality, and ensuring stable communication services under different network conditions. In scenarios with high real-time requirements, such as security monitoring or remote medical guidance, although adding FEC will increase the amount of data, it has less impact on real-time performance than the data retransmission delay caused by packet loss. By optimizing the algorithm and the proportion of redundant information, the impact on real-time performance is minimized while ensuring reliability, ensuring the timely delivery of key information.

[0034] In summary, the wearable badge-type audio and video device effectively realizes the process of adding FEC information to compressed data to obtain anti-packet loss compressed data by scientifically selecting the FEC encoding algorithm, carefully generating combined redundant information, and co-optimizing with the overall functions of the device.

[0035] In an embodiment of the present application, the data sending module 140 is used to send the anti-packet loss compressed data through a 4G network. It should be understood that sending the anti-packet loss compressed data through a 4G network is the core function of the wearable badge-type audio and video device. It can realize the transmission of audio and video information collected and processed locally to the receiving end, so as to realize the circulation of audio and video information between different devices or systems. For example, in a security scenario, the video and audio data collected by the badge-type equipment worn by on-site security personnel can be sent to the monitoring center through this module, so that the commander can understand the on-site situation in real time.

[0036] In the embodiment of the present application, the sending rate adjustment module 150 is used to monitor the RTT and the packet loss rate in real time, and dynamically adjust the sending rate based on the monitoring results of the RTT and the packet loss rate.

[0037] Accordingly, in the sending rate adjustment module, the technical concept of the present application is to obtain the time queue of RTT and the time queue of packet loss rate, and use data processing and coding technology based on deep learning to implicitly encode the RTT (main variable) and the packet loss rate (covariate) in time sequence, so as to intelligently recommend the sending rate at the next time point based on the dynamic attention interaction representation between the implicit modulation of the main variable of the sending rate and the implicit features of the covariate of the sending rate. In this way, the changes in the network status can be captured more accurately, which allows the sending rate to be dynamically adjusted according to the current network conditions to maximize the use of available bandwidth, reduce unnecessary delays and retransmissions, and thus improve the overall transmission efficiency.

[0038] Figure 2 The block diagram of the transmission rate adjustment module in the wearable badge-type audio and video device for realizing 4G network transmission according to the embodiment of the present application. Figure 2 As shown, the sending rate adjustment module 150 includes: an RTT packet loss rate data acquisition unit 151, which is used to obtain the time queue of RTT and the time queue of packet loss rate; a time series implicit coding unit 152, which is used to take the time queue of RTT as the main variable and the time queue of packet loss rate as the covariate, and perform multi-scale time series coding on the main variable and the covariate to obtain the time series implicit features of the sending rate main variable and the time series implicit features of the sending rate covariate; a dynamic interaction unit 153, which is used to perform dynamic memory feature attention interaction on the time series implicit features of the sending rate main variable and the time series implicit features of the sending rate covariate to obtain the time series fusion features of the sending rate main variable-covariate, and obtain an optimization instruction based on the time series fusion features of the sending rate main variable-covariate, and the optimization instruction includes a recommended value of the sending rate at the next time point.

[0039] In an embodiment of the present application, the RTT packet loss rate data acquisition unit 151 is used to obtain the time queue of RTT and the time queue of packet loss rate. It should be understood that RTT refers to the time it takes for data to be sent from the sender to the receiver and then returned from the receiver to the sender. It is an important indicator for measuring network delay. In more detail, when the wearable badge-type device sends a data packet to the server, the server immediately returns a confirmation data packet after receiving it, and the time spent in this round trip is RTT. The time queue information of RTT reflects the change of network delay at different time points. By analyzing this time queue, it can be understood whether the network delay is gradually increasing, decreasing or remaining stable. The packet loss rate specifically refers to the proportion of the number of lost data packets to the total number of data packets sent during the data transmission process. The time queue information of the packet loss rate reflects the dynamic changes in the network packet loss situation over a period of time. Considering that the network state is dynamically changing, only obtaining the RTT and packet loss rate at a single moment cannot fully understand the status of the network. By recording the time queue, the changing trend of the network state can be captured, providing a basis for more accurate dynamic adjustment of the sending rate. For example, if you only see that the RTT is low and the packet loss rate is zero at the current moment, but these two indicators fluctuated greatly in the previous period, then you cannot simply assume that the network condition has always been good and use a higher sending rate.

[0040] In the wearable badge-type audio and video equipment, obtaining the time queue of RTT (round trip time) and the time queue of packet loss rate is the core data collection link for dynamically adjusting the sending rate. The following is a detailed description of its implementation process:

[0041] When the device is running and sending data through the 4G network, it will strictly mark and record each data packet sent. When the data packet is sent from the device, the device will accurately record the timestamp T1 at this time. This timestamp is the departure mark of the data packet and records the exact moment it leaves the device. At the same time, on the receiving end, the device closely monitors the confirmation information of the corresponding data packet. Once the confirmation data packet is received, the device immediately records the timestamp T2 of receiving the confirmation data packet. By calculating the difference between these two key timestamps, that is, RTT=T1-T2, the time it takes for the data packet to go back and forth in the network can be accurately obtained.

[0042] During the continuous data transmission process, the device repeats the above operation and performs such detailed round-trip time calculation for each data packet. In addition, these calculated RTT values ​​will be recorded in chronological order, gradually forming a time series that reflects the changes in network delay, that is, the RTT time queue. For example, during a continuous working time, the device records the RTT value of data packet 1 as RTT1, the RTT value of data packet 2 as RTT2, and the RTT values ​​of subsequent data packets are also recorded in this way, eventually forming a complete time queue [RTT1, RTT2, RTT3, ...]. Each RTT value in this time queue represents the delay status of network transmission at the corresponding time point, which provides a rich and detailed data basis for subsequent in-depth analysis of network status.

[0043] In addition to monitoring and calculating RTT, the device also closely monitors and counts packet loss. During data transmission, the device will accurately count the data packets sent and record the total number of data packets sent, N1. At the same time, the device uses a specially designed mechanism to identify data packets that are lost during transmission. One common method is to set a timeout mechanism. When a data packet is sent, if the corresponding confirmation information is not received within a preset time range, the device determines that the data packet has been lost and counts it in the number of lost data packets, N2.

[0044] Based on the above accurate record of the total number of sent packets and the number of lost packets, the device regularly calculates the packet loss rate according to the specific formula PLR=N1 / N2. The calculation period is usually set according to the performance and actual needs of the device, for example, the packet loss rate in the current time period is calculated every 1 second. The packet loss rate value calculated each time will also be recorded in chronological order to form a complete packet loss rate time queue. For example, over a period of time, the packet loss rates calculated by the device in sequence are PLR1, PLR2, PLR3,..., which constitute the packet loss rate time queue [PLR1, PLR2, PLR3,...]. This time queue clearly shows the dynamic change trend of network packet loss over time, providing another important dimension of data information for the device to fully understand the network status.

[0045] In terms of time queue management, the device needs to find a balance between retaining enough historical information to accurately grasp the trend of network status changes and avoiding excessive data processing burdens due to long queues. To this end, the device sets a reasonable queue length. For example, the device may decide to retain only the most recent 100 RTT values ​​and packet loss rate values. As new measurement values ​​are continuously added to the queue, the oldest values ​​are removed in an orderly manner to ensure that the queue always maintains a fixed length and the data in it can accurately reflect the recent network status. This dynamic update mechanism ensures that the data in the time queue is always timely and representative, avoiding the risk of misjudgment of network status due to outdated data.

[0046] During the entire operation period, the device will continuously perform the above-mentioned operations such as data packet sending, receiving, RTT calculation, packet loss monitoring and statistics, and time queue update. This real-time update mechanism enables the data in the time queue to reflect the latest changes in the network status in a timely manner, providing timely, accurate and reliable data basis for subsequent key operations such as dynamic adjustment of the sending rate based on these data.

[0047] In an embodiment of the present application, the timing implicit coding unit 152 is used to take the time queue of the RTT as the main variable and the time queue of the packet loss rate as the covariate, and perform multi-scale timing coding on the main variable and the covariate to obtain the timing implicit characteristics of the main variable of the sending rate and the timing implicit characteristics of the covariate of the sending rate. Specifically, in an embodiment of the present application, the timing implicit coding unit is used to: take the time queue of the RTT as the main variable and the time queue of the packet loss rate as the covariate, and input the main variable and the covariate into the multi-scale timing encoder based on the RNN-LSTM hybrid model respectively to obtain the timing implicit coding vector of the main variable of the sending rate as the timing implicit characteristics of the main variable of the sending rate and the timing implicit coding vector of the sending rate covariate as the timing implicit characteristics of the sending rate covariate. It should be understood that since the network conditions are highly dynamic, that is, both RTT and packet loss rate have timing characteristics in the time dimension, and there is a mutual timing correlation in each time period. The timing changes of RTT and packet loss rate can reflect the current network status. Therefore, in order to better understand the temporal variation trend of network performance, in the technical solution of the present application, the time queue of the RTT is used as the main variable and the time queue of the packet loss rate is used as the covariate, and the main variable and the covariate are respectively input into the multi-scale temporal encoder based on the RNN-LSTM hybrid model to obtain the time series implicit coding vector of the main variable of the sending rate and the time series implicit coding vector of the covariate of the sending rate. It can be understood that RTT is a key indicator for measuring network performance. It directly reflects the time required from sending data to receiving confirmation information. Compared with the packet loss rate, RTT is easier to be accurately measured and continuously monitored. Each successful communication can be used to update the estimated value of RTT, making it a stable and reliable reference point, and thus as the main variable. Although RTT provides information about delay, it does not fully reflect the reliability of the network. That is, the packet loss rate directly indicates the data loss on the network path, which is very important for evaluating the quality of the network. As a covariate, it can provide additional information for the model to help understand the network status more comprehensively. It is worth mentioning that RNN (recurrent neural network) is good at processing sequence data, but the standard RNN has the problem of gradient disappearance and it is difficult to remember long-term dependencies. LSTM (Long Short-Term Memory) overcomes this problem and can effectively handle dependencies in long time series. Therefore, by combining the advantages of both, the hybrid model can simultaneously consider changes in different time windows, maintain high accuracy while processing longer time series, and thus capture more comprehensive network characteristics.

[0048] In the embodiment of the present application, the dynamic interaction unit 153 is used to perform dynamic memory feature attention interaction on the sending rate main variable time series implicit feature and the sending rate covariate time series implicit feature to obtain the sending rate main variable-covariate time series fusion feature, and obtain an optimization instruction based on the sending rate main variable-covariate time series fusion feature, and the optimization instruction includes a recommended value of the sending rate at the next time point. Specifically, Figure 3 FIG. 1 is a block diagram of a dynamic interaction unit in a wearable badge-type audio and video device for implementing 4G network transmission according to an embodiment of the present application. Figure 3 As shown, the dynamic interaction unit 153 includes: a sending rate timing dynamic attention interaction sub-unit 1531, which is used to perform dynamic memory feature attention interaction on the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector features to obtain a sending rate main variable-covariate timing fusion coding vector as the sending rate main variable-covariate timing fusion feature; a sending rate recommendation sub-unit 1532, which is used to obtain the optimization instruction based on the sending rate main variable-covariate timing fusion coding vector, and the optimization instruction includes a recommended value of the sending rate at the next time point.

[0049] In an embodiment of the present application, the sending rate timing dynamic attention interaction subunit 1531 is used to dynamically memorize the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector features to obtain the sending rate main variable-covariate timing fusion coding vector as the sending rate main variable-covariate timing fusion feature. It should be understood that there is a mutual correlation interaction relationship and an inherent implicit relationship between the sending rate main variable timing implicit feature and the sending rate covariate timing implicit feature. When the packet loss rate is high, TCP and other protocols will trigger a retransmission mechanism, which not only increases the actual data transmission time, but also may cause the congestion control algorithm to adjust the window size, further increasing the RTT. Therefore, in order to capture and mine the timing information of the two original features and the implicit pattern of interaction between them, in the technical solution of the present application, a dynamic memory feature attention interaction mechanism is introduced to process the sending rate main variable timing implicit feature and the sending rate covariate timing implicit feature to obtain the sending rate main variable-covariate timing fusion feature. In particular, by introducing a dynamic memory mechanism, the model is able to remember the network status at different time points and adjust its "memory" according to the current situation, thereby better understanding the complex dynamic relationship between RTT and packet loss rate, which helps to capture long-term dependencies and short-term fluctuations, providing support for more accurate predictions.

[0050] Specifically, in an embodiment of the present application, the sending rate timing dynamic attention interaction sub-unit includes: a sending rate collaborative feature extraction secondary sub-unit, which is used to perform collaborative feature extraction on the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector to obtain an implicit collaborative coding vector between the sending rate main variable and covariate timing features; a sending rate collaborative feature optimization secondary sub-unit, which is used to perform attention modulation optimization on the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector based on the implicit collaborative coding vector between the sending rate main variable and covariate timing features to obtain an optimized sending rate main variable timing implicit coding vector and an optimized sending rate covariate timing implicit coding vector; a sending rate main variable-covariate timing fusion coding vector generation secondary sub-unit, which is used to perform feature linear interaction on the optimized sending rate main variable timing implicit coding vector and the optimized sending rate covariate timing implicit coding vector to obtain the sending rate main variable-covariate timing fusion coding vector.

[0051] Specifically, in the embodiment of the present application, the transmission rate collaborative feature extraction secondary subunit is used to: cascade the transmission rate main variable time series implicit coding vector and the transmission rate covariate time series implicit coding vector to obtain the transmission rate main variable-covariate time series feature cascade coding vector; use the point convolution coding module and the LeakyReLU activation function to process the transmission rate main variable-covariate time series feature cascade coding vector to obtain the transmission rate main variable-covariate time series feature implicit collaborative coding vector. The above process can be expressed by the formula:

[0052]

[0053] in, and represent the time series implicit coding vector of the main variable of the transmission rate and the time series implicit coding vector of the covariate of the transmission rate respectively, , and They are point-by-point addition, point-by-point subtraction, and point-by-point multiplication. For cascade processing, is the point convolutional encoding operation, is the LeakyReLU activation function, It is the implicit co-coding vector between the sending rate main variable and the covariate time series features.

[0054] That is, by performing collaborative feature extraction on the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector, the deep correlation between the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector can be captured. This deep correlation is not limited to the simple linear combination of the two data timing features of RTT and packet loss rate, but involves complex nonlinear interactions between features. Specifically, through the convolution operation, the local correlation pattern between the two data timing features of RTT and packet loss rate can be identified, and the activation operation of the Leaky ReLU activation function can learn the nonlinear representation of the feature. The combined use of these two operations is conducive to capturing the complex interactive relationship between different data timing features that affect the sending rate.

[0055] Specifically, in the embodiment of the present application, the transmission rate collaborative feature optimization secondary subunit is used to:

[0056] The implicit collaborative coding vector between the transmission rate main variable and the covariate time series features is written into the dynamic memory unit to obtain the transmission rate main variable and the covariate time series dynamic key vector. The process can be expressed by the formula:

[0057]

[0058] in is the implicit co-coding vector between the sending rate main variable and the covariate time series features, is a 1×1 convolution operation, is the sending rate primary variable-covariate time series dynamic key vector.

[0059] The transmission rate main variable-covariate time series dynamic key vector is extracted from the dynamic memory unit, and the transmission rate main variable time series implicit encoding vector and the transmission rate main variable-covariate time series dynamic key vector are subjected to feature attention optimization based on the first converter structure to obtain the optimized transmission rate main variable time series implicit encoding vector. The process can be expressed by the formula:

[0060]

[0061]

[0062]

[0063] in, represents the timing implicit coding vector of the main variable of the sending rate, is the sending rate primary variable-covariate time series dynamic key vector, is the sending rate main variable query embedding matrix, is the first sending rate principal variable bias vector, is matrix multiplication, is the sending rate primary variable query vector, is the embedding matrix of the sending rate main variable value, is the second sending rate principal variable bias vector, is the value vector of the primary variable of the sending rate, yes The transposed vector of yes Length, yes function, It is the implicit coding vector of the main variable timing of the optimized sending rate.

[0064] The transmission rate main variable-covariate time series dynamic key vector is extracted from the dynamic memory unit, and the transmission rate covariate time series implicit encoding vector and the transmission rate main variable-covariate time series dynamic key vector are subjected to feature attention optimization based on the second converter structure to obtain the optimized transmission rate covariate time series implicit encoding vector. The process can be expressed by the formula:

[0065]

[0066]

[0067]

[0068] in, represents the sending rate covariate temporal implicit coding vector, is matrix multiplication, yes The transposed vector of yes Length, yes function, is the sending rate covariate query embedding matrix, is the first sending rate covariate bias vector, is the sending rate covariate query vector, is the embedding matrix of the sending rate covariate values, is the second sending rate covariate bias vector, is the vector of sending rate covariate values, is the optimized sending rate covariate timing implicit coding vector.

[0069] That is, the implicit collaborative coding vector between the main variable and covariate time series features of the sending rate is written into the dynamic memory unit, and the characteristic of the dynamic memory unit that can dynamically adjust its own memory content according to the current input state can be used to better understand the information of the implicit collaborative coding vector between the main variable and covariate time series features of the input sending rate, so as to more accurately grasp the actual network status and better serve the dynamic adjustment of the sending rate. Then, the sending rate main variable-covariate time series dynamic key vector is extracted from the dynamic memory unit, and the sending rate main variable time series implicit coding vector and the sending rate main variable-covariate time series dynamic key vector are optimized based on the first converter structure. At the same time, the sending rate main variable-covariate time series dynamic key vector is extracted from the dynamic memory unit, and the sending rate covariate time series implicit coding vector and the sending rate main variable-covariate time series dynamic key vector are optimized based on the second converter structure. Here, the converter structure uses its core self-attention mechanism to perform query attention-based feature modulation on the sending rate main variable time series implicit coding vector and the sending rate covariate time series implicit coding vector based on the sending rate main variable-covariate time series dynamic key vector. Specifically, the sending rate main variable-covariate time series dynamic key vector is used as a common attention space, which not only enhances the expression of key information in the original feature vector and reduces the influence of noise and irrelevant information, but also aligns the generated optimized sending rate main variable time series implicit coding vector and the optimized sending rate covariate time series implicit coding vector with the semantic space, which is beneficial to the subsequent feature interaction fusion operation.

[0070] Specifically, the optimized sending rate main variable time series implicit coding vector and the optimized sending rate covariate time series implicit coding vector are subjected to feature linear interaction to obtain the sending rate main variable-covariate time series fusion coding vector. The above process can be expressed by the formula:

[0071]

[0072] in, is the optimized sending rate main variable timing implicit coding vector, is the optimized sending rate covariate timing implicit coding vector, is the weighting coefficient, It is the sending rate main variable-covariate time series fusion coding vector.

[0073] That is, the information in the optimized sending rate main variable time series implicit coding vector and the optimized sending rate covariate time series implicit coding vector is further integrated through feature linear interaction to generate the final sending rate main variable-covariate time series fusion coding vector. In this way, while maintaining the unique information of each optimized feature vector, a comprehensive feature representation can be generated, which can provide high-quality input for subsequent decoding tasks.

[0074] In an embodiment of the present application, the sending rate recommendation subunit 1532 is used to obtain the optimization instruction based on the sending rate main variable-covariate timing fusion coding vector, and the optimization instruction includes the recommended value of the sending rate at the next time point. Specifically, in an embodiment of the present application, the sending rate recommendation subunit is used to: input the sending rate main variable-covariate timing fusion coding vector into the decoder-based sending rate dynamic optimization engine to obtain the optimization instruction, and the optimization instruction includes the recommended value of the sending rate at the next time point. More specifically, in an embodiment of the present application, the sending rate recommendation subunit is used to: multiply the decoding weight matrix of the decoder with the sending rate main variable-covariate timing fusion coding vector to obtain the decoded sending rate main variable-covariate timing fusion coding vector, and accumulate and sum all eigenvalues ​​of the decoded sending rate main variable-covariate timing fusion coding vector to obtain the recommended value of the sending rate at the next time point. It should be understood that the decoder is a neural network model, which decodes the sending rate main variable-covariate time series fusion coding vector containing network status information according to the pre-trained model parameters to obtain the recommended sending rate value that best matches the current network status. This recommended value enables the wearable badge-type audio and video device to adjust the sending rate in time according to the latest network status, avoiding problems such as low transmission efficiency or data loss caused by a fixed sending rate, which is conducive to ensuring stable data transmission and thus improving user experience.

[0075] In particular, when the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector distribution represent the cyclic multi-scale timing correlation characteristics of RTT and packet loss rate, after performing inter-feature attention interaction based on the dynamic memory mechanism, the sending rate main variable-covariate timing fusion coding vector will also have significant attention interaction distribution space structure differences due to the dynamic memory differences in the timing correlation feature distribution caused by the differences in the timing distribution of the source data, affecting the convergence consistency of the decoder, and thus affecting the accuracy of the optimization instructions obtained by the sending rate dynamic optimization engine based on the decoder.

[0076] Preferably, inputting the sending rate main variable-covariate timing fusion coding vector into a decoder-based sending rate dynamic optimization engine to obtain an optimization instruction comprises:

[0077] The absolute value sum and the square root of the square sum of all eigenvalues ​​of the transmission rate main variable-covariate time series fusion coding vector are calculated to obtain the first and second transmission rate main variable-covariate time series fusion spatial structure values; the above process can be expressed by the formula:

[0078]

[0079]

[0080] in, represents the first The eigenvalues ​​at the positions, is the spatial structure value of the first sending rate main variable-covariate time series fusion, is the spatial structure value of the second sending rate main variable-covariate time series fusion;

[0081] Determine the number of eigenvalues ​​of the sending rate main variable-covariate time series fusion coding vector ;

[0082] For each eigenvalue of the transmission rate main variable-covariate time series fusion coding vector, the first transmission rate main variable-covariate time series fusion spatial structure value minus the product of the eigenvalue and the number of eigenvalues ​​is calculated to obtain the first transmission rate main variable-covariate time series fusion long-range dependency value; the above process can be expressed by the formula:

[0083]

[0084] in, is the spatial structure value of the first sending rate main variable-covariate time series fusion, represents the first The eigenvalues ​​at the positions, is the number of eigenvalues ​​of the sending rate main variable-covariate time series fusion encoding vector, The long-range dependency value of the first sending rate main variable-covariate time series fusion;

[0085] The second transmission rate main variable-covariate time series fusion long-range dependency value is obtained by calculating the square root of the number of eigenvalues ​​multiplied by the product of the eigenvalues ​​minus the second transmission rate main variable-covariate time series fusion spatial structure value; the above process can be expressed by the formula:

[0086]

[0087] in, represents the first The eigenvalues ​​at the positions, is the number of eigenvalues ​​of the sending rate main variable-covariate time series fusion encoding vector, is the spatial structure value of the second sending rate main variable-covariate time series fusion, Fusion of long-range dependency values ​​for the second sending rate main variable-covariate time series;

[0088] The exponential value calculated by taking the first transmission rate main variable-covariate time series fusion long-range dependency value as the exponent of the natural constant and the inverse of the second transmission rate main variable-covariate time series fusion long-range dependency value are weighted summed to obtain the optimized eigenvalue corresponding to each eigenvalue; the above process can be expressed by the formula:

[0089]

[0090] in, is the long-range dependency value of the first sending rate main variable-covariate time series fusion, The long-range dependency value of the second sending rate main variable-covariate time series fusion, and is the weighted hyperparameter, is the first fusion encoding vector of the sending rate main variable-covariate time series The optimized eigenvalue corresponding to the eigenvalue at each position;

[0091] The optimized sending rate main variable-covariate timing fusion coding vector composed of the optimized characteristic values ​​is input into the decoder-based sending rate dynamic optimization engine to obtain the optimization instruction.

[0092] That is, in view of the possible spatial structure loss of the feature set of the sending rate main variable-covariate time series fusion coding vector in the high-dimensional space, which causes the decoder's weight matrix to implicitly infer spatial structure information based on the features, resulting in inconsistent convergence. A long-distance feature dependency relationship is established based on the overall feature scale of the sending rate main variable-covariate time series fusion coding vector relative to the spatial structure representation of the sending rate main variable-covariate time series fusion coding vector, so as to establish the feature local connectivity of the sending rate main variable-covariate time series fusion coding vector, and capture the spatial ambiguous information of the object feature value through the unstructured feature value point prediction of the sending rate main variable-covariate time series fusion coding vector, thereby improving the spatial inductive deviation perception ability of the feature set of the sending rate main variable-covariate time series fusion coding vector, improving the convergence consistency of the decoder, and improving the accuracy of the optimization instruction obtained by inputting the sending rate main variable-covariate time series fusion coding vector into the decoder-based sending rate dynamic optimization engine.

[0093] In summary, the sending rate adjustment module 150 is clearly explained, which uses deep learning-based data processing and encoding technology to perform time-series implicit encoding on the time queue of RTT (sending rate main variable) and the time queue of packet loss rate (sending rate covariate), so as to intelligently recommend the sending rate at the next time point based on the dynamic attention interaction representation between the timing implicit modulation of the sending rate main variable and the timing implicit features of the sending rate covariate. In this way, the changes in network status can be captured more accurately, which allows the sending rate to be dynamically adjusted according to the current network conditions to maximize the use of available bandwidth, reduce unnecessary delays and retransmissions, and thus improve overall transmission efficiency.

[0094] In summary, the wearable work badge-type audio and video device 100 for realizing 4G network transmission based on the embodiment of the present application is explained, which first collects data through the microphone and camera built into the device, then compresses the collected data, and then adds FEC information to the compressed data to obtain anti-packet loss compressed data, and then transmits the anti-packet loss compressed data through the 4G network. During the data transmission process, the RTT and packet loss rate are monitored in real time, and the sending rate is dynamically adjusted based on the monitoring results. In this way, the wearable work badge-type audio and video device can better realize efficient and reliable data transmission under the 4G network.

Claims

1. A wearable badge-type audio and video device that implements 4G network transmission, characterized in that: include: An audio and video data acquisition module, used to acquire audio data or video data using a built-in microphone and / or camera of the audio and video device; A data compression module, used for compressing the audio data or the video data to obtain compressed data; An information adding module, used for adding FEC information to the compressed data to obtain packet loss resistant compressed data; A data sending module, used for sending the anti-packet loss compressed data through a 4G network; A sending rate adjustment module, used for monitoring RTT and packet loss rate in real time, and dynamically adjusting the sending rate based on the monitoring results of the RTT and the packet loss rate; The sending rate adjustment module includes: an RTT packet loss rate data acquisition unit, which is used to acquire the time queue of RTT and the time queue of packet loss rate; a time series implicit coding unit, which is used to take the time queue of RTT as the main variable and the time queue of packet loss rate as the covariate, and perform multi-scale time series coding on the main variable and the covariate to obtain the time series implicit features of the sending rate main variable and the time series implicit features of the sending rate covariate; a dynamic interaction unit, which is used to perform dynamic memory feature attention interaction on the time series implicit features of the sending rate main variable and the time series implicit features of the sending rate covariate to obtain the time series fusion features of the sending rate main variable-covariate, and obtain an optimization instruction based on the time series fusion features of the sending rate main variable-covariate, wherein the optimization instruction includes a recommended value of the sending rate at the next time point; The timing implicit coding unit is used to: take the time queue of the RTT as the main variable and the time queue of the packet loss rate as the covariate, and input the main variable and the covariate into a multi-scale timing encoder based on the RNN-LSTM hybrid model to obtain a timing implicit coding vector of the main variable of the sending rate as the timing implicit feature of the main variable of the sending rate and a timing implicit coding vector of the covariate of the sending rate as the timing implicit feature of the covariate of the sending rate; Wherein, the dynamic interaction unit includes: A sending rate timing dynamic attention interaction subunit is used to perform dynamic memory feature attention interaction on the sending rate main variable timing implicit coding vector and the sending rate covariate timing implicit coding vector to obtain a sending rate main variable-covariate timing fusion coding vector as the sending rate main variable-covariate timing fusion feature; The sending rate recommendation subunit is used to obtain the optimization instruction based on the sending rate main variable-covariate time series fusion coding vector, and the optimization instruction includes a recommended value of the sending rate at the next time point.

2. The wearable badge-type audio and video device for realizing 4G network transmission according to claim 1 is characterized in that: The sending rate timing dynamic attention interaction subunit includes: A sending rate collaborative feature extraction secondary subunit is used to perform collaborative feature extraction on the sending rate main variable time series implicit coding vector and the sending rate covariate time series implicit coding vector to obtain an implicit collaborative coding vector between the sending rate main variable and covariate time series features; A sending rate collaborative feature optimization secondary subunit is used to perform attention modulation optimization on the sending rate main variable time series implicit coding vector and the sending rate covariate time series implicit coding vector based on the implicit collaborative coding vector between the sending rate main variable and covariate time series features to obtain an optimized sending rate main variable time series implicit coding vector and an optimized sending rate covariate time series implicit coding vector; The sending rate main variable-covariate timing fusion coding vector generates a secondary sub-unit, which is used to perform feature linear interaction on the optimized sending rate main variable timing implicit coding vector and the optimized sending rate covariate timing implicit coding vector to obtain the sending rate main variable-covariate timing fusion coding vector.

3. The wearable badge-type audio and video device for realizing 4G network transmission according to claim 2 is characterized in that: The transmission rate collaborative feature extraction secondary subunit is used to: Cascading the sending rate main variable time series implicit coding vector and the sending rate covariate time series implicit coding vector to obtain a sending rate main variable-covariate time series feature cascade coding vector; The point convolution encoding module and the Leaky ReLU activation function are used to process the cascade coding vector between the sending rate main variable and the covariate time series features to obtain the implicit collaborative coding vector between the sending rate main variable and the covariate time series features.

4. The wearable badge-type audio and video device for realizing 4G network transmission according to claim 3 is characterized in that: The transmission rate cooperative feature optimization secondary subunit is used to: Writing the implicit collaborative coding vector between the transmission rate main variable and covariate time series features into a dynamic memory unit to obtain a transmission rate main variable and covariate time series dynamic key vector; Extracting the sending rate main variable-covariate time series dynamic key vector from the dynamic memory unit, and performing feature attention optimization based on the first converter structure on the sending rate main variable time series implicit encoding vector and the sending rate main variable-covariate time series dynamic key vector to obtain the optimized sending rate main variable time series implicit encoding vector; The sending rate main variable-covariate timing dynamic key vector is extracted from the dynamic memory unit, and the sending rate covariate timing implicit encoding vector and the sending rate main variable-covariate timing dynamic key vector are subjected to feature attention optimization based on the second converter structure to obtain the optimized sending rate covariate timing implicit encoding vector.

5. The wearable badge-type audio and video device for realizing 4G network transmission according to claim 4 is characterized in that: The sending rate recommendation subunit is used to: input the sending rate main variable-covariate time series fusion coding vector into the decoder-based sending rate dynamic optimization engine to obtain the optimization instruction, and the optimization instruction includes a recommended value of the sending rate at the next time point.

6. The wearable badge-type audio and video device for realizing 4G network transmission according to claim 5 is characterized in that: The sending rate recommendation subunit is used to: multiply the decoding weight matrix of the decoder with the sending rate main variable-covariate timing fusion coding vector to obtain the decoded sending rate main variable-covariate timing fusion coding vector, and accumulate and sum all eigenvalues ​​of the decoded sending rate main variable-covariate timing fusion coding vector to obtain the recommended value of the sending rate at the next time point.

Images