Time delay monitoring system based on wireless image transmission

Abstract

The invention discloses a time delay monitoring system based on wireless image transmission, which comprises a time delay monitoring module and an emergency processing module, and the output of the time delay monitoring module is connected with the input of the emergency processing module. According to the invention, the time delay monitoring module and the emergency processing module are arranged, the time delay monitoring module captures time delay data between processes in a signal transmission process and some physical data which may influence time delay, an information source is provided for the emergency processing module, and the emergency processing module analyzes the data to obtain a reason for generating the time delay, so that the time delay can be accurately processed. And then a corresponding instruction is issued to solve the time delay. The emergency processing module comprises a deep learning analysis unit and a control unit, due to the fact that factors influencing time delay are many, efficient and accurate judgment cannot be achieved only through manpower, the deep learning analysis unit is designed, data of the time delay monitoring module is subjected to deep learning, and therefore time delay generation reasons are efficiently and accurately obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of data compression, and in particular to a time delay monitoring system based on wireless image transmission. Background Art

[0002] Wireless image transmission technology is mostly used in the fields of drones and aerospace. It can provide high-definition, real-time video signal transmission. However, if the delay in the transmission of real-time signals is too large, it will cause control lags, making it impossible for the device to move according to the established route or control, and the safety of the device will be greatly reduced.

[0003] Therefore, in the prior art, the delay is controlled within a very small range by optimizing the software and upgrading the communication method. However, in actual applications, depending on the scenario, the cause of the delay is not only in the software and communication method. Interference that cannot be prefabricated and physical problems of hardware equipment will increase the delay time. Therefore, there is an urgent need for a delay monitoring system that can be applied to complex scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a delay monitoring system based on wireless image transmission to solve the above problems.

[0005] The present invention is achieved through the following technical solutions: A delay monitoring system based on wireless image transmission comprises: a delay monitoring module and an emergency processing module, wherein the output of the delay monitoring module is connected to the input of the emergency processing module.

[0006] The present invention sets up a delay monitoring module and an emergency processing module. The delay monitoring module captures the delay data between various processes in the signal transmission process, as well as some physical data that may affect the delay, to provide an information source for the emergency processing module. The emergency processing module analyzes the data to obtain the cause of the delay, and then issues corresponding instructions to solve the delay.

[0007] Preferably, the emergency processing module includes a deep learning analysis unit and a control unit, and the delay monitoring module includes an interference monitoring unit, a hardware monitoring unit, and a delay dynamic monitoring unit, wherein the deep learning analysis unit performs deep learning on the preprocessed relevant information of the interference monitoring unit, the hardware monitoring unit, and the delay dynamic monitoring unit to find the cause of the unreasonable delay and provide the best solution, and then the deep learning analysis unit issues solution instructions to the control unit to achieve effective emergency processing.

[0008] The emergency handling module of the present invention includes a deep learning analysis unit and a control unit. Since there are many factors that affect the delay, efficient and accurate judgment cannot be achieved only by manpower. Therefore, a deep learning analysis unit is designed to perform deep learning on the data of the delay monitoring module, so as to efficiently and accurately obtain the cause of the delay.

[0009] Preferably, the interference monitoring unit is used to monitor the wireless signal strength in real time, the hardware monitoring unit is used to monitor the processor temperature information of the image acquisition end and the image display end in real time, and the delay dynamic monitoring unit is used to monitor the comprehensive time interval from the image signal being sent to the image display in real time.

[0010] The two most unstable factors affecting latency are wireless interference and hardware temperature. When there is obstruction or electromagnetic interference, the strength of the wireless signal will drop rapidly, resulting in increased latency. Similarly, if the hardware temperature is too high or too low, the processor's processing speed will slow down, resulting in increased latency.

[0011] Preferably, the deep learning analysis unit uses a long short-term memory network (LSTM) to process the information of the delay monitoring module, and the specific steps are as follows: S1: Through the latency monitoring module, wireless signal strength, processor temperature, and comprehensive latency are collected in real time; S2: preprocess the data collected in S1, including normalization, denoising, and segmenting the data into training and test sets; S3: Determine the number of network layers, the number of neurons in each layer, and the activation function in the LSTM model; S4: Design a loss function to train the LSTM model so that the LSTM model outputs the categories of causes of delay and the probability of occurrence of each category; S5: Evaluate the accuracy of the LSTM model.

[0012] Preferably, the number of neurons in each layer of the LSTM model in S3 is 256; The activation function uses the softmax function, the formula is as follows: in Represents the probability of belonging to category i when a given input sample z is given, where , K is the number of categories, Indicates the first Elements Perform exponential operations, Represents the i-th element in the input vector Z Perform exponential operation, j is the category index.

[0013] Since there are many factors that affect the delay, the model needs to be sufficiently stable. Therefore, the present invention uses a long short-term memory network. The LSTM network can learn long-term dependency information and avoid the problem of gradient disappearance. In addition, when processing sequence data, that is, wireless signal strength and processor temperature data, the parameters of the LSTM are shared across the entire sequence, which enables the model to process input sequences of variable length and reduces the number of parameters of the model.

[0014] Preferably, the loss function of the LSTM model is: Where L is the loss function.

[0015] is the total number of samples.

[0016] is the total number of categories.

[0017] is the weight of sample z belonging to category i.

[0018] is an indicator variable. If sample z belongs to category i, then ,otherwise .

[0019] is the probability that the model predicts that sample z belongs to category i.

[0020] is the logarithm of the predicted probability of the ith category.

[0021] Since there are significant differences in the number of samples of wireless signal strength and processor temperature data, the loss function of the present invention assigns different weights to each category, so that the model pays more attention to categories with fewer samples during training, thereby improving the model's recognition ability of these categories. In addition, by adjusting the weights, the performance degradation caused by category imbalance can be reduced. This helps to improve the overall accuracy and generalization ability of the model. In deep learning, category imbalance may lead to gradient vanishing or gradient exploding problems. Through this loss function, the contribution of different categories to the gradient can be balanced, which helps the model to be trained more stably.

[0022] Preferably, K in the activation function and loss function of the LSTM model is 2, where K represents the delay category caused by temperature and K represents the delay category caused by interference, respectively.

[0023] Preferably, the deep learning analysis unit outputs the relevant parameters collected in real time and the corresponding delay category analysis results to the emergency processing module, and the delay generated by the software is also input into the emergency processing module, where the delay generated by the software includes encoding and decoding delay, display processing delay, and network congestion delay. When the emergency processing module receives the above-mentioned delay information, it automatically selects the corresponding emergency measures through the meta-learning algorithm. The emergency measures include adjusting the antenna gain, adjusting the processor temperature, reporting the encoding and decoding delay problem and requesting repair, switching the display device, and switching the network type.

[0024] Preferably, the meta-learning algorithm comprises the following steps: S1: When used as the input for meta-training, the delay caused by temperature, the delay caused by interference, the encoding and decoding delay, the display processing delay, and the network congestion delay all include categories and the specific delay time generated by the corresponding categories; S2: When a single category is input, different inputs are matched to emergency measures through manual discrimination, and the meta-learning algorithm is trained and optimized so that it can automatically discern the emergency measures to be used when the information of S1 is input; S3: When multiple categories are input simultaneously, the meta-learning algorithm is extended by the Reptile algorithm; S3.1: Initialize the input and use the categories that affect the delay and their probabilities as the starting point of the meta-learning process; S3.2: Establish a task model, classify the categories that affect the delay and their probabilities again, and correspond them to the categories of emergency measures; S3.3: Train the task model using stochastic gradient descent; S4: Manually determine the accuracy and feasibility of the output results of the expanded meta-learning algorithm, and feed the erroneous data back to the meta-learning algorithm for further optimization.

[0025] Since the output results of the LSTM model are very sophisticated, it is also impossible to use only manual judgment when allocating emergency measures. Therefore, the present invention uses a meta-learning algorithm to correspond the output results of the LSTM model to the emergency measures, and through meta-training, achieves efficient and accurate emergency measure selection.

[0026] Preferably, when a new delay category is added, S2 to S4 of the meta-learning algorithm are repeated.

[0027] As application scenarios become more diverse, factors affecting latency will also increase. Therefore, the present invention selects a meta-learning algorithm to enable rapid learning with very few samples or iterations when faced with new tasks. In addition, in many practical applications, it is expensive or infeasible to obtain a large amount of labeled data. The meta-learning algorithm reduces the reliance on a large amount of labeled data and improves data utilization efficiency by learning from previous tasks. The learning algorithm enhances the model's transfer learning ability by learning common features and rules between tasks, enabling it to apply knowledge learned on one task to other related tasks.

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. The present invention sets a delay monitoring module and an emergency processing module. The delay monitoring module captures the delay data between various processes in the signal transmission process, as well as some physical data that may affect the delay, and provides an information source for the emergency processing module. The emergency processing module analyzes the data to obtain the cause of the delay, and then issues corresponding instructions to solve the delay. 2. The emergency processing module of the present invention includes a deep learning analysis unit and a control unit. Since there are many factors that affect the delay, efficient and accurate judgment cannot be achieved only by manpower. Therefore, by designing a deep learning analysis unit, the data of the delay monitoring module is deeply learned, so as to efficiently and accurately obtain the cause of the delay; 3. Since there are significant differences in the number of samples of wireless signal strength and processor temperature data, the loss function of the present invention assigns different weights to each category, so that the model pays more attention to categories with fewer samples during training, thereby improving the model's recognition ability of these categories. In addition, by adjusting the weights, the performance degradation caused by category imbalance can be reduced. This helps to improve the overall accuracy and generalization ability of the model. In deep learning, category imbalance may lead to gradient vanishing or gradient exploding problems. Through this loss function, the contribution of different categories to the gradient can be balanced, which helps the model to be trained more stably; 4. Since the output results of the LSTM model are very refined, it is also impossible to use only manual judgment when allocating emergency measures. Therefore, the present invention uses a meta-learning algorithm to correspond the output results of the LSTM model to the emergency measures, and through meta-training, achieve efficient and accurate selection of emergency measures; 5. With the enrichment of application scenarios, the factors affecting latency will also increase. Therefore, the present invention selects a meta-learning algorithm to enable rapid learning with very few samples or iterations when facing new tasks. In addition, in many practical applications, it is expensive or infeasible to obtain a large amount of labeled data. The meta-learning algorithm reduces the dependence on a large amount of labeled data and improves data utilization efficiency by learning from previous tasks. The learning algorithm enhances the transfer learning ability of the model by learning the common features and rules between tasks, enabling it to apply the knowledge learned on one task to other related tasks; DETAILED DESCRIPTION

[0029] The technical solution of the present invention is further described in detail below, but the protection scope of the present invention is not limited to the following.

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the embodiments described are only part of the embodiments of the present invention, rather than all the embodiments.

[0031] Therefore, the following detailed description of the embodiments of the present invention provided is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention. It should be noted that relational terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0032] Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0033] Example This embodiment includes: a delay monitoring system based on wireless image transmission, including: a delay monitoring module and an emergency processing module, wherein the output of the delay monitoring module is connected to the input of the emergency processing module. This embodiment is applied to the wireless image transmission scenario of a drone.

[0034] The present invention sets up a delay monitoring module and an emergency processing module. The delay monitoring module captures the delay data between various processes in the signal transmission process, as well as some physical data that may affect the delay, to provide an information source for the emergency processing module. The emergency processing module analyzes the data to obtain the cause of the delay, and then issues corresponding instructions to solve the delay.

[0035] In this embodiment, the emergency processing module includes a deep learning analysis unit and a control unit, and the delay monitoring module includes an interference monitoring unit, a hardware monitoring unit, and a delay dynamic monitoring unit. The deep learning analysis unit performs deep learning on the pre-processed relevant information of the interference monitoring unit, the hardware monitoring unit, and the delay dynamic monitoring unit to find the cause of the unreasonable delay and provide the best solution. Then the deep learning analysis unit issues solution instructions to the control unit to achieve effective emergency processing.

[0036] The emergency handling module of the present invention includes a deep learning analysis unit and a control unit. Since there are many factors that affect the delay, efficient and accurate judgment cannot be achieved only by manpower. Therefore, a deep learning analysis unit is designed to perform deep learning on the data of the delay monitoring module, so as to efficiently and accurately obtain the cause of the delay.

[0037] In this embodiment, the interference monitoring unit is used to monitor the wireless signal strength in real time, the hardware monitoring unit is used to monitor the processor temperature information of the image acquisition end and the image display end in real time, and the delay dynamic monitoring unit is used to monitor the comprehensive time interval from the image signal to the image display in real time.

[0038] The two most unstable factors affecting latency are wireless interference and hardware temperature. When there is obstruction or electromagnetic interference, the strength of the wireless signal will drop rapidly, resulting in increased latency. Similarly, if the hardware temperature is too high or too low, the processor's processing speed will slow down, resulting in increased latency.

[0039] In this embodiment, the deep learning analysis unit uses a long short-term memory network (LSTM) to process the information of the delay monitoring module, and the specific steps are as follows: S1: Through the latency monitoring module, wireless signal strength, processor temperature, and comprehensive latency are collected in real time; S2: preprocess the data collected in S1, including normalization, denoising, and segmenting the data into training and test sets; S3: Determine the number of network layers, the number of neurons in each layer, and the activation function in the LSTM model; S4: Design a loss function to train the LSTM model so that the LSTM model outputs the categories of causes of delay and the probability of occurrence of each category; S5: Evaluate the accuracy of the LSTM model.

[0040] Preferably, the number of neurons in each layer of the LSTM model in S3 is 256; The activation function uses the softmax function, the formula is as follows: in Represents the probability of belonging to category i when a given input sample z is given, where , K is the number of categories, Indicates the first Elements Perform exponential operations, Represents the i-th element in the input vector Z Perform exponential operation, j is the category index.

[0041] Since there are many factors that affect the delay, the model needs to be sufficiently stable. Therefore, the present invention uses a long short-term memory network. The LSTM network can learn long-term dependency information and avoid the problem of gradient disappearance. In addition, when processing sequence data, that is, wireless signal strength and processor temperature data, the parameters of the LSTM are shared across the entire sequence, which enables the model to process input sequences of variable length and reduces the number of parameters of the model.

[0042] In this embodiment, the loss function of the LSTM model is: Where L is the loss function.

[0043] is the total number of samples.

[0044] is the total number of categories.

[0045] is the weight of sample z belonging to category i.

[0046] is an indicator variable. If sample z belongs to category i, then ,otherwise .

[0047] is the probability that the model predicts that sample z belongs to category i.

[0048] is the logarithm of the predicted probability of the ith category.

[0049] Since there are significant differences in the number of samples of wireless signal strength and processor temperature data, the loss function of the present invention assigns different weights to each category, so that the model pays more attention to categories with fewer samples during training, thereby improving the model's recognition ability of these categories. In addition, by adjusting the weights, the performance degradation caused by category imbalance can be reduced. This helps to improve the overall accuracy and generalization ability of the model. In deep learning, category imbalance may lead to gradient vanishing or gradient exploding problems. Through this loss function, the contribution of different categories to the gradient can be balanced, which helps the model to be trained more stably.

[0050] In this embodiment, K in the activation function and loss function of the LSTM model is 2, where K represents the delay category caused by temperature and the delay category caused by interference, respectively.

[0051] In this embodiment, the deep learning analysis unit outputs the real-time collected relevant parameters and the corresponding delay category analysis results to the emergency processing module, and the delay generated by the software is also input into the emergency processing module, where the delay generated by the software includes encoding and decoding delay, display processing delay, and network congestion delay. When the emergency processing module receives the above-mentioned delay information, it automatically selects the corresponding emergency measures through the meta-learning algorithm. The emergency measures include adjusting the antenna gain, adjusting the processor temperature, reporting the encoding and decoding delay problem and requesting repair, switching the display device, and switching the network type.

[0052] In this embodiment, the meta-learning algorithm includes the following steps: S1: When used as the input for meta-training, the delay caused by temperature, the delay caused by interference, the encoding and decoding delay, the display processing delay, and the network congestion delay all include categories and the specific delay time generated by the corresponding categories; S2: When a single category is input, different inputs are matched to emergency measures through manual discrimination, and the meta-learning algorithm is trained and optimized so that it can automatically discern the emergency measures to be used when the information of S1 is input; S3: When multiple categories are input simultaneously, the meta-learning algorithm is extended by the Reptile algorithm; S3.1: Initialize the input and use the categories that affect the delay and their probabilities as the starting point of the meta-learning process; S3.2: Establish a task model, classify the categories that affect the delay and their probabilities again, and correspond them to the categories of emergency measures; S3.3: Train the task model using stochastic gradient descent; S4: Manually determine the accuracy and feasibility of the output results of the expanded meta-learning algorithm, and feed the erroneous data back to the meta-learning algorithm for further optimization.

[0053] Since the output results of the LSTM model are very sophisticated, it is also impossible to use only manual judgment when allocating emergency measures. Therefore, the present invention uses a meta-learning algorithm to correspond the output results of the LSTM model to the emergency measures, and through meta-training, achieves efficient and accurate emergency measure selection.

[0054] In this embodiment, when a new delay category is added, S2 to S4 of the meta-learning algorithm are repeated.

[0055] As application scenarios become more diverse, factors affecting latency will also increase. Therefore, the present invention selects a meta-learning algorithm to enable rapid learning with very few samples or iterations when faced with new tasks. In addition, in many practical applications, it is expensive or infeasible to obtain a large amount of labeled data. The meta-learning algorithm reduces the reliance on a large amount of labeled data and improves data utilization efficiency by learning from previous tasks. The learning algorithm enhances the model's transfer learning ability by learning common features and rules between tasks, enabling it to apply knowledge learned on one task to other related tasks.

[0056] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A delay monitoring system based on wireless image transmission, characterized in that: include: A time delay monitoring module and an emergency processing module, wherein the output of the time delay monitoring module is connected to the input of the emergency processing module.

2. According to claim 1, a delay monitoring system based on wireless image transmission is characterized in that: The emergency handling module includes a deep learning analysis unit and a control unit, and the delay monitoring module includes an interference monitoring unit, a hardware monitoring unit, and a delay dynamic monitoring unit. The deep learning analysis unit conducts deep learning on the pre-processed relevant information of the interference monitoring unit, the hardware monitoring unit, and the delay dynamic monitoring unit to find the cause of the unreasonable delay and give the best solution. Then the deep learning analysis unit issues solution instructions to the control unit to achieve effective emergency handling.

3. According to claim 2, a delay monitoring system based on wireless image transmission is characterized in that: The interference monitoring unit is used to monitor the wireless signal strength in real time, the hardware monitoring unit is used to monitor the processor temperature information of the image acquisition end and the image display end in real time, and the delay dynamic monitoring unit is used to monitor the comprehensive time interval from the image signal to the image display in real time.

4. The delay monitoring system based on wireless image transmission according to claim 3 is characterized in that: The deep learning analysis unit uses a long short-term memory network (LSTM) to process the information of the delay monitoring module. The specific steps are as follows: S1: Through the latency monitoring module, wireless signal strength, processor temperature, and comprehensive latency are collected in real time; S2: preprocess the data collected in S1, including normalization, denoising, and segmenting the data into training and test sets; S3: Determine the number of network layers, the number of neurons in each layer, and the activation function in the LSTM model; S4: Design a loss function to train the LSTM model so that the LSTM model outputs the categories of causes of delay and the probability of occurrence of each category; S5: Evaluate the accuracy of the LSTM model.

5. A delay monitoring system based on wireless image transmission according to claim 4, characterized in that: The number of neurons in each layer of the LSTM model in S3 is 256; The activation function uses the softmax function, the formula is as follows: in Represents the probability of belonging to category i when a given input sample z is given, where , K is the number of categories, Indicates the first Elements Perform exponential operations, Represents the i-th element in the input vector Z Perform exponential operation, j is the category index.

6. A delay monitoring system based on wireless image transmission according to claim 5, characterized in that: The loss function of the LSTM model is: Where L is the loss function; is the total number of samples; is the total number of categories; is the weight of sample z belonging to category i; is an indicator variable. If sample z belongs to category i, then ,otherwise ; is the probability that the model predicts that sample z belongs to category i; is the logarithm of the predicted probability of the ith category.

7. The delay monitoring system based on wireless image transmission according to claim 6 is characterized in that: In the activation function and loss function of the LSTM model, K is 2, where K represents the delay category caused by temperature and the delay category caused by interference, respectively.

8. The delay monitoring system based on wireless image transmission according to claim 7 is characterized in that: The deep learning analysis unit outputs the real-time collected relevant parameters and the corresponding delay category analysis results to the emergency processing module. The delay generated by the software is also input into the emergency processing module. The delay generated by the software includes encoding and decoding delay, display processing delay, and network congestion delay. When the emergency processing module receives the above delay information, it automatically selects the corresponding emergency measures through the meta-learning algorithm. The emergency measures include adjusting the antenna gain, adjusting the processor temperature, reporting the encoding and decoding delay problem and requesting repair, switching the display device, and switching the network type.

9. The delay monitoring system based on wireless image transmission according to claim 8, characterized in that: The meta-learning algorithm consists of the following steps: S1: When used as the input for meta-training, the delay caused by temperature, the delay caused by interference, the encoding and decoding delay, the display processing delay, and the network congestion delay all include categories and the specific delay time generated by the corresponding categories; S2: When a single category is input, different inputs are matched to emergency measures through manual discrimination, and the meta-learning algorithm is trained and optimized so that it can automatically discern the emergency measures to be used when the information of S1 is input; S3: When multiple categories are input at the same time, the meta-learning algorithm is expanded through the Reptile algorithm. The Reptile algorithm specifically includes the following steps: S3.1: Initialize the input and use the categories that affect the delay and their probabilities as the starting point of the meta-learning process; S3.2: Establish a task model, classify the categories that affect the delay and their probabilities again, and correspond them to the categories of emergency measures; S3.3: Train the task model using stochastic gradient descent; S4: Manually determine the accuracy and feasibility of the output results of the expanded meta-learning algorithm, and feed back the erroneous data into the meta-learning algorithm for further optimization.

10. A delay monitoring system based on wireless image transmission according to claim 9, characterized in that: When a new delay category is added, S2 to S4 of the meta-learning algorithm are repeated.

Images