A multi-modal data communication method and system applying the Beidou satellite system

Abstract

The present invention discloses a multi-modal data communication method and system applying the Beidou satellite system, which relates to the technical field of multi-modal data fusion. This method uses an unmanned aerial vehicle to carry sensors and high-definition cameras to collect communication data, environmental data, and on-site video data of the disaster area in real time, and transmits them to the command center through the Beidou satellite system. The visual transmission efficiency index scx, the acoustic wave transmission clarity index scz, and the environmental interference index hgz are calculated, and then the comprehensive transmission quality index czy is obtained, and data quality transmission evaluation is carried out with the preset first data transmission quality threshold A and the second data transmission quality threshold B. Combining the damage situation and communication requirements in the disaster area, and carrying out priority division for data transmission, so as to optimize limited communication resources and ensure stable and reliable communication in the disaster area. This method overcomes the deficiencies of traditional communication means when communication facilities in the disaster area are damaged, and improves the real-time performance, stability, and resource utilization efficiency of communication in the disaster area.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal data fusion, and particularly to a multimodal data communication method and system applying the Beidou satellite system. Background Art

[0002] In the current context of globalization, society faces increasingly complex challenges. Especially when dealing with natural disasters, public emergencies and other emergencies, the timeliness and accuracy of information transmission become particularly crucial. With the rapid development of technology, traditional emergency communication methods are difficult to meet the needs of modern disaster management. Especially in remote areas or post-disaster environments, communication facilities are severely damaged, and the complex natural environment will seriously interfere with information transmission. At this time, relying on advanced technical systems such as satellite communication and multimodal data transmission becomes the key to solving the problem. These technologies can not only break through geographical and environmental limitations, but also ensure the efficient transmission of various data from real-time videos to environmental monitoring data, providing timely decision-making support for disaster areas.

[0003] Current emergency communication systems usually face problems such as signal interference, transmission delay and insufficient network bandwidth. Especially in complex disaster scenarios, traditional communication means are often difficult to cope with diverse data requirements. In disaster areas, due to geographical location, weather conditions or the destruction of the disaster itself, communication facilities are often paralyzed, affecting the efficiency of post-disaster rescue. In addition, environmental factors in disaster areas will cause non-negligible interference to the signal transmission quality. At the same time, video and audio transmissions are also easily affected by factors such as compression, delay and bandwidth, resulting in unstable transmission quality. Existing communication quality assessment standards are difficult to comprehensively reflect the actual situation of transmission in the complex context of multimodal data, lack a reasonable division of data priorities, resulting in resource waste and low rescue efficiency. With the increase in the amount of multimodal data, how to ensure high-quality data transmission, especially in extreme environments, has become an urgent problem to be solved. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a multimodal data communication method and system applying the Beidou satellite system, which solves the problems in the above background art.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A multimodal data communication method applying the Beidou satellite system includes the following steps:

[0006] S1. Obtain communication data, environmental data and on-site video data in real time according to the Beidou positioning system and the unmanned aerial vehicle, and transmit them to the command center system in real time;

[0007] S2. Process the communication data, environmental data, and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group, and a signal data group respectively;

[0008] S3. Analyze the impact of video data transmission, audio transmission, and environmental factors on transmission respectively according to S2, and obtain a visual transmission efficiency index scx, a sound wave transmission clarity index scz, and an environmental interference index hgz;

[0009] S4. Perform summary calculations based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz, and environmental interference index hgz to obtain a comprehensive transmission quality index czy, and conduct data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B;

[0010] S5. When the data quality transmission evaluation shows that the data transmission quality is stable, conduct comprehensive analysis in combination with the data transmission quality, and perform priority division on the data transmission according to the disaster situation and communication requirements with a preset first priority threshold X and a second priority threshold Z.

[0011] Preferably, S1 includes S11 and S12;

[0012] S11. After a disaster occurs and the communication facilities in the disaster area are paralyzed, use the Beidou satellite positioning system to perform real-time positioning on the location of the disaster area, obtain the longitude and latitude coordinates of the disaster area and the command center, and establish a communication channel with the outside world through Beidou satellite communication equipment to obtain communication data;

[0013] S12. After obtaining the real-time positioning of the disaster area, use a drone equipped with a sensor group and a high-definition camera to collect the environmental data and on-site video data of the disaster area in real time, and compress the on-site video data and communication data according to the encoder built in the drone, and transmit the environmental data, on-site video data, and communication data to the command center system in real time through satellite communication;

[0014] The sensor group includes a temperature sensor, a humidity sensor, a pressure sensor, and a spectrum analyzer.

[0015] Preferably, S2 includes S21, S22, and S23;

[0016] S21. The command center system receives the environmental data, on-site video data, and communication data in real time, conducts data verification on the environmental data, on-site video data, and communication data, decodes the video data and communication data through a decoder, decodes the video data into individual image frames, extracts the frame pictures containing buildings, and then uses semantic segmentation technology to identify the building areas in the images to obtain building frame images;

[0017] S22. Build a building damage model through a convolutional neural network. Collect a large amount of labeled building damage image data and import it into the building damage model. Iteratively train the building damage model, and then import the obtained building frame images into the trained building damage model to automatically identify the damaged areas of the building, identify the damaged buildings, obtain a set of building damage images. Extract the edges of building cracks and building deficiencies in the set of building damage images through image processing methods to obtain the building damage degree sh;

[0018] S23. Obtain the latitude and longitude coordinates of the disaster area and the command center, and use the Haversine formula for calculation to obtain the distance L between the disaster area and the satellite communication station and the distance jl from the disaster area to the command center respectively. Then, combine the signal propagation speed to calculate the time cs required for data transmission to the satellite link, and use the recorded data transmission time to obtain the data waiting time ps;

[0019] The distance L between the disaster area and the satellite communication station, the data waiting time ps, and the time cs required for data transmission to the satellite link form a transmission delay data group;

[0020] The building damage degree sh and the distance jl from the disaster area to the command center form a building damage data group;

[0021] Obtain the video frame rate rv and video resolution fb through the high-definition camera carried by the drone. Obtain the video frame size fv and video compression ratio cv through the size of each frame calculated during the encoding process by the encoder and the ratio before and after data compression. Real-time monitor the video data bandwidth bw through a bandwidth monitoring tool, which together form a video transmission data group;

[0022] Based on the obtained communication data, use the audio processing tool FFmpeg to obtain the voice duration ts, and display the communication data size before and after compression. Calculate the voice compression ratio ys to form an audio data group;

[0023] Temperature sensors, humidity sensors, and pressure sensors are respectively used to collect the ambient temperature T, ambient humidity H, and atmospheric pressure P to form an environmental data group. A spectrum analyzer is used to collect the interference signal power gr, signal power xh, and noise power zg to form a signal data group.

[0024] Preferably, the said S3 includes S31, S32, and S33;

[0025] S31. After performing dimensionless processing on the obtained transmission delay data group, summarize and calculate to obtain the data transmission delay ∆t, which is used to analyze the quality of data transmission. The specific formula is , where c represents the speed of light;

[0026] After dimensionless processing of the obtained signal data group, the signal interference xg and the signal-to-noise ratio sn are calculated by summarization. Specifically, , ;

[0027] After dimensionless processing of the obtained video transmission data group, it is calculated by summarization with the obtained data transmission delay ∆t and signal interference xg to obtain the visual transmission efficiency index scx, and the non-linear influence of video data transmission quality is comprehensively analyzed;

[0028] The visual transmission efficiency index scx is obtained by calculation with the following formula;

[0029] ;

[0030] In the formula, a represents the influence index of transmission delay on video quality;

[0031] S32. After dimensionless processing of the obtained audio data group, it is calculated by summarization with the obtained transmission delay ∆t, signal interference xg and signal-to-noise ratio sn to obtain the acoustic wave transmission clarity index scz, and the influencing factors and non-linear relationships of audio data are analyzed;

[0032] The acoustic wave transmission clarity index scz is obtained by calculation with the following formula;

[0033] ;

[0034] In the formula, e represents the exponential function, b1 represents the non-linear influence coefficient of transmission delay and noise, and b2 represents the influence coefficient of voice compression on quality;

[0035] S33. After dimensionless processing of the obtained environmental data group, the environmental interference index hgz is calculated by summarization to analyze the non-linear cumulative influence of environmental factors on data transmission quality;

[0036] The environmental interference index hgz is obtained by calculation with the following formula;

[0037] ;

[0038] In the formula, T 0 , H 0 and P 0 represent the standard environmental temperature, standard environmental humidity and standard atmospheric pressure respectively, e represents the exponential function, and dt represents the differential symbol in the integral formula.

[0039] Preferably, the S4 includes S41 and S42;

[0040] S41. After performing dimensionless processing on the obtained visual transmission efficiency index scx, acoustic wave transmission clarity index scz, and environmental interference index hgz, summarize and calculate to obtain the comprehensive transmission quality index czy, and analyze and comprehensively evaluate the data transmission quality under the comprehensive influencing factors of multi-modal data;

[0041] The comprehensive transmission quality index czy is obtained through the following formula;

[0042] ;

[0043] In the formula, ln represents the logarithmic function, sin represents the sine function, e represents the exponential function, h1 represents the adjustment coefficient of the interaction between the visual transmission efficiency index scx, acoustic wave transmission clarity index scz, and environmental interference index hgz, and h2 represents the scaling coefficient that adjusts the entire comprehensive transmission quality index, represents the pi, with two decimal places after the decimal point.

[0044] Preferably, S42. Based on the emergency communication quality standard in the field of emergency communication, preset the first data transmission quality threshold A and the second data transmission quality threshold B according to the minimum transmission quality requirement and the ideal transmission quality requirement, where the first data transmission quality threshold A is the minimum transmission quality requirement, the second data transmission quality threshold B is the ideal transmission quality requirement, and perform data quality transmission evaluation with the obtained comprehensive transmission quality index czy, and trigger response measures according to the evaluation results. The specific evaluation scheme is as follows;

[0045] When the comprehensive transmission quality index czy < the first data transmission quality threshold A, the data transmission quality is unqualified and cannot meet the data transmission requirements. At this time, reconstruct the transmission channel;

[0046] When the first data transmission quality threshold A ≤ the comprehensive transmission quality index czy ≤ the second data transmission quality threshold B, the data transmission quality is unstable. At this time, the command center system performs iterative optimization and evaluation according to S3 until the data transmission quality is stable;

[0047] When the comprehensive transmission quality index czy > the second data transmission quality threshold B, it means that the data transmission quality is stable. At this time, trigger the priority transmission of data.

[0048] Preferably, the S5 includes S51 and S52;

[0049] S51. When the data quality transmission evaluation shows that the data transmission quality is stable, after performing dimensionless processing on the obtained building damage data group, summarize and calculate to obtain the communication priority index tsy, and divide the data transmission priority according to the disaster situation and communication requirements to optimize the limited data transmission resources;

[0050] The communication priority index tsy is calculated and obtained through the following formula;

[0051] ;

[0052] In the formula, f1 represents the environmental impact factor, f2 represents the non-linear impact coefficient of the severity of the disaster area, and f3 represents the non-linear impact coefficient of distance on signal transmission.

[0053] Preferably, in S52, a comprehensive analysis of the data transmission quality and priority is performed, and priority division is carried out, specifically including S521 and S522;

[0054] In S521, after dimensionless processing of the obtained comprehensive transmission quality index czy and communication priority index tsy, the comprehensive communication index zht is calculated and obtained by summarization. By comprehensively considering two important factors of transmission quality and priority, the communication requirements are analyzed;

[0055] The comprehensive communication index zht is calculated and obtained through the following formula;

[0056] ;

[0057] In the formula, β1 represents the influence adjustment constant of the communication priority index tsy on the comprehensive communication index zht, and β2 represents the power index of the interaction strength between the comprehensive transmission quality index czy and the communication priority index tsy, which is used to generate a non-linear feedback effect between the comprehensive transmission quality index czy and the communication priority index tsy.

[0058] Preferably, in S522, based on all historical comprehensive communication indices zht, they are sorted from large to small, and the historical comprehensive communication indices zht at the 20% and 50% positions are respectively set as the preset first priority threshold X and the second priority threshold Z through the percentile method, and a priority evaluation is carried out with the obtained comprehensive communication index zht. The specific evaluation scheme is as follows;

[0059] When the comprehensive communication index zht > the first priority threshold X, it is the first priority;

[0060] When the first priority threshold X ≤ the comprehensive communication index zht ≤ the second priority threshold Z, it is the second priority;

[0061] When the comprehensive communication index zht < the second priority threshold Z, it is the third priority.

[0062] A multi-modal data communication system applying the Beidou satellite system includes a data acquisition module, a data processing module, a transmission impact analysis module, a comprehensive transmission analysis module, and a priority division module;

[0063] The data acquisition module is used to obtain communication data, environmental data, and on-site video data in real time based on the Beidou positioning system and the unmanned aerial vehicle (UAV), and transmit them to the command center system in real time;

[0064] The data processing module processes the communication data, environmental data, and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group, and a signal data group respectively;

[0065] The transmission impact analysis module is used to analyze the impact of video data transmission, audio transmission, and environmental factors on transmission respectively based on the data processing module, and obtain a visual transmission efficiency index scx, a sound wave transmission clarity index scz, and an environmental interference index hgz;

[0066] The comprehensive transmission analysis module is used to perform summary calculations based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz, and environmental interference index hgz to obtain a comprehensive transmission quality index czy, and perform data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B;

[0067] The priority division module is used to perform comprehensive analysis in combination with the data transmission quality when the data quality transmission evaluation shows stable data transmission quality, and divide the priority of data transmission according to the disaster situation and communication requirements with a preset first priority threshold X and a second priority threshold Z.

[0068] The present invention provides a multi-modal data communication method and system applying the Beidou satellite system. It has the following beneficial effects:

[0069] (1) This method can ensure that even in the case of communication facility paralysis in the disaster area, key information can still be stably transmitted to the command center by using the UAV to collect environmental data, video data, and communication data in the disaster area in real time and combining the positioning function of the Beidou satellite. The command center system comprehensively processes the received multi-modal data, and uses advanced technologies such as image recognition and convolutional neural network to accurately analyze the building damage degree, environmental interference, and signal quality, so as to provide an accurate disaster situation assessment.

[0070] (2) This method comprehensively analyzes the transmission effect of multi-modal data through the calculation of the visual transmission efficiency index scx, the sound wave transmission clarity index scz, and the environmental interference index hgz. By calculating the comprehensive transmission quality index czy obtained through comprehensive calculation and performing data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B, the stability of data transmission quality can be monitored in real time to ensure the timely and accurate transmission of disaster area information. At the same time, according to the results of data quality evaluation, optimization measures can be intelligently triggered.

[0071] (3)This method conducts priority division based on the building damage data set, the comprehensive transmission quality index czy, and the communication priority index tsy. The calculation of the communication priority index tsy takes into account factors such as the environmental impact factor f1, the non-linear impact coefficient f2 of the disaster area severity, and the non-linear impact coefficient f3 of the distance on signal transmission. Finally, the comprehensive communication index zht is sorted according to historical data, presetting the first priority threshold X and the second priority threshold Z, and allocating communication requirements by evaluating the magnitude of the comprehensive communication index zht. When the comprehensive communication index zht is greater than the second priority threshold Z, the data will be preferentially transmitted to ensure that the emergency communication requirements in the disaster area are promptly met. The successful implementation of this solution not only improves the reliability and efficiency of communication in the disaster area but also ensures the timely transmission of critical data through optimized resource allocation, providing strong data support for post-disaster reconstruction and emergency response. Description of the Drawings

[0072] Figure 1 Schematic diagram of the steps of a multi-modal data communication method applying the Beidou satellite system according to the present invention;

[0073] Figure 2 Schematic diagram of the process of a multi-modal data communication system applying the Beidou satellite system according to the present invention;

[0074] Figure 3 Schematic diagram of signal interference and signal-to-noise ratio varying with frequency according to the present invention;

[0075] Figure 4 Schematic diagram of the spots for evaluating the quality of multi-modal data transmission according to the present invention. Detailed Embodiments

[0076] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0077] Embodiment 1

[0078] Please refer to Figure 1 , the present invention provides a multi-modal data communication method applying the Beidou satellite system. To achieve the above objectives, the present invention is implemented through the following technical solutions: including the following steps:

[0079] S1. Obtain communication data, environmental data, and on-site video data in real time based on the Beidou positioning system and the unmanned aerial vehicle, and transmit them to the command center system in real time;

[0080] S2. Process the communication data, environmental data, and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group, and a signal data group respectively;

[0081] S3. Analyze the influence of video data transmission, audio transmission, and environmental factors on transmission respectively according to S2, and obtain a visual transmission efficiency index scx, a sound wave transmission clarity index scz, and an environmental interference index hgz;

[0082] S4. Conduct a summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz, and environmental interference index hgz to obtain a comprehensive transmission quality index czy, and conduct a data quality transmission assessment with a preset first data transmission quality threshold A and a second data transmission quality threshold B;

[0083] S5. When the data quality transmission assessment shows that the data transmission quality is stable, conduct a comprehensive analysis in combination with the data transmission quality, and divide the data transmission priorities according to the disaster situation and communication requirements with a preset first priority threshold X and a second priority threshold Z.

[0084] In this embodiment, by integrating the Beidou positioning system and UAV technology, it is ensured that communication data, environmental data, and on-site video data can be obtained in real time after a disaster occurs, and these key data are transmitted to the command center in real time. In the case of the collapse of communication facilities in the disaster area, satellite communication is relied on to maintain smooth information transmission, so as to provide accurate and timely disaster information for the command center. This efficient data transmission mechanism enhances the timeliness of disaster emergency response and can significantly improve the monitoring and assessment capabilities of the disaster scene. The command center system calculates the visual transmission efficiency index scx, the acoustic wave transmission clarity index scz, and the environmental interference index hgz by comprehensively analyzing multi-dimensional data such as transmission delay, building damage, video and audio transmission quality, and environmental interference, and further calculates the comprehensive transmission quality index czy, and conducts data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B. This process can coordinate and optimize among multi-modal data compared with the existing single data transmission method, avoiding the decline of data transmission quality caused by a single factor. By dynamically adjusting the data transmission priority, the most urgent communication needs in the disaster area are effectively guaranteed, thereby improving the stability and efficiency of emergency communication. Compared with the existing technology, the innovation of this method in data quality assessment and priority management can comprehensively consider environmental factors, building damage conditions, and real-time communication needs, greatly improving the accuracy and flexibility of data transmission. Especially in the disaster emergency scenario, it can adjust the transmission channel or transmission priority in real time according to the data quality to ensure the priority transmission of key data. Through this mechanism, it can quickly respond and ensure the reliability of information transmission in a complex and changeable disaster environment. Finally, it not only greatly improves the communication efficiency in disaster emergency response, but also provides an expandable and intelligent solution for future emergency communication, effectively supporting the post-disaster recovery and reconstruction work.

[0085] Embodiment 2

[0086] This embodiment is an explanatory description based on Embodiment 1. Please refer to Figure 1 , specifically: S1 includes S11 and S12;

[0087] S11. After a disaster occurs and the communication facilities in the disaster area are paralyzed, the Beidou satellite positioning system is used to perform real-time positioning on the location of the disaster area, obtain the longitude and latitude coordinates of the disaster area and the command center, and establish a communication channel with the outside world based on the Beidou satellite communication equipment to obtain communication data;

[0088] S12. After obtaining the real-time positioning of the disaster area, a UAV equipped with a sensor group and a high-definition camera is used to collect environmental data and on-site video data of the disaster area in real time, and the on-site video data and communication data are compressed based on the encoder built in the UAV, and the environmental data, on-site video data, and communication data are transmitted to the command center system in real time through satellite communication;

[0089] The sensor group includes a temperature sensor, a humidity sensor, a pressure sensor, and a spectrum analyzer.

[0090] In this embodiment, the disaster area is located in real time through the Beidou satellite positioning system, and a stable communication channel is established with the outside world through satellite communication equipment to ensure that the communication between the disaster area and the command center is not interrupted, overcoming the communication obstacles caused by the damage of traditional communication facilities. An unmanned aerial vehicle equipped with a sensor group and a high-definition camera is used to collect environmental data and on-site video data of the disaster area in real time, the data is compressed by an internal encoder, and then these data are transmitted to the command center in real time through satellite communication, further enhancing the ability to obtain real-time information in the disaster area. This innovative method not only ensures the timely transmission of disaster area information, but also provides more comprehensive and accurate support for disaster assessment through multi-dimensional data collection, greatly improving the efficiency and accuracy of post-disaster emergency response, and ensuring that rescue operations can make rapid decisions based on real-time data.

[0091] Embodiment 3

[0092] This embodiment is an explanatory description based on Embodiment 2. Please refer to Figure 1 , specifically: S2 includes S21, S22, and S23;

[0093] S21. The command center system receives environmental data, on-site video data, and communication data in real time. After data verification of the environmental data, on-site video data, and communication data, the video data and communication data are decoded through a decoder. The video data is decoded into individual image frames, the frame pictures containing buildings are extracted, and then the building areas in the images are identified through semantic segmentation technology to obtain building frame images;

[0094] S22. A building damage model is constructed through a convolutional neural network. A large number of labeled building damage image data are collected and imported into the building damage model for iterative training of the building damage model. Then the obtained building frame images are imported into the trained building damage model to automatically identify the building damage areas, identify the damaged buildings, obtain a set of building damage images, and extract the building crack edges and building deficiencies in the set of building damage images through image processing methods to obtain the building damage degree sh;

[0095] S23. By obtaining the longitude and latitude coordinates of the disaster area and the command center, and using the Haversine formula for calculation, the distances L between the disaster area and the satellite communication station and jl from the disaster area to the command center are respectively obtained. Then, combined with the signal propagation speed, the time cs required for data to be transmitted to the satellite link is calculated, and the data waiting time ps is obtained by recording the data transmission time;

[0096] The transmission delay data group consists of the distance L between the disaster area and the satellite communication station, the data waiting time ps, and the time cs required for data to be transmitted to the satellite link;

[0097] The building damage data group consists of the building damage degree sh and the distance jl from the disaster area to the command center;

[0098] The video frame rate rv and the video resolution fb are obtained through a high-definition camera carried by a drone. The video frame size fv and the video compression ratio cv are obtained by calculating the size of each frame during the encoding process by an encoder and the ratio before and after data compression. The video data bandwidth bw is monitored in real time through a bandwidth monitoring tool, jointly forming the video transmission data group;

[0099] Based on the obtained communication data, the audio processing tool FFmpeg is used to obtain the voice duration ts, and the communication data sizes before and after compression are displayed. The voice compression ratio ys is obtained through calculation, forming the audio data group;

[0100] The temperature sensor, humidity sensor, and pressure sensor are respectively used to collect the environmental temperature T, environmental humidity H, and atmospheric pressure P, forming the environmental data group. The spectrum analyzer is used to collect the interference signal power gr, signal power xh, and noise power zg, forming the signal data group.

[0101] In this embodiment, multi-modal data from the disaster area can be comprehensively and accurately processed, and data verification and analysis can be effectively carried out. The command center system decodes the video and communication data, combines semantic segmentation technology and convolutional neural networks, and automatically identifies and evaluates the building damage degree sh, ensuring the efficient and accurate identification of damaged buildings in the disaster area. This technology not only improves the speed of building damage assessment in the disaster area but also reduces manual intervention, significantly enhancing the accuracy and consistency of the assessment results. At the same time, by accurately calculating the distance L between the disaster area and the satellite communication station, the time cs required for data to be transmitted to the satellite link, and the data waiting time ps, combined with real-time bandwidth monitoring, the optimization of the video data transmission bandwidth bw, video frame size fv, and video compression ratio cv is ensured, reducing transmission delay and improving the stability of data transmission. In addition, the voice duration ts and voice compression ratio ys obtained through the audio processing tool FFmpeg, the environmental temperature T, humidity H, atmospheric pressure P collected by environmental sensors, and the signal data such as the interference signal power gr, signal power xh, and noise power zg collected by the spectrum analyzer provide a comprehensive and real-time situation of the disaster area for the command center, enabling command decisions to be based on more comprehensive and accurate real-time data, thus greatly improving the efficiency and decision-making quality of disaster emergency response.

[0102] Embodiment 4

[0103] This embodiment is an explanatory description based on Embodiment 3. Please refer to Figure 1, specifically: S3 includes S31, S32, and S33;

[0104] After the obtained transmission delay data group is dimensionless processed, the data transmission delay ∆t is obtained by summary calculation for analyzing the quality of data transmission. The specific formula is , where c represents the speed of light;

[0105] After the obtained signal data group is dimensionless processed, the signal interference xg and the signal-to-noise ratio sn are respectively obtained by summary calculation, specifically , ;

[0106] After the obtained video transmission data group is dimensionless processed, it is summarized and calculated with the obtained data transmission delay ∆t and signal interference xg to obtain the visual transmission efficiency index scx, and the non-linear influence of video data transmission quality is comprehensively analyzed;

[0107] The visual transmission efficiency index scx is obtained by calculating through the following formula;

[0108] ;

[0109] , where a represents the influence index of transmission delay on video quality;

[0110] After the obtained audio data group is dimensionless processed, it is summarized and calculated with the obtained transmission delay ∆t, signal interference xg, and signal-to-noise ratio sn to obtain the sound wave transmission clarity index scz, and the influencing factors and non-linear relationships of audio data are analyzed;

[0111] The sound wave transmission clarity index scz is obtained by calculating through the following formula;

[0112] ;

[0113] , where e represents the exponential function, b1 represents the non-linear influence coefficient of transmission delay and noise, and b2 represents the influence coefficient of voice compression on quality;

[0114] After the obtained environmental data group is dimensionless processed, the environmental interference index hgz is obtained by summary calculation to analyze the non-linear cumulative influence of environmental factors on data transmission quality;

[0115] The environmental interference index hgz is obtained by calculating through the following formula;

[0116] ;

[0117] , where T 0 , H 0 , and P 0respectively represent the standard environmental temperature, standard environmental humidity, and standard atmospheric pressure, e represents the exponential function, and dt represents the differential symbol in the integral formula.

[0118] In this embodiment, through the dimensionless processing and comprehensive analysis of data transmission delay, signal interference, signal-to-noise ratio, video data, and audio data, the transmission efficiency of multimodal data can be accurately evaluated. In particular, the calculation of the visual transmission efficiency index scx and the acoustic wave transmission clarity index scz can comprehensively consider the non-linear effects of multiple factors such as transmission delay, signal interference, and audio data compression, thereby providing an accurate quantitative evaluation of the quality of video and audio data. The environmental interference index hgz further enhances the prediction ability of data transmission quality in disaster environments by analyzing the non-linear cumulative effects of environmental factors such as temperature, humidity, and air pressure on transmission quality. This comprehensive and refined evaluation method not only improves the accuracy of data transmission compared to traditional single transmission quality monitoring methods but also can intelligently adjust transmission strategies in complex environments to ensure the stable transmission of key information, greatly enhancing the reliability and timeliness of post-disaster emergency communication.

[0119] Embodiment 5

[0120] This embodiment is an explanatory description based on Embodiment 4. Please refer to Figure 1 and Figure 4 , specifically: S4 includes S41 and S42;

[0121] S41. After dimensionless processing of the obtained visual transmission efficiency index scx, acoustic wave transmission clarity index scz, and environmental interference index hgz, summarize and calculate to obtain the comprehensive transmission quality index czy, and analyze the transmission quality of data under the comprehensive influencing factors of multimodal data;

[0122] The comprehensive transmission quality index czy is calculated through the following formula;

[0123] ;

[0124] In the formula, ln represents the logarithmic function, sin represents the sine function, e represents the exponential function, h1 represents the adjustment coefficient of the interaction between the visual transmission efficiency index scx, acoustic wave transmission clarity index scz, and environmental interference index hgz, h2 represents the scaling coefficient for adjusting the entire comprehensive transmission quality index, represents pi, with two decimal places in the value.

[0125] S42. Based on the emergency communication quality standard in the field of emergency communication, preset the first data transmission quality threshold A and the second data transmission quality threshold B according to the minimum transmission quality requirement and the ideal transmission quality requirement. Among them, the first data transmission quality threshold A is the minimum transmission quality requirement, and the second data transmission quality threshold B is the ideal transmission quality requirement. Then, conduct data quality transmission evaluation with the obtained comprehensive transmission quality index czy, and trigger response measures according to the evaluation results. The specific evaluation scheme is as follows;

[0126] When the comprehensive transmission quality index czy < the first data transmission quality threshold A, the data transmission quality is unqualified and cannot meet the data transmission requirements. At this time, reconstruct the transmission channel;

[0127] When the first data transmission quality threshold A ≤ the comprehensive transmission quality index czy ≤ the second data transmission quality threshold B, the data transmission quality is unstable. At this time, the command center system performs iterative optimization and evaluation according to S3 until the data transmission quality is stable;

[0128] When the comprehensive transmission quality index czy > the second data transmission quality threshold B, it means that the data transmission quality is stable. At this time, trigger the data priority transmission.

[0129] In this embodiment, the comprehensive transmission quality index czy is obtained by comprehensively calculating the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz, and the data quality transmission evaluation is performed with the preset first data transmission quality threshold A and the second data transmission quality threshold B, so that the overall quality of data transmission can be accurately evaluated. This multi-dimensional evaluation system significantly improves the refined management of data transmission, and can take corresponding optimization measures according to different data quality states. When the transmission quality is unqualified, the transmission channel can be actively adjusted; when the data quality is unstable, the command center can perform iterative optimization until the transmission quality is stable; and when the transmission quality reaches the ideal standard, data priority transmission is triggered to ensure that key communication needs are met first. The scatter plot shows the relationship between the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz on the comprehensive transmission quality index czy. Each point in the scatter plot represents a data point, and its x-axis and y-axis correspond to the visual transmission efficiency index scx and the sound wave transmission clarity index scz, respectively. The color of the point reflects the comprehensive transmission quality index czy. The points in the figure show different colors according to the different values ​​of their positions on the x-axis and y-axis. The lighter color represents a higher comprehensive transmission quality index czy, while the darker color represents a lower comprehensive transmission quality index czy. It not only shows the relationship between the two factors of the visual transmission efficiency index scx and the sound wave transmission clarity index scz, but also provides an overall evaluation of the transmission quality through color changes, which is easy to observe and analyze. This process ensures that the communication in disaster emergency response is always kept in the best state, improves the reliability and flexibility of communication, greatly reduces the risks caused by unstable data transmission, makes emergency communication more efficient and intelligent, and can respond quickly in complex environments. The introduction of this dynamic adjustment mechanism not only improves the system's resilience, but also effectively supports the rational allocation of emergency communication resources, providing strong technical support for post-disaster rescue and reconstruction work.

[0130] Example 6

[0131] This embodiment is explained in Example 5. Please refer to Figure 1 , specifically: the S5 includes S51 and S52;

[0132] S51. When the data transmission quality is evaluated as stable, dimensionless processing is performed on the acquired building damage data group, and the communication priority index tsy is obtained by summarizing and calculating, and data transmission is prioritized according to the disaster situation and communication needs, so as to optimize limited data transmission resources;

[0133] The communication priority index tsy is calculated by the following formula:

[0134] ;

[0135] In the formula, f1 represents the environmental impact factor, f2 represents the non-linear impact coefficient of the severity of the disaster area, and f3 represents the non-linear impact coefficient of distance on signal transmission.

[0136] S52. Comprehensively analyze the data transmission quality and priority, and conduct priority division, specifically including S521 and S522;

[0137] S521. After dimensionless processing based on the obtained comprehensive transmission quality index czy and communication priority index tsy, summarize and calculate to obtain the comprehensive communication index zht, and analyze the communication requirements by comprehensively considering two important factors of transmission quality and priority;

[0138] The comprehensive communication index zht is calculated through the following formula;

[0139] ;

[0140] In the formula, β1 represents the influence adjustment constant of the communication priority index tsy on the comprehensive communication index zht, and β2 represents the power exponent of the interaction strength between the comprehensive transmission quality index czy and the communication priority index tsy, which is used to generate a non-linear feedback effect between the comprehensive transmission quality index czy and the communication priority index tsy.

[0141] S522. Sort all historical comprehensive communication indexes zht from largest to smallest, and respectively set the preset first priority threshold X and second priority threshold Z for the historical comprehensive communication indexes zht at the 20% and 50% positions through the percentile method, and conduct priority evaluation with the obtained comprehensive communication index zht. The specific evaluation scheme is as follows;

[0142] When the comprehensive communication index zht > the first priority threshold X, it is the first priority;

[0143] When the first priority threshold X ≤ the comprehensive communication index zht ≤ the second priority threshold Z, it is the second priority;

[0144] When the comprehensive communication index zht < the second priority threshold Z, it is the third priority.

[0145] In this embodiment, by calculating the communication priority index tsy, dynamic priority division of data transmission can be carried out according to the environmental impact factor f1, the non-linear impact coefficient f2 of the severity of the disaster area, and the non-linear impact coefficient f3 of distance on signal transmission, effectively optimizing the limited communication resources. This measure ensures that the most urgent communication needs in the disaster area can be preferentially met, avoiding resource waste and delayed transmission of critical data. Further, the comprehensive communication index zht combines two key factors, transmission quality and priority, ensuring that communication needs under different disaster conditions are scientifically and reasonably handled. By presetting the first priority threshold X and the second priority threshold Z through sorting historical data and the percentile method, and conducting priority evaluation, the communication priority can be flexibly adjusted and respond to changes in real time according to the comprehensive communication index zht, thereby improving the adaptability and emergency response efficiency of the communication network in the disaster environment. This innovative method maximizes communication efficiency under limited resources, ensuring communication stability and timeliness during post-disaster reconstruction and emergency response.

[0146] Embodiment 7

[0147] Please refer to Figure 2 , a multi-modal data communication system applying the Beidou satellite system, including a data acquisition module, a data processing module, a transmission impact analysis module, a comprehensive transmission analysis module, and a priority division module;

[0148] The data acquisition module is used to obtain communication data, environmental data, and on-site video data in real time based on the Beidou positioning system and unmanned aerial vehicles, and transmit them to the command center system in real time;

[0149] The data processing module processes the communication data, environmental data, and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group, and a signal data group respectively;

[0150] The transmission impact analysis module is used to analyze the impact of video data transmission, audio transmission, and environmental factors on transmission respectively based on the data processing module, and obtain a visual transmission efficiency index scx, an acoustic wave transmission clarity index scz, and an environmental interference index hgz;

[0151] The comprehensive transmission analysis module is used to perform summary calculations based on the obtained visual transmission efficiency index scx, acoustic wave transmission clarity index scz, and environmental interference index hgz to obtain a comprehensive transmission quality index czy, and conduct data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B;

[0152] The priority division module is used to perform comprehensive analysis in combination with the data transmission quality when the data quality transmission assessment shows that the data transmission quality is stable, and perform priority division on the data transmission according to the preset first priority threshold X and the second priority threshold Z based on the disaster situation and communication requirements.

[0153] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-modal data communication method using the Beidou satellite system, characterized in that: The following steps are involved: S1. Obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and drones, and transmit them to the command center system in real time; S2. Process the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; The video frame rate rv and video resolution fb are obtained through the high-definition camera carried by the drone. The video frame size fv and video compression ratio cv are obtained through the size of each frame calculated by the encoder during the encoding process and the ratio before and after data compression. The video data bandwidth bw is monitored in real time through the bandwidth monitoring tool, which together form a video transmission data group. According to the acquired communication data, the audio processing tool FFmpeg is used to obtain the speech duration ts, and the communication data size before and after compression is displayed. The speech compression ratio ys is obtained by calculation to form an audio data group; The temperature sensor, humidity sensor and pressure sensor are used to collect the ambient temperature T, ambient humidity H and atmospheric pressure P respectively, forming an environmental data group; S3, according to S2, respectively analyze the impact of video data transmission, audio transmission and environmental factors on transmission, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; ； ； ； Where a represents the influence index of transmission delay on video quality, b1 represents the nonlinear influence coefficient of transmission delay and noise, b2 represents the influence coefficient of speech compression on quality, T0, H0 and P0 represent standard ambient temperature, standard ambient humidity and standard atmospheric pressure respectively, e represents exponential function, dt represents the differential symbol in the integral, ∆t represents data transmission delay, xg represents signal interference, and sn represents signal-to-noise ratio; S4, performing summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtaining a comprehensive transmission quality index czy, and performing data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B; S5. When the data quality transmission is evaluated as stable data transmission quality, a comprehensive analysis is performed in combination with the data transmission quality, and a first priority threshold X and a second priority threshold Z are preset according to the disaster situation and communication needs to prioritize data transmission.

2. A multimodal data communication method using the BeiDou satellite system according to claim 1, characterized in that: Said S1 includes S11 and S12; S11. After a disaster occurs, when the communication facilities in the disaster area are paralyzed, the Beidou satellite positioning system is used to locate the disaster area in real time, obtain the longitude and latitude coordinates of the disaster area and the command center, and establish a communication channel with the outside world based on the Beidou satellite communication equipment to obtain communication data; S12. After obtaining the real-time location of the disaster area, collect the environmental data and on-site video data of the disaster area in real time through a drone carrying a sensor group and a high-definition camera, compress the on-site video data and communication data according to the built-in encoder of the drone, and transmit the environmental data, on-site video data and communication data to the command center system in real time through satellite communication; The sensor group includes a temperature sensor, a humidity sensor, a pressure sensor and a spectrum analyzer.

3. A multi-modal data communication method using the BeiDou satellite system according to claim 2, characterized in that: The S2 includes S21, S22 and S23; S21, the command center system receives the environmental data, on-site video data and communication data in real time, and after data verification of the environmental data, on-site video data and communication data, decodes the video data and communication data through a decoder, decodes the video data into separate image frames, extracts the frame image containing the building, and then identifies the building area in the image through semantic segmentation technology to obtain the building frame image; S22, constructing a building damage model through a convolutional neural network, collecting a large amount of labeled building damage image data and importing it into the building damage model, iteratively training the building damage model, and then importing the acquired building frame image into the trained building damage model, automatically identifying the building damage area, identifying the damaged building, obtaining a building damage image set, extracting the building crack edges and building missing in the building damage image set through an image processing method, and obtaining the building damage degree sh; S23, by obtaining the longitude and latitude coordinates of the disaster area and the command center, and using the Haversing formula to calculate, respectively obtain the distance L between the disaster area and the satellite communication station and the distance jl between the disaster area and the command center, and then calculate the time cs required for data transmission to the satellite link in combination with the signal propagation speed, and use the recorded data transmission time to obtain the data waiting time ps; The distance L between the disaster area and the satellite communication station, the data waiting time ps, and the time cs required for data transmission to the satellite link constitute the transmission delay data group; The building damage degree sh and the distance from the disaster area to the command center jl constitute the building damage data group; The spectrum analyzer is used to collect interference signal power gr, signal power xh and noise power zg to form a signal data group.

4. The multimodal data communication method using the BeiDou satellite system according to claim 3, characterized in that: The S3 includes S31, S32 and S33; S31. After dimensionless processing is performed on the acquired transmission delay data group, the data transmission delay ∆t is obtained by summarizing and calculating, which is used to analyze the quality of data transmission. The specific formula is: , where c represents the speed of light; After dimensionless processing of the acquired signal data set, the signal interference xg and signal-to-noise ratio sn are obtained by summarizing and calculating, specifically: , ; After dimensionless processing of the acquired video transmission data group, the obtained data transmission delay ∆t and signal interference xg are summarized and calculated to obtain the visual transmission efficiency index scx, and the nonlinear influence of the video data transmission quality is comprehensively analyzed; S32, after dimensionless processing is performed on the acquired audio data group, the acquired transmission delay ∆t, signal interference xg and signal-to-noise ratio sn are summarized and calculated to obtain the sound wave transmission clarity index scz, and analyze the influencing factors and nonlinear relationships of the audio data; S33. After dimensionless processing is performed on the acquired environmental data group, the environmental interference index hgz is obtained by summary calculation, and the nonlinear cumulative impact of environmental factors on data transmission quality is analyzed.

5. The multimodal data communication method using the BeiDou satellite system according to claim 4, characterized in that: The S4 includes S41 and S42; S41, after dimensionless processing based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, the comprehensive transmission quality index czy is obtained by summary calculation, and the transmission quality of the data is comprehensively evaluated under the comprehensive influencing factors of multimodal data; The comprehensive transmission quality index czy is calculated by the following formula: ； Where ln represents the logarithmic function, sin represents the sine function, h1 represents the adjustment coefficient of the interaction between the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz, and h2 represents the scaling factor for adjusting the entire comprehensive transmission quality index. Represents pi, with two decimal places.

6. The multimodal data communication method using the BeiDou satellite system according to claim 5, characterized in that: S42. Based on the emergency communication quality standard in the field of emergency communication, and according to the minimum transmission quality requirement and the ideal transmission quality requirement, a first data transmission quality threshold A and a second data transmission quality threshold B are preset, wherein the first data transmission quality threshold A is the minimum transmission quality requirement, and the second data transmission quality threshold B is the ideal transmission quality requirement, and a data quality transmission evaluation is performed with the obtained comprehensive transmission quality index czy, and response measures are triggered according to the evaluation results. The specific evaluation scheme is as follows; When the comprehensive transmission quality index czy is less than the first data transmission quality threshold A, the data transmission quality is unqualified and cannot meet the data transmission requirements, and the transmission channel is rebuilt at this time; When the first data transmission quality threshold A≤comprehensive transmission quality index czy≤second data transmission quality threshold B, the data transmission quality is unstable. At this time, the command center system performs iterative optimization and evaluation according to S3 until the data transmission quality is stable. When the comprehensive transmission quality index czy>the second data transmission quality threshold B, it indicates that the data transmission quality is stable, and data priority transmission is triggered at this time.

7. The multi-modal data communication method using the BeiDou satellite system according to claim 6, characterized in that: The S5 includes S51 and S52; S51. When the data transmission quality is evaluated as stable, dimensionless processing is performed on the acquired building damage data group, and the communication priority index tsy is obtained by summarizing and calculating, and data transmission is prioritized according to the disaster situation and communication needs, so as to optimize limited data transmission resources; The communication priority index tsy is calculated by the following formula: ； Where f1 represents the environmental impact factor, f2 represents the nonlinear impact coefficient of the severity of the disaster area, and f3 represents the nonlinear impact coefficient of distance on signal transmission.

8. The multi-modal data communication method using the BeiDou satellite system according to claim 7, characterized in that: S52, comprehensively analyzing the data transmission quality and priority, and performing priority division, specifically including S521 and S522; S521, after dimensionless processing based on the obtained comprehensive transmission quality index czy and communication priority index tsy, a comprehensive communication index zht is obtained by summarizing and calculating, and the communication demand is analyzed by comprehensively considering the two important factors of transmission quality and priority; The comprehensive communication index zht is calculated by the following formula: ； Wherein, β1 represents the regulating constant of the influence of the communication priority index tsy on the comprehensive communication index zht, and β2 represents the power exponent of the interaction strength between the comprehensive transmission quality index czy and the communication priority index tsy, which is used to produce a nonlinear feedback effect between the comprehensive transmission quality index czy and the communication priority index tsy.

9. The multi-modal data communication method using the BeiDou satellite system according to claim 8, characterized in that: S522, sorting all historical comprehensive communication indexes zht from large to small, and using the percentile method to preset the first priority threshold X and the second priority threshold Z for the historical comprehensive communication indexes zht at 20% and 50%, respectively, and performing priority evaluation with the obtained comprehensive communication index zht, the specific evaluation scheme is as follows; When the comprehensive communication index zht> the first priority threshold X, it is the first priority; When the first priority threshold X≤comprehensive communication index zht≤second priority threshold Z, it is the second priority; When the comprehensive communication index zht<the second priority threshold Z, it is the third priority.

10. A multimodal data communication system using the BeiDou satellite system, comprising a multimodal data communication method using the BeiDou satellite system according to any one of claims 1 to 9, characterized in that: It includes data acquisition module, data processing module, transmission impact analysis module, comprehensive transmission analysis module and priority division module; The data acquisition module is used to obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and the UAV, and transmit it to the command center system in real time; The data processing module processes the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; The transmission impact analysis module is used to analyze the impact of video data transmission, audio transmission and environmental factors on transmission according to the data processing module, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; The comprehensive transmission analysis module is used to perform summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtain the comprehensive transmission quality index czy, and perform data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B; The priority division module is used to perform a comprehensive analysis based on the data transmission quality when the data quality transmission assessment shows that the data transmission quality is stable, and to prioritize the data transmission by presetting a first priority threshold X and a second priority threshold Z according to the disaster situation and communication needs.

Images