A multi-modal data communication method and system using Beidou satellite system

Abstract

The present invention discloses a multimodal data communication method and system using the Beidou satellite system, and relates to the technical field of multimodal data fusion. The method uses unmanned aerial vehicles carrying sensors and high-definition cameras to collect communication data, environmental data and on-site video data in the disaster area in real time, and transmits them to the command center through the Beidou satellite system, calculates the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz, and then obtains the comprehensive transmission quality index czy, and performs data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B. Combined with the damage situation and communication needs of the disaster area, and data transmission is prioritized, so as to optimize limited communication resources and ensure stable and reliable communication in the disaster area. The method overcomes the shortcomings of traditional communication means when communication facilities in the disaster area are damaged, and improves the real-time, stability and resource utilization efficiency of communication in the disaster area.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal data fusion, and in particular to a multimodal data communication method and system using a Beidou satellite system. Background Art

[0002] In the current global context, society is facing increasingly complex challenges, especially when responding to natural disasters, public emergencies and other emergencies, the timeliness and accuracy of information transmission have become particularly critical. With the rapid development of science and technology, traditional emergency communication methods can no longer meet the needs of modern disaster management. Especially in remote areas or post-disaster environments, communication facilities are severely damaged, and the complex natural environment will cause serious interference to information transmission. At this time, relying on advanced technical systems such as satellite communications and multimodal data transmission becomes the key to solving the problem. These technologies can not only break through geographical and environmental limitations, but also ensure the efficient transmission of various types of data from real-time video to environmental monitoring data, providing timely decision-making support for disaster areas.

[0003] Current emergency communication systems often face problems such as signal interference, transmission delays, and insufficient network bandwidth. Especially in complex disaster scenarios, traditional communication methods often find it difficult to cope with diverse data needs. In disaster areas, communication facilities are often paralyzed due to geographical location, weather conditions, or damage caused by the disaster itself, affecting the efficiency of post-disaster rescue. In addition, environmental factors in disaster areas can cause significant interference to signal transmission quality. At the same time, video and audio transmission are also susceptible to factors such as compression, delay, and bandwidth, resulting in unstable transmission quality. Existing communication quality evaluation standards are difficult to fully reflect the actual status of transmission in the complex context of multimodal data, lack a reasonable division of data priorities, and result in waste of resources and low rescue efficiency. With the increase in the amount of multimodal data, how to ensure high-quality data transmission, especially in extreme environments, has become an urgent problem to be solved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a multimodal data communication method and system using the BeiDou satellite system, which solves the problems in the above-mentioned background technology.

[0005] To achieve the above purpose, the present invention is implemented by the following technical scheme: a multimodal data communication method using the Beidou satellite system, comprising the following steps: S1. Obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and drones, and transmit them to the command center system in real time; S2. Process the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; S3, according to S2, respectively analyze the impact of video data transmission, audio transmission and environmental factors on transmission, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; S4, performing summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtaining a comprehensive transmission quality index czy, and performing data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B; S5. When the data quality transmission is evaluated as stable data transmission quality, a comprehensive analysis is performed in combination with the data transmission quality, and a first priority threshold X and a second priority threshold Z are preset according to the disaster situation and communication needs to prioritize data transmission.

[0006] Preferably, said S1 includes S11 and S12; S11. After a disaster occurs, when the communication facilities in the disaster area are paralyzed, the Beidou satellite positioning system is used to locate the disaster area in real time, obtain the longitude and latitude coordinates of the disaster area and the command center, and establish a communication channel with the outside world based on the Beidou satellite communication equipment to obtain communication data; S12. After obtaining the real-time location of the disaster area, collect the environmental data and on-site video data of the disaster area in real time through a drone carrying a sensor group and a high-definition camera, compress the on-site video data and communication data according to the built-in encoder of the drone, and transmit the environmental data, on-site video data and communication data to the command center system in real time through satellite communication; The sensor group includes a temperature sensor, a humidity sensor, a pressure sensor and a spectrum analyzer.

[0007] Preferably, S2 includes S21, S22 and S23; S21, the command center system receives the environmental data, on-site video data and communication data in real time, and after data verification of the environmental data, on-site video data and communication data, decodes the video data and communication data through a decoder, decodes the video data into separate image frames, extracts the frame image containing the building, and then identifies the building area in the image through semantic segmentation technology to obtain the building frame image; S22, constructing a building damage model through a convolutional neural network, collecting a large amount of labeled building damage image data and importing it into the building damage model, iteratively training the building damage model, and then importing the acquired building frame image into the trained building damage model, automatically identifying the building damage area, identifying the damaged building, obtaining a building damage image set, extracting the building crack edges and building missing in the building damage image set through an image processing method, and obtaining the building damage degree sh; S23, by obtaining the longitude and latitude coordinates of the disaster area and the command center, and using the Haversing formula to calculate, respectively obtain the distance L between the disaster area and the satellite communication station and the distance jl between the disaster area and the command center, and then calculate the time cs required for data transmission to the satellite link in combination with the signal propagation speed, and use the recorded data transmission time to obtain the data waiting time ps; The distance L between the disaster area and the satellite communication station, the data waiting time ps, and the time cs required for data transmission to the satellite link constitute the transmission delay data group; The building damage degree sh and the distance from the disaster area to the command center jl constitute the building damage data group; The video frame rate rv and video resolution fb are obtained through the high-definition camera carried by the drone. The video frame size fv and video compression ratio cv are obtained through the size of each frame calculated by the encoder during the encoding process and the ratio before and after data compression. The video data bandwidth bw is monitored in real time through the bandwidth monitoring tool, which together form a video transmission data group. According to the acquired communication data, the audio processing tool FFmpeg is used to obtain the speech duration ts, and the communication data size before and after compression is displayed. The speech compression ratio ys is obtained by calculation to form an audio data group; The temperature sensor, humidity sensor and pressure sensor are used to collect the ambient temperature T, ambient humidity H and atmospheric pressure P respectively to form an environmental data group. The spectrum analyzer is used to collect the interference signal power gr, signal power xh and noise power zg to form a signal data group.

[0008] Preferably, said S3 includes S31, S32 and S33; S31. After dimensionless processing is performed on the acquired transmission delay data group, the data transmission delay ∆t is obtained by summarizing and calculating, which is used to analyze the quality of data transmission. The specific formula is: , where c represents the speed of light; After dimensionless processing of the acquired signal data set, the signal interference xg and signal-to-noise ratio sn are obtained by summarizing and calculating, specifically: , ; After dimensionless processing of the acquired video transmission data group, the obtained data transmission delay ∆t and signal interference xg are summarized and calculated to obtain the visual transmission efficiency index scx, and the nonlinear influence of the video data transmission quality is comprehensively analyzed; The visual transmission efficiency index scx is calculated by the following formula: ; In the formula, a represents the impact index of transmission delay on video quality; S32, after dimensionless processing is performed on the acquired audio data group, the acquired transmission delay ∆t, signal interference xg and signal-to-noise ratio sn are summarized and calculated to obtain the sound wave transmission clarity index scz, and analyze the influencing factors and nonlinear relationships of the audio data; The sound wave transmission clarity index scz is calculated and obtained by the following formula: ; In the formula, e represents the exponential function, b1 represents the nonlinear influence coefficient of transmission delay and noise, and b2 represents the influence coefficient of speech compression on quality; S33, after dimensionless processing based on the acquired environmental data group, the environmental interference index hgz is obtained by summary calculation, and the nonlinear cumulative impact of environmental factors on data transmission quality is analyzed; The environmental interference index hgz is calculated and obtained by the following formula: ; Wherein, T0, H0 and P0 represent standard ambient temperature, standard ambient humidity and standard atmospheric pressure respectively, e represents exponential function, and dt represents the differential sign in the integral.

[0009] Preferably, said S4 includes S41 and S42; S41, after dimensionless processing based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, the comprehensive transmission quality index czy is obtained by summary calculation, and the transmission quality of the data is comprehensively evaluated under the comprehensive influencing factors of multimodal data; The comprehensive transmission quality index czy is calculated by the following formula: ; Wherein, ln represents logarithmic function, sin represents sine function, e represents exponential function, h1 represents the adjustment coefficient of the interaction between visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, h2 represents the scaling coefficient for adjusting the overall comprehensive transmission quality index, Represents pi, with two decimal places.

[0010] Preferably, S42, based on the emergency communication quality standard in the field of emergency communication, and according to the minimum transmission quality requirement and the ideal transmission quality requirement, a first data transmission quality threshold A and a second data transmission quality threshold B are preset, wherein the first data transmission quality threshold A is the minimum transmission quality requirement, and the second data transmission quality threshold B is the ideal transmission quality requirement, and a data quality transmission evaluation is performed with the obtained comprehensive transmission quality index czy, and a response measure is triggered according to the evaluation result, and the specific evaluation scheme is as follows; When the comprehensive transmission quality index czy is less than the first data transmission quality threshold A, the data transmission quality is unqualified and cannot meet the data transmission requirements, and the transmission channel is rebuilt at this time; When the first data transmission quality threshold A≤comprehensive transmission quality index czy≤second data transmission quality threshold B, the data transmission quality is unstable. At this time, the command center system performs iterative optimization and evaluation according to S3 until the data transmission quality is stable. When the comprehensive transmission quality index czy>the second data transmission quality threshold B, it indicates that the data transmission quality is stable, and data priority transmission is triggered at this time.

[0011] Preferably, said S5 includes S51 and S52; S51. When the data transmission quality is evaluated as stable, dimensionless processing is performed on the acquired building damage data group, and the communication priority index tsy is obtained by summarizing and calculating, and data transmission is prioritized according to the disaster situation and communication needs, so as to optimize limited data transmission resources; The communication priority index tsy is calculated by the following formula: ; Where f1 represents the environmental impact factor, f2 represents the nonlinear impact coefficient of the severity of the disaster area, and f3 represents the nonlinear impact coefficient of distance on signal transmission.

[0012] Preferably, S52, comprehensively analyzing the data transmission quality and priority, and performing priority division, specifically includes S521 and S522; S521, after dimensionless processing based on the obtained comprehensive transmission quality index czy and communication priority index tsy, a comprehensive communication index zht is obtained by summarizing and calculating, and the communication demand is analyzed by comprehensively considering the two important factors of transmission quality and priority; The comprehensive communication index zht is calculated by the following formula: ; Wherein, β1 represents the regulating constant of the influence of the communication priority index tsy on the comprehensive communication index zht, and β2 represents the power exponent of the interaction strength between the comprehensive transmission quality index czy and the communication priority index tsy, which is used to produce a nonlinear feedback effect between the comprehensive transmission quality index czy and the communication priority index tsy.

[0013] Preferably, S522, sorting from large to small based on all historical comprehensive communication indexes zht, and presetting the first priority threshold X and the second priority threshold Z of the historical comprehensive communication indexes zht at 20% and 50% respectively by percentile method, and performing priority evaluation with the obtained comprehensive communication index zht, the specific evaluation scheme is as follows; When the comprehensive communication index zht> the first priority threshold X, it is the first priority; When the first priority threshold X≤comprehensive communication index zht≤second priority threshold Z, it is the second priority; When the comprehensive communication index zht<the second priority threshold Z, it is the third priority.

[0014] A multi-modal data communication system using the Beidou satellite system, comprising a data acquisition module, a data processing module, a transmission impact analysis module, a comprehensive transmission analysis module and a priority division module; The data acquisition module is used to obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and the UAV, and transmit it to the command center system in real time; The data processing module processes the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; The transmission impact analysis module is used to analyze the impact of video data transmission, audio transmission and environmental factors on transmission according to S2, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; The comprehensive transmission analysis module is used to perform summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtain the comprehensive transmission quality index czy, and perform data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B; The priority division module is used to perform a comprehensive analysis based on the data transmission quality when the data quality transmission assessment shows that the data transmission quality is stable, and to prioritize the data transmission by presetting a first priority threshold X and a second priority threshold Z according to the disaster situation and communication needs.

[0015] The present invention provides a multi-modal data communication method and system using the Beidou satellite system. It has the following beneficial effects: (1) This method uses drones to collect environmental data, video data, and communication data from disaster areas in real time. Combined with the positioning function of Beidou satellites, it can ensure that even if the communication facilities in the disaster area are paralyzed, key information can still be stably transmitted to the command center. The command center system comprehensively processes the received multimodal data and uses advanced image recognition, convolutional neural network and other technologies to accurately analyze the degree of building damage, environmental interference, and signal quality, thereby providing an accurate disaster assessment.

[0016] (2) This method comprehensively analyzes the transmission effect of multimodal data by calculating the visual transmission efficiency index scx, the sound wave transmission clarity index scz, and the environmental interference index hgz. The comprehensive transmission quality index czy obtained by comprehensive calculation is used to evaluate the data transmission quality with the preset first data transmission quality threshold A and the second data transmission quality threshold B. The stability of the data transmission quality can be monitored in real time to ensure the timely and accurate transmission of disaster area information. At the same time, based on the results of the data quality evaluation, optimization measures can be triggered intelligently.

[0017] (3) This method prioritizes the building damage data group, the comprehensive transmission quality index czy, and the communication priority index tsy. The calculation of the communication priority index tsy takes into account factors such as the environmental impact factor f1, the nonlinear impact coefficient of the severity of the disaster area f2, and the nonlinear impact coefficient of distance on signal transmission f3. Finally, the comprehensive communication index zht will be sorted according to the historical data, and the first priority threshold X and the second priority threshold Z will be preset. The communication demand will be allocated by evaluating the size of the comprehensive communication index zht. When the comprehensive communication index zht is greater than the second priority threshold Z, the data will be transmitted first to ensure that the emergency communication needs of the disaster area are met in a timely manner. The successful implementation of this scheme not only improves the reliability and efficiency of communication in the disaster area, but also ensures the timely transmission of key data by optimizing resource allocation, providing strong data support for post-disaster reconstruction and emergency response. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the steps of a multi-modal data communication method using the Beidou satellite system of the present invention; Figure 2 A schematic diagram of a multi-modal data communication system using the Beidou satellite system according to the present invention; Figure 3 It is a schematic diagram of signal interference and signal-to-noise ratio changing with frequency according to the present invention; Figure 4 A schematic diagram of multimodal data transmission quality assessment spots according to the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example 1 See also Figure 1 The present invention provides a multi-modal data communication method using the Beidou satellite system. To achieve the above purpose, the present invention is implemented by the following technical scheme: comprising the following steps: S1. Obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and drones, and transmit them to the command center system in real time; S2. Process the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; S3, according to S2, respectively analyze the impact of video data transmission, audio transmission and environmental factors on transmission, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; S4, performing summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtaining a comprehensive transmission quality index czy, and performing data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B; S5. When the data quality transmission is evaluated as stable data transmission quality, a comprehensive analysis is performed in combination with the data transmission quality, and a first priority threshold X and a second priority threshold Z are preset according to the disaster situation and communication needs to prioritize data transmission.

[0021] In this embodiment, by integrating the Beidou positioning system and drone technology, it is ensured that communication data, environmental data and on-site video data can be obtained in real time after a disaster occurs, and these key data are transmitted to the command center in real time. In the case of paralysis of communication facilities in the disaster area, satellite communication is relied on to maintain smooth information transmission, thereby providing the command center with accurate and timely disaster information. This efficient data transmission mechanism enhances the timeliness of disaster emergency response and can significantly improve the monitoring and evaluation capabilities of disaster site conditions. The command center system calculates the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz by comprehensively analyzing multidimensional data such as transmission delay, building damage, video and audio transmission quality, and environmental interference, and further calculates the comprehensive transmission quality index czy, and performs data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B. Compared with the existing single data transmission method, this process can coordinate and optimize between multimodal data to avoid the decline in data transmission quality caused by a single factor. By dynamically adjusting the data transmission priority, the most urgent communication needs in the disaster area are effectively guaranteed, thereby improving the stability and efficiency of emergency communications. Compared with existing technologies, this method's innovations in data quality assessment and priority management can comprehensively consider environmental factors, building damage conditions, and real-time communication needs, greatly improving the accuracy and flexibility of data transmission. Especially in disaster emergency scenarios, the transmission channel or transmission priority can be adjusted in real time according to data quality to ensure that critical data is transmitted first. Through this mechanism, it is possible to respond quickly and ensure the reliability of information transmission in complex and changing disaster environments. Ultimately, it not only greatly improves communication efficiency in disaster emergency response, but also provides a scalable and intelligent solution for future emergency communications, effectively supporting post-disaster recovery and reconstruction.

[0022] Example 2 This embodiment is explained in Example 1, please refer to Figure 1 , specifically: the S1 includes S11 and S12; S11. After a disaster occurs, when the communication facilities in the disaster area are paralyzed, the Beidou satellite positioning system is used to locate the disaster area in real time, obtain the longitude and latitude coordinates of the disaster area and the command center, and establish a communication channel with the outside world based on the Beidou satellite communication equipment to obtain communication data; S12. After obtaining the real-time location of the disaster area, collect the environmental data and on-site video data of the disaster area in real time through a drone carrying a sensor group and a high-definition camera, compress the on-site video data and communication data according to the built-in encoder of the drone, and transmit the environmental data, on-site video data and communication data to the command center system in real time through satellite communication; The sensor group includes a temperature sensor, a humidity sensor, a pressure sensor and a spectrum analyzer.

[0023] In this embodiment, the disaster area is located in real time through the Beidou satellite positioning system, and a stable communication channel is established with the outside world through satellite communication equipment to ensure that the communication between the disaster area and the command center is not interrupted, overcoming the communication barriers caused by the damage of traditional communication facilities. The environmental data and on-site video data of the disaster area are collected in real time by drones equipped with sensor groups and high-definition cameras, and the data is compressed in combination with the built-in encoder, and then the data is transmitted to the command center in real time through satellite communication, further enhancing the ability to obtain real-time information in the disaster area. This innovative method not only ensures the timely transmission of information from the disaster area, but also provides more comprehensive and accurate support for disaster assessment through multi-dimensional data collection, greatly improving the efficiency and accuracy of post-disaster emergency response, and ensuring that rescue operations can make quick decisions based on real-time data.

[0024] Example 3 This embodiment is explained in Example 2. Please refer to Figure 1 Specifically: S2 includes S21, S22 and S23; S21, the command center system receives the environmental data, on-site video data and communication data in real time, and after data verification of the environmental data, on-site video data and communication data, decodes the video data and communication data through a decoder, decodes the video data into separate image frames, extracts the frame image containing the building, and then identifies the building area in the image through semantic segmentation technology to obtain the building frame image; S22, constructing a building damage model through a convolutional neural network, collecting a large amount of labeled building damage image data and importing it into the building damage model, iteratively training the building damage model, and then importing the acquired building frame image into the trained building damage model, automatically identifying the building damage area, identifying the damaged building, obtaining a building damage image set, extracting the building crack edges and building missing in the building damage image set through an image processing method, and obtaining the building damage degree sh; S23, by obtaining the longitude and latitude coordinates of the disaster area and the command center, and using the Haversing formula to calculate, respectively obtain the distance L between the disaster area and the satellite communication station and the distance jl between the disaster area and the command center, and then calculate the time cs required for data transmission to the satellite link in combination with the signal propagation speed, and use the recorded data transmission time to obtain the data waiting time ps; The distance L between the disaster area and the satellite communication station, the data waiting time ps, and the time cs required for data transmission to the satellite link constitute the transmission delay data group; The building damage degree sh and the distance from the disaster area to the command center jl constitute the building damage data group; The video frame rate rv and video resolution fb are obtained through the high-definition camera carried by the drone. The video frame size fv and video compression ratio cv are obtained through the size of each frame calculated by the encoder during the encoding process and the ratio before and after data compression. The video data bandwidth bw is monitored in real time through the bandwidth monitoring tool, which together form a video transmission data group. According to the acquired communication data, the audio processing tool FFmpeg is used to obtain the speech duration ts, and the communication data size before and after compression is displayed. The speech compression ratio ys is obtained by calculation to form an audio data group; The temperature sensor, humidity sensor and pressure sensor are used to collect the ambient temperature T, ambient humidity H and atmospheric pressure P respectively to form an environmental data group. The spectrum analyzer is used to collect the interference signal power gr, signal power xh and noise power zg to form a signal data group.

[0025] In this embodiment, multimodal data from the disaster area can be processed comprehensively and accurately, and data verification and analysis can be effectively performed. The command center system automatically identifies and evaluates the degree of building damage sh by decoding video and communication data, combining semantic segmentation technology and convolutional neural networks, ensuring efficient and accurate identification of building damage in the disaster area. This technology not only improves the speed of building damage assessment in the disaster area, but also reduces manual intervention, significantly improving the accuracy and consistency of the assessment results. At the same time, by accurately calculating the distance L between the disaster area and the satellite communication station, the time cs required for data transmission to the satellite link, and the data waiting time ps, combined with real-time bandwidth monitoring, the optimization of the video data transmission bandwidth bw, the video frame size fv, and the video compression ratio cv is ensured, reducing transmission delays and improving the stability of data transmission. In addition, the signal data such as speech duration ts and speech compression ratio ys obtained by the audio processing tool FFmpeg, ambient temperature T, humidity H, atmospheric pressure P collected by environmental sensors, and interference signal power gr, signal power xh and noise power zg collected by the spectrum analyzer provide the command center with comprehensive and real-time information on the disaster area, enabling command decisions to be based on more comprehensive and accurate real-time data, thereby greatly improving the efficiency and decision-making quality of disaster emergency response.

[0026] Example 4 This embodiment is explained in Example 3, please refer to Figure 1 , specifically: S3 includes S31, S32 and S33; S31. After dimensionless processing is performed on the acquired transmission delay data group, the data transmission delay ∆t is obtained by summarizing and calculating, which is used to analyze the quality of data transmission. The specific formula is: , where c represents the speed of light; After dimensionless processing of the acquired signal data set, the signal interference xg and signal-to-noise ratio sn are obtained by summarizing and calculating, specifically: , ; After dimensionless processing of the acquired video transmission data group, the obtained data transmission delay ∆t and signal interference xg are summarized and calculated to obtain the visual transmission efficiency index scx, and the nonlinear influence of the video data transmission quality is comprehensively analyzed; The visual transmission efficiency index scx is calculated by the following formula: ; In the formula, a represents the impact index of transmission delay on video quality; S32, after dimensionless processing is performed on the acquired audio data group, the acquired transmission delay ∆t, signal interference xg and signal-to-noise ratio sn are summarized and calculated to obtain the sound wave transmission clarity index scz, and analyze the influencing factors and nonlinear relationships of the audio data; The sound wave transmission clarity index scz is calculated and obtained by the following formula: ; In the formula, e represents the exponential function, b1 represents the nonlinear influence coefficient of transmission delay and noise, and b2 represents the influence coefficient of speech compression on quality; S33, after dimensionless processing based on the acquired environmental data group, the environmental interference index hgz is obtained by summary calculation, and the nonlinear cumulative impact of environmental factors on data transmission quality is analyzed; The environmental interference index hgz is calculated and obtained by the following formula: ; Wherein, T0, H0 and P0 represent standard ambient temperature, standard ambient humidity and standard atmospheric pressure respectively, e represents exponential function, and dt represents the differential sign in the integral.

[0027] In this embodiment, the transmission efficiency of multimodal data can be accurately evaluated by dimensionless processing and comprehensive analysis of data transmission delay, signal interference, signal-to-noise ratio, video data and audio data. In particular, the calculation of the visual transmission efficiency index scx and the sound wave transmission clarity index scz can fully consider the nonlinear effects of multiple factors such as transmission delay, signal interference and audio data compression, thereby providing an accurate quantitative evaluation of the quality of video and audio data. The environmental interference index hgz further improves the predictive ability of data transmission quality under disaster environments by analyzing the nonlinear cumulative effects of environmental factors such as temperature, humidity and air pressure on transmission quality. Compared with the traditional single transmission quality monitoring method, this comprehensive and refined evaluation method not only improves the accuracy of data transmission, but also can intelligently adjust the transmission strategy in a complex environment to ensure that key information can be stably transmitted, greatly enhancing the reliability and timeliness of post-disaster emergency communications.

[0028] Example 5 This embodiment is explained in Example 4. Please refer to Figure 1 and Figure 4 , specifically: the S4 includes S41 and S42; S41, after dimensionless processing based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, the comprehensive transmission quality index czy is obtained by summary calculation, and the transmission quality of the data is comprehensively evaluated under the comprehensive influencing factors of multimodal data; The comprehensive transmission quality index czy is calculated by the following formula: ; Wherein, ln represents logarithmic function, sin represents sine function, e represents exponential function, h1 represents the adjustment coefficient of the interaction between visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, h2 represents the scaling coefficient for adjusting the overall comprehensive transmission quality index, Represents pi, with two decimal places.

[0029] S42. Based on the emergency communication quality standard in the field of emergency communication, and according to the minimum transmission quality requirement and the ideal transmission quality requirement, a first data transmission quality threshold A and a second data transmission quality threshold B are preset, wherein the first data transmission quality threshold A is the minimum transmission quality requirement, and the second data transmission quality threshold B is the ideal transmission quality requirement, and a data quality transmission evaluation is performed with the obtained comprehensive transmission quality index czy, and response measures are triggered according to the evaluation results. The specific evaluation scheme is as follows; When the comprehensive transmission quality index czy is less than the first data transmission quality threshold A, the data transmission quality is unqualified and cannot meet the data transmission requirements, and the transmission channel is rebuilt at this time; When the first data transmission quality threshold A≤comprehensive transmission quality index czy≤second data transmission quality threshold B, the data transmission quality is unstable. At this time, the command center system performs iterative optimization and evaluation according to S3 until the data transmission quality is stable. When the comprehensive transmission quality index czy>the second data transmission quality threshold B, it indicates that the data transmission quality is stable, and data priority transmission is triggered at this time.

[0030] In this embodiment, the comprehensive transmission quality index czy is obtained by comprehensively calculating the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz, and the data quality transmission evaluation is performed with the preset first data transmission quality threshold A and the second data transmission quality threshold B, so that the overall quality of data transmission can be accurately evaluated. This multi-dimensional evaluation system significantly improves the refined management of data transmission, and can take corresponding optimization measures according to different data quality states. When the transmission quality is unqualified, the transmission channel can be actively adjusted; when the data quality is unstable, the command center can perform iterative optimization until the transmission quality is stable; and when the transmission quality reaches the ideal standard, data priority transmission is triggered to ensure that key communication needs are met first. The scatter plot shows the relationship between the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz on the comprehensive transmission quality index czy. Each point in the scatter plot represents a data point, and its x-axis and y-axis correspond to the visual transmission efficiency index scx and the sound wave transmission clarity index scz, respectively. The color of the point reflects the comprehensive transmission quality index czy. The points in the figure show different colors according to the different values ​​of their positions on the x-axis and y-axis. The lighter color represents a higher comprehensive transmission quality index czy, while the darker color represents a lower comprehensive transmission quality index czy. It not only shows the relationship between the two factors of the visual transmission efficiency index scx and the sound wave transmission clarity index scz, but also provides an overall evaluation of the transmission quality through color changes, which is easy to observe and analyze. This process ensures that the communication in disaster emergency response is always kept in the best state, improves the reliability and flexibility of communication, greatly reduces the risks caused by unstable data transmission, makes emergency communication more efficient and intelligent, and can respond quickly in complex environments. The introduction of this dynamic adjustment mechanism not only improves the system's resilience, but also effectively supports the rational allocation of emergency communication resources, providing strong technical support for post-disaster rescue and reconstruction work.

[0031] Example 6 This embodiment is explained in Example 5. Please refer to Figure 1 , specifically: the S5 includes S51 and S52; S51. When the data transmission quality is evaluated as stable, dimensionless processing is performed on the acquired building damage data group, and the communication priority index tsy is obtained by summarizing and calculating, and data transmission is prioritized according to the disaster situation and communication needs, so as to optimize limited data transmission resources; The communication priority index tsy is calculated by the following formula: ; Where f1 represents the environmental impact factor, f2 represents the nonlinear impact coefficient of the severity of the disaster area, and f3 represents the nonlinear impact coefficient of distance on signal transmission.

[0032] S52, comprehensively analyzing the data transmission quality and priority, and performing priority division, specifically including S521 and S522; S521, after dimensionless processing based on the obtained comprehensive transmission quality index czy and communication priority index tsy, a comprehensive communication index zht is obtained by summarizing and calculating, and the communication demand is analyzed by comprehensively considering the two important factors of transmission quality and priority; The comprehensive communication index zht is calculated by the following formula: ; Wherein, β1 represents the regulating constant of the influence of the communication priority index tsy on the comprehensive communication index zht, and β2 represents the power exponent of the interaction strength between the comprehensive transmission quality index czy and the communication priority index tsy, which is used to produce a nonlinear feedback effect between the comprehensive transmission quality index czy and the communication priority index tsy.

[0033] S522, sorting all historical comprehensive communication indexes zht from large to small, and using the percentile method to preset the first priority threshold X and the second priority threshold Z for the historical comprehensive communication indexes zht at 20% and 50%, respectively, and performing priority evaluation with the obtained comprehensive communication index zht, the specific evaluation scheme is as follows; When the comprehensive communication index zht> the first priority threshold X, it is the first priority; When the first priority threshold X≤comprehensive communication index zht≤second priority threshold Z, it is the second priority; When the comprehensive communication index zht<the second priority threshold Z, it is the third priority.

[0034] In this embodiment, by calculating the communication priority index tsy, the data transmission can be dynamically prioritized according to the environmental impact factor f1, the nonlinear impact coefficient f2 of the severity of the disaster area, and the nonlinear impact coefficient f3 of the distance on the signal transmission, effectively optimizing the limited communication resources. This measure ensures that the most urgent communication needs in the disaster area can be met first, avoiding waste of resources and delayed transmission of key data. Furthermore, the comprehensive communication index zht combines the two key factors of transmission quality and priority to ensure that the communication needs under different disaster situations are handled scientifically and reasonably. The first priority threshold X and the second priority threshold Z are preset by sorting historical data and the percentile method, and priority evaluation is performed. The communication priority can be flexibly adjusted, and the changes can be responded to in real time according to the comprehensive communication index zht, thereby improving the adaptability of the communication network in the disaster environment and the emergency response efficiency. This innovative method maximizes communication efficiency under limited resources and ensures the stability and timeliness of communication during post-disaster reconstruction and emergency response.

[0035] Example 7 Please refer to Figure 2 , a multimodal data communication system using the BeiDou satellite system, comprising a data acquisition module, a data processing module, a transmission impact analysis module, a comprehensive transmission analysis module and a priority division module; The data acquisition module is used to obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and the UAV, and transmit it to the command center system in real time; The data processing module processes the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; The transmission impact analysis module is used to analyze the impact of video data transmission, audio transmission and environmental factors on transmission according to S2, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; The comprehensive transmission analysis module is used to perform summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtain the comprehensive transmission quality index czy, and perform data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B; The priority division module is used to perform a comprehensive analysis based on the data transmission quality when the data quality transmission assessment shows that the data transmission quality is stable, and to prioritize the data transmission by presetting a first priority threshold X and a second priority threshold Z according to the disaster situation and communication needs.

[0036] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-modal data communication method using the Beidou satellite system, characterized in that: The following steps are involved: S1. Obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and drones, and transmit them to the command center system in real time; S2. Process the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; S3, according to S2, respectively analyze the impact of video data transmission, audio transmission and environmental factors on transmission, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; S4, performing summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtaining a comprehensive transmission quality index czy, and performing data quality transmission evaluation with a preset first data transmission quality threshold A and a second data transmission quality threshold B; S5. When the data quality transmission is evaluated as stable data transmission quality, a comprehensive analysis is performed in combination with the data transmission quality, and a first priority threshold X and a second priority threshold Z are preset according to the disaster situation and communication needs to prioritize data transmission.

2. A multimodal data communication method using the BeiDou satellite system according to claim 1, characterized in that: Said S1 includes S11 and S12; S11. After a disaster occurs, when the communication facilities in the disaster area are paralyzed, the Beidou satellite positioning system is used to locate the disaster area in real time, obtain the longitude and latitude coordinates of the disaster area and the command center, and establish a communication channel with the outside world based on the Beidou satellite communication equipment to obtain communication data; S12. After obtaining the real-time location of the disaster area, collect the environmental data and on-site video data of the disaster area in real time through a drone carrying a sensor group and a high-definition camera, compress the on-site video data and communication data according to the built-in encoder of the drone, and transmit the environmental data, on-site video data and communication data to the command center system in real time through satellite communication; The sensor group includes a temperature sensor, a humidity sensor, a pressure sensor and a spectrum analyzer.

3. A multi-modal data communication method using the BeiDou satellite system according to claim 2, characterized in that: The S2 includes S21, S22 and S23; S21, the command center system receives the environmental data, on-site video data and communication data in real time, and after data verification of the environmental data, on-site video data and communication data, decodes the video data and communication data through a decoder, decodes the video data into separate image frames, extracts the frame image containing the building, and then identifies the building area in the image through semantic segmentation technology to obtain the building frame image; S22, constructing a building damage model through a convolutional neural network, collecting a large amount of labeled building damage image data and importing it into the building damage model, iteratively training the building damage model, and then importing the acquired building frame image into the trained building damage model, automatically identifying the building damage area, identifying the damaged building, obtaining a building damage image set, extracting the building crack edges and building missing in the building damage image set through an image processing method, and obtaining the building damage degree sh; S23, by obtaining the longitude and latitude coordinates of the disaster area and the command center, and using the Haversing formula to calculate, respectively obtain the distance L between the disaster area and the satellite communication station and the distance jl between the disaster area and the command center, and then calculate the time cs required for data transmission to the satellite link in combination with the signal propagation speed, and use the recorded data transmission time to obtain the data waiting time ps; The distance L between the disaster area and the satellite communication station, the data waiting time ps, and the time cs required for data transmission to the satellite link constitute the transmission delay data group; The building damage degree sh and the distance from the disaster area to the command center jl constitute the building damage data group; The video frame rate rv and video resolution fb are obtained through the high-definition camera carried by the drone. The video frame size fv and video compression ratio cv are obtained through the size of each frame calculated by the encoder during the encoding process and the ratio before and after data compression. The video data bandwidth bw is monitored in real time through the bandwidth monitoring tool, which together form a video transmission data group. According to the acquired communication data, the audio processing tool FFmpeg is used to obtain the speech duration ts, and the communication data size before and after compression is displayed. The speech compression ratio ys is obtained by calculation to form an audio data group; The temperature sensor, humidity sensor and pressure sensor are used to collect the ambient temperature T, ambient humidity H and atmospheric pressure P respectively to form an environmental data group. The spectrum analyzer is used to collect the interference signal power gr, signal power xh and noise power zg to form a signal data group.

4. The multimodal data communication method using the BeiDou satellite system according to claim 3, characterized in that: The S3 includes S31, S32 and S33; S31. After dimensionless processing is performed on the acquired transmission delay data group, the data transmission delay ∆t is obtained by summarizing and calculating, which is used to analyze the quality of data transmission. The specific formula is: , where c represents the speed of light; After dimensionless processing of the acquired signal data set, the signal interference xg and signal-to-noise ratio sn are obtained by summarizing and calculating, specifically: , ; After dimensionless processing of the acquired video transmission data group, the obtained data transmission delay ∆t and signal interference xg are summarized and calculated to obtain the visual transmission efficiency index scx, and the nonlinear influence of the video data transmission quality is comprehensively analyzed; The visual transmission efficiency index scx is calculated by the following formula: ； In the formula, a represents the impact index of transmission delay on video quality; S32, after dimensionless processing is performed on the acquired audio data group, the acquired transmission delay ∆t, signal interference xg and signal-to-noise ratio sn are summarized and calculated to obtain the sound wave transmission clarity index scz, and analyze the influencing factors and nonlinear relationships of the audio data; The sound wave transmission clarity index scz is calculated and obtained by the following formula: ； In the formula, e represents the exponential function, b1 represents the nonlinear influence coefficient of transmission delay and noise, and b2 represents the influence coefficient of speech compression on quality; S33, after dimensionless processing based on the acquired environmental data group, the environmental interference index hgz is obtained by summary calculation, and the nonlinear cumulative impact of environmental factors on data transmission quality is analyzed; The environmental interference index hgz is calculated and obtained by the following formula: ； Wherein, T0, H0 and P0 represent standard ambient temperature, standard ambient humidity and standard atmospheric pressure respectively, e represents exponential function, and dt represents the differential sign in the integral.

5. The multimodal data communication method using the BeiDou satellite system according to claim 4, characterized in that: The S4 includes S41 and S42; S41, after dimensionless processing based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, the comprehensive transmission quality index czy is obtained by summary calculation, and the transmission quality of the data is comprehensively evaluated under the comprehensive influencing factors of multimodal data; The comprehensive transmission quality index czy is calculated by the following formula: ； Wherein, ln represents logarithmic function, sin represents sine function, e represents exponential function, h1 represents the adjustment coefficient of the interaction between visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, h2 represents the scaling coefficient for adjusting the overall comprehensive transmission quality index, Represents pi, with two decimal places.

6. The multimodal data communication method using the BeiDou satellite system according to claim 5, characterized in that: S42. Based on the emergency communication quality standard in the field of emergency communication, and according to the minimum transmission quality requirement and the ideal transmission quality requirement, a first data transmission quality threshold A and a second data transmission quality threshold B are preset, wherein the first data transmission quality threshold A is the minimum transmission quality requirement, and the second data transmission quality threshold B is the ideal transmission quality requirement, and a data quality transmission evaluation is performed with the obtained comprehensive transmission quality index czy, and response measures are triggered according to the evaluation results. The specific evaluation scheme is as follows; When the comprehensive transmission quality index czy is less than the first data transmission quality threshold A, the data transmission quality is unqualified and cannot meet the data transmission requirements, and the transmission channel is rebuilt at this time; When the first data transmission quality threshold A≤comprehensive transmission quality index czy≤second data transmission quality threshold B, the data transmission quality is unstable. At this time, the command center system performs iterative optimization and evaluation according to S3 until the data transmission quality is stable. When the comprehensive transmission quality index czy>the second data transmission quality threshold B, it indicates that the data transmission quality is stable, and data priority transmission is triggered at this time.

7. A multi-modal data communication method using the BeiDou satellite system according to claim 6, characterized in that: The S5 includes S51 and S52; S51. When the data transmission quality is evaluated as stable, dimensionless processing is performed on the acquired building damage data group, and the communication priority index tsy is obtained by summarizing and calculating, and data transmission is prioritized according to the disaster situation and communication needs, so as to optimize limited data transmission resources; The communication priority index tsy is calculated by the following formula: ； Where f1 represents the environmental impact factor, f2 represents the nonlinear impact coefficient of the severity of the disaster area, and f3 represents the nonlinear impact coefficient of distance on signal transmission.

8. The multi-modal data communication method using the BeiDou satellite system according to claim 7, characterized in that: S52, comprehensively analyzing the data transmission quality and priority, and performing priority division, specifically including S521 and S522; S521, after dimensionless processing based on the obtained comprehensive transmission quality index czy and communication priority index tsy, a comprehensive communication index zht is obtained by summarizing and calculating, and the communication demand is analyzed by comprehensively considering the two important factors of transmission quality and priority; The comprehensive communication index zht is calculated by the following formula: ； Wherein, β1 represents the regulating constant of the influence of the communication priority index tsy on the comprehensive communication index zht, and β2 represents the power exponent of the interaction strength between the comprehensive transmission quality index czy and the communication priority index tsy, which is used to produce a nonlinear feedback effect between the comprehensive transmission quality index czy and the communication priority index tsy.

9. The multi-modal data communication method using the BeiDou satellite system according to claim 8, characterized in that: S522, sorting all historical comprehensive communication indexes zht from large to small, and using the percentile method to preset the first priority threshold X and the second priority threshold Z for the historical comprehensive communication indexes zht at 20% and 50%, respectively, and performing priority evaluation with the obtained comprehensive communication index zht, the specific evaluation scheme is as follows; When the comprehensive communication index zht> the first priority threshold X, it is the first priority; When the first priority threshold X≤comprehensive communication index zht≤second priority threshold Z, it is the second priority; When the comprehensive communication index zht<the second priority threshold Z, it is the third priority.

10. A multimodal data communication system using the BeiDou satellite system, comprising a multimodal data communication method using the BeiDou satellite system according to any one of claims 1 to 9, characterized in that: It includes data acquisition module, data processing module, transmission impact analysis module, comprehensive transmission analysis module and priority division module; The data acquisition module is used to obtain communication data, environmental data and on-site video data in real time based on the Beidou positioning system and the UAV, and transmit it to the command center system in real time; The data processing module processes the communication data, environmental data and on-site video data through the command center system to obtain a transmission delay data group, a building damage data group, a video transmission data group, an audio data group, an environmental data group and a signal data group respectively; The transmission impact analysis module is used to analyze the impact of video data transmission, audio transmission and environmental factors on transmission according to S2, and obtain the visual transmission efficiency index scx, the sound wave transmission clarity index scz and the environmental interference index hgz; The comprehensive transmission analysis module is used to perform summary calculation based on the obtained visual transmission efficiency index scx, sound wave transmission clarity index scz and environmental interference index hgz, obtain the comprehensive transmission quality index czy, and perform data quality transmission evaluation with the preset first data transmission quality threshold A and the second data transmission quality threshold B; The priority division module is used to perform a comprehensive analysis based on the data transmission quality when the data quality transmission assessment shows that the data transmission quality is stable, and to prioritize the data transmission by presetting a first priority threshold X and a second priority threshold Z according to the disaster situation and communication needs.

Images