A wireless networking optimization method and system based on multimodal data analysis
By adopting the wireless network optimization method of multimodal data analysis at the construction site, and using technologies such as self-supervised learning and dynamic convolutional neural networks, the problem of differentiation of multimodal data fusion is solved, efficient and accurate data fusion and decision support are achieved, and wireless network optimization capabilities at the construction site are improved.
Patent Information
- Application Number
- CN202411890756.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-04-01
AI Technical Summary
There is a fusion difference between multimodal data at the construction site, resulting in poor data fusion effect and low decision-making accuracy.
The wireless network optimization method based on multimodal data analysis is adopted, and multi-level linkage optimization and differentiated fusion are carried out through self-supervised learning, dynamic convolutional neural network, dynamic adaptive comparison loss function, edge computing and task offloading, modal feature weighting strategies and reinforcement learning algorithms to achieve multi-stage linkage optimization and differentiated fusion.
It improves the understanding and alignment capabilities of multimodal data, enhances the overall reliability and accuracy of data fusion, improves the adaptability and computing efficiency of sensor networks, and solves the problems of dynamic allocation of spectrum resources and signal interference.
Smart Images
Figure CN119743777A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent management technology, and in particular to a wireless networking optimization method and system based on multimodal data analysis. Background Art
[0002] With the development of emerging technologies such as the Internet of Things, artificial intelligence, and edge computing, intelligent management of construction sites has become a key component of the current Industry 4.0 construction. The construction environment is complex and changeable, and various sensors, cameras, smart terminals and other equipment deployed on site collect a large amount of multimodal data (such as video, audio, sensor data, etc.) through wireless networks to achieve comprehensive monitoring and analysis of construction progress, safety conditions and environmental parameters.
[0003] There are a large number of heterogeneous data sources at construction sites, such as video surveillance systems, environmental sensors, positioning devices, and audio acquisition equipment. These multimodal data have characteristics such as diverse formats, different acquisition frequencies, and synchronization difficulties, making data fusion extremely difficult. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a wireless networking optimization method and system based on multimodal data analysis, which effectively solves the fusion differences between multimodal data and improves the fusion effect and decision accuracy.
[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0006] A wireless networking optimization method based on multimodal data analysis comprises the following steps:
[0007] S1. Acquire multimodal data of a construction site, and process the data to obtain processed multimodal data, which includes channels and frequencies.
[0008] S2. Using self-supervised learning, feature representation learning is performed on the unlabeled feature data in the processed multimodal data to obtain corresponding data features.
[0009] S3. Based on the data features in step S2, a dynamic convolutional neural network is used to dynamically generate a convolution kernel and extract the corresponding feature vector.
[0010] S4. Using a dynamic adaptive contrast loss function, dynamically adjust the weight of the feature vector in step S3.
[0011] S5. Based on the adjusted result in step S4, multi-level linkage optimization is performed by combining edge computing with task offloading.
[0012] S6. Using the modal feature weighting strategy, the results of the multi-level linkage optimization in step S5 are differentially fused to obtain a fused multi-modal feature representation.
[0013] S7. Use the reinforcement learning algorithm to learn the fused results in step S6, and summarize the learning results to the cloud. Use the adaptive sensor network that collaborates with the edge and cloud to optimize the learning results, obtain the global optimal scheduling and processing strategy, and complete the optimization of the wireless network.
[0014] Furthermore, in step S1, the processed multimodal data includes the following contents:
[0015] Based on edge nodes, computing tasks and hardware loads are used to make task offloading decisions between edge nodes. Based on historical spectrum usage and network status, deep learning models are used to predict spectrum usage within a set time, and the spectrum is adjusted in combination with long short-term memory network time series prediction technology. According to the signal strength and interference of the network, channels and frequencies are determined to obtain processed multimodal data.
[0016] Furthermore, in step S2, the data features obtained include the following:
[0017] Self-supervised learning is used to design pre-training tasks to obtain the intrinsic structure and temporal relationship of unlabeled feature data. The pre-training tasks include predicting the order of video frames and reconstructing missing video frames. A neural network is used to train the data, and the corresponding data features are obtained by calculating the task error and updating the network weights.
[0018] Furthermore, in step S4, adjusting the weight includes the following:
[0019] The dynamic adaptive contrast loss function in the dynamic convolutional neural network is used to dynamically adjust the convolution kernel parameters in the feature vector. The specific formula is:
[0020]
[0021] Among them, L contrastive represents the dynamic adaptive contrast loss function, w i represents the weight parameter of the i-th modal data, represents the feature alignment error of the i-th modal data, represents the eigenvector corresponding to the first sample in the i-th modal data, Represents the eigenvector corresponding to the second sample in the i-th modal data.
[0022] Furthermore, in step S5, the linkage optimization includes the following contents:
[0023] A star topology is used in the central area, a mesh topology is used in the edge area, and automated equipment is used to obtain environmental factor data and adjust user locations and communication parameters.
[0024] The first feature is obtained by performing data fusion and analysis tasks on the edge node where the data source is located in real time; the second feature is obtained by processing the computing tasks through task offloading using the central cloud server.
[0025] Furthermore, in step S6, the fusion includes the following contents:
[0026] Through the coordinated control of antennas and the use of beamforming technology, the signal transmission direction and shape are adjusted according to the user's location and interference conditions; an adaptive directional antenna system is introduced to dynamically adjust the direction of the antenna through real-time perception of user location information.
[0027] The attention mechanism is used to calculate the weight of each feature in step S5 to obtain the weighted features. The weighted features are processed using graph neural networks and dynamic convolutional neural networks to obtain a fused multimodal feature representation.
[0028] Furthermore, in step S7, the optimization of the wireless networking includes the following:
[0029] The Q-learning-based reinforcement learning algorithm is used to process the fused multimodal feature representation to obtain the optimal configuration strategy, and then obtain the corresponding reward. The reward is adjusted using the reward function weighted by multi-dimensional environmental factors. The specific formula is:
[0030]
[0031] Among them, R t represents the reward function at time t, f s (S t , A t ) represents the state S of the sth environmental factor at time t t and action A t The feedback value under the condition, Q represents the total number of environmental factors; w s represents the adaptive weight of the sth environmental factor, q s It represents the priority of the sth environmental factor, and β represents the control parameter used to adjust the weight of the priority.
[0032] Sensors are arranged at the construction site to form a sensor network for real-time data collection of environmental factors. The data is sent to the edge node through the local wireless network for data filtering, denoising and analysis. If the data exceeds the set threshold, the edge node immediately takes action to issue an alarm and control the equipment. Adaptive modulation and coding technology is used to calculate the adaptive noise power estimate, select the modulation method according to the channel status, and adjust the sensor sampling frequency, data upload frequency or task offloading strategy to obtain the global optimal scheduling and processing strategy.
[0033] The adaptive noise power estimation formula is:
[0034]
[0035] Among them, P noise (t) represents the noise power at time t, S c (t) and They represent the received signal and expected signal of the cth sample respectively, and N represents the total number of samples.
[0036] Furthermore, the present invention also proposes a wireless networking optimization system based on multimodal data analysis, comprising:
[0037] The data processing module is used to acquire multimodal data of the construction site and process the data to obtain processed multimodal data.
[0038] The feature vector extraction module is used to use self-supervised learning to perform feature representation learning on the unlabeled feature data in the processed multimodal data to obtain the corresponding data features; based on the data features, the dynamic convolutional neural network is used to dynamically generate convolution kernels to extract the corresponding feature vectors.
[0039] The linkage optimization module is used to dynamically adjust the weights of the feature vectors in the feature vector extraction module by using a dynamic adaptive contrast loss function; based on the adjusted results, multi-level linkage optimization is performed by combining edge computing with task offloading.
[0040] The fusion module is used to utilize the modal feature weighting strategy to perform differential fusion on the results of the multi-level linkage optimization in the optimization module to obtain the fused multi-modal feature representation.
[0041] The wireless networking optimization module is used to use the reinforcement learning algorithm to learn the fusion results in the fusion module, and summarize the learning results to the cloud. It uses the adaptive sensor network that collaborates with the edge and cloud to optimize the learning results, obtain the global optimal scheduling and processing strategy, and complete the optimization of the wireless network.
[0042] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the wireless networking optimization method based on multimodal data analysis when executing the computer program.
[0043] Furthermore, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes the wireless networking optimization method based on multimodal data analysis.
[0044] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:
[0045] 1. The present invention can effectively alleviate the dependence on manual labeling by introducing self-supervised learning, and improve the ability to understand multimodal data by learning feature representation on unlabeled data. In addition, the present invention also improves the ability to align multimodal data, better adapts to the data characteristics of the current environment, effectively alleviates the load of edge devices, can maximize the computing power of the central server, improves the overall reliability and accuracy of data fusion, and enhances the adaptability of sensor networks.
[0046] 2. The present invention enhances the real-time response capability to different load conditions through dynamic load balancing of reinforcement learning, and solves the problems of limited edge computing resources and uneven load distribution. By introducing a dynamic reasoning path selection mechanism, the problem of long reasoning time of complex algorithms in multimodal data is solved. The intelligent caching and pre-fetching mechanism significantly reduces the time consumption caused by data access delays, thereby ensuring that data can be responded to in a timely manner when needed, solving the problems of network delays and bandwidth bottlenecks. Through dynamic model switching and adaptive quantization mechanisms, real-time requirements are significantly improved, so that construction site equipment can flexibly adapt under different computing load conditions, solving the problem of long reasoning time of complex algorithms. Through federated reinforcement learning and cloud-edge collaborative optimization, edge nodes can not only quickly adapt to changes in the on-site environment, but also further optimize decision-making and scheduling strategies through cloud collaboration, achieving a comprehensive improvement in computing efficiency and real-time performance.
[0047] 3. Through deep spectrum learning, the present invention can not only perceive the spectrum status in real time, but also predict and actively select the optimal channel in advance, solving the problem of dynamic allocation of spectrum resources. In a dynamic and drastic environment such as a construction site, the decision-making process of network operation and maintenance and configuration becomes more intelligent and predictable through the digital twin network. Operation and maintenance personnel can quickly adjust the configuration of on-site equipment through simulation results in a virtual environment to reduce the risk of configuration errors. Through the combination of modularization and self-healing networks, the network system at the construction site has a strong self-healing ability, and can quickly adjust and repair when problems occur in equipment or connections, thereby maintaining the stability and continuous operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is an overall implementation flow chart of the present invention. DETAILED DESCRIPTION
[0049] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0050] Existing data fusion methods usually rely on manual annotation and supervised learning models, but the cost of construction site data annotation is high and the quality is difficult to guarantee. At the same time, due to the inconsistency of features between different modal data, traditional data alignment and fusion methods cannot effectively solve the differences between multimodal data, thus affecting the fusion effect and the accuracy of decision-making.
[0051] To achieve the above objectives, the present invention proposes a wireless networking optimization method based on multimodal data analysis, such as Figure 1 As shown, the specific steps are as follows:
[0052] S1. Acquire multimodal data of the construction site and process the data to obtain processed multimodal data; the specific contents are:
[0053] Based on edge nodes, the computing tasks and hardware loads in the perceived environment are utilized to obtain real-time task offloading decisions between edge nodes. Based on historical spectrum usage and current network status, a deep learning model is used to obtain spectrum utilization in the future, and the spectrum is adjusted in combination with long short-term memory network time series prediction technology. According to the signal strength and interference of the network, the optimal channel and frequency are dynamically selected to obtain processed multimodal data.
[0054] S2. The amount of data on construction sites is huge and diverse, but the lack of high-quality labeled data makes it difficult for traditional supervised learning methods to work. The introduction of self-supervised learning can effectively alleviate the dependence on manual labeling and further improve the model's ability to understand multimodal data by learning feature representations on unlabeled data.
[0055] Using self-supervised learning, feature representation learning is performed on the unlabeled feature data in the processed multimodal data to obtain the corresponding data features; the specific content is:
[0056] Self-supervised learning is used to design pre-training tasks to obtain the intrinsic structure and temporal relationship of unlabeled feature data. The pre-training tasks include predicting the order of video frames and reconstructing missing video frames. A neural network is used to train the data, and important features such as the motion patterns of objects in the data and scene changes are obtained by calculating the task errors and updating the network weights. Then, the corresponding data features are obtained. These feature vectors represent the deep information of the data content or other modal data.
[0057] It has high adaptability in actual construction scenarios, especially when combined with federated learning, it can improve the efficiency of data utilization. Especially when deployed on edge devices, self-supervised learning can effectively improve the model's feature representation capabilities without the need to upload large amounts of data for centralized labeling, thereby reducing bandwidth usage and ensuring data security.
[0058] S3. Based on the data features in step S2, a dynamic convolutional neural network is used to dynamically generate a convolution kernel and extract the corresponding feature vector.
[0059] S4. The environment and data characteristics of the construction site often change, and fixed convolution kernels cannot adapt to these changes. An adaptive model based on dynamic convolutional neural network is introduced to dynamically generate convolution kernels according to the characteristics of the input data in order to better adapt to the data characteristics of the current environment. The dynamic convolutional neural network combined with the semantic relationship modeling of the graph neural network can improve the adaptability of the overall feature extraction, especially when different modal data are fused, it can automatically adapt to different data characteristics, thereby improving the fusion accuracy and robustness.
[0060] In order to better achieve the alignment of multimodal data in the feature space, the contrast optimization method is used to improve the alignment ability of multimodal data. The consistent feature representation between different modalities is learned by maximizing the similarity of similar samples and minimizing the similarity of different samples. In the construction scene, contrast optimization can be used to align visual features such as in the video with the environmental data of the sensor. To enhance the contrast optimization effect, the dynamic adaptive contrast loss function is used to dynamically adjust the weights of the feature vectors in step S3 to adapt to the environmental characteristics and data quality to ensure more accurate alignment; the specific contents are as follows:
[0061] Based on features such as brightness and contrast, the dynamic adaptive contrast loss function in the dynamic convolutional neural network is used to dynamically adjust the convolution kernel parameters in the feature vector to better extract useful features. The specific formula is:
[0062]
[0063] Among them, L contrastive represents the dynamic adaptive contrast loss function, w i represents the weight parameter of the i-th modal data, represents the feature alignment error of the i-th modal data, represents the eigenvector corresponding to the first sample in the i-th modal data, Represents the eigenvector corresponding to the second sample in the i-th modal data.
[0064] Through the dynamic adaptive contrast loss function, the dynamic weight adjustment of the modal features is realized, so that the influence of high-quality features can be adaptively enhanced and the weight of noise or disturbed features can be reduced during the feature alignment process. This significantly improves the alignment ability of contrast optimization in the case of large fluctuations in data quality and complex environments such as construction sites.
[0065] S5. The computing resources of edge devices at construction sites are limited. Edge computing can be combined with task offloading to form a multi-level linkage optimization mechanism.
[0066] A star topology is used in the central area to ensure high bandwidth and low latency of key nodes, and a mesh topology is used in the edge area to expand signal coverage through multi-hop nodes and reduce signal attenuation and interference in long-distance transmission. Automated equipment or mobile robots are used to dynamically deploy relay nodes to areas with weak signals or severe interference, perceive environmental changes, and adjust user locations and communication parameters.
[0067] Edge computing is used to process simple data fusion and analysis tasks in real time near the data source, while complex computing tasks (such as deep model training or large-scale feature extraction) are transferred to more powerful central cloud servers through task offloading. Therefore, data fusion and analysis tasks are performed in real time on the edge node where the data source is located to obtain the first feature; the computing tasks are processed by the central cloud server through task offloading to obtain the second feature.
[0068] This mechanism combines federated learning and TinyML technology to allow edge devices to perform local lightweight model reasoning and preliminary training, and ensure the secure transmission of important data when tasks are offloaded. Through intelligent task scheduling algorithms, the system can dynamically determine when to perform tasks at the edge and when to offload to the cloud, thereby optimizing the system's computing efficiency and bandwidth usage. This can not only effectively alleviate the load on edge devices, but also maximize the computing power of central servers.
[0069] S6. At the construction site, different data modalities contribute differently to the final decision. The modal feature weighting strategy can dynamically adjust the weight of each modality in the fusion and achieve differentiated fusion of multimodal data. For example, in a noisy environment, video data is more reliable than audio data, while in temperature and humidity monitoring, sensor data is more important than visual data.
[0070] Using the modal feature weighting strategy, the results of the multi-level linkage optimization in step S5 are differentiated and fused to obtain the fused multi-modal feature representation; the specific content is:
[0071] Through the coordinated control of multiple antennas and the use of beamforming technology, the signal transmission direction and shape are adjusted according to the current user location and interference conditions to concentrate energy for transmission to the target user; an adaptive directional antenna system is introduced to dynamically adjust the direction of the antenna through real-time perception of user location information to maximize the received signal strength of the target device and minimize interference.
[0072] The attention mechanism is used to calculate the weight of each feature in step S5 to obtain the weighted features. The weighted features are processed using graph neural networks and dynamic convolutional neural networks to obtain a fused multimodal feature representation, which is used in decision-making processes such as wireless networking optimization and task scheduling. It helps reduce dependence on low-quality or interfered data and improves the overall reliability and accuracy of data fusion.
[0073] S7. At the construction site, the layout and quantity of sensors have a great impact on data quality and fusion effect. Through the collaborative computing capabilities of the edge and cloud, the status and environmental conditions of the sensors can be analyzed in real time, the working mode and data transmission strategy of the sensors can be adjusted dynamically, and adaptive sensor network optimization can be achieved. For example, by analyzing the battery status of the sensor nodes and the importance of the data, it is decided that some sensors enter low-power mode, while sensors in key areas maintain high-frequency acquisition and transmission. This optimization scheme collects the operating status of the sensors through edge devices and combines it with the global data analysis of the central cloud to form a closed-loop control system to ensure the optimal energy consumption of the sensor network while ensuring the timeliness and integrity of the data. Combined with blockchain technology, the work log of each sensor node is recorded to ensure the credibility and traceability of the entire system. This collaborative optimization not only improves the adaptability of the sensor network, but also ensures the stability and reliability of long-term operation in complex environments.
[0074] The reinforcement learning algorithm is used to learn the fused results in step S6, and the obtained learning results are summarized to the cloud. The learning results are optimized by using the adaptive sensor network coordinated by the edge and the cloud to obtain the global optimal scheduling and processing strategy, and complete the optimization of the wireless network; the specific contents are as follows:
[0075] The Q-learning-based reinforcement learning algorithm is used to enable the network system to find the optimal configuration strategy through continuous attempts under different environmental changes. Each successful configuration will receive positive rewards, and failures will be punished. The network system gradually optimizes its adjustment strategy through accumulated experience. For complex multi-dimensional network parameter adjustments, a deep Q network is used to make decisions. The reward function weighted by multi-dimensional environmental factors is used to dynamically adjust the reward value according to the different environmental conditions of the construction site. The specific formula is:
[0076]
[0077] Among them, R t represents the reward function at time t, f s (S t , A t ) represents the state S of the sth environmental factor at time t t and action A t The feedback value under the condition, Q represents the total number of environmental factors; w s represents the adaptive weight of the sth environmental factor, q s It represents the priority of the sth environmental factor, and β represents the control parameter used to adjust the weight of the priority.
[0078] Sensors such as temperature, humidity, and video surveillance are arranged at the construction site to form a sensor network for real-time environmental data collection. The data is sent to the edge computing node through the local wireless network Wi-Fi and LoRa for data filtering, denoising, and basic analysis. If the data belongs to real-time task environment anomaly detection or state change, the edge node immediately takes action to issue an alarm and control the equipment; adaptive modulation and coding technology is used to calculate the adaptive noise power estimate, select the modulation mode in real time according to the channel status, and adjust the sensor sampling frequency, data upload frequency or task offloading strategy to obtain the global optimal scheduling and processing strategy. For example, the sampling frequency is reduced in areas with good signals, and the sampling frequency is increased in areas with poor signals or heavy tasks.
[0079] The adaptive noise power estimation formula is:
[0080]
[0081] Among them, P noise (t) represents the noise power at time t, S c (t) and They represent the received signal and expected signal of the cth sample respectively, and N represents the total number of samples.
[0082] The method proposed in the present invention improves the flexibility and adaptability of the system: through the coordinated work of spectrum management, dynamic task allocation and antenna optimization, the system can flexibly respond to complex changes in the construction site, reduce the impact of signal interference, and maintain the stability of the communication link.
[0083] The method proposed in the present invention enhances the real-time processing capability: through intelligent caching, reasoning path optimization, dynamic model switching and other means, the computing delay is greatly reduced to ensure the real-time performance of model reasoning and data processing.
[0084] The method proposed in the present invention reduces signal interference and improves anti-interference capability: through channel optimization, collaborative interference elimination and dynamic adjustment of antenna direction, the system can effectively self-adjust when environmental interference is severe, thereby reducing the interference impact on the communication link.
[0085] The method proposed in the present invention improves the quality and stability of multimodal data fusion: through the combination of modal feature weighting, contrast optimization and beamforming, high-quality multimodal data fusion is still maintained in a high-interference environment, providing a reliable basis for decision-making.
[0086] The method proposed in the present invention enhances the overall intelligence and creativity of the system: through the coordinated solution of four technical problems, the system shows a higher level of intelligence and automation in actual application on the construction site, and can achieve global optimal real-time communication, data fusion and signal interference management, ensuring the efficient operation of the system.
[0087] The embodiment of the present invention also proposes a wireless networking optimization system based on multimodal data analysis, including a data processing module, a feature vector extraction module, a linkage optimization module, a fusion module, a wireless networking optimization module and a computer program that can be run on a processor. It should be noted that each module in the above system corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.
[0088] The embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that when the processor executes the computer program, the specific steps of the method provided in the embodiment of the present invention are corresponding to the specific steps of the method provided in the embodiment of the present invention, and the processor has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.
[0089] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. It should be noted that when the computer program is executed by the processor, it corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.
[0090] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.
[0091] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0092] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0093] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A wireless networking optimization method based on multimodal data analysis, characterized in that: include: S1, acquiring multimodal data of a construction site, and processing the data to obtain processed multimodal data, the data including channels and frequencies; S2. Using self-supervised learning, feature representation learning is performed on the unlabeled feature data in the processed multimodal data to obtain corresponding data features; S3, based on the data features in step S2, using a dynamic convolutional neural network to dynamically generate a convolution kernel and extract a corresponding feature vector; S4, using a dynamic adaptive contrast loss function to dynamically adjust the weight of the feature vector in step S3; S5. Based on the adjusted result in step S4, multi-level linkage optimization is performed by combining edge computing with task offloading; S6, using the modal feature weighting strategy, differentially fuse the results of the multi-level linkage optimization in step S5 to obtain a fused multi-modal feature representation; S7. Use the reinforcement learning algorithm to learn the fused results in step S6, and summarize the learning results to the cloud. Use the adaptive sensor network that collaborates with the edge and cloud to optimize the learning results, obtain the global optimal scheduling and processing strategy, and complete the optimization of the wireless network.
2. The wireless networking optimization method based on multimodal data analysis according to claim 1, characterized in that: In step S1, the processed multimodal data includes the following contents: Based on edge nodes, computing tasks and hardware loads are used to make task offloading decisions between edge nodes. Based on historical spectrum usage and network status, deep learning models are used to predict spectrum usage within a set time, and the spectrum is adjusted in combination with long short-term memory network time series prediction technology. According to the signal strength and interference of the network, channels and frequencies are determined to obtain processed multimodal data.
3. The wireless networking optimization method based on multimodal data analysis according to claim 1, characterized in that: In step S2, the data features obtained include the following: Self-supervised learning is used to design pre-training tasks to obtain the intrinsic structure and temporal relationship of unlabeled feature data. The pre-training tasks include predicting the order of video frames and reconstructing missing video frames. A neural network is used to train the data, and the corresponding data features are obtained by calculating the task error and updating the network weights.
4. The wireless networking optimization method based on multimodal data analysis according to claim 1, characterized in that: In step S4, adjusting the weight includes the following: The dynamic adaptive contrast loss function in the dynamic convolutional neural network is used to dynamically adjust the convolution kernel parameters in the feature vector. The specific formula is: Among them, L contrastive represents the dynamic adaptive contrast loss function, w i represents the weight parameter of the i-th modal data, represents the feature alignment error of the i-th modal data, represents the eigenvector corresponding to the first sample in the i-th modal data, Represents the eigenvector corresponding to the second sample in the i-th modal data.
5. The wireless networking optimization method based on multimodal data analysis according to claim 1, characterized in that: In step S5, the linkage optimization includes the following contents: Use star topology in the central area and mesh topology in the edge area, and use automation equipment to obtain environmental factor data and adjust user locations and communication parameters; Perform data fusion and analysis tasks on the edge node where the data source is located in real time to obtain the first feature; The second feature is obtained by processing the computing tasks by task offloading using the central cloud server.
6. The wireless networking optimization method based on multimodal data analysis according to claim 5, characterized in that: In step S6, fusion Includes the following: Through the coordinated control of antennas, beamforming technology is used to adjust the transmission direction and shape of the signal according to the user's location and interference conditions; an adaptive directional antenna system is introduced to dynamically adjust the direction of the antenna through real-time perception of user location information; The attention mechanism is used to calculate the weight of each feature in step S5 to obtain the weighted features. The weighted features are processed using graph neural networks and dynamic convolutional neural networks to obtain a fused multimodal feature representation.
7. The wireless networking optimization method based on multimodal data analysis according to claim 1, characterized in that: In step S7, the optimization of the wireless networking includes the following: The Q-learning-based reinforcement learning algorithm is used to process the fused multimodal feature representation to obtain the optimal configuration strategy, and then obtain the corresponding reward. The reward is adjusted using the reward function weighted by multi-dimensional environmental factors. The specific formula is: Among them, R t represents the reward function at time t, f s (S t , A t ) represents the state S of the sth environmental factor at time t t and action A t The feedback value under the condition, Q represents the total number of environmental factors; w s represents the adaptive weight of the sth environmental factor, q s represents the priority of the sth environmental factor, β represents the control parameter, which is used to adjust the weight of the priority; Sensors are deployed at the construction site to form a sensor network for real-time data collection of environmental factors. The data is sent to the edge node through the local wireless network for data filtering, denoising and analysis. If the data exceeds the set threshold, the edge node immediately takes action to issue an alarm and control the equipment. Adaptive modulation and coding technology is used to calculate adaptive noise power estimation, select the modulation mode according to the channel state, and adjust the sensor sampling frequency, data upload frequency or task offloading strategy to obtain the global optimal scheduling and processing strategy. The adaptive noise power estimation formula is: Among them, P noise (t) represents the noise power at time t, S c (t) and They represent the received signal and expected signal of the cth sample respectively, and N represents the total number of samples.
8. A system applied to the wireless networking optimization method based on multimodal data analysis as claimed in claim 1, characterized in that: include: A data processing module is used to obtain multimodal data of the construction site and process the data to obtain processed multimodal data; The feature vector extraction module is used to use self-supervised learning to perform feature representation learning on the unlabeled feature data in the processed multimodal data to obtain the corresponding data features; based on the data features, the dynamic convolutional neural network is used to dynamically generate the convolution kernel to extract the corresponding feature vector; A linkage optimization module is used to dynamically adjust the weights of feature vectors in the feature vector extraction module using a dynamic adaptive contrast loss function; based on the adjusted results, multi-level linkage optimization is performed by combining edge computing with task offloading; The fusion module is used to use the modal feature weighting strategy to perform differential fusion on the results of multi-level linkage optimization in the optimization module to obtain the fused multi-modal feature representation; The wireless networking optimization module is used to use the reinforcement learning algorithm to learn the fusion results in the fusion module, and summarize the learning results to the cloud. It uses the adaptive sensor network that collaborates with the edge and cloud to optimize the learning results, obtain the global optimal scheduling and processing strategy, and complete the optimization of the wireless network.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the wireless networking optimization method based on multimodal data analysis described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the wireless networking optimization method based on multimodal data analysis according to any one of claims 1 to 7 is executed.