Communication indicator prediction method based on multi-modal large model and related device

Abstract

The application provides a communication index prediction method based on a multi-modal large model and related equipment, relates to the technical field of data processing, and can obtain multi-dimensional time sequence data, geographical space data and corresponding text description data in a user communication process; performs time sequence feature extraction processing on the multi-dimensional time sequence data to obtain a time sequence feature vector, and performs spatial feature extraction processing on the geographical space data to obtain a spatial feature vector; encodes the text description data based on a word embedding layer of a large language model to obtain a semantic feature vector; performs cross-modal attention fusion on the three types of feature vectors based on a text prototype to generate a joint feature representation; and inputs the joint feature representation into the large language model to generate communication index prediction information, which can effectively model the coupling relationship between multi-modal indexes, improve prediction accuracy, and thus ensure the effectiveness of network optimization decisions.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and in particular to a communication index prediction method based on a multi-modal large model and related equipment. BACKGROUND

[0002] In practical application of a communication network, in order to guarantee service quality and optimize resource allocation, network operators need to predict future network congestion, signal attenuation and performance problems based on user historical communication indexes, so as to formulate forward-looking network optimization strategies, so as to improve user experience and reduce operation and maintenance costs.

[0003] Existing user-side performance modeling methods usually analyze traffic time sequence characteristics in an internal communication environment, but such single-modal single-index prediction model can only capture local information and is difficult to comprehensively analyze user behavior rules in complex network scenarios, resulting in limited prediction accuracy and affecting the effectiveness of network optimization decisions. SUMMARY

[0004] Embodiments of the present application provide a communication index prediction method based on a multi-modal large model and related equipment, which can effectively model the coupling relationship between multi-modal indexes, improve prediction accuracy, and thus ensure the effectiveness of network optimization decisions.

[0005] In a first aspect, embodiments of the present application provide a communication index prediction method based on a multi-modal large model, comprising: acquiring multi-dimensional time sequence data, geographical space data and text description data corresponding to the multi-dimensional time sequence data in a user communication process; performing time sequence feature extraction processing on the multi-dimensional time sequence data to obtain a time sequence feature vector; performing spatial feature extraction processing on the geographical space data to obtain a spatial feature vector; encoding the text description data based on a word embedding layer of a pre-trained large language model to obtain a semantic feature vector; performing cross-modal attention fusion on the time sequence feature vector, the spatial feature vector and the semantic feature vector based on a text prototype of the large language model to generate a joint feature representation; inputting the joint feature representation into the large language model, so that the large language model obtains a corresponding reasoning vector based on the joint feature representation, and outputs communication index prediction information according to the reasoning vector.

[0006] In some embodiments, the time sequence feature extraction processing on the multi-dimensional time sequence data to obtain a time sequence feature vector comprises: constructing a convolutional neural network based on an attention mechanism according to the feature dimensions of the multi-dimensional time sequence data; dividing the multi-dimensional time sequence data into a plurality of local time sequence blocks along the time sequence dimension, and extracting local time sequence features from each of the local time sequence blocks through the convolutional neural network, and obtaining a time sequence feature vector according to all the local time sequence features.

[0007] In some embodiments, the spatial feature extraction process on the geospatial data to obtain a spatial feature vector comprises: dividing the geospatial data into a plurality of grid units, and marking a service cell grid involved in the user communication process; performing convolution feature extraction on all the grid units to generate local spatial features; superimposing a learnable embedding vector on the local spatial features corresponding to all the service cell grids to obtain local enhanced features; and obtaining a spatial feature vector according to the local spatial features of all the unmarked grid units and the local enhanced features of all the service cell grids.

[0008] In some embodiments, the cross-modal attention fusion of the time sequence feature vector, the spatial feature vector, and the semantic feature vector based on the text prototype of the large language model to generate a joint feature representation comprises: generating a first text prototype and a second text prototype based on the text dictionary data of the large language model; taking the time sequence feature vector as a query vector and the first text prototype as a key value vector to generate a time sequence feature in a text modality through attention weight calculation; taking the spatial feature vector as a query vector and the second text prototype as a key value vector to generate a spatial feature in a text modality; and splicing the time sequence feature in the text modality, the spatial feature in the text modality, and the semantic feature vector to generate a multi-modal joint feature representation.

[0009] In some embodiments, the process of obtaining the geospatial data comprises: obtaining base station location data involved in the user communication process; determining a target map area according to the base station location data and a preset engineering parameter, and obtaining geospatial data corresponding to the target map area from a preset map library.

[0010] In some embodiments, the process of obtaining text description data corresponding to the multi-dimensional time sequence data comprises: constructing a dataset introduction text containing the meanings of a plurality of communication indicators in the multi-dimensional time sequence data; constructing a task description text containing a historical sequence length definition and a predicted sequence length definition in communication indicator prediction; and splicing the dataset introduction text and the task description text to obtain text description data.

[0011] In some embodiments, the training process of the large language model comprises: dividing the multi-dimensional time sequence data in the training data into a plurality of samples according to a time sliding window, each sample containing an input sequence and an output sequence; calculating a predicted output of the large language model to be trained on the input sequence, and calculating a model loss according to the predicted output and the output sequence; updating the input-output mapping layer parameters of the large language model based on the model loss to obtain a trained large language model.

[0012] In a second aspect, an embodiment of the present application provides a communication index prediction device based on a multi-modal large model, comprising: a data acquisition module configured to acquire multi-dimensional time series data, geographical space data, and text description data corresponding to the multi-dimensional time series data in a user communication process; a time series processing module configured to perform time series feature extraction processing on the multi-dimensional time series data to obtain a time series feature vector; a space processing module configured to perform spatial feature extraction processing on the geographical space data to obtain a spatial feature vector; a text processing module configured to encode the text description data based on a word embedding layer of a pre-trained large language model to obtain a semantic feature vector; a feature fusion module configured to perform cross-modal attention fusion on the time series feature vector, the spatial feature vector, and the semantic feature vector based on a text prototype of the large language model to generate a joint feature representation; and a prediction and reasoning module configured to input the joint feature representation into the large language model, so that the large language model obtains a corresponding reasoning vector based on the joint feature representation, and outputs communication index prediction information according to the reasoning vector.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; at least one memory configured to store at least one program; and when the at least one program is executed by the at least one processor, the electronic device is configured to implement the communication index prediction method based on the multi-modal large model according to any one of the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer executable instructions for executing the communication index prediction method based on the multi-modal large model according to any one of the first aspect.

[0015] The application provides a communication index prediction method based on a multi-modal large model and related equipment. The method can integrate multi-dimensional time sequence data, geographical space data and text description data in the user communication process, construct a multi-modal feature fusion framework, and solve the problem that the existing single-modal single-index prediction model can only capture local information and is difficult to comprehensively analyze the user behavior rules in a complex network scenario. In the application, the attention mechanism and the convolutional neural network can be used to extract time sequence feature vectors from multi-dimensional time sequence data, extract spatial feature vectors from geographical space data, and analyze the context information of the text description data by combining the semantic coding ability of the large language model to obtain semantic feature vectors. It can be understood that the time sequence feature vectors reflect the communication index at the time of communication, the geographical space data reflects the environmental information at the time of communication, and the semantic feature vectors reflect the data set background knowledge of the communication index and the prediction task, so that the large language model can have semantic reasoning ability for the communication index and the environmental information. Further, based on the text prototype of the large language model, the time sequence feature, the spatial feature and the semantic feature can be dynamically interacted through the cross-modal attention mechanism, the time sequence feature and the spatial feature are associated with the semantic feature, and the joint feature representation of the text modal is obtained. The joint feature representation can be input into the pre-trained large language model, and the semantic reasoning ability of the large language model can map the multi-modal information including time sequence, space and semantics into the future communication index prediction result, so as to model the spatio-temporal association and semantic logic in the network environment, improve the prediction accuracy, and enable the operator to optimize the base station resource configuration in advance based on the high-precision prediction result, thereby reducing the network congestion risk and improving the user experience and operation and maintenance efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The method flowchart of the communication index prediction method based on the multi-modal large model provided by an embodiment of the application is provided.

[0017] Figure 2 The method flowchart of obtaining the time sequence feature vector in the communication index prediction method based on the multi-modal large model provided by an embodiment of the application is provided.

[0018] Figure 3 The method flowchart of obtaining the spatial feature vector in the communication index prediction method based on the multi-modal large model provided by an embodiment of the application is provided.

[0019] Figure 4 The method flowchart of generating the joint feature representation in the communication index prediction method based on the multi-modal large model provided by an embodiment of the application is provided.

[0020] Figure 5 The method flowchart of obtaining the geographical space data in the communication index prediction method based on the multi-modal large model provided by an embodiment of the application is provided.

[0021] Figure 6 The method flow chart for obtaining text description data corresponding to the multi-dimensional time sequence data in the communication index prediction method based on a multi-modal large model provided by an embodiment of the present application;

[0022] Figure 7 The method flow chart for training the large language model in the communication index prediction method based on a multi-modal large model provided by an embodiment of the present application;

[0023] Figure 8 The overall model flow chart of the communication index prediction method based on a multi-modal large model provided by an embodiment of the present application;

[0024] Figure 9 The overall flow chart for obtaining a spatial feature vector in the communication index prediction method based on a multi-modal large model provided by an embodiment of the present application;

[0025] Figure 10 The structural schematic diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0027] In some embodiments, although the functional modules are divided in the system schematic diagram, and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the system or the order in the flow chart. The terms first, second, etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0028] In addition, unless otherwise explicitly specified and limited, the term "connection / connected" should be interpreted broadly, for example, it can be a fixed connection or a movable connection, or a detachable connection or a non-detachable connection, or an integral connection; it can be a mechanical connection, or an electrical connection or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium.

[0029] In the description of the embodiments of the present application, the reference terms "one embodiment / implementation", "another embodiment / implementation" or "certain embodiments / implementations", "in the above-mentioned embodiments / implementations" and the like mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least two embodiments or implementations disclosed in the present application. In the disclosure of this application, the schematic representation of the above terms does not necessarily refer to the same embodiment or implementation. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that in the flowchart.

[0030] In the practical application of communication networks, to ensure service quality and optimize resource allocation, network operators need to predict future network congestion, signal attenuation, and performance issues based on historical user communication metrics. This allows them to develop forward-looking network optimization strategies to improve user experience and reduce operation and maintenance costs. Existing user-side performance modeling methods typically analyze traffic timing characteristics within the internal communication environment, using isolated modeling to predict communication metrics. However, in addition to traffic characteristics, the internal communication environment also includes other metrics such as signal level, latency, packet loss rate, and retransmission rate. Furthermore, the internal communication environment also includes metrics from other modalities, such as the external communication environment. These metrics are correlated in time and space and collectively reflect the dynamic characteristics of the network environment. However, existing single-modality, single-metric prediction models can only capture local information and are unable to fully analyze user behavior patterns in complex network scenarios. This results in limited prediction accuracy, which in turn affects the effectiveness of network optimization decisions.

[0031] In order to solve the above problems, this application notes that the prediction of communication indicators requires the integration of time dimension fluctuations, spatial distribution characteristics and semantic description information. Existing single-modal models cannot effectively characterize the correlation of multi-dimensional data, especially the lack of quantitative analysis of geographic spatial features. Through analysis, it is found that the spatial correlation between base station location data and user movement trajectory affects the signal strength, while text descriptions can provide semantic interpretation of communication scenarios. Based on this, this application proposes to combine time series feature extraction, spatial feature modeling and natural language processing, and use the cross-modal fusion capabilities of large language models to construct a joint representation. Further exploration found that the attention mechanism can effectively align the distribution of different modal features in the semantic space, guide cross-modal interaction through text prototypes, and realize the deep coupling of spatiotemporal features and semantic information.

[0032] Therefore, the embodiment of the present application provides a communication index prediction method based on a multi-modal large model and related equipment. The present application can integrate multi-dimensional time series data, geographical space data and text description data in the user communication process to construct a multi-modal feature fusion framework, so as to solve the problem that the existing single-modal single-index prediction model can only capture local information and is difficult to comprehensively analyze the user behavior rules in complex network scenarios. The present application can use attention mechanism and convolutional neural network to extract time series feature vectors from multi-dimensional time series data and spatial feature vectors from geographical space data, and combine the semantic encoding ability of the large language model to analyze the context information of the text description data to obtain semantic feature vectors. The coupling relationship between multi-modal indexes can be effectively modeled, the prediction accuracy can be improved, and the effectiveness of network optimization decision can be ensured.

[0033] The embodiments of the present application will be further described below with reference to the accompanying drawings.

[0034] Reference Figure 1 , Figure 1 The method flowchart of the communication index prediction method based on the multi-modal large model provided by an embodiment of the present application; in some embodiments, the present application proposes a communication index prediction method based on a multi-modal large model, at least including the following steps:

[0035] Step S110, obtaining multi-dimensional time series data, geographical space data in the user communication process and text description data corresponding to the multi-dimensional time series data;

[0036] Step S120, performing time series feature extraction processing on the multi-dimensional time series data to obtain a time series feature vector;

[0037] Step S130, performing spatial feature extraction processing on the geographical space data to obtain a spatial feature vector;

[0038] Step S140, encoding the text description data based on the word embedding layer of the pre-trained large language model to obtain a semantic feature vector;

[0039] Step S150, based on the text prototype of the large language model, performing cross-modal attention fusion on the time series feature vector, the spatial feature vector and the semantic feature vector to generate a joint feature representation;

[0040] Step S160, inputting the joint feature representation into the large language model, so that the large language model obtains a corresponding reasoning vector based on the joint feature representation, and outputs communication index prediction information according to the reasoning vector.

[0041] The multi-dimensional time sequence data can be continuous time sequence data containing multiple communication indicators such as traffic, delay and packet loss rate, and can be implemented by using minute-level sampling data collected by a sensor to represent the dynamic change of the communication network state; the geographic space data can be geographic information data containing base station positions, terrain elevations and building distributions, and can be implemented by using vector layer data provided by a GIS system to analyze the influence of the space environment on signal propagation; the text description data can be natural language texts describing the characteristics of the communication scene, and can be implemented by using manually annotated or automatically generated scene description texts to provide semantic context information; the time sequence feature extraction processing can be capturing local patterns in time sequences by using a convolutional neural network, and can be implemented by dividing time sequence segments by using a sliding window and extracting periodic features to identify short-term fluctuation rules of communication indicators; the spatial feature extraction processing can be extracting local spatial features of geographic grids by using convolution operations, and can be implemented by dividing geographic grids and superimposing service cell embedding vectors to enhance the spatial representation capability of key areas; and the cross-modal attention fusion can be calculating attention weights of different modal features by using text prototypes as key-value vectors, and can be implemented by aligning time sequence features and semantic space distributions by using a multi-head attention mechanism to eliminate the semantic gap between modalities.

[0042] It can be understood that the time sequence feature vector reflects the communication indicators at the time of communication, the geographic space data reflects the environmental information at the time of communication, and the semantic feature vector reflects the communication indicators and the background knowledge of the data set of the prediction task, so that the large language model can have semantic reasoning capability for the communication indicators and the environmental information; further, the present application can dynamically interact the time sequence features, the spatial features and the semantic features based on the text prototype of the large language model by using the cross-modal attention mechanism, associate the time sequence features and the spatial features with the semantic features, obtain the joint feature representation of the text modal, and input the joint feature representation to the large language model after pre-training, so as to map the multi-modal information including time sequence, space and semantics into the future communication indicator prediction result by using the semantic reasoning capability, model the spatio-temporal correlation and semantic logic in the network environment, improve the prediction accuracy, and enable the operator to optimize the base station resource configuration in advance based on the high-precision prediction result, thereby reducing the risk of network congestion and improving the user experience and operation and maintenance efficiency.

[0043] In some embodiments, the multi-dimensional time sequence data in the communication process is divided into multiple local time sequence blocks, periodic features and trend features are extracted by using an attention convolutional network to form a feature vector with time dependence; the geographic space data is converted into a grid representation, and a learnable position embedding is superimposed on the grid where the service cell is located, and a spatial distribution pattern is extracted by using a convolutional layer; the text description data is converted into a semantic vector by using a word embedding layer of a large language model to retain the context relationship of the natural language description.

[0044] In some embodiments, in the feature fusion stage, the time sequence feature vector and the space feature vector can respectively interact with the text prototype with attention, generate cross-modal features aligned with the semantic space, and input the fused joint feature representation into a large language model for autoregressive decoding to output the communication index prediction value of the future period through sequence generation, so as to realize the collaborative modeling of the time dimension dynamic change, the space distribution characteristics and the semantic description information.

[0045] It can be understood that, compared with the prior art, the traditional single-modal prediction method only analyzes the isolated time sequence change of the traffic index, without considering the mutual influence between multi-dimensional indexes, for example, the negative correlation between the packet loss rate and the signal strength is not modeled, and the existing spatial analysis method usually processes the base station location data independently, without joint optimization with time sequence features; while the present scheme maps the geographical space features to the text semantic space through the cross-modal attention mechanism, so that the base station engineering parameter features can be associated with the signal attenuation trend in the time dimension; in addition, the traditional text processing method is difficult to associate with structured communication data, while the present scheme uses the text prototype of the large language model to guide feature fusion, so that concepts such as "peak period congestion" in the semantic description can be directly associated with the traffic mutation pattern in the time sequence features.

[0046] It is worth noting that, through the above technical scheme, the present application can effectively integrate the multi-modal data features in the communication scene, solve the problem of the separation of space-time features and semantic information, i.e. through time sequence feature extraction to capture the dynamic evolution law of the communication index, spatial feature modeling to quantify the influence of the geographical environment on the signal quality, text encoding to provide semantic interpretation of the scene, and then cross-modal attention fusion to eliminate the representation difference between different modalities, so that the base station distribution features can be associated with the time sequence fluctuation to take advantage of the reasoning ability of the large language model to convert the joint features into interpretable prediction results, thereby improving the accuracy of network congestion prediction in complex scenarios and providing a reliable basis for dynamically adjusting the base station load distribution.

[0047] Reference Figure 2 , Figure 2 In the multi-modal large model-based communication index prediction method provided by an embodiment of the present application, a method flowchart for obtaining a time sequence feature vector is provided; in some embodiments, multi-dimensional time sequence data is subjected to time sequence feature extraction processing to obtain a time sequence feature vector, including at least the following steps:

[0048] Step S210, constructing a convolutional neural network based on an attention mechanism according to the feature dimension of the multi-dimensional time sequence data;

[0049] In step S220, the multi-dimensional time series data is divided into multiple local time series blocks along the time sequence dimension, and local time series features are extracted from each local time series block by a convolutional neural network, and a time series feature vector is obtained according to all the local time series features.

[0050] The convolutional neural network based on the attention mechanism can refer to a network architecture composed of a convolutional layer and an attention module in series. Specifically, the convolutional neural network based on the attention mechanism can be implemented by connecting a one-dimensional convolutional layer for extracting local features and a self-attention module for realizing time sequence dimension weight distribution. This structure can adaptively focus on the feature dimension correlation of key time nodes, and solve the problem of insufficient sensitivity of traditional convolutional networks to time sequence dynamic changes.

[0051] The local time series block can be a continuous data segment divided according to a preset time window length. The local time series block can be implemented by using a sliding window or a fixed length block method. Through the block operation, a long sequence can be decomposed into multiple short segments, so as to facilitate the capture of local time domain features and reduce the difficulty of long-range dependence modeling.

[0052] In some embodiments, the process of obtaining multi-dimensional time series data in the user communication process can include collecting all user measurement reports of the communication process between the base station and the user equipment through the base station, classifying different users, extracting the time sequence user measurement report sequence of one user, and then selecting multiple key communication indicator fields from the time sequence user measurement report sequence to form multi-dimensional time series data. The multi-dimensional time series data is used to construct a multivariate time series prediction task to capture the communication service situation of the user in the corresponding time period. Specifically, the process can include the following 17 key communication indicator fields: DL GrantCount (downlink scheduling permission times), DL RB Num (downlink resource block quantity), UL Grant Count (uplink scheduling permission times), UL RB Num (uplink resource block quantity), DL MCS AVG (downlink average modulation and coding strategy), DL RANK (downlink rank), UL MCS AVG (uplink average modulation and coding strategy), UL RANK (uplink rank), CQI (channel quality indicator), DL IBLER (downlink initial block error rate), DL RBLER (downlink retransmission block error rate), UL IBLER (uplink initial block error rate), UL RBLER (uplink retransmission block error rate), UL DMRS RSRP (uplink demodulation reference signal received power), UL DMRS SINR (uplink demodulation reference signal signal-to-noise ratio), UL SRS RSRP (uplink sounding reference signal received power), and UL SRS SINR (uplink sounding reference signal signal-to-noise ratio). The multivariate time series prediction task is constructed for these fields to capture the communication service situation of the user in the corresponding time period.

[0053] It can be understood that after the multi-dimensional time series data is divided into multiple time segments, local feature extraction is performed along the feature dimension through the convolution kernel, each convolution kernel corresponds to local pattern recognition of a feature dimension, the attention module recalibrates the weight of the convolution output, key features are screened by calculating the correlation between time steps, and after all local time sequence blocks are fused through weighting, a global time sequence feature vector is formed, which not only retains local dynamic characteristics but also constructs global time correlation.

[0054] In some embodiments, corresponding to steps S210 to S220, feature processing is performed through the designed attention mechanism-based convolutional neural network, time series data is extracted through the Attention mechanism of the feature dimension, and then divided into multiple patch small blocks along the time sequence dimension, for example, the time length of each small block can be 2 time sequence records.

[0055] It is worth noting that, compared with the prior art, the traditional method uses a single convolution layer or a fully connected network to process time series data, cannot distinguish the contribution difference of different feature dimensions to time series changes, and lacks fine-grained modeling capability for local time windows. The scheme realizes local feature focusing through block operation, enhances the correlation modeling between feature dimensions through the attention mechanism, and can solve the problem of insufficient capture of multi-dimensional index coupling relationship.

[0056] Reference Figure 3 , Figure 3 In the communication index prediction method based on a multi-modal large model provided by an embodiment of the present application, a method flowchart for obtaining a spatial feature vector is provided. In some embodiments, spatial feature extraction is performed on geographic spatial data to obtain a spatial feature vector, including at least the following steps:

[0057] Step S310, dividing the geographic spatial data into multiple grid units and marking the service cell grids involved in the user communication process;

[0058] Step S320, performing convolution feature extraction on all grid units to generate local spatial features;

[0059] Step S330, superimposing a learnable embedding vector on the local spatial features corresponding to all service cell grids to obtain local enhanced features;

[0060] Step S340, obtaining a spatial feature vector according to the local spatial features of all unmarked grid units and the local enhanced features of all service cell grids.

[0061] The grid unit can be a discrete area formed by dividing geographic space data according to a preset resolution, and can be implemented by using a square grid of a fixed size or a dynamically adjusted size. The continuous space is converted into structured data through discretization processing. The service cell grid can be a grid unit corresponding to a base station coverage area actually connected in the user communication process, and can be marked by a preset mapping relationship between the base station coordinates and the grid, and is used for positioning the core influence area. The convolution feature extraction can be understood as using a convolutional neural network to encode the spatial features of the local neighborhood of the grid unit, and can be implemented by using a multi-layer convolution kernel stacking structure. The spatial correlation between adjacent grids is captured by a sliding window operation. The learnable embedding vector can be a feature enhancement parameter automatically optimized through model training, which can be initialized as a random vector and updated in the back propagation process, and is used to assign a dynamic weight to the service cell grid to distinguish its importance.

[0062] It can be understood that the geographic space data is first divided into grid units, wherein the service cell grid is marked as a key area through the mapping relationship between the base station position and the grid coordinates, and each grid unit and its adjacent area are input into a convolutional neural network for local spatial feature extraction, so as to capture the influence of spatial attributes such as terrain and building distribution on signal propagation. For the service cell grid, a learnable embedding vector is superimposed on the basis of the extracted local spatial features. The vector is automatically adjusted through gradient descent in the training process, so that the model can adaptively enhance the expression of spatial features related to the service cell. The unmarked grid unit retains the original convolutional features, avoiding excessive interference of the features of non-key areas, and then the features of all grid units can be spliced to form a complete spatial feature vector containing spatial heterogeneity information, which highlights the local influence of the service cell while retaining the global spatial distribution pattern.

[0063] In some embodiments, dividing the geographic space data into a plurality of grid units includes dynamically adjusting the grid size according to the base station coverage density, or using a sliding window method to perform multi-scale division on the map; wherein a small size grid is used for high-density coverage area, and a large size grid is used for low-density coverage area. It can be understood that through the dynamic grid division strategy, the communication activity characteristics of different areas are adaptively matched, the flexibility and environmental adaptability of spatial feature extraction are enhanced, and the prediction accuracy is improved.

[0064] It is worth noting that the traditional method usually extracts global features or uniformly processes geospatial data, and cannot distinguish the spatial correlation difference between the service cell and the surrounding environment. The scheme dynamically adjusts the weight of the service cell grid while retaining the overall spatial structure, so that the spatial feature vector can more accurately represent the interaction between the core area and the surrounding environment. Through the combination of grid division and feature enhancement, not only the global spatial distribution features are retained, but also the key information of the area where the service cell is located is strengthened, avoiding the loss of core area features caused by uniform processing in traditional methods, so that the spatial feature vector can reflect the spatial correlation between the core area and the overall environment of network coverage, providing a more complete spatial information basis for subsequent cross-modal fusion.

[0065] Reference Figure 4 , Figure 4 In an embodiment of the communication index prediction method based on a multi-modal large model provided by the present application, a method flowchart for generating a joint feature representation; in some embodiments, based on a text prototype of a large language model, a cross-modal attention fusion is performed on the time series feature vector, the spatial feature vector and the semantic feature vector to generate a joint feature representation, including at least the following steps:

[0066] Step S410, generating a first text prototype and a second text prototype based on text dictionary data of a large language model;

[0067] Step S420, taking the time series feature vector as a query vector and the first text prototype as a key value vector, generating a time series feature of a text modality through attention weight calculation;

[0068] Step S430, taking the spatial feature vector as a query vector and the second text prototype as a key value vector, generating a spatial feature of a text modality;

[0069] Step S440, concatenating the time series feature of the text modality, the spatial feature of the text modality and the semantic feature vector to generate a multi-modal joint feature representation.

[0070] The text prototype can be a vector set with semantic representation extracted from the pre-training dictionary of the large language model, which can be constructed by selecting word embedding vectors corresponding to high-frequency words, and is used to establish a semantic mapping relationship of cross-modal features. Cross-modal attention fusion can be a mapping of different modal features to a unified semantic space through an attention mechanism, which specifically adopts a query-key attention calculation method to realize the alignment of non-text modal features and text prototypes. Attention weight calculation can be to determine the matching degree of the query vector and the key value vector through dot product operation combined with scaling operation, which can adopt a softmax function to obtain a weight distribution for weighted aggregation of the key value vector to generate modal fusion features.

[0071] In some embodiments, a large language model stores a large number of pre-trained vocabulary vectors in a text dictionary, and two groups of vocabulary vectors with semantic representation are selected from the pre-trained vocabulary vectors to construct a first text prototype and a second text prototype; the time sequence feature vector is taken as a query vector to perform attention calculation with the first text prototype to obtain a mapping representation of each time sequence feature in a text semantic space, forming a time sequence feature of the text modality; similarly, the spatial feature vector is taken as a query vector to perform attention calculation with the second text prototype to generate a spatial feature of the text modality; in this way, the time sequence feature and the spatial feature are converted into vector representations aligned with the text semantic space, eliminating the semantic gap between different modalities; and then the time sequence feature and the spatial feature of the text modality are concatenated with the original semantic feature vector in the feature dimension to form a joint feature representation containing multi-modal information, providing a unified semantic input for subsequent prediction tasks.

[0072] In some embodiments, generating the first text prototype and the second text prototype based on the text dictionary data of the large language model can include copying the dictionary of the large language model and the corresponding embedding vectors twice as the first text prototype and the second text prototype, and storing them in the proposed model; this step can be implemented by using the get_input_embeddings() method of the large language model class provided by Huggingface, and two text prototypes are obtained, which can be used in the feature extraction process of the time sequence data and the map data to convert their modalities from their respective modalities to the text modality, so that the large language model can understand and reason.

[0073] Further, the process of generating the time sequence feature of the text modality by taking the time sequence feature vector as a query vector and the first text prototype as a key vector through attention weight calculation can include: using a one-dimensional convolution layer and an MLP to extract a Token vector from each patch in the time sequence feature vector, and performing Cross Attention processing with the preset first text prototype to obtain the time sequence feature of the text modality, and the formula of Cross Attention can be:

[0074]

[0075]

[0076]

[0077]

[0078] wherein Q is the time sequence feature vector extracted in the foregoing obtained through a linear layer Linear, and K and V are obtained through a linear layer Linear from the first text prototype (i.e., P in the formula).​​​​

[0079] Further, the process of generating the spatial features of the text modality by taking the spatial feature vector as the query vector and the second text prototype as the key value vector can include: performing Cross Attention processing on the preset second text prototype to obtain the spatial features of the text modality. The Cross Attention here is consistent with the Cross Attention process in the time series data feature extraction stage, and only needs to replace with the spatial feature vector obtained above and replace P with the second text prototype.

[0080] It can be understood that through the above technical solutions, the semantic fragmentation problem between multi-modal data can be solved, so that the time series dynamic change features, geographical spatial distribution features and text semantic features can be jointly modeled in a unified semantic space. By establishing a cross-modal mapping relationship, the comprehensive analysis capability of the model on the multi-dimensional influence factors of the communication index is enhanced, and the accuracy and reliability of the communication index prediction in a complex network environment are improved.

[0081] Reference Figure 5 , Figure 5 In the multi-modal large model-based communication index prediction method provided by an embodiment of the present application, a method flowchart for obtaining geographical spatial data is provided. In some embodiments, the process of obtaining geographical spatial data at least includes the following steps:

[0082] Step S510, obtaining base station position data involved in the user communication process;

[0083] Step S520, determining a target map area according to the base station position data and a preset engineering parameter, and obtaining geographical spatial data corresponding to the target map area from a preset map database.

[0084] The base station position data can be three-dimensional coordinate information of a base station establishing a wireless connection with the user communication device, which can be obtained by using a base station positioning interface or a network topology database, and is used to establish a basic association between the user communication behavior and the physical location. The preset engineering parameter can be an effective action area of the base station signal propagation, which can be calculated and generated by using a cellular network hexagonal coverage model or a signal strength attenuation model, and is used to demarcate a spatial boundary related to the service capability of the base station. The preset map database can be a data set storing standardized geographical information, which can be a multi-layer database containing elevation data, building contour vector maps and terrain classification raster maps, and is used to provide key environmental feature data affecting the wireless signal propagation.

[0085] It can be understood that when the user initiates a communication request, the system obtains the location coordinates of the service base station by analyzing the signaling message. According to the transmission power of the base station and the propagation model, the effective coverage radius of the signal is calculated, for example, based on the free space loss formula to determine the standard coverage area with an engineering parameter of 1 km. After superimposing this engineering parameter on the geographic coordinate system, a spatial polygon containing the center point of the base station and the surrounding affected area is generated. By calling the pre-constructed map database interface, the rasterized data of all geographic features within the polygon range can be extracted, including terrain elevation values, building heights, and vegetation distribution density, etc. Layer information.

[0086] In some embodiments, the process of obtaining geospatial data in the user communication process can include: according to the base station position involved in the communication process recorded by the single user measurement report, combining the engineering parameters (such as latitude and longitude) of the base station, obtaining the map from the open source database OpenStreetMap website. For example, taking the base station position as the center, the standard coverage area map (such as 1km*1km) is expanded outwardly, wherein OpenStreetMap is an open source map library, only the latitude and longitude range of the target map needs to be calculated according to the latitude and longitude of the base station and the selected range size in advance, and the corresponding map can be framed by inputting in the website, and then the screenshot software is used for interception.

[0087] In some embodiments, the preset engineering parameters include the base station transmission power and the wireless propagation model; determining the target map area specifically includes: calculating the effective coverage radius of the signal based on the base station transmission power and the wireless propagation model; taking the base station position as the center, determining the range of the target map area according to the effective coverage radius of the signal; it can be understood that the signal coverage range is accurately calculated by combining the propagation model and the transmission power, which avoids the error of manually presetting the coverage range, makes the division of geospatial data more suitable for the actual network environment, and improves the accuracy of spatial feature representation.

[0088] It can be understood that through the above technical solutions, the present application can solve the problem of the correlation between geospatial data and user communication behavior, and based on the base station engineering parameters to determine the target area, the geographical feature interference of the communication irrelevant area can be excluded, and the subsequent spatial feature extraction is focused on the actual influence area of signal propagation. The structured data source provided by the preset map library enhances the integrity of the geographic features, so that the key environmental factors such as terrain undulation and building shielding can be accurately modeled, thereby improving the analysis ability of spatial attenuation effect in communication index prediction.

[0089] Reference Figure 6 , Figure 6In the method for predicting communication indicators based on a multi-modal large model provided by an embodiment of the present application, a flowchart of a method for obtaining text description data corresponding to multi-dimensional time series data is provided. In some embodiments, the process of obtaining text description data corresponding to multi-dimensional time series data includes at least the following steps:

[0090] Step S610: constructing dataset introduction text containing the meanings of multiple communication indicators in the multi-dimensional time series data;

[0091] Step S620: constructing task description text containing the definitions of historical sequence length and predicted sequence length in the communication indicator prediction;

[0092] Step S630: concatenating the dataset introduction text and the task description text to obtain the text description data.

[0093] The dataset introduction text can be a text passage that semantically explains each communication indicator in the multi-dimensional time series data. Specifically, natural language generation technology can be used to convert indicator names, units, and physical meanings into readable descriptions. For example, “RSRP” can be explained as “reference signal received power, reflecting the strength of the base station signal received by the user, with a unit of dBm”. This helps the large language model establish a mapping relationship between data features and semantic concepts by explicitly defining the meanings of indicators. The task description text can be a text passage that defines the time window for time series modeling tasks. Specifically, a structured template can be used to generate an explanatory statement containing the length of the historical sequence and the length of the predicted sequence. This limits the time range and constrains the modeling boundaries of the model in terms of the correlation between data time series.

[0094] Specifically, when generating the dataset introduction text, technical documents of each communication indicator in the multi-dimensional time series data are parsed to extract indicator names, measurement units, and physical meanings, and form standardized indicator explanation statements. For example, “RTT” can be converted to “network round-trip time, indicating the time for a data packet to travel from the sender to the receiver and back, with a unit of milliseconds”. Such statements can map technical parameters in the original data to semantic concepts that can be understood by the language model. When generating the task description text, based on the preset parameters of the prediction task, the time span of the historical sequence and the time span of the prediction target are defined, for example, “based on the communication indicators in the past 60 minutes to predict the indicator changes in the next 15 minutes”. Such descriptions provide context constraints for time series modeling for the model. By concatenating the two types of text, a complete text description containing data structure explanations and task parameter definitions is formed, allowing the large language model to understand the physical meaning of the time series data based on natural language prior knowledge and establish cross-modal feature associations within the preset time window.

[0095] In some embodiments, corresponding to steps S610 to S630, the present application can construct a dataset introduction text for the meaning of the communication data indicator, store it in the model, and by introducing the introduction text for the dataset in the prediction, introduce the background knowledge of the dataset through the understanding ability of the large language model for the text mode, thereby improving the prediction accuracy, and then splice the task description information after the introduction text. For example, the task description information can include the historical sequence length and the future sequence length. It can be understood that through the above technical solution, the present application solves the problem of insufficient semantic understanding caused by the lack of text mode in the traditional prediction model, enables the large language model to accurately analyze the technical meaning of each indicator in the multi-dimensional time series data, and based on the task description, clearly constrains the time window of time series modeling, thereby improving the reasoning accuracy for complex communication scenarios. For example, when predicting network congestion, the model can focus on the extraction of short-term burst traffic characteristics through the clear indication of "predicting the next 10 minutes" in the task description, avoiding prediction deviation caused by ambiguous time window.

[0096] Reference Figure 7 , Figure 7 In the communication indicator prediction method based on the multi-modal large model provided by an embodiment of the present application, a method flowchart for training a large language model is provided. In some embodiments, the training process of the large language model at least includes the following steps:

[0097] Step S710, dividing the multi-dimensional time series data in the training data into multiple samples according to a time sliding window, each sample containing an input sequence and an output sequence;

[0098] Step S720, calculating the prediction output of the large language model to be trained for the input sequence, and calculating the model loss according to the prediction output and the output sequence;

[0099] Step S730, updating the input-output mapping layer parameters of the large language model based on the model loss to obtain a trained large language model.

[0100] Wherein, the division of the time sliding window into multiple samples can be understood as the sliding segmentation of the continuous time series data according to a fixed window length, which can be realized by using a time series slicing method. Each window contains a historical input sequence and a corresponding future output sequence. This division method can preserve the dynamic time series correlation in the data and provide a clear mapping relationship between the historical state and the future prediction target for the model. The input-output mapping layer parameters can be the network layer parameters in the large language model responsible for converting the input features into the prediction output, which can be realized by using a linear transformation layer or a fully connected layer. This parameter updating method enables the model to focus on learning the non-linear mapping relationship from multi-modal features to prediction results, avoiding excessive adjustment of the bottom feature extraction layer and destroying the learned cross-modal correlation features. ​

[0101] In some embodiments, the training process of the large language model further comprises: freezing the backbone network parameters of the large language model, and only updating the parameters of the input mapping layer and the output mapping layer; the input mapping layer is used to adapt the multi-modal joint feature representation to the embedding space of the large language model, and the output mapping layer is used to map the inference vector of the large language model back to the time series data modality. It can be understood that by freezing the backbone network parameters, the training calculation cost is significantly reduced while the pre-training semantic ability of the large language model is retained, and the model training efficiency is improved.

[0102] Specifically, in the training phase, the original multi-dimensional time series data is cut into multiple samples containing input and output sequences, and these samples retain the dynamic time series pattern in the data through the time sliding window division method. The model generates a predicted output after receiving the input sequence, and the deviation of the model in capturing the time series features can be quantified by calculating the loss function between the predicted output and the real output sequence. When the model loss is backpropagated, only the input and output mapping layer parameters are adjusted, rather than updating the entire network parameters. This selective parameter update strategy enables the model to maintain the stability of the underlying feature extraction layer during optimization, while strengthening the mapping ability between the input features and the prediction target, thereby improving the generalization performance of the model in multi-modal data joint modeling.

[0103] In some embodiments, before training the large language model, the present application can obtain the large language model backbone network, for example, by obtaining the large language model as the prediction backbone through the Huggingface website. Python's transformers can realize the interaction with the Huggingface website, which provides the ModelConfig class and the Model class. In Python, only by setting the Config object and the Model object, the required large language model can be simply obtained, for example, LLaMA 3 7B. Further, the Tokenizer of the obtained large language model can be used as the Text Embedder to extract semantic feature vectors. It is worth noting that the large language model can provide a Tokenizer that can be used directly, which is responsible for text data segmentation and feature extraction, so that the Token processed by the Tokenizer can be directly processed by the large language model backbone

[0104] In some embodiments, dividing the multi-dimensional time series data in the training data into multiple samples according to the time sliding window can include: dividing the multi-dimensional time series data in the training data, dividing multiple samples according to the time sliding window, and dividing each sample into an input sequence and an output sequence, for example: the input sequence is 96 time points, and the output sequence is 24 time points.

[0105] It can be understood that, compared with the prior art, the traditional single-mode single-index prediction model usually adopts a global parameter updating strategy in the training process, which causes the bottom feature extraction layer and the high-level mapping layer to be adjusted at the same time, easily destroys the learned cross-modal correlation features, and this method still relies on a large amount of pre-training data. The scheme divides the samples, and limits the parameter updating range, so that the bottom feature extraction layer remains stable, and the input and output mapping layer learns the mapping rule of the multi-modal time sequence feature, effectively avoids the overfitting problem and improves the adaptability of the model to complex time sequence patterns.

[0106] In some embodiments, in the large language model inference stage corresponding to steps S710 to S730, the present application can input the time sequence features of the text modality corresponding to the sequence, the spatial features and the semantic feature vectors of the text modality, splice the multi-modal joint feature representation, and send it to the large language model backbone for inference and prediction to obtain the corresponding inference vector. The prediction output obtained by the above process And the true value Calculate the mean square error loss function Training, where N is the number of samples, H is the length of the prediction time window, and D is the number of data feature dimensions. Then, based on the model loss, update the input and output mapping layer parameters of the large language model, so that the large language model can map the inference vector back to the time sequence data modality through a linear layer as the final prediction output of the entire model, to obtain the trained large language model.

[0107] Through the above technical solution, the present application solves the problem that the existing model cannot effectively utilize the dynamic features of multi-dimensional time sequence data. By time sliding window division, the time sequence correlation is preserved, and by optimizing the parameters of specific network layers, the multi-modal data joint modeling capability is enhanced. The training mechanism enables the model to more accurately capture the coupling relationship between multi-modal indicators, thereby improving the accuracy of communication index prediction and providing a reliable basis for network optimization decision-making.

[0108] Reference Figure 8 , Figure 8The overall model flowchart of the communication index prediction method based on the multi-modal large model provided by an embodiment of the present application is provided. In some embodiments, the specific implementation process of the communication index prediction method based on the multi-modal large model can be as follows: multi-dimensional time series data, geographic space data, and corresponding text description data in the user communication process can be obtained; time series feature extraction processing is performed on the multi-dimensional time series data to obtain a time series feature vector, and space feature extraction processing is performed on the geographic space data to obtain a space feature vector; the text description data is encoded based on a large language model word embedding layer to obtain a semantic feature vector; a cross-modal attention fusion is performed on the three types of feature vectors based on a text prototype to generate a joint feature representation; and the joint feature representation is input into a large language model to generate communication index prediction information, which can effectively model the coupling relationship between multi-modal indexes, improve the prediction accuracy, and further ensure the effectiveness of network optimization decisions.

[0109] Reference Figure 9 , Figure 9 In the communication index prediction method based on the multi-modal large model provided by an embodiment of the present application, the overall flowchart for obtaining the space feature vector can be understood as follows: corresponding to steps S310 to S340, in the map data feature extraction stage: the map in the above is divided into small blocks, and according to the position information in the user measurement report, a 01 mark of “whether it is a service cell” is divided to each small block in the map, which is called a cell mark. If the user measurement report of the user is taken from the position of this small block, it is marked as 1, otherwise it is marked as 0. Then, a Patch Embedder module is used for feature extraction on each small block, i.e., two-dimensional convolution, multi-layer perception, and ReLU function. Then, only the small blocks with the cell mark of 1 can be additionally further extracted by a Cell Embedder module, i.e., a learnable CellEmbedding vector is added, to obtain a map Patch vector. The map Patch vector is the space feature vector mentioned in the above embodiment.

[0110] The application also provides a communication index prediction device based on a multi-modal large model, comprising: a data acquisition module for acquiring multi-dimensional time series data, geographical space data and text description data corresponding to the multi-dimensional time series data in the user communication process; a time series processing module for performing time series feature extraction processing on the multi-dimensional time series data to obtain a time series feature vector; a space processing module for performing spatial feature extraction processing on the geographical space data to obtain a spatial feature vector; a text processing module for encoding the text description data based on the word embedding layer of the pre-trained large language model to obtain a semantic feature vector; a feature fusion module for fusing the time series feature vector, the spatial feature vector and the semantic feature vector based on the text prototype of the large language model to generate a joint feature representation; and a prediction reasoning module for inputting the joint feature representation into the large language model to make the large language model obtain a corresponding reasoning vector based on the joint feature representation, and outputting communication index prediction information according to the reasoning vector.

[0111] The multi-dimensional time series data can be continuous monitoring data containing signal level, time delay, packet loss rate, retransmission rate and other communication indexes, and can be stored and called by a time series database to capture the dynamic change law of the communication environment. The geographical space data can be geographical information of base station positions and their coverage areas involved in the user communication process, and can be mapped by a geographic information system to represent the spatial distribution characteristics of the communication environment. The text description data can be explanatory text defining the meaning of the communication index and the prediction task, and can be constructed by natural language generation technology to provide the model with task understanding at the semantic level. The cross-modal attention fusion can use the text prototype of the large language model as a key-value vector to map the time series and spatial features to the text semantic space through attention weight calculation, and can be implemented by a multi-head attention mechanism to eliminate the semantic gap between different modalities. The joint feature representation can be a multi-dimensional vector that fuses the time series, spatial and semantic features, and can be generated by feature concatenation and normalization operations to provide a unified input representation for the large language model.

[0112] It can be understood that the data acquisition module synchronously collects the time sequence, space and text multi-source data of the user communication behavior, and realizes structured storage through a multi-dimensional time sequence database and a geographic information system. The time sequence processing module extracts local features of the time sequence data based on a convolutional neural network based on an attention mechanism, divides data blocks along the time dimension, and extracts dynamic change patterns. The spatial processing module divides the geographic space data into grid units, extracts local spatial features by convolution operation, and superimposes a learnable embedding vector on the service cell grid to enhance the spatial correlation. The text processing module converts the text description into a semantic vector using the word embedding layer of a large language model, and establishes a mapping relationship between the communication indicators and the natural language description. The feature fusion module takes the text prototype as a semantic bridge, aligns the time sequence and spatial features to the text semantic space through two cross-modal attention calculations, and then splices the semantic features to form a joint representation. The prediction and reasoning module inputs the joint features into the large language model for sequence reasoning, and outputs the prediction results including future network congestion, signal attenuation and other indicators.

[0113] It is worth noting that, compared with the prior art, the traditional communication indicator prediction device can only process a single type of time sequence data, uses an isolated model for single indicator prediction, and cannot model the spatio-temporal correlation between multi-modal data. The present scheme realizes the collaborative modeling of time sequence dynamic features, spatial distribution features and semantic task understanding by constructing a multi-modal data processing module and a cross-modal fusion mechanism. The existing device lacks the ability to structure the geographic space data, and does not introduce text description information to assist the model in understanding and predicting the task, making it difficult to capture the multi-factor coupling relationship in the complex network environment. The grid-based spatial feature extraction and text prototype guided attention fusion enable the large language model to integrate spatio-temporal features and task semantics for reasoning and decision-making, improving the accuracy of network congestion prediction and signal attenuation warning, and providing a reliable basis for dynamic network resource scheduling.

[0114] The implementation of the communication indicator prediction device based on the multi-modal large model is the same as the previous "communication indicator prediction method based on multi-modal large model", and specific reference is made to the previous description, which will not be repeated here.

[0115] Reference Figure 10 , Figure 10 The structure schematic diagram of the electronic device provided by an embodiment of the present application includes a memory, a processor and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the communication indicator prediction method based on the multi-modal large model of any one of the embodiments is realized, for example, the method steps S110 to S160 in the above description Figure 1 are executed, the method steps S210 to S220 in the above description Figure 2 are executed, Figure 3the method steps S310 to S340 in the method 300, Figure 4 the method steps S410 to S440 in the method 400, Figure 5 the method steps S510 to S520 in the method 500, Figure 6 the method steps S610 to S630 in the method 600, Figure 7 the method steps S710 to S730 in the method 700.

[0116] The electronic device 1000 of the embodiments of the present application includes one or more processors 1010 and memories 1020, Figure 10 In the method 300, the processor 1010 and the memory 1020 are taken as examples.

[0117] The processor 1010 and the memory 1020 can be connected through a bus or other means, Figure 10 In the method 300, the connection through the bus is taken as an example.

[0118] The memory 1020 as a kind of non-transient computer readable storage medium, it can be used to store non-transient software programs and non-transient computer executable programs.In addition, the memory 1020 can include high-speed random access memory, and can also include non-transient memory, such as at least one magnetic disk storage device, flash memory device or other non-transient solid-state memory device.In some embodiments, the memory 1020 can optionally include a memory 1020 arranged remotely relative to the processor 1010, and these remote memories can be connected to the electronic device 1000 through a network, and examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0119] In some embodiments, the processor executes the computer program according to the preset interval time to execute the communication index prediction method based on the multi-modal large model of any one of the above-mentioned embodiments.

[0120] Those skilled in the art can understand that, Figure 10 The device structure shown in the method 300 does not constitute a limitation on the electronic device 1000, and can include more or fewer components than those shown, or combine certain components, or different component arrangements.

[0121] In the method 300, Figure 10 In the electronic device 1000 shown in the method 300, the processor 1010 can be used to call the communication index prediction method based on the multi-modal large model stored in the memory 1020, so as to realize the communication index prediction based on the multi-modal large model.

[0122] Based on the hardware structure of the electronic device 1000, various embodiments of the communication index prediction apparatus based on the multi-modal large model of the present application are proposed, and at the same time, the non-transitory software programs and instructions required to implement the communication index prediction method based on the multi-modal large model of the above embodiments are stored in the memory, and when executed by the processor, the communication index prediction method based on the multi-modal large model of the above embodiments is executed.

[0123] The embodiments of the present application also provide a computer readable storage medium, which stores computer executable instructions for executing the communication index prediction method based on the multi-modal large model described above, so that the one or more processors execute the communication index prediction method based on the multi-modal large model of any one of the above embodiments, for example, execute the method steps S110 to S160 in the method described above Figure 1 Figure 2 The embodiments of the present application also provide a computer readable storage medium, which stores computer executable instructions for executing the communication index prediction method based on the multi-modal large model described above, so that the one or more processors execute the communication index prediction method based on the multi-modal large model of any one of the above embodiments, for example, execute the method steps S210 to S220 in the method described above Figure 3 Figure 4 The embodiments of the present application also provide a computer readable storage medium, which stores computer executable instructions for executing the communication index prediction method based on the multi-modal large model described above, so that the one or more processors execute the communication index prediction method based on the multi-modal large model of any one of the above embodiments, for example, execute the method steps S410 to S440 in the method described above Figure 5 Figure 6 The embodiments of the present application also provide a computer readable storage medium, which stores computer executable instructions for executing the communication index prediction method based on the multi-modal large model described above, so that the one or more processors execute the communication index prediction method based on the multi-modal large model of any one of the above embodiments, for example, execute the method steps S610 to S630 in the method described above Figure 7

[0124] The above described device embodiments are only schematic, and the units described as separate components can or can not be physically separate, that is, they can be located in one place, or can be distributed on multiple network nodes. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0125] ​​​​As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, etc. in the methods disclosed above can be embodied in software, firmware, hardware, and / or suitable combinations thereof. Some or all of the physical components can be implemented with a microprocessor, such as a central processing unit, a digital signal processor, or a microprocessor executing software, or with hardware, or with an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer readable storage media (or non-transitory media), and communication media (or transitory media). As is known to those of ordinary skill in the art, computer readable storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0126] The above description is that of the preferred embodiments of the present application. Various equivalents substitutions and modifications can be conceived by those skilled in the art without departing from the spirit of the present application, and these equivalents substitutions and modifications are included in the scope of the claims.

Claims

1. A communication index prediction method based on a multimodal large model, characterized in that: include: Obtain multi-dimensional time series data, geospatial data, and text description data corresponding to the multi-dimensional time series data during user communication; wherein the geospatial data is geographic information data including base station location, terrain elevation, and building distribution, and the text description data is natural language text describing communication scenario characteristics; The process of acquiring the geographic spatial data includes: acquiring the base station location data involved in the user communication process; determining a target map area based on the base station location data and preset engineering parameters, and acquiring the geographic spatial data corresponding to the target map area from a preset map library; The preset engineering parameters include base station transmission power and a wireless propagation model, and determining the target map area based on the base station location data and the preset engineering parameters includes: calculating a signal effective coverage radius based on the base station transmission power and the wireless propagation model; and determining the target map area based on the signal effective coverage radius with the base station location as the center; Performing time series feature extraction processing on the multi-dimensional time series data to obtain a time series feature vector; wherein, performing time series feature extraction processing on the multi-dimensional time series data to obtain a time series feature vector includes: constructing a convolutional neural network based on an attention mechanism according to the feature dimension of the multi-dimensional time series data; dividing the multi-dimensional time series data into multiple local time series blocks along the time series dimension, and extracting local time series features from each of the local time series blocks through the convolutional neural network, and obtaining a time series feature vector based on all the local time series features; the local time series blocks are continuous data segments divided according to a preset time window length; Performing spatial feature extraction processing on the geographic spatial data to obtain a spatial feature vector; wherein the performing spatial feature extraction processing on the geographic spatial data to obtain the spatial feature vector includes: dividing the geographic spatial data into multiple grid units according to a grid size dynamically adjusted according to base station coverage density, and marking the service cell grids involved in the user communication process according to the location information in the user measurement report; performing convolution feature extraction on all the grid units to generate local spatial features; superimposing a learnable embedding vector on the local spatial features corresponding to all the service cell grids to obtain local enhanced features; and obtaining the spatial feature vector based on the local spatial features of all unmarked grid units and the local enhanced features of all the service cell grids; Encoding the text description data based on the word embedding layer of a pre-trained large language model to obtain a semantic feature vector; Based on the text prototype of the large language model, cross-modal attention fusion is performed on the temporal feature vector, the spatial feature vector, and the semantic feature vector to generate a joint feature representation; The joint feature representation is input into the large language model, so that the large language model obtains a corresponding inference vector based on the joint feature representation, and outputs communication indicator prediction information according to the inference vector.

2. The communication index prediction method based on the multimodal large model according to claim 1 is characterized in that: The text prototype based on the large language model performs cross-modal attention fusion on the temporal feature vector, the spatial feature vector, and the semantic feature vector to generate a joint feature representation, including: generating a first text prototype and a second text prototype based on the text dictionary data of the large language model; Using the time series feature vector as the query vector and the first text prototype as the key value vector, the time series features of the text modality are generated through attention weight calculation; Using the spatial feature vector as the query vector and the second text prototype as the key-value vector, the spatial feature of the text modality is generated; The temporal features of the text modality, the spatial features of the text modality and the semantic feature vector are spliced ​​to generate a multimodal joint feature representation.

3. The communication index prediction method based on the multimodal large model according to claim 1 is characterized in that: The process of obtaining the text description data corresponding to the multi-dimensional time series data includes: Constructing a data set introduction text containing the meanings of multiple communication indicators in the multi-dimensional time series data; Construct a task description text that includes the definition of the length of the historical sequence and the length of the predicted sequence in the communication indicator prediction; The dataset introduction text and the task description text are concatenated to obtain text description data.

4. The communication index prediction method based on the multimodal large model according to claim 1 is characterized in that: The training process of the large language model includes: The multi-dimensional time series data in the training data is divided into multiple samples according to the time sliding window, and each sample contains an input sequence and an output sequence; Calculating the predicted output of the large language model to be trained for the input sequence, and calculating the model loss based on the predicted output and the output sequence; The input-output mapping layer parameters of the large language model are updated based on the model loss to obtain a trained large language model.

5. A communication index prediction device based on a multimodal large model, characterized in that: include: A data acquisition module is used to acquire multi-dimensional time series data, geospatial data, and text description data corresponding to the multi-dimensional time series data during the user communication process; wherein the geospatial data is geographic information data including base station location, terrain elevation, and building distribution; and the text description data is natural language text describing the characteristics of the communication scenario; The process of acquiring the geographic spatial data includes: acquiring the base station location data involved in the user communication process; determining a target map area based on the base station location data and preset engineering parameters, and acquiring the geographic spatial data corresponding to the target map area from a preset map library; The preset engineering parameters include base station transmission power and a wireless propagation model, and determining the target map area based on the base station location data and the preset engineering parameters includes: calculating a signal effective coverage radius based on the base station transmission power and the wireless propagation model; and determining the target map area based on the signal effective coverage radius with the base station location as the center; A time series processing module, configured to perform time series feature extraction processing on the multi-dimensional time series data to obtain a time series feature vector; wherein the above-mentioned time series feature extraction processing on the multi-dimensional time series data to obtain the time series feature vector includes: constructing a convolutional neural network based on the attention mechanism according to the feature dimension of the multi-dimensional time series data; dividing the multi-dimensional time series data into multiple local time series blocks along the time series dimension, and extracting local time series features from each of the local time series blocks through the convolutional neural network, and obtaining a time series feature vector based on all the local time series features; the local time series block is a continuous data segment divided according to a preset time window length; A spatial processing module is configured to perform spatial feature extraction processing on the geographic spatial data to obtain a spatial feature vector; wherein the performing spatial feature extraction processing on the geographic spatial data to obtain the spatial feature vector comprises: dividing the geographic spatial data into a plurality of grid cells according to a grid size dynamically adjusted according to base station coverage density, and marking the serving cell grids involved in the user communication process according to the location information in the user measurement report; performing convolution feature extraction on all the grid cells to generate local spatial features; superimposing a learnable embedding vector on the local spatial features corresponding to all the serving cell grids to obtain local enhanced features; and obtaining the spatial feature vector based on the local spatial features of all unmarked grid cells and the local enhanced features of all the serving cell grids; A text processing module, configured to encode the text description data based on a word embedding layer of a pre-trained large language model to obtain a semantic feature vector; a feature fusion module, configured to perform cross-modal attention fusion on the temporal feature vector, the spatial feature vector, and the semantic feature vector based on the text prototype of the large language model to generate a joint feature representation; The prediction and reasoning module is used to input the joint feature representation into the large language model so that the large language model obtains a corresponding reasoning vector based on the joint feature representation, and outputs communication indicator prediction information according to the reasoning vector.

6. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the communication indicator prediction method based on the multimodal large model as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and the computer-executable instructions are used to execute the communication indicator prediction method based on the multimodal large model as described in any one of claims 1 to 4.

Images