Multimodal spatial data fusion method based on large model technology

Abstract

The present invention discloses a multimodal spatial data fusion method based on a large model technology. The present invention relates to the field of multimodal spatial data technology and solves the problem that multimodal data are all generated at different time points and fusion errors are very likely to occur during the actual fusion process. The present invention confirms the multimodal spatial data associated with different spatial feature points, preferentially determines a first set of data sets, and then determines the time difference of adjacent selection types based on the relevant features of the first set of data sets, and then confirms other subsequent multimodal data sets of the same spatial feature points in turn based on the output frequencies of different multimodal data. In this way, it can be effectively guaranteed that the corresponding data set will not be confused, and it can be effectively guaranteed that the data associated with its spatial model in the subsequent updating process does not have any errors. The subsequent different data sets can be effectively confirmed in turn to achieve a better selection effect.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal spatial data, and in particular to a multimodal spatial data fusion method based on large model technology. Background Art

[0002] Multimodal spatial data refers to a data set composed of multiple different types of data modalities in a spatial environment; these data modalities can be obtained through different sensors, collection methods or data sources, and they jointly describe objects, phenomena or processes in space; the spatial dimension is a key link for these data, allowing data of different modalities to be related to each other in the same geographic space or spatial reference system.

[0003] The application with announcement number CN113378989B discloses a multimodal data fusion method based on a complex collaborative structure feature recombination network; the existing multimodal data fusion technology mainly focuses on direct feature fusion while ignoring the two-way interaction between modalities, and there is a problem of semantic gap between features during multimodal fusion. The invention uses a deep neural network to extract unimodal features of images and texts, establishes a two-way interactive attention model of images and texts based on the transformer mechanism, mines the feature connection between images and texts, performs multimodal semantic association, and introduces a complex collaborative structure network to deepen the connection of interactive information between modalities, performs two-way feature recombination under multimodal deep fusion, realizes the alignment of image and text semantic space, better adapts to the neural network's search for complementary information between different modalities, enhances the model's understanding and generalization ability of multimodal semantics, and further improves the classification accuracy of the multimodal feature network.

[0004] For the data fusion process of multimodal spatial data, based on the different multimodal data associated with different spatial feature points, the multiple groups of multimodal data associated with the corresponding feature points are directly fused to complete the specific data fusion process. However, in the actual processing process, the data sources associated with the multimodal data have different frequencies, so the multimodal data are generated at different time points. In the actual fusion process, fusion errors are very likely to occur, resulting in the inability to guarantee the data output effect of the model. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a multimodal spatial data fusion method based on large model technology, which solves the problem that multimodal data are generated at different time points and fusion errors are likely to occur in the actual fusion process.

[0006] To achieve the above objectives, the present invention is implemented by the following technical scheme: A multimodal spatial data fusion method based on large model technology comprises the following steps:

[0007] Step 1: Based on the preset spatial model, determine the multimodal data associated with different spatial feature points in the spatial model. Based on the determination result, lock the multimodal data set associated with the corresponding spatial feature point. Then, based on the data generation frequency of the corresponding multimodal data source, confirm the multimodal data set generated by the specified spatial feature point in the subsequent stage in turn. The sub-steps are:

[0008] S11, based on the preset spatial model and the internally set spatial feature points, different multimodal data associated with the spatial feature points are confirmed from the generated multimodal data, the multimodal data contains preset spatial features, the spatial features correspond to the spatial feature points, and the first group of multimodal data sets confirmed by the corresponding spatial feature points are confirmed by the relevant operators;

[0009] S12, using the first group of multimodal data sets confirmed based on the spatial feature points as a feature data set, confirming the types of different multimodal data in the feature data set, using them as selected types, and recording the time difference between adjacent selected types in the feature data set, using the time difference as the feature time of adjacent multimodal data;

[0010] S13, confirm the data generation frequency of the multimodal data source associated with the corresponding selection type, and then confirm the data generation frequency difference associated with the corresponding adjacent selection types based on the feature time between the corresponding adjacent selection types, and convert: feature time + data generation frequency difference = selection time. Based on the selection type confirmed by this feature data set and the selection time confirmed between the corresponding adjacent selection types, data is selected from several groups of multimodal data subsequently associated with the feature point of this space, and used as the second group of multimodal data sets of the feature point of this space. The specific sub-steps are:

[0011] S131. According to the determined selection type, randomly select data from a plurality of groups of multimodal data to generate a plurality of pending data sets;

[0012] S132, based on the selection time between the corresponding adjacent selection types, confirm the time difference between the corresponding selection types in the corresponding pending data set, and when there is a pending data set: the time difference of all the selection types is consistent with the selection time, then this pending data set is used as the second group of multimodal data sets;

[0013] S14, calibrating the second group of multimodal data sets as feature data sets, and selecting subsequent multimodal data sets in sequence in the same manner as steps S12-S13;

[0014] Step 2: According to the different multimodal data sets associated with different spatial feature points, and according to the data types that can be read by this spatial model, the types of different multimodal data in the multimodal data set are confirmed, and the optimal data type of this spatial feature point is determined according to the type conversion rate associated with the historical data, and data fusion is performed. The specific sub-steps are as follows:

[0015] S21, determining different selection types from the corresponding multimodal data set within the spatial feature point, and then confirming the data type that can be read by the present spatial model. If there are multiple groups of data types, executing step S22; if there is only one group of data types, directly unifying the types of multimodal data of different selection types from the data type. After completing the type unification, directly merging the multiple groups of multimodal data of the present multimodal data set;

[0016] S22, randomly select a group of data types from multiple groups of data types as the pending type, identify the conversion rate of the selected type to the pending type from historical data, select the minimum value and the maximum value from the identified groups of conversion rates, confirm the rate interval of the conversion of the selected type to the pending type, and then determine the conversion rate of the selected type to the pending type according to the multimodal data capacity R associated with the selected type. i , where i represents different selection types, according to the multimodal data capacity R i and the confirmed rate range, locking the time range associated with this selected type;

[0017] Then confirm the time intervals corresponding to other selected types in turn, then sum the minimum values ​​of several confirmed time intervals to confirm the minimum eigenvalue, then sum the maximum values ​​of several time intervals to confirm the maximum eigenvalue, and confirm the characteristic interval of this undetermined type based on the minimum eigenvalue and the maximum eigenvalue;

[0018] S23, mark the other data types as pending types in turn, and confirm the feature intervals associated with the other pending types in the same manner as step S22;

[0019] S24, based on different feature intervals associated with different pending types, the feature intervals are calibrated as [Q k min,Q k max], where k represents different pending types, using TZ k =Q k min×C1+Q k max×C2 confirms the eigenvalue TZ associated with the pending type k , where C1 and C2 are preset fixed coefficient factors, from the different eigenvalues ​​TZ associated with different types to be determined k In the menu, select TZ kThe undetermined type associated with min is used as the optimal data type of this spatial feature point, and based on the determined optimal data type, the multimodal data set associated with this spatial feature point is unified in type, and the multimodal data set with unified type is fused;

[0020] Step 3: Synchronously fuse the multimodal data sets of different spatial feature points, and confirm the real-time fusion rate based on the specific fusion process. Then, according to the correlation difference of the fusion rate, adjust the internal computing power in real time so that several different spatial feature points can complete the fusion process within the set time difference. The specific sub-steps are:

[0021] S31, real-time determination of the total data capacity Z of different multimodal data sets in different spatial feature points q , where q represents different spatial feature points. Based on the computing power held by different spatial feature points, the associated multimodal data sets are fused and the fusion rate V is confirmed in real time. q , using T q =Z q ÷V q Confirm its associated time T q ;

[0022] S32, the associated time T of the confirmed several spatial feature points q Sort by groups of associated time T. q Maximum confirmation time T q max and time minimum value T q min, if (T q max-T q min)>Y1, where Y1 is the preset value, then the minimum time value T q min is associated with the spatial feature point holding power to improve (T q max-T q min)≤Y1, and based on the real-time fusion monitoring process, the holding computing power of different spatial feature points is adjusted in real time to complete the relevant process of data fusion.

[0023] Preferably, if (T q max-T q min)≤Y1, the real-time fusion process is monitored.

[0024] Preferably, in step S31, the total data capacity value Z q It will gradually become smaller with the corresponding data fusion process.

[0025] The present invention provides a multimodal spatial data fusion method based on large model technology. Compared with the prior art, it has the following beneficial effects:

[0026] The present invention confirms the multimodal spatial data associated with different spatial feature points, prioritizes the first set of data sets, and then determines the time difference of adjacent selection types based on the relevant features of the first set of data sets, and then confirms other subsequent multimodal data sets of the same spatial feature points in sequence based on the output frequency of different multimodal data. In this way, it can be effectively guaranteed that the corresponding data sets will not be confused, and it can be effectively guaranteed that there are no errors in the data associated with the spatial model during the subsequent update process, and different subsequent data sets can be effectively confirmed in sequence to achieve better selection effects.

[0027] Subsequently, before data fusion, the data features of the corresponding spatial feature points are confirmed, and the best data type is selected to ensure the fastest conversion rate of the corresponding spatial feature points, thereby fully ensuring the subsequent data fusion rate and achieving a better processing method;

[0028] The method of real-time confirmation of the time of the corresponding spatial feature points is used to confirm the specific fusion situations associated with different fusion processes in real time, and then the computing power associated with the fusion process is adjusted in real time based on the time sequence, so as to ensure the corresponding real-time fusion effect and ensure that the corresponding spatial model can complete the corresponding overall data fusion process within a shorter time difference. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the present invention;

[0030] Figure 2 This is a schematic diagram of adjusting the computing power of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] First embodiment

[0033] See also Figure 1 , the present application provides a multimodal spatial data fusion method based on large model technology, comprising the following steps:

[0034] Step 1: Based on a preset spatial model, determine the multimodal data associated with different spatial feature points in the spatial model. Based on the determination result, lock the multimodal data set associated with the corresponding spatial feature point. Subsequently, based on the data generation frequency of the corresponding multimodal data source source, confirm the multimodal data set generated by the specified spatial feature point in the subsequent stage in turn. Specifically, the spatial model is a preset model, which is prepared in advance by relevant operators, and there are multiple different spatial feature points in the corresponding spatial model, which are also calibrated in advance in the corresponding model. The corresponding multimodal data corresponding to different feature points are subsequently combined based on the associated corresponding features to confirm the multimodal data set associated with the corresponding feature point.

[0035] The specific method of confirming the multimodal data set associated with the spatial feature points is as follows:

[0036] S11. Based on the preset spatial model and the spatial feature points set internally, different multimodal data associated with the spatial feature points are confirmed from the generated multimodal data. The multimodal data has preset spatial features, and the spatial features correspond to the spatial feature points. The first set of multimodal data sets confirmed by the corresponding spatial feature points are confirmed by the relevant operators. Because different multimodal data have different output frequency characteristics, some data may have been generated in two or three groups, and the subsequent data with the same position features have not been generated. Therefore, the operator needs to give priority to confirming the first set of multimodal data sets. For example, there is a spatial feature in its spatial model. Feature point A, point A needs to fuse three sets of multimodal data, and the three sets of multimodal data are proposed to be A1, A2 and A3, where the output frequency of A1 is once per second, the output frequency of A2 is once per 2 seconds, and the output frequency of A3 is once per 5 seconds. In this case, A1 outputs five sets and A3 outputs only one set. Therefore, when the system selects itself, it is easy to directly select two sets of A1 and one set of A2 as the data to be fused this time. Then there is no A3, which will cause data errors. In order to avoid such situations, the first set of data sets corresponding to the spatial feature points is fitted by the operator based on the preset spatial features to confirm the first set of data sets belonging to this spatial feature point;

[0037] S12, based on the first group of multimodal data sets confirmed by the spatial feature points as the feature data set, confirm the types of different multimodal data in the feature data set, use them as the selected types, and record the time difference between adjacent selected types in the feature data set (different multimodal data are all accompanied by corresponding timestamps after generation, and the absolute value of the difference between their timestamps is the corresponding time difference), and use this time difference as the feature time of adjacent multimodal data;

[0038] S13, confirm the data generation frequency of the multimodal data source associated with the corresponding selection type, and then confirm the data generation frequency difference associated with the corresponding adjacent selection types based on the feature time between the corresponding adjacent selection types (the data generation frequency difference is determined in the same way as the time difference between the selection types), and convert: feature time + data generation frequency difference = selection time. Based on the selection type confirmed by this feature data set and the selection time confirmed between the corresponding adjacent selection types, data is selected from several groups of multimodal data subsequently associated with the feature point of this space, and used as the second group of multimodal data sets of the feature point of this space, wherein the specific method of data selection is:

[0039] S131. According to the determined selection type, randomly select data from a plurality of groups of multimodal data to generate a plurality of pending data sets;

[0040] S132, based on the selection time between the corresponding adjacent selection types, confirm the time difference between the corresponding selection types in the corresponding pending data set. When there is a pending data set: the time difference of all selection types is consistent with the selection time, then this pending data set is used as the second group of multimodal data sets. Specifically, combined with specific examples, first, different spatial feature points have different multimodal data, and their data sources have different frequency characteristics. When the characteristics are consistent, it is easy to confirm that when the time difference of the associated frequency characteristics is 0, according to the determined selection types A1, A2 and A3, select from the subsequent data to ensure that their time differences are all 0. However, in the actual verification selection process, the time differences are not all 0. Combined with the example content of the above step S11, the frequency of A1 is 1 second, the frequency of A2 is 2 seconds, and the frequency of A3 is 5 seconds. After the first group of data sets is determined, it is planned to be {A1, A2, A3}, A1-A 2 is 1, and the characteristic time between A2-A3 is 3. The frequency difference is also: A1-A2 is 1, A2-A3 is 3, then the confirmed selection time A1-A2 is 2, A2-A3 is 6, then in the subsequent selection, the data set can be confirmed and selected based on this time difference, and there will be no confusion. The initial time is set to 0, and the timestamp of the first set of data sets A1-A2-A3 is: 1-2-5, and the timestamp of the next set of data sets A1-A2-A3 is: 2-4-10, and the next set is determined based on the time characteristics of the previous set, that is: 3-6-15, and so on. The subsequent selected data sets will not be chaotic, because in some cases, there is a data update, and the data of the spatial model is updated in real time. If an error occurs during the fusion of multimodal data, it will cause an update error. Therefore, this method can effectively confirm the subsequent different data sets in turn to achieve a better selection effect;

[0041] S14, calibrating the second group of multimodal data sets as feature data sets, and selecting subsequent multimodal data sets in sequence in the same manner as steps S12-S13;

[0042] Step 2: According to the different multimodal data sets associated with different spatial feature points, and according to the data types that can be read by this spatial model, the types of different multimodal data in the multimodal data set are confirmed, and the optimal data type of this spatial feature point is determined according to the type conversion rate associated with the historical data, and data fusion is performed (the data fusion method is to integrate multiple groups of multimodal data and put them into the same batch for operation and display. Since this type of processing method is the same as the multimodal data method associated with the prior art, it will not be described in detail here). The specific sub-steps for determining the optimal data type of the spatial feature point are:

[0043] S21, determining different selection types from the multimodal data set corresponding to the spatial feature point, and then confirming the data type that can be read by the present spatial model. If there is only one set of data types, then directly unify the types of multimodal data of selection types different from the data types. After completing the type unification, directly fuse multiple groups of multimodal data of the present multimodal data set. If there are multiple groups of data types, execute step S22;

[0044] S22, randomly select a group of data types from multiple groups of data types as the pending type, identify the conversion rate of the selected type to the pending type from historical data, select the minimum value and the maximum value from the identified groups of conversion rates, confirm the rate interval of the conversion of the selected type to the pending type, and then determine the conversion rate of the selected type to the pending type according to the multimodal data capacity R associated with the selected type. i , where i represents different selection types, according to the multimodal data capacity R i and the confirmed rate range, locking the time range associated with this selected type;

[0045] Then confirm the time intervals corresponding to other selected types in turn, then sum the minimum values ​​of several confirmed time intervals to confirm the minimum eigenvalue, then sum the maximum values ​​of several time intervals to confirm the maximum eigenvalue, and confirm the characteristic interval of this undetermined type based on the minimum eigenvalue and the maximum eigenvalue;

[0046] S23, mark the other data types as pending types in turn, and confirm the feature intervals associated with the other pending types in the same manner as step S22;

[0047] S24, based on different feature intervals associated with different pending types, the feature intervals are calibrated as [Q k min,Q kmax], where k represents different pending types, using TZ k =Q k min×C1+Q k max×C2 confirms the eigenvalue TZ associated with the pending type k , where C1 and C2 are preset fixed coefficient factors, and their specific values ​​are determined by the operator based on experience, from the different characteristic values ​​TZ associated with different types to be determined k In the menu, select TZ k The undetermined type associated with min is used as the optimal data type of this spatial feature point, and based on the determined optimal data type, the multimodal data set associated with this spatial feature point is unified in type, and the multimodal data set with unified type is fused;

[0048] Specifically, the multimodal data set confirmed by the corresponding spatial feature points is proposed to be {A1, A2, A3}, among which the data types acceptable to this spatial model are B1 and B2. B1 is preferentially selected as the pending type. From the historical data, the conversion rates of A1-B1, A2-B1 and A3-B1 are confirmed. The minimum and maximum values ​​are selected from the confirmed groups of conversion rates, so that the corresponding rate interval can be confirmed. The corresponding multimodal data has a corresponding data capacity. Based on the corresponding data capacity and rate interval, the time interval associated with the corresponding multimodal data can be locked, so that the corresponding feature interval can be confirmed. Subsequently, B2 is used as the pending type, and the feature interval associated with this pending type B2 is simultaneously confirmed. Based on the feature intervals confirmed by B1 and B2, the corresponding feature values ​​are confirmed, so as to select the best data type to ensure the fastest conversion rate of the corresponding spatial feature points, thereby fully ensuring the subsequent data fusion rate and achieving a better processing method.

[0049] Second embodiment

[0050] In the specific implementation process of this embodiment, compared with the above embodiment, this embodiment mainly performs relevant processing on a specific fusion process. In order to ensure that the corresponding models can complete the corresponding data fusion process within a unified timeline, the rates of different spatial feature points are monitored and adjusted in real time;

[0051] Step 3: Combination Figure 2 , the multimodal data sets of different spatial feature points are synchronously fused, and the real-time fusion rate is confirmed based on the specific fusion process. Then, according to the correlation difference of the fusion rate, the internal computing power is adjusted in real time, so that several different spatial feature points can complete the fusion process within the set time difference. The specific sub-steps of real-time adjustment are:

[0052] S31, real-time determination of the total data capacity Z of different multimodal data sets in different spatial feature points q , where q represents different spatial feature points. Based on the computing power held by different spatial feature points, the associated multimodal data sets are fused and the fusion rate V is confirmed in real time. q , using T q =Z q ÷V q Confirm its associated time T q , its Z q It will gradually become smaller as the corresponding data fusion process progresses;

[0053] S32, the associated time T of the confirmed several spatial feature points q Sort by groups of associated time T. q Maximum confirmation time T q max and time minimum value T q min, if (T q max-T q min)>Y1, where Y1 is a preset value, and its specific value is determined by the operator based on experience. q min is associated with the spatial feature point holding power to improve (T q max-T q min)≤Y1, and based on the real-time fusion process, the computing power held by different spatial feature points is adjusted in real time to complete the relevant process of data fusion (if the corresponding Tq confirmed later still has such a situation, the same method is used for adjustment to reduce the time difference in real time to achieve the effect of synchronously completing the fusion process).

[0054] Specifically, the method of real-time confirmation of the time of the corresponding spatial feature points is adopted to confirm the specific fusion situations associated with different fusion processes in real time, and then the computing power associated with the fusion process is adjusted in real time based on the time sequence, so as to ensure the corresponding real-time fusion effect and ensure that the corresponding spatial model can complete the corresponding overall data fusion process within a shorter time difference.

[0055] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0056] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A multimodal spatial data fusion method based on large model technology, characterized in that: The following steps are involved: Step 1: Based on the preset spatial model, determine the multimodal data associated with different spatial feature points in the spatial model. Based on the determination result, lock the multimodal data set associated with the corresponding spatial feature point. Then, based on the data generation frequency of the corresponding multimodal data source, confirm the multimodal data set generated by the specified spatial feature point in the subsequent stage in turn. The sub-steps are: S11, based on the preset spatial model and the internally set spatial feature points, different multimodal data associated with the spatial feature points are confirmed from the generated multimodal data, the multimodal data contains preset spatial features, the spatial features correspond to the spatial feature points, and the first group of multimodal data sets confirmed by the corresponding spatial feature points are confirmed by the relevant operators; S12, using the first group of multimodal data sets confirmed based on the spatial feature points as a feature data set, confirming the types of different multimodal data in the feature data set, using them as selected types, and recording the time difference between adjacent selected types in the feature data set, using the time difference as the feature time of adjacent multimodal data; S13, confirm the data generation frequency of the multimodal data source associated with the corresponding selection type, and then confirm the data generation frequency difference associated with the corresponding adjacent selection types based on the feature time between the corresponding adjacent selection types, and convert: feature time + data generation frequency difference = selection time, based on the selection type confirmed by this feature data set and the selection time confirmed between the corresponding adjacent selection types, select data from several groups of multimodal data subsequently associated with the feature point of this space, and use them as the second group of multimodal data sets of the feature point of this space; S14, calibrating the second group of multimodal data sets as feature data sets, and selecting subsequent multimodal data sets in sequence in the same manner as steps S12-S13; The spatial dimension is a key link in these data, allowing data from different modalities to be related to each other in the same geographic space or spatial reference system; Step 2: According to the different multimodal data sets associated with different spatial feature points, and according to the data types that can be read by this spatial model, the types of different multimodal data in the multimodal data sets are confirmed, and the optimal data type of this spatial feature point is determined according to the type conversion rate associated with the historical data, and data fusion is performed; Step 3: Synchronously fuse the multimodal data sets of different spatial feature points, and confirm the real-time fusion rate based on the specific fusion process. Then, according to the correlation difference of the fusion rate, adjust the internal computing power in real time so that several different spatial feature points can complete the fusion process within the set time difference.

2. The multimodal spatial data fusion method based on large model technology according to claim 1 is characterized in that: In step S13, the specific sub-steps of selecting data from the subsequent associated groups of multimodal data are: S131, randomly selecting data from a plurality of groups of multimodal data according to the determined selection type, to generate a plurality of pending data sets; S132. Based on the selection time between corresponding adjacent selection types, confirm the time difference between corresponding selection types in the corresponding pending data set. When there is a pending data set in which the time difference of all selection types is consistent with the selection time, use this pending data set as the second multimodal data set.

3. The multimodal spatial data fusion method based on large model technology according to claim 1 is characterized in that: In step 2, the specific sub-steps for determining the optimal data type of the spatial feature point are: S21, determining different selection types from the multimodal data set corresponding to the spatial feature point, and then confirming the data types that can be read by the spatial model. If there are multiple groups of data types, executing step S22; S22, randomly select a group of data types from multiple groups of data types as the pending type, identify the conversion rate of the selected type to the pending type from historical data, select the minimum value and the maximum value from the identified groups of conversion rates, confirm the rate interval of the conversion of the selected type to the pending type, and then determine the conversion rate of the selected type to the pending type according to the multimodal data capacity R associated with the selected type. i , where i represents different selection types, according to the multimodal data capacity R i and the confirmed rate range, locking the time range associated with this selected type; Then confirm the time intervals corresponding to other selected types in turn, then sum the minimum values ​​of several confirmed time intervals to confirm the minimum eigenvalue, then sum the maximum values ​​of several time intervals to confirm the maximum eigenvalue, and confirm the characteristic interval of this undetermined type based on the minimum eigenvalue and the maximum eigenvalue; S23, mark the other data types as pending types in turn, and confirm the feature intervals associated with the other pending types in the same manner as step S22; S24, based on different feature intervals associated with different pending types, the feature intervals are calibrated as [Q k min,Q k max], where k represents different pending types, using TZ k =Q k min×C1+Q k max×C2 confirms the eigenvalue TZ associated with the pending type k , where C1 and C2 are preset fixed coefficient factors, from the different eigenvalues ​​TZ associated with different types to be determined k In the menu, select TZ k The undetermined type associated with min is used as the optimal data type for this spatial feature point, and based on the determined optimal data type, the multimodal data sets associated with this spatial feature point are unified in type, and the multimodal data sets that have completed type unification are fused.

4. The multimodal spatial data fusion method based on large model technology according to claim 3 is characterized in that: In the step S21, if there is only one group of data types, the multimodal data of a selected type different from the data type are directly unified in type. After the type unification is completed, the multiple groups of multimodal data of the multimodal data set are directly fused.

5. The multimodal spatial data fusion method based on large model technology according to claim 1 is characterized in that: In step 3, the specific sub-steps for real-time adjustment of the internal computing power are: S31, real-time determination of the total data capacity Z of different multimodal data sets in different spatial feature points q , where q represents different spatial feature points. Based on the computing power held by different spatial feature points, the associated multimodal data sets are fused and the fusion rate V is confirmed in real time. q , using T q =Z q ÷V q Confirm its associated time T q ; S32, the associated time T of the confirmed several spatial feature points q Sort by groups of associated time T. q Maximum confirmation time T q max and time minimum value T q min, if (T q max-T q min)>Y1, where Y1 is the preset value, then the minimum time value T q min is associated with the spatial feature point holding power to improve (T q max-T q min)≤Y1, and based on the real-time fusion monitoring process, the holding computing power of different spatial feature points is adjusted in real time to complete the relevant process of data fusion.

6. The multimodal spatial data fusion method based on large model technology according to claim 5 is characterized in that: In step S32, if (T q max-T q min)≤Y1, the real-time fusion process is monitored.

7. The multimodal spatial data fusion method based on large model technology according to claim 5 is characterized in that: In step S31, the total data capacity value Z q It will gradually become smaller with the corresponding data fusion process.

Images