Multi-modal data fusion method and system based on low-fidelity finite element model

Abstract

The application belongs to the technical field of data processing. A multi-modal data fusion method and system based on a low-fidelity finite element model are provided. In the low-fidelity finite element model of an airplane, aerodynamic load data, structural response data, physical property data and flight parameter data are extracted in units of each section of the airplane. After preliminary processing of the aerodynamic load data, the structural response data, the physical property data and the flight parameter data, the different modal data after the preliminary processing are mapped to a common representation space through a joint representation learning mechanism. Through a cross-modal attention mechanism, the attention of different modal data is dynamically adjusted in the representation space to obtain a feature tensor that completely describes each section. A training set is constructed according to the feature tensor of each section of the airplane for airplane state evaluation, thereby ensuring the accuracy of the extracted data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-modal data fusion method and system based on a low-fidelity finite element model. BACKGROUND

[0002] The statements in this section merely provide background technology related to the present application and do not necessarily constitute prior art.

[0003] In the field of aircraft design, due to the difference of various maneuvering requirements, hundreds of load cases need to be considered in the design stage of each type, and finite element analysis is carried out. When the dangerous degree of the working condition is evaluated and quantified, since the relative dangerous degree between working conditions is quantified and sorted, when a new load case is added, all working conditions need to be re-quantified and sorted, which consumes additional computing resources, and such working condition analysis is only for one type of aircraft.

[0004] Training a model that can accurately predict and evaluate the dangerous degree of the working condition in real time and offline for multiple types of aircraft can greatly shorten the design cycle and provide important optimization indicators for design engineers, but due to the excessive internal structure details of the high-fidelity model of the aircraft, data extraction and analysis consume a lot of computing and time costs. SUMMARY

[0005] In order to solve the problems of the prior art, the present application provides a multi-modal data fusion method and system based on a low-fidelity finite element model, which extracts data based on a low-fidelity model and fuses multi-modal data, takes each section of the aircraft as a unit, each sample is a certain structure section of the aircraft under a certain working condition, and ensures the accuracy of the extracted data, providing training set data with complete features for subsequent deep learning model training.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a multi-modal data fusion method based on a low-fidelity finite element model.

[0008] A multi-modal data fusion method based on a low-fidelity finite element model, comprising the following processes:

[0009] From the low-fidelity finite element model of the aircraft, taking each section of the aircraft as a unit, extracting aerodynamic load data, structural response data, physical property data and flight parameter data;

[0010] After the aerodynamic load data, the structural response data, the physical property data and the flight parameter data are preliminarily processed, the preliminarily processed different modal data are mapped to a common representation space through a joint representation learning mechanism;

[0011] Through the cross-modal attention mechanism, the attention of different modal data in the representation space is dynamically adjusted to obtain a complete feature tensor describing each section, and a training set is constructed according to the feature tensors of each section of the aircraft for aircraft state evaluation.

[0012] As a further limitation of the first aspect of the application, each section of the aircraft includes:

[0013] For the wing section and the tail section, the section density gradually increases from the wing tip region to the wing root region along the spanwise distribution;

[0014] For the fuselage section, the spacing is less than a set threshold value with the bulkhead position as the reference along the longitudinal direction.

[0015] At least one section is additionally added to the stress concentration area.

[0016] As a further limitation of the first aspect of the application, the aerodynamic load data includes: six-component load of the section and resultant force and moment of the six-component load components;

[0017] The flight parameter data includes: Mach number, angle of attack, sideslip angle, altitude and maneuver type.

[0018] As a further limitation of the first aspect of the application, the structural response data includes: maximum Von Mises stress of all structural elements within a set distance range in the two-dimensional plane of the section; displacement vector of all nodes within a set distance range in the two-dimensional plane of the section; equivalent strain of all structural elements within a set distance range in the two-dimensional plane of the section.

[0019] As a further limitation of the first aspect of the application, the stress or strain data is preliminarily processed, including:

[0020] Obtain the stress or strain data set, wherein each row represents a section under a certain working condition, and each column represents the element stress or strain value, and the element stress or strain value of each section under each working condition is extracted and then corrected by label stamping;

[0021] According to the element stress or strain value corrected by label stamping, threshold division is performed, the number of elements contained in each threshold value is calculated, the influence factor of the threshold range is assigned by using an expert experience model, the number of elements in each threshold range is weighted and summed with the influence factor to obtain a danger degree score, and the danger degree score of each section of each working condition is output and stored to complete stress or strain feature extraction.

[0022] As a further limitation of the first aspect of the application, the displacement data is preliminarily processed, including:

[0023] The displacement data set is obtained, wherein each row represents a section under a certain working condition, and each column represents the displacement value of each node. Assuming that the displacement of the node is isotropic, the nodes are calculated as scalars. For each section under each working condition, the average displacement of the section involving the nodes is calculated, and the average displacement is used as the displacement feature.

[0024] As a further limitation of the first aspect of the application, the physical property data includes: section area; extraction of the section area, including:

[0025] The scale is determined according to the size of each section, the scale is added in the view using a marking tool, the pixel length occupied by the scale in the scale area is identified, and the actual scale factor is determined;

[0026] The contour of the structural section is extracted using edge detection, the structural section area is accurately divided, the number of pixels in the structural section contour is counted, and the actual area is calculated using the actual scale factor of the scale.

[0027] As a further limitation of the first aspect of the application, the physical property data further includes: moment of inertia and material property, extraction of the moment of inertia, including:

[0028] The centroid axis position of the section is determined by combining the image recognition algorithm and the HHO optimization algorithm, the section is meshed into a plurality of square and rectangular sub-sections; the centroid axis position of each sub-section is determined according to the symmetry axis method; the distance between the centroid axis of each sub-section and the centroid axis of the whole section is calculated according to the pixel conversion, the moment of inertia of each sub-section is calculated, and the moment of inertia is converted to the centroid axis according to the translation axis formula, and the moment of inertia is converted and superimposed to obtain the moment of inertia of the section.

[0029] As a further limitation of the first aspect of the application, the attention of different modal data in the representation space is dynamically adjusted by a cross-modal attention mechanism, including:

[0030] When performing attention initialization calculation, initial weight assignment is performed according to the relative importance relationship between different modal features, the dangerous degree of each section under the load working condition is evaluated, sorted and classified according to the dangerous degree evaluation index, and the sorting result and the dangerous degree classification result are obtained;

[0031] The Pearson correlation coefficient and the Spearman correlation coefficient are used to analyze the correlation between each modal feature and the sorting result and the dangerous degree classification result, and the attention weight is dynamically updated according to the correlation degree between each modal feature and the sorting result and the dangerous degree classification result on the basis of the initial weight assignment.

[0032] The second aspect of the application provides a multi-modal data fusion system based on a low-fidelity finite element model.

[0033] A multi-modal data fusion system based on a low-fidelity finite element model, comprising:

[0034] A multi-modal data extraction unit is configured to extract aerodynamic load data, structural response data, physical property data and flight parameter data from the low-fidelity finite element model of the aircraft in units of each cross section of the aircraft.

[0035] A multi-modal data mapping unit is configured to map different modal data after preliminary processing of the aerodynamic load data, the structural response data, the physical property data and the flight parameter data to a common representation space through a joint representation learning mechanism.

[0036] A cross-modal data fusion unit is configured to dynamically adjust the attention of different modal data in the representation space through a cross-modal attention mechanism to obtain a feature tensor that completely describes each cross section, and to construct a training set from the feature tensors of each cross section of the aircraft for aircraft state assessment.

[0037] Compared with the prior art, the beneficial effects of the present application are:

[0038] The present application uses a low-fidelity model for data extraction, which has lower computational cost than a high-fidelity model, faster data extraction and analysis, and is suitable for large-scale data processing. Through multi-modal data fusion, different types of data are fused into a complete feature tensor, improving the efficiency of data processing and providing a complete and reasonable training set for subsequent deep learning.

[0039] The present application is suitable for new aircraft structures and adopts a data extraction and fusion method based on aircraft cross sections, with each sample being a certain structural cross section of the aircraft under certain conditions, which adapts to the structural characteristics of new aircraft and adds additional cross sections in high stress concentration areas such as cabin door edges and joints to ensure accurate analysis of critical areas.

[0040] By extracting and fusing aerodynamic load data, structural response data, physical property data and flight parameter data, a more comprehensive feature description is obtained, providing more feature learning for subsequent deep learning models. The cross-modal attention mechanism is introduced to adjust the attention weights of different data according to known basic rules, further optimizing the feature representation and providing an excellent data set for subsequent models.

[0041] The present application enhances the data through aerodynamic elasticity equations, extreme condition simulation and Latin hypercube sampling method, increases the sample size and ensures the generalization ability of the model. The singular value decomposition (SVD) technique is used to select principal components and add new features, further enriching the feature space and improving the prediction ability of the model.

[0042] The present application extracts stress / strain features, displacement features, cross-sectional area features and moment of inertia features, combines expert experience, pre-processes and extracts features from data, ensures the accuracy and reliability of the data, uses intelligent optimization algorithms such as Harris Hawk Optimization (HHO), and combines image processing technology to accurately calculate the physical properties of the cross section (such as cross-sectional area and moment of inertia), and improve the accuracy of data processing.

[0043] The present application maps different modal data into a common representation space through a joint representation learning mechanism, so that different modal data has similar representation in the space, supports cross-modal tasks, introduces a cross-modal attention mechanism, adjusts the attention weight of different data according to known basic rules, further optimizes the feature representation, and improves the subsequent prediction performance of the model, provides important optimization indicators for design engineers, and helps them better evaluate and optimize the structural performance of the aircraft in the design stage.

[0044] The advantages of the additional aspects of the present application will be partially given in the following description, partially become obvious from the following description, or be known through the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0045] The drawings accompanying the specification of the present application serve to provide a further understanding of the present application, and the illustrative embodiments of the present application and their descriptions serve to explain the present application, and do not constitute an improper limitation of the present application.

[0046] Figure 1 A flowchart of a multi-modal data fusion method based on a low-fidelity finite element model is provided for an exemplary embodiment of the present application;

[0047] Figure 2 A flowchart of a stress / strain feature extraction method is provided for an exemplary embodiment of the present application;

[0048] Figure 3 A flowchart of a cross-sectional area extraction method is provided for an exemplary embodiment of the present application;

[0049] Figure 4 A flowchart of a moment of inertia extraction method is provided for an exemplary embodiment of the present application;

[0050] Figure 5 A flowchart of a multi-modal data fusion method based on a joint representation learning is provided for an exemplary embodiment of the present application;

[0051] Figure 6 A schematic diagram of a multi-modal data fusion system based on a low-fidelity finite element model is provided for an exemplary embodiment of the present application. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0054] In modern aircraft, the distinctions between wings, vertical stabilizers, and horizontal stabilizers are no longer significant; their structures are highly integrated and more unified. Therefore, previous methods for analyzing the hazard level of aircraft operating conditions based on structural components are no longer applicable. In view of this, this implementation proposes a multimodal data fusion method based on a low-fidelity finite element model. All data extraction and fusion are performed on an aircraft cross-section basis, with each sample representing a specific structural section of the aircraft under a particular operating condition. For example... Figure 1 As shown, the specific process includes the following:

[0055] S1: Extract aerodynamic load data, structural response data, physical property data, and flight parameter data from the low-fidelity finite element model of the aircraft, taking each section of the aircraft as a unit.

[0056] S2: After preliminary processing of aerodynamic load data, structural response data, physical property data, and flight parameter data, the different modal data after preliminary processing are mapped to a common representation space through a joint representation learning mechanism.

[0057] S3: Through a cross-modal attention mechanism, the attention of different modal data is dynamically adjusted in the representation space to obtain a feature tensor that fully describes each cross section. A training set is constructed based on the feature tensors of each cross section of the aircraft for aircraft state assessment.

[0058] In this implementation, the cross-section division standard is as follows:

[0059] (1) Wing / Tail section: uniformly distributed along the span, with denser sections at the wing root (1 section per 10% of the chord length) and sparser sections at the wingtip (1 section per 20% of the chord length).

[0060] (2) Fuselage section: along the longitudinal direction, with the position of the partition frame as the reference, the spacing shall not exceed 1 meter;

[0061] (3) Special areas: Additional cross sections are added to areas with high stress concentration (such as the edge of the hatch and the joint).

[0062] In this implementation, the aerodynamic load data includes: in the low-fidelity model after the aerodynamic load data has been applied and balanced, the six-component load of a certain structural section of the aircraft can be extracted using tools. , , , , , ) and the resultant force and moment of six load components ( , ) are output in JSON data format using a secondary development plug-in of the tool.

[0063] In the present implementation, the structural response data (digital text) include:

[0064] Stress: the maximum Von Mises stress of all structural elements within the distance of 2-3 adjacent section elements in the two-dimensional plane of the section (i.e., the discrete structural elements in the section);

[0065] Displacement: the displacement vector of all nodes within the distance of 2-3 adjacent section elements in the two-dimensional plane of the section;

[0066] Strain: the equivalent strain of all structural elements within the distance of 2-3 adjacent section elements in the two-dimensional plane of the section;

[0067] The above data are output in JSON data format, facilitating data extraction and further processing by scripts.

[0068] In the present implementation, the physical property data extraction (image and digital text) include:

[0069] Section image: the section is located in the center of the image, the image scale is determined, the image size is set, and the existing tool is used to take an image of the section, preparing for subsequent calculation of the section area and moment of inertia;

[0070] Material properties of elements near the section: elastic modulus , Poisson's ratio , shear modulus , and density .

[0071] In the present implementation, the flight parameter data (digital text) include: Mach number, angle of attack, side slip angle, altitude, and maneuver type (text data need further processing).

[0072] For the above data, first, data processing is performed through the respective corresponding expert models, and then a joint representation learning mechanism is used to map the data of different modalities to a common representation space, so that the data of different modalities have similar representations in the space; in addition, a cross-modal attention mechanism is introduced to adjust the attention of different data according to known basic rules; finally, data fusion is completed to obtain a feature tensor that can completely describe a structural section in a working condition.

[0073] More specifically, the aerodynamic load data is not further processed, and the eight data , , , , , , , are directly taken as sample features.

[0074] The stress / strain feature extraction process, as shown in Figure 2 , obtains stress and strain structural response data through finite element software, and the output includes all working conditions and all structural sections of the aircraft. Starting from the overall working condition and section data, the processing path is divided into single-section stress data or single-section strain data, followed by data extraction and label correction, threshold limit division and setting, and unit quantity statistics in the threshold region. Then, the threshold 1 unit quantity to the threshold 7 unit quantity are respectively cross-operated with the influence factors 1 to 7 after expert experience evaluation, to obtain the class danger degree score, and finally the overall working condition and all section score data are archived. More specifically, it includes:

[0075] Firstly, the extracted data needs to be processed to obtain the final data set, wherein each row represents a certain structural section under a certain working condition (i.e. a sample), and each column represents the unit stress / strain value. After extracting the unit stress / strain value of each section under each working condition, label correction is needed to avoid the problem of label and data not corresponding;

[0076] Secondly, threshold division is performed according to the unit stress / strain value. For example, in the coarse grid model, the unit stress is roughly distributed in the range of 50-700 MPa, and according to this, 5 threshold limits are uniformly set in the range of 200-700, and 2 threshold limits are uniformly set in the range of 50-200.

[0077] Thirdly, the number of units contained in each threshold is calculated. Generally speaking, the larger the stress / strain threshold range, the more units in the range, which means that the section is more dangerous. Therefore, the invention introduces expert experience evaluation to assign influence factors to these thresholds according to design experience.

[0078] Finally, the number of units in each threshold range is weighted and summed with the influence factor to obtain the class danger degree score. The class danger degree score of each section under each working condition is output and stored, and the feature extraction is completed.

[0079] In the present implementation, the displacement feature extraction includes: in the displacement data set, the meaning of the row is the same as that of the stress / strain data set, and the column represents the displacement value of each node. In the feature extraction process, it is assumed that the displacement of the node is isotropic, and therefore the nodes are calculated as scalars. For each cross section under each working condition, the average value of the displacement of the nodes involved in the cross section is calculated, and the average value is used as a feature representation.

[0080] In the present implementation, the physical property data extraction includes cross-sectional area extraction and moment of inertia feature extraction. The cross-sectional area extraction is as shown in the following table: Figure 3

[0081] More specifically, it includes: determining the appropriate scale according to the size of each cross section, then adding the scale in the view using the annotation tool in the software, then using the TCL secondary development plug-in to complete the image capture of each cross section, and saving; generally, the scale is located at the lower left or lower right of the image, so the ROI region segmentation can be used to find the area where the scale is located, and then the OCR recognition is performed on the area to read the length on the scale;

[0082] In order to accurately obtain the cross-sectional profile, it is necessary to convert the color image to a grayscale image to simplify subsequent processing, and to denoise and enhance the image to improve the accuracy of edge detection. In addition, the pixel length occupied by the scale in the scale area needs to be identified, and the actual length corresponding to a unit pixel is calculated according to this;

[0083] After the image is preprocessed, the edge detection is used to extract the profile of the structural section, and the structural section area is accurately divided. Then, the number of pixels in the structural section profile is counted, and the conversion coefficient of the scale is used to calculate the actual area.

[0084] The moment of inertia feature extraction is as shown in the following table: Figure 4 ​As shown, starting from the structural cross-sectional screenshot (with scale bar), after image preprocessing, the process enters the OpenCV image processing model and HHO optimization algorithm stage. First, the centroid coordinates are determined using the HHO optimization algorithm, thus determining the centroidal axis of the overall cross-section. Then, the cross-section is meshed (divided into sub-sections), divided into internal square meshes and edge rectangular meshes. Next, the centroidal axes of the sub-sections are determined (using the symmetry axis method), and the distance between the sub-section centroidal axis and the overall cross-section centroidal axis is calculated. Using the translation axis formula, the moments of inertia are combined and superimposed to finally obtain the cross-sectional moment of inertia. More specifically, this includes:

[0085] Because the structural cross-section is irregular and the boundary curve function is difficult to define, a combination method is used to calculate the moment of inertia of the section. This involves dividing the cross-section into multiple sub-sections, solving for their moments of inertia, and then superimposing them according to the translation axis theorem to obtain the final moment of inertia. Before calculating the moment of inertia, it is necessary to determine the position of the centroidal axis or centroid of the entire cross-section.

[0086] Since the cross-sectional area is indirectly obtained from the image processing algorithm, it can only guarantee that the relative size of each cross-section is correct, but its absolute size cannot be verified. Therefore, the centroid method is not used as the main method to determine the centroid position. Instead, an image processing model combined with the HHO algorithm is used to iteratively derive the centroid position through an optimization problem. The centroid coordinates obtained by the centroid method are used as auxiliary verification.

[0087] Assume that the cross-sectional area obtained from the area calculation model is... Establish a coordinate system using the (0,0) coordinate point in the image to obtain the static moments of the cross section with respect to the x-axis and y-axis:

[0088] (1);

[0089] in, Representative cross section The static moment of an axis is the product of the areas of all infinitesimal elements within its cross-section and its constant. Integral of axial distance; Representative cross section The static moment of an axis is the product of the areas of all infinitesimal elements within its cross-section and its constant. Integral of axial distance; The area dA of the infinitesimal element within the cross-section in the image coordinate system (with the top left corner of the image as (0,0)) Coordinate; y represents the y-coordinate of the infinitesimal area dA within the cross-section in the image coordinate system (with the top left corner of the image as (0,0)); dA represents the infinitesimal area within the cross-section; A Represents the total cross-sectional area.

[0090] The centroid method calculation expression is as follows:

[0091] (2).

[0092] The same as the cross-sectional area calculation, first add the scale in the software, take a screenshot of the structural section, and pretreat. First, the centroid axis position of the overall section needs to be determined. For irregular sections, the centroid position is not easy to determine, and then the position of the centroid axis is not easy to determine. According to the definition of the centroid axis, the static moment of the section to the centroid axis is zero. The present application takes the static moment as the optimization index, adds the necessary constraint conditions, combines OpenCV image recognition and HHO optimization algorithm, and finds the overall centroid axis position.

[0093] Harris' hawk optimization algorithm (HHO) is a meta-heuristic optimization algorithm inspired by the hunting behavior of Harris' hawk. By simulating the behavior patterns of surrounding, pursuing and attacking prey, combined with the dynamic balance strategy of exploration and development stage, it realizes the effective solution to complex optimization problems.

[0094] Similarly, the coordinate system is established with the image (0, 0) point, and the initial centroid coordinates are (x, y). The coordinate system is established, and the area of each sub-section is , the coordinate range of each point on the section in the coordinate system is to .

[0095] Among them, the optimization function is as follows:

[0096] (3);

[0097] Among them, and represent the coordinates of the nth point on the section.

[0098] The specific steps are as follows:

[0099] Step 1: initialization: generation of initial population, evaluation of candidate solutions.

[0100] The initial population is generated and represented as:

[0101] (4);

[0102] Among them, represents the position vector of the ith candidate solution at the tth iteration, respectively represent the position vectors of the 1st, 2nd, 3rd and nth candidate solutions at the tth iteration.

[0103] Here the elements in the initial population are centroid coordinates, evaluate the candidate solutions, calculate the objective function value of each candidate solution, and determine the initial optimal solution, denoted as When calculating the objective function value, the initial population needs to be passed into the image recognition model, and a coordinate system is established according to the initial population coordinates, the cross section is gridded, and the large cross section is divided into multiple sub-cross sections (the same as the model calculation of the cross section area described above).

[0104] Step 2: Iterative process (loop until termination condition).

[0105] Step 2.1: Calculate the energy parameter E:

[0106] (5);

[0107] wherein, is the randomized energy, t is the current iteration number, and T is the maximum iteration number.

[0108] Step 2.2: Determine the exploration phase or development phase.

[0109] If , enter the exploration phase;

[0110] If , enter the development phase.

[0111] Step 3: Exploration phase.

[0112] Random selection strategy: based on random number :

[0113] Step 3.1: When :

[0114] (6);

[0115] wherein, is the randomly selected candidate solution, is the random number, represents the candidate solution of the tth iteration round, represents the candidate solution of the t+1th iteration round.

[0116] Step 3.2: When :

[0117] (7);

[0118] wherein, is the population average position, is the random number, LU and UB are the search space boundaries; represents the optimal candidate solution at the tth iteration; N represents the number of candidate solutions.

[0119] Step 4: Development phase.

[0120] Based on the escape energy of prey and strategy selection:

[0121] Step 4.1: Calculate random number: ;

[0122] Step 4.2: Determine development strategy.

[0123] When , soft surround:

[0124] (8);

[0125] Wherein, .

[0126] Soft surround with gradual rapid dive:

[0127] (9);

[0128] (10);

[0129] Wherein, S represents a random scaling factor; Y represents a newly generated candidate solution intermediate variable; Levy flight function; D represents the dimension of the search space.

[0130] According to and fitness selection of new candidate solution:

[0131] (11).

[0132] Wherein, Y represents the fitness value corresponding to Y; Z represents the fitness value corresponding to Z; The fitness value of the current candidate solution at the tth iteration.

[0133] Step 5: Update and evaluation.

[0134] Evaluate new candidate solution: calculate the fitness value of the new candidate solution;

[0135] Update the optimal solution: if the fitness of the new candidate solution is better than the current optimal solution , update the optimal solution to the new candidate solution.

[0136] Step 6: Termination condition.

[0137] Check termination condition: if the termination condition is met (such as reaching the maximum number of iterations T), stop the algorithm and output the optimal solution ; Otherwise, continue the iteration process.

[0138] So far, the coordinates of the cross-sectional centroid can be obtained by combining the image processing model with the HHO optimization algorithm, and the coordinates are compared and verified with the coordinates obtained by the centroid method. The final centroid is determined.

[0139] After obtaining the centroid axis of the overall section, the moment of inertia of the overall section can be calculated. Similarly, the combination method is used for moment of inertia calculation, and the specific process is as follows:

[0140] (1) As described above for calculating the cross-sectional area, first, the cross-section with a scale is intercepted from the software;

[0141] (2) The image is binarized, denoised, and enhanced;

[0142] (3) According to the above image processing model and HHO optimization algorithm, the centroid axis position is determined;

[0143] (4) The cross-section is gridded and decomposed into multiple square (internal) and rectangular (edge) sub-sections;

[0144] (5) According to the symmetry axis method, the centroid axis position of each sub-section is determined;

[0145] (6) According to the pixel conversion, the distance between the centroid axis of each sub-section and the overall cross-sectional centroid axis is calculated;

[0146] (7) The moment of inertia of each sub-section is calculated, and according to the translation axis formula, it is converted to the overall centroid axis;

[0147] (8) After the moment of inertia conversion and superposition, the moment of inertia of the overall section is obtained.

[0148] In this implementation, the elastic modulus E, Poisson's ratio v, shear modulus G, and density p in the material properties are all numerical values of the section, so no further processing is required.

[0149] In this implementation, the data extraction of flight parameters includes: Mach number, attack angle, side slip angle, height, and maneuver type (text data needs further processing). Among these flight parameters, except for the maneuver type, the remaining parameters are numerical data and can be directly put into the data set. For the maneuver type, it is encoded. Different maneuver types cause different parts to fail easily, so according to the maneuver type and the position of the structure profile, combined with the stress cloud diagram of the coarse grid model, a danger level is assigned. For example, the wing root part in the rolling maneuver has the highest danger level of 5, and the wing tip part has a danger level of 1.

[0150] For example, Figure 5As shown, the present embodiment proposes a multi-modal data fusion method of joint representation learning. Starting from aerodynamic load data, flight parameters, material properties, section physical properties, and structural response data, the aerodynamic load data is obtained through data retention to obtain section force and torque data, the flight parameters are obtained through a text processing model to obtain one-dimensional flight parameter data, the material properties are obtained through data retention to obtain material basic attribute data, the section physical properties are obtained through an OpenCV and HHO joint processing model to obtain section area and moment of inertia, and the structural response data is obtained through a stress and strain processing scoring system to obtain stress, strain, and displacement evaluation. Finally, these data are integrated into working condition 1 section 1, working condition 1 section n, working condition n section 1, and working condition n section n, covering the dimensions of aerodynamic load data, flight parameters, material properties, physical properties, and structural response.

[0151] More specifically, the extracted data is processed through its own expert model, and image, text, and digital data are collectively mapped to a unified modal vector space, that is, a shared embedding layer is used to map data of different modalities to the same low-dimensional representation space. The multi-modal data obtained from the aircraft is retained through data analysis models, image processing models, intelligent optimization algorithms, and other methods, and the key information of each modality is retained and mapped to a one-dimensional space, establishing a connection between different modalities to support cross-modal tasks. This method can effectively utilize the complementary information of multi-modal data and improve the performance of the model.

[0152] The present embodiment provides a multi-modal data fusion method based on cross-modal attention mechanism, specifically including:

[0153] According to the basic rule, the attention is initialized and valued. When performing attention initialization calculation, the different features can be weighted according to the coarse grid model. That is, the important relative relationship between the features is weighted. First, the aerodynamic load data, flight parameter data, and section material properties are obtained directly without secondary processing, and are highly reliable. The physical properties of the section and the structural response data are obtained through image processing, optimization algorithms, and secondary models, and cannot guarantee absolute accuracy. Therefore, compared with the first three (aerodynamic load, flight parameter, and section material property), the weight is lower. For the aerodynamic load data, flight parameter data, and section material property, the aerodynamic load contains more information, and the difference between the working conditions is larger. Therefore, the initialization weight of the aerodynamic load is increased. At this point, all the above data are encoded according to the attention level, among which the aerodynamic load data has the highest attention level, the flight parameter and material property have the second highest attention level, and the physical property and structural response have the lowest attention level. According to this basic rule, a random function is used to generate the initial weight of each index.

[0154] In the present implementation, the method of fusing machine learning and reinforcement learning can evaluate, sort and grade the cross sections under load conditions according to the above indicators. Based on the sorting results and the dangerous level division results of the cross sections under load conditions, the correlation between each feature and the sorting results and the dangerous level division results is analyzed through Pearson correlation coefficient and Spearman correlation coefficient. Attention is calculated according to the correlation between each feature and the sorting results and the dangerous level division. The features of different modal data are multiplied by the corresponding attention weights to obtain the weighted feature representation.

[0155] In the present implementation, the obtained data is also enhanced, including:

[0156] First, the synthetic data enhancement (i.e. sample size enhancement) includes: based on the aeroelastic equation, a small disturbance (±10%) is applied to the aerodynamic load, and the new displacement and stress are calculated through the static equation:

[0157] (12);

[0158] where K is the stiffness matrix obtained by the coarse mesh model, is the disturbed load, represents the new displacement vector.

[0159] The present application also generates more dangerous working conditions by selecting the feature dimension, ensuring that untested scenarios are covered.

[0160] The present application also uses the Latin hypercube sampling method to divide the range of each variable into several equal probability intervals, ensuring that there is only one sample point in each interval, thereby ensuring that the sample is uniformly distributed in each dimension. In a multi-dimensional space, by independently stratifying sampling in each dimension, the generated sample points can better cover the entire parameter space, avoiding the sample aggregation or sparsity problem that may occur in traditional random sampling methods.

[0161] Second, feature space enhancement (feature quantity), specifically, including: using SVD decomposition on the aerodynamic load data matrix, selecting the principal components, and adding new features according to the combination of principal component singular values and contribution rates.

[0162] Figure 6 A multi-modal data fusion system based on a low-fidelity finite element model is shown, including:

[0163] The multi-modal data extraction unit is configured to extract aerodynamic load data, structural response data, physical property data and flight parameter data from the low-fidelity finite element model of the aircraft in units of each cross section of the aircraft;

[0164] The multi-modal data mapping unit is configured to map the different modal data after preliminary processing of the aerodynamic load data, the structural response data, the physical property data and the flight parameter data to a common representation space through a joint representation learning mechanism.

[0165] The cross-modal data fusion unit is configured to dynamically adjust the attention of the different modal data in the representation space through a cross-modal attention mechanism to obtain a feature tensor that completely describes each section, and construct a training set according to the feature tensors of each section of the aircraft for aircraft state evaluation.

[0166] The specific working methods of the above units are described in Embodiment 1, which will not be repeated here.

[0167] It can be understood that the above units can be combined into one or several other units respectively or entirely, or some of the units can be further split into a plurality of units with smaller functions to constitute, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above units are logically divided, and the functions of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the system can also include other units, and these functions can also be realized by other units in actual application, and can be realized by multiple units.

[0168] According to another embodiment of the present application, the system of the embodiment can be constructed and the method of Embodiment 1 of the present application can be implemented by running a computer program (including program code) capable of performing each step involved in the corresponding method of the present application on a general computing device such as a computer including processing elements and storage elements such as a Central Processing Unit (CPU), a Random Access Memory (RAM), a Read Only Memory (ROM), etc., the computer program can be recorded on a computer readable recording medium such as a computer readable recording medium, and loaded into the above computing device through the computer readable recording medium and run therein.

[0169] The above is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-modal data fusion method based on low-fidelity finite element model, characterized in that, The method comprises the following processes: extracting aerodynamic load data, structural response data, physical property data and flight parameter data from each section of the aircraft in a low-fidelity finite element model of the aircraft; after preliminary processing of the aerodynamic load data, the structural response data, the physical property data and the flight parameter data, mapping the preliminary processed different modal data to a unified modal vector space through a joint representation learning mechanism, and mapping the data of different modes to the same low-dimensional representation space using a shared embedding layer; dynamically adjusting the attention of different modal data in the representation space through a cross-modal attention mechanism to obtain a feature tensor that completely describes each section, and constructing a training set from the feature tensor of each section of the aircraft for aircraft state evaluation; the cross-modal attention mechanism dynamically adjusts the attention of different modal data in the representation space, including: when performing attention initialization calculation, initial weight assignment is performed according to the relative importance relationship between different modal features, the dangerous degree of each section under the load working condition is evaluated, sorted and classified according to the dangerous degree evaluation index, and a sorting result and a dangerous degree classification result are obtained; by using Pearson correlation coefficient and Spearman correlation coefficient, correlation analysis is performed on the correlation between each modal feature and the sorting result and the dangerous degree classification result, and the attention weight is dynamically updated according to the correlation between each modal feature and the sorting result and the dangerous degree classification result on the basis of the initial weight assignment.

2. The multi-modal data fusion method based on a low-fidelity finite element model according to claim 1, wherein the sections of the aircraft comprise: for the wing section and the tail section, the section density gradually increases from the wing tip area to the wing root area along the spanwise direction; for the fuselage section: along the longitudinal direction, the interval is less than a set threshold based on the position of the bulkhead; at least one section is additionally added to the stress concentration area.

3. The multi-modal data fusion method based on a low-fidelity finite element model according to claim 1, wherein the aerodynamic load data comprises: six-component load of the section and resultant force and moment of the six-component load components; the flight parameter data comprises: Mach number, angle of attack, side slip angle, height and maneuver type.

4. The multi-modal data fusion method based on a low-fidelity finite element model according to claim 1, wherein the structural response data comprises: maximum Von Mises stress of all structural elements within a set distance range in the two-dimensional plane of the section; displacement vector of all nodes within a set distance range in the two-dimensional plane of the section; equivalent strain of all structural elements within a set distance range in the two-dimensional plane of the section.

5. The multi-modal data fusion method based on a low-fidelity finite element model according to claim 4, wherein the preliminary processing of the stress or strain data comprises: obtaining a stress or strain data set, wherein each row represents a section under a certain working condition, and each column represents an element stress or strain value, and after extracting the element stress or strain value of each section under each working condition, label tag correction is performed. ​ ​ ​ ​ According to the unit stress or strain value corrected according to the label stamp, threshold division is performed, the number of units contained in each threshold is calculated, an expert experience model is used to assign influence factors to the threshold range, the number of units in each threshold range is weighted and summed with the influence factors, a danger degree score is obtained, and the danger degree score of each section under each working condition is output and stored, and stress or strain feature extraction is completed.

6. The low-fidelity finite element model-based multi-modal data fusion method of claim 4, wherein, The displacement data is preliminarily processed, including: Obtaining a displacement data set, wherein each row represents a section under a certain working condition, and each column represents the displacement value of each node. Assuming that the displacement of the node is isotropic, the nodes are calculated as scalars. For the section under each working condition, the average displacement of the nodes involved in the section is calculated, and the average value is used as the displacement feature.

7. The low-fidelity finite element model-based multi-modal data fusion method of claim 1, wherein, The physical property data includes: section area; The section area is extracted, including: A scale is determined according to the size of each section, a scale is added to the view, and the actual scale factor is determined by identifying the pixel length occupied by the scale in the scale area after the section image with the added scale is preprocessed; The contour of the structural profile is extracted using edge detection, the structural profile area is accurately divided, the number of pixels in the structural profile contour is counted, and the actual area is calculated using the actual scale factor of the scale.

8. The low-fidelity finite element model-based multi-modal data fusion method of claim 7, wherein, The physical property data further includes: moment of inertia and material properties, and the moment of inertia is extracted, including: The centroid axis position of the section is determined by combining the image recognition algorithm and the HHO optimization algorithm, the section is meshed into multiple square and rectangular sub-sections, the centroid axis position of each sub-section is determined according to the symmetry axis method, the distance between the centroid axis of each sub-section and the centroid axis of the whole section is calculated according to the pixel conversion, the moment of inertia of each sub-section is calculated, and the moment of inertia is converted to the centroid axis according to the translation axis formula and is superimposed to obtain the moment of inertia of the section.

9. A low-fidelity finite element model based multi-modal data fusion system, characterized in that, Including: The multi-modal data extraction unit is configured to extract aerodynamic load data, structural response data, physical property data, and flight parameter data from the low-fidelity finite element model of the aircraft in units of sections of the aircraft; The multi-modal data mapping unit is configured to preliminarily process the aerodynamic load data, the structural response data, the physical property data, and the flight parameter data, and map the preliminarily processed different modal data to a unified modal vector space through a joint representation learning mechanism, and map the data of different modalities to the same low-dimensional representation space using a shared embedding layer; The cross-modal data fusion unit is configured to dynamically adjust the attention of different modal data in the representation space through a cross-modal attention mechanism, obtain a feature tensor that completely describes each section, and construct a training set according to the feature tensor of each section of the aircraft for aircraft state evaluation; By the cross-modal attention mechanism, the attention of different modal data is dynamically adjusted in the representation space, including: In the attention initialization calculation, the initial weight assignment is performed according to the relative importance relationship between different modal features, the dangerous degree evaluation, sorting and grade division of each section under the load working condition are performed according to the dangerous degree evaluation index, and the sorting result and the dangerous grade division result are obtained; Through the Pearson correlation coefficient and the Spearman correlation coefficient, the correlation analysis of each modal feature and the sorting result and the dangerous grade division result is performed, and on the basis of the initial weight assignment, the attention weight is dynamically updated according to the correlation degree of each modal feature and the sorting result and the dangerous grade division result.

Images