Magnetic levitation operation control intelligent system based on multiple modes

Abstract

The invention belongs to the technical field of artificial intelligence and magnetic levitation traffic, and discloses a multi-mode-based magnetic levitation operation control intelligent system which comprises a user subsystem, a user interface, an artificial intelligence subsystem, a ubiquitous perception subsystem, a magnetic levitation traffic physical subsystem and a communication subsystem. The user subsystem is used for submitting a transportation demand, a user demand and a transportation object, the artificial intelligence subsystem is used for completing a system-level safety-critical task, a vehicle-level safety-critical task and a vehicle-level non-safety-critical task, and the ubiquitous perception subsystem provides multi-modal perception information of the user subsystem and the artificial intelligence subsystem. The magnetic suspension traffic physical subsystem provides physical reality for completing transportation requirements, and the communication subsystem provides a ubiquitous interconnection basis for other subsystems. Potential implementation of multi-mode artificial intelligence in the field of safety critical requirements is effectively promoted, intelligent transformation of a magnetic levitation traffic system is accelerated, and deep fusion of calculation, information and physics is promoted.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology and magnetic levitation transportation technology, and in particular to a multi-modal magnetic levitation transportation control intelligent system. Background Art

[0002] Maglev transportation uses electromagnetic control to achieve train suspension, traction and braking. Compared with traditional wheeled and wheel-rail transportation, it has the advantages of no contact wear, safety and reliability, low vibration and noise, and a wider operating speed range. China's transportation network is becoming increasingly complex and complete, and traditional transportation has encountered bottlenecks in future transportation needs. There is an urgent need to develop a maglev transportation system. As the supporting technologies related to maglev transportation gradually mature, such as superconducting technology, automatic control technology, and communication technology, maglev transportation has gradually become the focus of the next generation of national infrastructure. In addition, while tackling the problem of increasing the operating speed of maglev trains, it is also necessary to pay attention to improving the intelligence level of the transportation system.

[0003] The Large Language Model (LLM) ChatGPT launched by OpenAI has demonstrated excellent language understanding, generation and knowledge reasoning capabilities, making it a new generation of knowledge representation and call method after databases and search engines. ChatGPT has basically overcome the problems of poor robustness and lack of reasoning ability of deep learning models, and provides targeted services according to user needs, including copywriting and code writing. Its infrastructure as a service operation mode marks that the large language model is gradually moving towards artificial general intelligence (AGI), that is, human-like intelligence that can learn and evolve autonomously, generate autonomous consciousness, and have human-like emotions to solve various complex problems in different fields and complete corresponding tasks is defined as general artificial intelligence.

[0004] General artificial intelligence models continue to emerge, allowing artificial intelligence technology to gradually evolve into a new stage of development. The controllability and commercialization of general artificial intelligence to assist in various fields, especially safety-critical fields, will be the focus of future research. For the maglev transportation system, its application is both a developmental technology empowerment and a fusion technology challenge. Leveraging general artificial intelligence, coupling multiple information technologies, assisting in the design of intelligent systems, empowering the maglev physical system, creating a new industry ecology, improving the level of intelligence, and meeting more potentially complex transportation scenarios have become urgent needs.

[0005] The existing maglev transportation system is based on written programs and cannot provide end-to-end (demand side - service end value side) real-time intelligent interaction for system users, cannot respond to system user needs in a timely manner, and is difficult to meet the increasingly complex maglev transportation needs. Summary of the invention

[0006] In response to the increasingly complex transportation scenarios, this application provides a multi-modal maglev transportation control intelligent system, aiming to promote the integration of artificial intelligence computing processes and maglev transportation physical processes.

[0007] The present invention provides a multi-modal maglev transportation control intelligent system, which is an intelligent maglev transportation system, including: a user subsystem, a user interface, an artificial intelligence subsystem, a ubiquitous perception subsystem, a maglev transportation physical subsystem, and a communication subsystem; wherein:

[0008] The user subsystem is used to submit transportation requirements, user requirements and transportation objects, including system users and transportation objects;

[0009] The user interface is used to realize the interaction and information exchange between the user subsystem and the artificial intelligence subsystem;

[0010] The artificial intelligence subsystem is used to complete system-level safety-critical tasks, vehicle-level safety-critical tasks, and vehicle-level non-safety-critical tasks, including system-level multimodal dedicated artificial intelligence MDAI-MS (Multimodal Dedicated Artificial Intelligence-Maglev System), vehicle-level multimodal dedicated artificial intelligence MDAI-MV (Multimodal Dedicated Artificial Intelligence-Maglev Vehicle), and vehicle-level multimodal general artificial intelligence MAGI-MV (Multimodal Artificial General Intelligence-Maglev Vehicle);

[0011] The ubiquitous perception subsystem collects real-time multimodal information from the user subsystem and the maglev transportation physical subsystem at a large spatial and temporal scale, and transmits it to the artificial intelligence subsystem.

[0012] The magnetic levitation physical subsystem is used to complete the physical reality of the system user's transportation needs;

[0013] The communication subsystem provides a ubiquitous interconnection foundation for perception interaction and data transmission between subsystems of the maglev operation and control intelligent system.

[0014] Advantages of the present invention include:

[0015] The present invention provides an intelligent maglev transportation system, a potential implementation of artificial intelligence technology in the maglev transportation system, real-time perception information is associated with transportation physical entities, general artificial intelligence and special artificial intelligence work together to complete non-safety-critical tasks and safety-critical tasks of the maglev transportation system, and realize information autonomy, control autonomy, and traffic process autonomy in the end-to-end service process, providing a system solution for the application of large-scale models to end-to-end services in the field of safety-critical requirements;

[0016] The present invention provides a solution for mapping the inherent generalized knowledge of general artificial intelligence into the physical reality behavior of a maglev transportation system. Specifically, the uncertain expression of system user requirements and ubiquitous perception information in the semantic space is mapped into the deterministic expression of control instructions in the semantic space, and then mapped into the deterministic expression of mechanical equipment in the behavior space, wherein the model calculation reasoning process and the physical process influence each other and interact in real time, realizing the deep integration of calculation, information and physics. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the specific structure and connection relationship of a multi-modal magnetic levitation transportation control intelligent system of the present application;

[0018] Figure 2 A schematic diagram of the network structure of the multimodal artificial intelligence model of this application;

[0019] Figure 3 A schematic diagram of the specific structure of the multimodal information interaction layer in the multimodal artificial intelligence model of this application;

[0020] Figure 4 Schematic diagram of the training steps of the multimodal artificial intelligence model of this application:

[0021] Figure 5 A schematic diagram of the structure and connection relationship of the communication subsystem of this application;

[0022] Figure 6 A schematic diagram of the operation scenario of the maglev transportation control intelligent system provided in the implementation of this application;

[0023] Figure 7 A schematic diagram of the operation flow of the maglev transportation control intelligent system provided in the implementation of this application;

[0024] Attached Figure 1 The meanings of the various symbols are as follows:

[0025] Icons: 1 represents the user subsystem; 2 represents the user interface; 3 represents the artificial intelligence subsystem; 4 represents the ubiquitous perception subsystem; 5 represents the magnetic levitation physics subsystem; 6 represents the communication subsystem;

[0026] System-level multimodal dedicated artificial intelligence MDAI-MS (Multimodal Dedicated Artificial Intelligence-Maglev System);

[0027] Vehicle-level multimodal dedicated artificial intelligence MDAI-MV (Multimodal Dedicated Artificial Intelligence-Maglev Vehicle);

[0028] Vehicle-level multimodal general artificial intelligence MAGI-MV (Multimodal Artificial General Intelligence-Maglev Vehicle). DETAILED DESCRIPTION

[0029] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0030] like Figure 1 As shown, a multi-modal maglev transportation control intelligent system is an intelligent maglev transportation system, comprising: a user subsystem (1), a user interface (2), an artificial intelligence subsystem (3), a ubiquitous perception subsystem (4), a maglev transportation physical subsystem (5), and a communication subsystem (6).

[0031] The user subsystem (1) includes system users and transport objects. The system users submit transport requirements to the maglev transportation system through a user interface. The system users include but are not limited to real users (humans) or virtual users (artificial intelligence models). The transport objects are the contents transported by the maglev transportation system and are defined by the system users, including but not limited to humans or goods.

[0032] Furthermore, the transportation demand is presented in text or voice mode, and the content of the transportation demand at least includes the transportation object, the transportation origin, the transportation destination, and may also include the transportation time limit, etc.

[0033] The user interface (2) realizes the interaction and information exchange between the user subsystem (1) and the artificial intelligence subsystem (3), and the interface form includes but is not limited to software interface, brain-computer interface, etc.

[0034] The artificial intelligence subsystem (3) includes a system-level multimodal dedicated artificial intelligence MDAI-MS (Multimodal Dedicated Artificial Intelligence-Maglev System), a vehicle-level multimodal dedicated artificial intelligence MDAI-MV (Multimodal Dedicated Artificial Intelligence-Maglev Vehicle), and a vehicle-level multimodal general artificial intelligence MAGI-MV (Multimodal Artificial General Intelligence-Maglev Vehicle);

[0035] Furthermore, the MDAI-MS serves the entire maglev transportation system and is deployed in the control center of the maglev transportation system to complete system-level safety-critical tasks involved in the operation of the maglev transportation system;

[0036] Specifically, the system-level safety-critical tasks include: dispatching maglev vehicles according to transportation needs, assigning maglev vehicle IDs to users, solving the optimal path from the transportation origin to the destination, calculating the static operation curve from the transportation origin to the destination, etc., and transmitting the above-generated information to the corresponding MDAI-MV in the form of a message;

[0037] Specifically, the MDAI-MS is constructed based on a neural network, with a decoder-encoder as the basic architecture, and performs self-supervised pre-training through a large amount of unlabeled multimodal (text, voice, video, etc.) data and supervised fine-tuning with labeled multimodal data. It especially focuses on learning general and professional knowledge in the fields of vehicles, transportation, control, communication, computers, and electrical engineering, and has semantic understanding and logical reasoning capabilities. It also uses reinforcement learning based on artificial feedback to make its generation strategy safety-oriented. Downstream task interfaces are set according to the functions of MDAI-MS, including: a transportation demand analysis interface, a static operation curve generation interface, etc.

[0038] Furthermore, the MDAI-MV corresponds to the maglev vehicle included in the maglev transportation system one by one and is deployed on the on-board computer of the corresponding maglev vehicle, for example, the MDAI-MV i Serving the i-th maglev vehicle to complete vehicle-level safety-critical tasks related to transportation needs;

[0039] Specifically, the vehicle-level safety-critical task refers to MDAI-MV i Based on the information from MDAI-MS and the real-time perception information of the ubiquitous perception subsystem, and based on the communication technology and the MDAI-MV of the front neighboring vehicle, iInteractive information, dynamically adjust the static operation curve, generate a dynamic operation curve, and convert the kinematic parameters such as the speed and acceleration of the magnetic levitation vehicle into motor voltage, current, etc., generate control instructions and send them to the corresponding safety-critical equipment, as well as transport object identity recognition and transport object behavior monitoring;

[0040] Specifically, the safety-critical equipment includes but is not limited to doors, traction motors, suspension electromagnets, guide electromagnets, brakes, turnout mechanisms, etc.;

[0041] Specifically, the MDAI-MV i Based on neural network construction, with decoder-encoder as the basic architecture, self-supervised pre-training is performed through a large amount of unlabeled multimodal data, especially focusing on learning general and professional knowledge in the fields of vehicles, transportation, control, communication, computer, and electrical, with semantic understanding and logical reasoning capabilities, and reinforcement learning based on artificial feedback makes its generation strategy safety-oriented, and sets downstream task interfaces according to MDAI-MS functions, including: message parsing interface, dynamic operation curve generation interface, and control instruction generation interface;

[0042] Furthermore, the MAGI-MV also corresponds to the maglev vehicles included in the maglev transportation system, and is also deployed in the on-board computer of the corresponding maglev vehicle, such as MAGI-MV i Serves the i-th maglev vehicle, and MAGI-MV i With MDAI-MV i Together they serve the i-th maglev vehicle to complete the vehicle-level non-safety-critical tasks involved in the transportation process;

[0043] Specifically, the vehicle-level non-safety-critical tasks may come from system requirements or system user requirements, and are defined as tasks that do not affect the driving safety of the maglev vehicle, including user requirement reasoning, user non-safety-critical requirements, non-safety-critical vehicle-mounted equipment control, etc.;

[0044] Specifically, the non-safety-critical vehicle-mounted equipment includes but is not limited to lighting, air conditioning, audio, etc.;

[0045] Specifically, the MAGI-MV i Based on neural network construction, with decoder-encoder as the basic architecture, self-supervised pre-training is performed through a large amount of unlabeled multimodal data to learn general knowledge in the fields of humans, society, environment, vehicles, medicine, transportation, etc., with semantic understanding and logical reasoning capabilities, and reinforcement learning based on artificial feedback so that its generation strategy does not violate ethics, etc. Downstream task interfaces are set according to MDAI-MS functions, including: user demand analysis interface, control command generation interface, multi-modal signal generation interface, etc.;

[0046] It should be noted that system users can only access MDAI-MS and MAGI-MV allocated by MDAI-MS according to transportation needs through the user interface. i , unable to access MDAI-MV.

[0047] The ubiquitous perception subsystem (4) collects real-time multimodal information of the user subsystem (1) and the magnetic levitation traffic physical subsystem (5) at a large spatial and temporal scale, and transmits the information to the artificial intelligence subsystem (3), and is composed of a plurality of perception submodules, including at least a first perception module, a second perception module, a third perception module, and a fourth perception module; wherein:

[0048] The first sensing module is used to estimate the running speed and time-space position of the maglev vehicle, and is composed of a speed sensor, a synchronous clock, a positioning sensor, a satellite navigation, etc.;

[0049] The second sensing module is used to estimate the guide gap and suspension gap of the magnetic levitation vehicle, and is composed of a guide gap sensor and a suspension gap sensor;

[0050] The third sensing module is used to collect dynamic shape information of the transport object, track line, and external environment in the maglev vehicle, and is composed of a 3D depth camera in the vehicle, a drone vision camera, an infrared vision camera, a laser radar, etc.

[0051] The fourth sensing module is used to monitor the operation and fault conditions of the maglev vehicle, track line, switch mechanism, motor and other equipment and auxiliary sub-equipment, and is composed of a vibration sensor, an acoustic emission sensor, a voltage sensor, a current sensor, etc.;

[0052] Furthermore, the ubiquitous sensing subsystem can be classified into a vehicle-mounted sensing module, a trackside sensing module, and a far-field sensing module (satellite) according to the deployment location.

[0053] The magnetic levitation physical subsystem (5) is used to complete the physical reality of the transportation needs of system users, including but not limited to magnetic levitation vehicles, on-board equipment, track lines, linear motors, switch mechanisms, traction substations, 5G communication base stations and related ancillary equipment.

[0054] The communication subsystem (6) provides a ubiquitous interconnection foundation for the ubiquitous interconnection formed by sensing interaction and data transmission between the various subsystems of the maglev operation and control intelligent system;

[0055] Specifically, the vehicle base station is connected to the vehicle computer, and the vehicle computer is connected to the vehicle safety-critical device, the vehicle non-safety-critical device and the vehicle sensing module through bus technology, and the bus adopts optical fiber transmission medium;

[0056] Specifically, the vehicle-mounted base station is connected to the ground base station through 38GHz millimeter wave communication technology, and the ground base station is connected to the line safety-critical equipment (traction motor, switch mechanism, etc.) and the trackside sensing module through the trackside switch;

[0057] Specifically, the trackside switch is connected to the MDAI-MS via IP-based Ethernet communication technology, and information transmission passes through the Ethernet switch, routing equipment and firewall;

[0058] Specifically, the far-field sensing module is connected to the vehicle-mounted computer according to the information transmission specific medium of the far-field sensing device, for example, the vehicle-mounted computer receives the satellite navigation signal through the vehicle-mounted antenna and the vehicle-mounted receiver;

[0059] Specifically, the user subsystem (1) is connected to the MDAI-MS and MAGI-MV via IP-based Ethernet communication technology.

[0060] It should be noted that the MAGI-MS, MDAI-MS and MDAI-MV provided in the implementation of this application use the same network structure and training method, but differ in the scale of the training data set, the scale of the model parameters and the downstream task interface; the network structure is the network structure of a multimodal artificial intelligence model.

[0061] The specific network structure and computing principle of the multimodal artificial intelligence model provided in the implementation of this application are explained in detail below.

[0062] like Figure 2 As shown in the figure, the network structure of the multimodal artificial intelligence model (including special and general) consists of a set of m modal signals as the network input, and the network structure can be divided into an encoder and feature layer for each modal signal, a multimodal information interaction layer, an output layer for each modal signal, and a downstream task interface.

[0063] Taking the mth modal encoder as an example, the mth modal encoder discretizes, vector encodes and position encodes the mth modal signal;

[0064] Specifically, there is the mth modal signal x m As the input of the multimodal artificial intelligence model, the mth modal encoding first discretizes it into N sub-samples of the same size to obtain the sample sequence Where N is determined according to the characteristics of the mth modal signal. For example, when the signal is a two-dimensional image data, N = 14. m Convert the discretized sample sequence into a computable vector in Encode the mth modal signal vector E m The weight parameter is then concatenated with the category label cls mTo represent the semantic information represented by the sample sequence, the position code E is added at the end p To represent the position information of the sub-sample in the sample sequence, the output y is obtained m =[cls m ,V m ]+E p .

[0065] Taking the mth modal feature layer as an example, the mth modal feature layer is the output from the mth modal encoder Perform feature extraction;

[0066] Specifically, for the mth modal signal, the output y of the mth modal encoder is m As input, the mth modal feature layer is decoded by L layers of Transformer neural network T en The output of each layer is used as the input of the next layer, i.e., the feature in It represents the weight parameter of the jth layer Transformer neural network decoder of the mth modal feature layer. L is determined according to the characteristics of the mth modal signal. For example, when the signal is a two-dimensional image data, L = 6. Finally, the mth modal signal feature is output through the mth modal feature layer.

[0067] It should be noted that the Transformer neural network T is an existing technology, in which a layer of Transformer neural network consists of a layer of decoder T en and a decoder layer T de It is stacked and will not be described in detail here.

[0068] The multimodal information interaction layer encodes the features from each modal signal by modal category, modal fusion, and modal position. Then the multimodal information feature layer extracts fusion features to achieve unified expression of the same semantic information in the interactive space by different modal signals.

[0069] Figure 3 The specific structure of the multimodal information interaction layer provided by this application is shown.

[0070] Specifically, the multimodal information interaction layer takes the output of the multimodal signal feature layer as input; assuming that the feature pair is obtained from n modal signals with the same semantics The multimodal information interaction layer first generates signal features for each modality. Add the corresponding modal category tag mod j ,get Then sequentially concatenate the signal features of each modality to obtain the fusion sequence Then add the modal position code E′ to the features of different modal signalsp To characterize the position information of different modalities in the fusion sequence, we can get Then it is input to the multimodal information feature layer, which is composed of Z layers of Transformer neural network T stacked together, and the output of each layer is used as the input of the next layer, that is, y 1 ′=T(y′;θ 1 ),…,y′ Z =T(y′ Z-1 ,θ Z ), where θ j , j = 1, ..., Z represents the weight parameter of the jth layer Transformer neural network of the multimodal interaction layer. In this embodiment, Z = 3. Finally, the multimodal feature y′ is output through the multimodal information interaction layer. Z .

[0071] Taking the m-th modal output layer as an example, the m-th modal output layer analyzes the multimodal features from the multimodal interaction layer and extracts the implicit features of the m-th modal signal to achieve cross-modal feature conversion;

[0072] Specifically, assuming that the first modal signal is used as the input of the multimodal artificial intelligence model, the multimodal feature is obtained through the output of the first modal encoder, the first modal feature layer and the multimodal information interaction layer, and the mth modal output layer takes the multimodal feature as input; there is a multimodal feature y′ Z , the m-th modal output layer consists of a G-layer Transformer neural network decoder T de The output of each layer is used as the input of the next layer, that is, in represents the weight parameter of the jth layer Transformer neural network decoder of the mth modal output layer. G is determined according to the characteristics of the mth modal signal. For example, when the signal is a two-dimensional image data, G = 3. Finally, the mth modal implicit feature is output through the mth modal output layer. The downstream task interface takes the implicit features output by each modal output layer as input. The downstream task interface can call a single modal output layer or multiple modal output layers to complete specific safety-critical tasks or non-safety-critical tasks, generally including user demand analysis tasks, path planning tasks, speed target curve estimation tasks, generative tasks, etc.

[0073] Specifically, assuming that the qth downstream task interface needs to output a signal in the mth mode, the mth mode implicit feature As input, the qth downstream task interface can be composed of a B-layer Transformer neural network decoder T de And Q layers of multilayer perceptrons M are stacked, and the output of each layer is used as the input of the next layer, that is, in It represents the weight parameters of the j-th layer Transformer neural network decoder or multi-layer perceptron of the q-th downstream task interface. B and Q are determined according to the characteristics of the m-th modal signal. For example, when the signal is two-dimensional image data, B=2, Q=2. Finally, the signal in the m-th modal form is output through the q-th downstream task interface.

[0074] It should be noted that the multi-layer perceptron is an existing technology implemented by a neural network and will not be described in detail here.

[0075] Figure 4 The training steps of the multimodal artificial intelligence model provided in the implementation of this application are shown, and a specific explanation is given below:

[0076] Step S101, pre-training each modality encoder, each modality feature layer, multimodal information interaction layer and each modality output layer;

[0077] Specifically, each modality encoder, each modality feature layer, and each modality output are trained only by data consisting of the corresponding modality signal. For example, the first modality encoder and the first modality feature layer are trained only by a large amount of unlabeled data or structured knowledge of the first modality type in a self-supervised manner, and the same is true for the remaining modality encoders and feature layers.

[0078] Among them, only the vector encoding E i Parameters The training goal is to map the corresponding modal signal into a computable vector suitable for extracting features in the corresponding modal feature layer; each modal feature layer has L layers of Transformer neural network decoders T en The weight parameter Training is required, and the training goal is to enable each modal feature layer to fully extract the shallow texture features and deep semantic features of the corresponding modal signal, and to have the generalization ability to understand the corresponding modal signal;

[0079] Among them, the weight parameter θ of the Z-layer Transformer neural network T in the multimodal information interaction layer j , j = 1, ..., Z is trained by a large number of unlabeled matching sample pairs. For a matching sample pair {x 1 ,…,x m}, where x i ,i=1,…,m represents the sample from the i-th modal signal. Mutual matching means that the samples of different modal signals contain the same content semantically or can describe each other in pairs. The training goal of the multimodal information interaction layer is to eliminate the inconsistency between the different modal features extracted by different modal feature layers based on their respective modal signals for a mutually matched sample pair.

[0080] Among them, the weight parameters of the G-layer Transformer neural network decoder in each modality output layer are Training is required, and its training goal is to enable each modal output layer to extract implicit features that can be converted into corresponding modal signals based on the multimodal features output by the multimodal information interaction layer;

[0081] The self-supervised pre-training process of each modality encoder, each modality feature layer, multimodal information interaction layer and each modality output layer uses the prediction loss function and contrastive loss function to enable the multimodal artificial intelligence model to have cross-modal understanding capabilities;

[0082] Taking the mth modal signal as an example, there are samples x m , after discretization, we get the sample sequence N is the number of sub-samples. The purpose of the prediction loss function is to enable the multimodal artificial intelligence model to have the ability to understand a single modal signal. Specifically, it refers to maximizing the multimodal artificial intelligence model according to the sample sequence. The conditional probability P of the jk-th to j-1-th sub-samples predicting the j-th sub-sample is as follows:

[0083]

[0084] It should be noted that each modal signal has a corresponding prediction loss function The total prediction loss function is as follows:

[0085]

[0086] The purpose of the contrast loss function is to ensure that the semantics of different modal features extracted from matching modal signals have a high correlation, and the semantics of different modal features extracted from mismatching modal signals have a low correlation, so that different modal features are aligned in the multimodal space, and the multimodal artificial intelligence model has cross-modal understanding capabilities; taking the contrast loss function of the first modal signal and the mth modal signal as an example, there are b matching sample pairs in a Batch_size, as follows:

[0087]

[0088] Among them, Sr(·,·) is a function for calculating the semantic relevance of features. The higher the relevance, the higher the Sr value, and vice versa. and are the implicit features of the 1st and mth modal signal features in the ith sample pair, respectively. and are the implicit features of the 1st and mth modal signal features in the jth sample pair respectively;

[0089] Total contrast loss function It is the sum of the contrast losses calculated pairwise between all modal signals, that is:

[0090]

[0091] It should be noted that the above step S101 is a multimodal artificial intelligence model pre-training process, and the training data can come from web page crawling, public databases, public data sets, etc.; the scale of the training data set is in the GB level, and the number of model parameters is in the 100 million level, and the order is: MAGI-MS>MDAI-MS>MDAI-MV;

[0092] Step S102, supervised training of weight parameters of downstream task interfaces, and fine-tuning of multimodal artificial intelligence models in combination with instruction fine-tuning;

[0093] Specifically, according to the type of task that the system needs to complete, set the downstream task interface, and supervise the training of the weight parameters of the downstream task interface through labeled training data. The training goal is to make the labels predicted by the multimodal artificial intelligence model consistent with the true labels to adapt to downstream tasks in dedicated fields; for example, for a given m-th modal sample x m , with a corresponding true label l m , the cross entropy loss function of this sample is as follows:

[0094]

[0095] Among them, P(x m ) is the multimodal artificial intelligence model for sample x m The probability of class prediction;

[0096] In addition, instruction tuning is used to fine-tune the weight parameters of the multimodal artificial intelligence model. Instruction tuning is to add additional instructions to the input to guide the multimodal artificial intelligence model to understand human intentions. The goal is to help the multimodal artificial intelligence model better complete downstream tasks in specific fields through prompts. Given an input x presented in the form of the mth modality m , after fine-tuning the command, we get Multimodal AI model analysis m , and generate correct responses based on instructions to complete specific downstream tasks;

[0097] Specifically, for MDAI-MS, taking user demand analysis as an example:

[0098] Input: Transportation requirements (in text or voice form);

[0099] After fine-tuning the instructions: ① "Transportation demand | Please allocate a maglev vehicle ID", ② "Transportation demand, please confirm the transportation object, transportation origin, transportation destination, etc.", ③ "Transportation object, transportation origin, transportation destination, etc. | Request to solve the optimal route from the transportation origin to the destination"...

[0100] Predefined answers: ① “Maglev vehicle ID”, ② “transport object, transport origin, transport destination, etc.”, ③ “optimal route from transport origin to destination”;

[0101] Specifically, for MDAI-MV i , taking vehicle-level safety-critical tasks as an example:

[0102] Input: section route instructions, static operation curves, real-time perception information (in text form);

[0103] After fine-tuning the instructions: ① "Section route instructions, static operation curve, real-time perception information | Please dynamically adjust the static operation curve according to the perception information", ② "Section route instructions, optimal speed target curve | Please generate control instructions for safety-critical equipment in the section"...

[0104] Predefined answers: ① “optimal speed target curve within the section”, ② “control instructions for safety-critical equipment such as motors”;

[0105] Step S103, constraining the multimodal artificial intelligence model through reinforcement learning with human feedback;

[0106] Reinforcement learning based on artificial feedback refers to: establishing a reward model that takes actions as input and outputs scalar reward values. During the training process, the reward model continuously fits the safety-critical thinking in human intentions (for example, EN 50126, EN50128, IEC 61508, etc.) based on artificial feedback; further, under the reinforcement learning framework, the intelligent maglev transportation system is regarded as the environment, the weight parameters of the multimodal artificial intelligence model are regarded as the strategy, and the output is regarded as the action. The reward model evaluates the action to calculate the reward, and the loss function aims to maximize the reward, thereby constraining the output of the multimodal artificial intelligence model;

[0107] For example, for MDAI-MS, the track line, virtual section setting, maglev vehicle hardware rated parameters, power supply system parameters, etc. are used as boundary conditions, and the optimization of traction, suspension, guidance smoothness, and operating energy consumption is used as the objective function, and combined with reinforcement learning with human feedback to generate optimal paths, static operating curves, etc. guided by safety requirements; for MDAI-MV, real-time perception information is used as dynamic parameters, and the quantitative relationship between the kinematic parameters of the maglev vehicle-coil current and voltage is used as a model, and the optimization of traction, suspension, guidance smoothness, and operating energy consumption is used as the objective function, and combined with reinforcement learning with human feedback to generate the optimal speed target curve and control instructions guided by safety requirements.

[0108] The following is a detailed introduction to the operation process of the multi-modal maglev transportation control intelligent system provided in the implementation of this application. Figure 6 As shown, taking the conventional electromagnetic suspension (EMS) rail transit as an example, the train traction work is completed by a linear synchronous motor; the line in the figure spans 4 power supply sections, each power supply section is composed of 6 traction motors (stator segments), each traction motor is 2 km long, the length of the maglev car is 100 m, and each traction motor is provided with 2 auxiliary parking points; in this embodiment, the maglev car needs to run from station A to station B first, and then run from station B to the traction motor before the switch mechanism, and after turning the switch mechanism, finally run to station E to complete the transportation demand.

[0109] like Figure 7 As shown in the figure, the operation process of the multi-modal maglev operation and control intelligent system is as follows:

[0110] Step S201, the system user submits the transportation demand to MDAI-MS through the user interface;

[0111] Specifically, three users submit transportation requirements to the system, namely x 1 = "From Station A to Station C", x 2 = "I got on the bus at station B, please take me to station C" and x 3 = "Send the goods from station A to station C", the above requirements are transmitted to MDAI-MS through the user interface;

[0112] Step S202, MDAI-MS analyzes the transportation demand, allocates the maglev vehicle, generates the required driving information according to the physical static parameters and transmits it to the corresponding MDAI-MV;

[0113] Specifically, MDAI-MS analyzes the transportation demand and obtains the transportation object, transportation origin, and transportation destination, which are "system user 1, station A, station C", "system user 2, station B, station C", and "cargo, station A, station C". It dispatches Maglev Vehicle 01 to station A to serve the system user, and the system user obtains access to MAGI-MV 01 MDAI-MS generates the required information for driving according to the physical static parameters from station A to station C, such as track line parameters, speed limit parameters, maglev vehicle traction, braking performance, etc., such as the preferred route for maglev vehicle 01 to run, the static running curve from the transportation origin to the destination, etc., and packages the above information and transmits it to MDAI-MV 01 ;

[0114] In addition, the standard 3D shape information of the transport object is uploaded by the system user or the 3D face database is called, and transmitted from MDAI-MS to MDAI-MV. 01 ;

[0115] Step S203, MDAI-MV determines whether the identity, behavior, location and other information of the transport object meet the requirements for safe departure of the maglev vehicle through the visual information inside the vehicle;

[0116] Specifically, MDAI-MV 01 The 3D depth camera in the maglev vehicle obtains the 3D modeling information of the objects in the vehicle and determines whether it matches the standard 3D shape information of the transported object. After the identity matching is successful, MDAI-MV 01 Based on the visual monitoring information provided by the 3D depth camera inside the vehicle, the position information of the transport object, including behavior and position information, is inferred to determine whether it meets the requirements for safe driving of the maglev vehicle;

[0117] Step S204: close the vehicle door, and the MDAI-MV generates the control instructions required for the maglev vehicle to travel according to the real-time ubiquitous sensing information and sends them to the relevant safety-critical equipment to complete the transportation needs;

[0118] Specifically, MDAI-MV 01 After confirming that the conditions for the safe departure of the maglev vehicle are met, a control command to close the door is generated and transmitted to the door controller; MDAI-MV 01 Based on the static operation curve and track line resource information provided by MDAI-MS, and based on real-time ubiquitous perception information, including track line parameters, weather environment around the line, maglev vehicle time and space positioning information, operating speed, turnout mechanism status, etc., a dynamic operation curve is generated with a step-by-step strategy, and the kinematic parameters such as maglev vehicle speed and acceleration are converted into electromagnetic control quantities such as traction voltage and current according to the mapping relationship between the maglev vehicle operating speed and the electromagnetic control quantity of the traction motor, and directly converted by MDAI-MV 01Generate control instructions and send them to corresponding safety-critical equipment, such as enabling the traction motor to provide the required electrical resources and moving the turnout to the correct position;

[0119] It should be noted that step-forward means that, in general, the end point of the current movement authorization of the maglev vehicle is the forward auxiliary parking point, and the current dynamic operation curve is calculated based on this; if there is a switch mechanism between the maglev vehicle and the forward auxiliary parking point, the switch mechanism is the end point of the movement authorization; if there are other maglev vehicles between the maglev vehicle and the forward auxiliary parking point, the braking parking point with the worst braking performance of the previous vehicle is the end point of the movement authorization, forming a tracking of the previous vehicle, but the same traction motor is not allowed to provide traction to two maglev vehicles at the same time, and the two adjacent vehicles must be separated by a traction motor. At this time, the generation of the dynamic operation curve also needs to consider the information provided by the MDAI-MV of the previous vehicle, including: the running status of the previous vehicle, the running speed of the previous vehicle, the driving acceleration of the previous vehicle, the driving transportation destination of the previous vehicle, etc.

[0120] It should be noted that, based on the communication subsystem, the MDAI-MVs of different maglev vehicles can exchange information with each other through the MDAI-MS, and the control instructions of the MDAI-MV can be directly issued to the traction motor in the maglev physical subsystem and the traction motor and switch mechanism in the forward direction;

[0121] It should be noted that based on the vehicle-motor communication strategy, the MDAI-MV directly issues control instructions to the traction motors where it is located and the traction motors in the forward direction according to the generated dynamic operation curve, so as to reduce the impact of switching power supply partitions on the running smoothness;

[0122] It should be noted that the MDAI-MV is only allowed to communicate with the traction motor where the maglev vehicle is currently located and the traction motor adjacent to it in the forward direction. When the maglev vehicle leaves the traction motor, the communication with the traction motor is interrupted and the electromagnetic control quantity of the traction motor is reset. A traction motor or switch mechanism is only allowed to communicate with the MDAI-MV of one maglev vehicle to avoid control confusion of the traction motor or switch mechanism.

[0123] It should be noted that the format of the control instruction is: control instruction identifier + communication protocol identifier + magnetic levitation vehicle ID + control object IP address + length identifier + content + check code;

[0124] It should be noted that during the operation of the magnetic levitation vehicle, the guide gap and the suspension gap are stabilized near the set value by the guide electromagnet and the suspension electromagnet based on the closed-loop control technology;

[0125] Step S205: During the operation of the maglev vehicle, the MAGI-MV completes the non-safety-critical requirements submitted by the system user;

[0126] During the operation of the maglev vehicle, the system user submits user requirements to the MAGI-MV assigned to the maglev vehicle 01. The MAGI-MV analyzes whether the user requirements include safety-critical tasks and only allows the MAGI-MV to perform non-safety-critical tasks in the user requirements, such as turning on the vehicle air conditioner, audio services, checking the status of cargo, etc.

[0127] Step S206: The maglev vehicle runs to the transportation destination, completes the transportation requirements submitted by the system user, and closes the system user's access rights to the MAGI-MV.

[0128] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A multi-modal maglev transportation control intelligent system, which is an intelligent maglev transportation system, characterized in that: include: User subsystem (1), user interface (2), artificial intelligence subsystem (3), ubiquitous perception subsystem (4), magnetic levitation transportation physics subsystem (5), communication subsystem (6); wherein: The user subsystem (1) is used to submit transportation requirements, user requirements and transportation objects, including system users and transportation objects; The user interface (2) is used to realize interaction and information exchange between the user subsystem (1) and the artificial intelligence subsystem (3); The artificial intelligence subsystem (3) is used to complete system-level safety-critical tasks, vehicle-level safety-critical tasks, and vehicle-level non-safety-critical tasks, including system-level multimodal dedicated artificial intelligence MDAI-MS, vehicle-level multimodal dedicated artificial intelligence MDAI-MV, and vehicle-level multimodal general artificial intelligence MAGI-MV; The ubiquitous sensing subsystem (4) collects real-time multimodal information of the user subsystem (1) and the magnetic levitation transportation physical subsystem (5) at a large spatial and temporal scale, and transmits the information to the artificial intelligence subsystem (3). The magnetic levitation physical subsystem (5) is used to realize the physical reality of the transportation needs of system users; The communication subsystem (6) provides a ubiquitous interconnection foundation for perception interaction and data transmission between subsystems of the maglev operation and control intelligent system.

2. The multi-modal maglev intelligent control system according to claim 1, characterized in that: The artificial intelligence subsystem (3): The MDAI-MS serves the entire maglev transportation system and is deployed in the maglev transportation system control center to complete system-level safety-critical tasks involved in the operation of the maglev transportation system. The MDAI-MS is built on the basis of a neural network, with a decoder-encoder as the basic architecture, and performs self-supervised pre-training through a large amount of unlabeled multimodal data and supervised fine-tuning through labeled multimodal data. It has the ability of semantic understanding and logical reasoning, and its generation strategy is safety-oriented based on reinforcement learning based on artificial feedback, and the downstream task interface is set according to the MDAI-MS function; The MDAI-MV corresponds to the maglev vehicle included in the maglev transportation system one by one and is deployed on the on-board computer of the corresponding maglev vehicle. i Serving the i-th maglev vehicle to complete vehicle-level safety-critical tasks related to transportation needs; The MDAI-MV i Based on neural network construction, with decoder-encoder as the basic architecture, it performs self-supervised pre-training through a large amount of unlabeled multimodal data, has semantic understanding and logical reasoning capabilities, and uses reinforcement learning based on human feedback to make its generation strategy safety-oriented, and sets the downstream task interface according to the MDAI-MS function; The MAGI-MV corresponds to the maglev vehicle included in the maglev transportation system one by one and is deployed on the on-board computer of the corresponding maglev vehicle. i Serves the i-th maglev vehicle, and MAGI-MV i With MDAI-MV i Together they serve the i-th maglev vehicle to complete the vehicle-level non-safety-critical tasks involved in the transportation process; The MAGI-MV i It is built on a neural network with a decoder-encoder as the basic architecture. It uses a large amount of unlabeled multimodal data for self-supervised pre-training. It has the ability of semantic understanding and logical reasoning. It also uses reinforcement learning based on artificial feedback to ensure that its generation strategy does not violate ethics. The downstream task interface is set according to the MDAI-MS function.

3. The multi-modal maglev transportation control intelligent system according to claim 1, characterized in that: The ubiquitous sensing subsystem (4): The first sensing module is used to estimate the running speed and time-space position of the maglev vehicle, including a speed sensor, a synchronous clock, a positioning sensor, and a satellite navigation; The second sensing module is used to estimate the guide gap and suspension gap of the magnetic levitation vehicle, including a guide gap sensor and a suspension gap sensor; The third sensing module is used to collect dynamic shape information of the transport object, track line and external environment in the maglev vehicle, including a 3D depth camera in the vehicle, a drone vision camera, an infrared vision camera and a laser radar; The fourth sensing module is used to monitor the operation and fault conditions of the maglev vehicle, track line, switch mechanism, motor equipment and auxiliary equipment, and includes a vibration sensor, an acoustic emission sensor, a voltage sensor, and a current sensor.

4. The multi-modal maglev transportation control intelligent system according to claim 1, characterized in that: MAGI-MS, MDAI-MS and MDAI-MV use the same network structure, which is a network structure of a multimodal artificial intelligence model. A set of m modal signals is used as the network input. The network structure is divided into each modal signal encoder and feature layer, multimodal information interaction layer, each modal signal output layer and downstream task interface. Specifically: The m-th modal signal encoder discretizes, vectorizes and positions the m-th modal signal; The m-th modal signal feature layer extracts features from the output of the m-th modal signal encoder; The multimodal information interaction layer encodes the features from each modal signal by modal category, modal fusion, and modal position. Then the multimodal information feature layer extracts fusion features to achieve unified expression of the same semantic information in the interactive space by different modal signals. The m-th modal signal output layer analyzes the multimodal features from the multimodal interaction layer and extracts the implicit features of the m-th modal signal to achieve cross-modal feature conversion; The downstream task interface takes the implicit features output by each modal output layer as input, and calls a single modal output layer or multiple modal output layers to complete specific safety-critical tasks or non-safety-critical tasks.

5. The multi-modal maglev intelligent control system according to claim 4, characterized in that: The training steps of a multimodal AI model include: Step S101: pre-training each modality encoder, each modality feature layer, multimodal information interaction layer and each modality output layer; Step S102: supervised training of weight parameters of downstream task interfaces, and fine-tuning of multimodal artificial intelligence models in combination with instruction fine-tuning; Step S103: Constrain the multimodal artificial intelligence model through reinforcement learning with human feedback.

6. The multi-modal maglev intelligent control system according to claim 5, characterized in that: The step S101: Each modality encoder, each modality feature layer, and each modality output are trained with data consisting only of the corresponding modality signal; The self-supervised pre-training process of each modality encoder, each modality feature layer, multimodal information interaction layer and each modality output layer uses prediction loss function and contrast loss function to enable the multimodal artificial intelligence model to have cross-modal understanding capabilities.

7. The multi-modal maglev intelligent control system according to claim 1, characterized in that: The system operation process is: Step S201: The system user submits a transportation demand to MDAI-MS through the user interface; Step S202: MDAI-MS analyzes the transportation demand, allocates the maglev vehicle, generates the required driving information according to the physical static parameters and transmits it to the corresponding MDAI-MV; Step S203: MDAI-MV determines whether the identity, behavior, and location information of the transport object meet the requirements for safe departure of the maglev vehicle through the visual information inside the vehicle; Step S204: close the vehicle door, and the MDAI-MV generates the control instructions required for the maglev vehicle to travel according to the real-time ubiquitous sensing information and sends them to the relevant safety-critical equipment to complete the transportation needs; Step S205: During the operation of the maglev vehicle, the MAGI-MV completes the non-safety-critical requirements submitted by the system user; Step S206: The maglev vehicle runs to the transportation destination, completes the transportation requirements submitted by the system user, and closes the system user's access rights to the MAGI-MV.

Images