Remote diagnosis management method and system based on vehicle-mounted terminal

Abstract

The invention relates to the technical field of terminal management, and particularly discloses a remote diagnosis management method and system based on a vehicle-mounted terminal, and the method comprises the steps: obtaining multi-source heterogeneous data collected by the vehicle-mounted terminal in real time; performing dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set; constructing an edge-cloud collaborative multi-modal model according to the multi-source data priority set, and obtaining multiple pieces of dynamic data information according to the edge-cloud collaborative multi-modal model; and acquiring data association information and semantic information of each piece of dynamic data distribution information, and constructing a vehicle data association topological graph according to the data association information and the semantic information. According to the invention, the analysis and utilization capability of the vehicle data is effectively improved. The constructed edge-cloud collaborative multi-modal model can efficiently process multi-source data, reduce transmission delay and cloud burden, and improve real-time performance and decision accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of terminal management, and in particular to a remote diagnosis management method and system based on a vehicle-mounted terminal. Background Art

[0002] With the rapid development of vehicle intelligence and networking, vehicle-mounted terminals play an increasingly crucial role during vehicle operation. Vehicle-mounted terminals can collect various data of vehicles, including information in multiple aspects such as the power system, chassis system, and electronic system, providing a rich data source for vehicle fault diagnosis and performance optimization. At the same time, the rise of edge computing and cloud computing technologies also provides strong support for the processing and analysis of massive vehicle-mounted data.

[0003] Existing remote diagnosis management technologies based on vehicle-mounted terminals have key defects in many aspects. In terms of data processing, multi-source heterogeneous data collection is incomplete, and there is a lack of effective priority classification and preprocessing mechanisms. The data types collected by traditional methods are limited, making it difficult to reflect the overall picture of vehicle operation. Insufficient attention is paid to data timeliness and potential risks, and high-noise data interference is large, resulting in insufficient diagnostic basis. Summary of the Invention

[0004] The purpose of the present invention is to provide a remote diagnosis management method and system based on a vehicle-mounted terminal to solve the technical problems mentioned in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions: A remote diagnosis management method based on a vehicle-mounted terminal, comprising: Obtaining multi-source heterogeneous data collected in real time by the vehicle-mounted terminal; Performing dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set; Constructing an edge-cloud collaborative multimodal model according to the multi-source data priority set, and obtaining multiple dynamic data information according to the edge-cloud collaborative multimodal model; Obtaining data association information and semantic information of each dynamic data allocation information, and constructing a vehicle data association topology graph according to the data association information and semantic information; Performing feature extraction on the vehicle data association topology graph to obtain cross-modal fault feature information, and obtaining diagnostic fault information according to the cross-modal fault feature information; Performing feature extraction on the diagnostic fault information to obtain fault classification data, and generating fault resource scheduling information according to the fault classification data; Dynamically adjusting the vehicle-mounted terminal according to the fault resource scheduling information.

[0006] Preferably, the step of dynamically classifying the multi-source heterogeneous data to obtain a multi-source data priority set includes: Extract data type identifiers from the multi-source heterogeneous data, where the data type identifiers include sensor data, image data, communication protocol data, and log data; Assign basic priority weights to each data type identifier according to a preset rule library, where the preset rule library includes data real-time requirements, data volume, and security level; Obtain the network bandwidth resource occupancy rate and resource load value of the real-time monitoring vehicle terminal, and dynamically adjust the basic priority weights according to the resource occupancy rate and load value to generate dynamic priority weights; Sort the multi-source heterogeneous data according to the dynamic priority weights to generate a multi-source data priority set, and divide the multi-source data priority set into a real-time processing queue, a cache queue, and an asynchronous processing queue in descending order of priority.

[0007] Preferably, the step of constructing an edge-cloud collaborative multi-modal model according to the multi-source data priority set includes: Extract real-time processing queue data from the multi-source data priority set and distribute the data to edge computing nodes; Obtain the computing power distribution according to the edge computing nodes; Generate intermediate feature vectors based on the processing results of the edge computing nodes and upload the intermediate feature vectors to the cloud; construct a multi-modal fusion model in the cloud, where the multi-modal fusion model includes a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch; Input the intermediate feature vectors into the multi-modal fusion model, and generate dynamic data information through cross-modal feature alignment and weight allocation strategies.

[0008] Preferably, the step of constructing a vehicle data association topology graph according to the data association information and semantic information includes: Extract device identifiers, data flow paths, and timestamp sequences from the data association information; Generate device function description tags and data dependency information according to the semantic information; Map the device identifiers to the function description tags to construct device-function nodes; Generate data transmission edges between nodes according to the data flow paths and timestamp sequences, and label the transmission frequency and delay parameters; construct a vehicle data association topology graph based on the device-function nodes, data transmission edges, and data dependency information.

[0009] Preferably, the step of extracting features from the vehicle data association topology graph to obtain cross-modal fault feature information includes: Extract the node degree distribution, edge weight distribution, and subgraph clustering features from the vehicle data association topology graph; Map the topology graph to a low-dimensional vector space based on a graph embedding algorithm to generate a graph structure feature vector; Combine the temporal features and image features in the dynamic data information, and perform feature fusion through an attention mechanism to obtain dynamic fusion feature data; Obtain a preset fault mode library, and match the dynamic fusion feature data according to the preset fault mode library to filter out abnormal feature segments that deviate from the normal mode; Calculate the similarity between the abnormal feature segments and historical fault cases to generate cross-modal fault feature information.

[0010] Preferably, the step of generating fault resource scheduling information according to the fault classification data includes: Obtain a preset resource allocation policy model, and input the fault classification data into the preset resource allocation policy model to obtain resource allocation information, where the resource allocation information includes fault level information, repair time-consuming information, and impact range parameter information; Allocate redundant computing resources for high-priority faults, allocate dynamic elastic resources for medium-priority faults, and allocate asynchronous repair queues for low-priority faults according to the fault level information; Obtain the real-time position information and network coverage status of the in-vehicle terminal to generate a resource scheduling path plan; Generate fault resource scheduling information according to the resource scheduling path plan and the resource allocation information, where the fault resource scheduling information includes computing node switching instructions, data transmission link switching instructions, and service degradation instructions.

[0011] Preferably, the step of dynamically adjusting the in-vehicle terminal according to the fault resource scheduling information includes: Obtain the software service response delay according to the in-vehicle terminal; Obtain the in-vehicle terminal status according to the software service response delay, where the in-vehicle terminal status includes resource overload, network link interruption, and critical service failure; If resource overload is detected, migrate some tasks to adjacent edge nodes or the cloud according to the fault resource scheduling information; if network link interruption is detected, enable local cached data and switch to an alternative communication protocol; If critical service failure is detected, trigger the service degradation mode to give priority to ensuring the operation of core functions; Feed back the adjusted operating status to the cloud according to the software service response delay.

[0012] The present invention also provides a remote diagnosis management system based on an in-vehicle terminal, including: The first acquisition module is used to acquire multi-source heterogeneous data collected in real time by the vehicle terminal; The classification module is used to perform dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set; The establishment module is used to construct an edge-cloud collaborative multimodal model according to the multi-source data priority set, and obtain multiple dynamic data information according to the edge-cloud collaborative multimodal model; The second acquisition module is used to acquire the data association information and semantic information of each dynamic data allocation information, and construct a vehicle data association topology graph according to the data association information and semantic information; The first extraction module is used to extract features from the vehicle data association topology graph to obtain cross-modal fault feature information, and obtain diagnostic fault information according to the cross-modal fault feature information; The second extraction module is used to extract features from the diagnostic fault information to obtain fault classification data, and generate fault resource scheduling information according to the fault classification data; The mobilization module is used to dynamically adjust the vehicle terminal according to the fault resource scheduling information.

[0013] Preferably, the classification module includes: The extraction unit is used to extract data type identifiers from the multi-source heterogeneous data, where the data type identifiers include sensor data, image data, communication protocol data, and log data; The allocation unit is used to allocate basic priority weights for each data type identifier according to a preset rule library, and the preset rule library includes data real-time requirements, data volume, and security level; The first acquisition unit is used to acquire the network bandwidth resource occupancy rate and resource load value of the vehicle terminal monitored in real time, dynamically adjust the basic priority weight according to the resource occupancy rate and load value, and generate a dynamic priority weight; The sorting unit is used to sort the multi-source heterogeneous data according to the dynamic priority weight, generate a multi-source data priority set, and divide the multi-source data priority set into a real-time processing queue, a cache queue, and an asynchronous processing queue in descending order of priority.

[0014] Preferably, the establishment module includes: The second acquisition unit is used to extract real-time processing queue data from the multi-source data priority set and distribute the data to the edge computing nodes; The third acquisition unit is used to obtain the computing power distribution according to the edge computing nodes; The generation unit is used to generate an intermediate feature vector based on the processing results of the edge computing nodes and upload the intermediate feature vector to the cloud; A building unit is used to build a multimodal fusion model in the cloud. The multimodal fusion model includes a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch. An input unit is configured to input the intermediate feature vector into the multimodal fusion model, and generate dynamic data information through a cross-modal feature alignment and weight assignment strategy.

[0015] The beneficial effects of this application are as follows: The present invention effectively improves the ability to analyze and utilize vehicle data. The constructed edge-cloud collaborative multimodal model can efficiently process multi-source data, reduce transmission latency and cloud burden, and improve real-time performance and decision-making accuracy. The constructed vehicle data association topology graph intuitively shows the data association and collaborative working methods between devices, facilitating fault troubleshooting and system optimization. The extracted cross-modal fault feature information synthesizes various features, providing strong support for accurate fault diagnosis. In terms of resource scheduling and system adjustment, it ensures the efficiency of fault handling and the stability of the system. Reasonable fault resource scheduling information is generated based on the fault classification data, realizing differential allocation of resources and scientific path planning to ensure the rational use of resources. The in-vehicle terminal is dynamically adjusted according to the fault resource scheduling information, the system status is monitored in real time, corresponding measures are taken for different problems, and the adjusted status is fed back to the cloud, realizing dynamic optimization and collaborative working of the system, and improving the reliability and operating efficiency of the in-vehicle system. Description of the Drawings

[0016] Figure 1 It is a schematic flowchart of the method according to an embodiment of this application.

[0017] Figure 2 It is a schematic structural diagram of the system according to an embodiment of this application.

[0018] The realization, functional features, and advantages of the purpose of this application will be further described with reference to the embodiments and the accompanying drawings. Detailed Embodiments

[0019] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] As Figure 1 shown, this application provides a remote diagnosis and management method based on an in-vehicle terminal, including: S1. Obtain multi-source heterogeneous data collected in real time by the in-vehicle terminal; S2. Perform dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set; S3. Construct an edge-cloud collaborative multimodal model according to the multi-source data priority set, and obtain multiple dynamic data information according to the edge-cloud collaborative multimodal model; S4. Obtain the data association information and semantic information of each of the dynamic data allocation information, and construct a vehicle data association topology graph according to the data association information and semantic information; S5. Extract features from the vehicle data association topology graph to obtain cross-modal fault feature information, and obtain diagnostic fault information according to the cross-modal fault feature information; S6. Extract features from the diagnostic fault information to obtain fault classification data, and generate fault resource scheduling information according to the fault classification data; S7. Dynamically adjust the in-vehicle terminal according to the fault resource scheduling information.

[0021] As described in the above steps S1 - S7, in the field of in-vehicle terminal remote diagnosis management, there are many defects in the prior art. First, in terms of data collection, the data types collected by traditional methods are single, only focusing on some key vehicle data, such as engine speed, vehicle speed, etc., ignoring information such as in-vehicle camera video streams and user interaction logs, resulting in the inability to comprehensively reflect the vehicle operation status, making the diagnostic results one-sided and difficult to discover potential fault hazards. Secondly, there is a lack of a dynamic priority classification mechanism during data processing. All data is processed uniformly, and resources cannot be allocated according to data characteristics and actual requirements. The processing of key data is not timely, seriously affecting the diagnostic efficiency and accuracy, and may delay the processing time in case of emergency faults, endangering driving safety; The present invention obtains multi-source heterogeneous data collected in real time by the in-vehicle terminal. The data collection of traditional in-vehicle diagnostic systems has limitations, relying only on some key data, resulting in incomplete and inaccurate diagnostic results and difficulty in discovering potential fault hazards.

[0022] Among them, multi-source heterogeneous data collection covers various aspects such as vehicle CAN bus signals, acceleration sensor data, battery status parameters, in-vehicle camera video streams, user interaction logs, tire pressure monitoring data, and in-vehicle audio system operation records. Taking an autonomous vehicle as an example, CAN bus signals feedback power and braking system parameters, acceleration sensors monitor the vehicle attitude, camera video streams identify road conditions, user interaction logs record operation habits, and tire pressure monitoring data ensures driving safety. By collecting these data, the vehicle operation status can be reflected comprehensively, providing rich basis for subsequent diagnosis, improving the diagnostic accuracy and timeliness, and effectively ensuring driving safety.

[0023] Next, the multi-source heterogeneous data is dynamically prioritized to obtain the multi-source data priority set. Traditional data processing methods do not consider data characteristics and actual needs, which may lead to untimely processing of key data, affecting diagnostic efficiency and accuracy. This step generates a priority weight matrix based on data type, potential risk level of fault, real-time vehicle operating conditions and driving environment complexity, and pre-processes high-noise data. For example, when the vehicle is driving at high speed, data closely related to safety, such as engine temperature and vehicle speed, are given high priority. Once the engine temperature approaches the warning value, the system prioritizes the relevant data and issues an alarm in time; while relatively minor data such as the operation records of the on-board audio system are processed when resources are sufficient. This dynamic priority classification can reasonably allocate computing resources and processing order, ensure timely analysis of key information, improve fault diagnosis efficiency, reduce fault processing time, enhance system response speed, and ensure safe and stable operation of the vehicle. Then, an edge-cloud collaborative multimodal model is constructed based on the multi-source data priority set, and multiple dynamic data information is obtained. Traditional diagnostic models cannot effectively integrate multi-source heterogeneous data, make it difficult to mine potential associations between data, and the diagnostic results are one-sided. The edge-cloud collaborative multimodal model gives full play to the real-time nature of edge computing and the powerful computing power of cloud computing. In electric vehicle fault diagnosis, edge computing nodes process battery and motor data in real time to generate intermediate feature vectors. The cloud multimodal fusion model combines video streams and other data for in-depth analysis, which can accurately determine whether the battery abnormality is caused by internal faults, charging system problems, or driving conditions, thereby improving the accuracy and reliability of fault diagnosis and providing a more reliable basis for subsequent fault handling.

[0024] After that, obtain the data association information and semantic information of the dynamic data allocation information, and construct a vehicle data association topology graph. Traditional methods are difficult to intuitively present the complex relationships among multi-source data, which is not conducive to fault diagnosis. When constructing the topology graph, extract the device identifier, data flow path, and timestamp sequence from the data association information, generate device function description tags and data dependency information according to the semantic information, map the device identifier and function description tags to construct device-function nodes, generate data transmission edges based on the data flow path and timestamp sequence and annotate the parameters, and finally construct the topology graph. In the troubleshooting of vehicle networks, through the topology graph, the scope of the fault impact can be quickly traced, and it can be determined whether the fault is a problem with the node itself or a connection or data transmission fault, improving the efficiency and accuracy of fault diagnosis. Then, extract the features of the vehicle data association topology graph to obtain cross-modal fault feature information and obtain the diagnostic fault information. Traditional fault diagnosis methods are based on single-modal data, which easily leads to missed or misdiagnosed faults. The extraction of cross-modal fault feature information integrates the features of multiple data modalities. In the fault diagnosis of hybrid vehicles, by integrating the data of the power system, battery, and driving conditions and extracting the features of the topology graph, it can accurately determine that the battery anomaly is a fault of the energy recovery device, rather than simply a battery or engine problem, improving the accuracy of fault diagnosis, reducing misjudgment, providing accurate guidance for vehicle maintenance and repair, reducing maintenance costs, and extending the service life of the vehicle.

[0025] Next, extract the features of the diagnostic fault information to obtain fault classification data and generate fault resource scheduling information. Traditional fault handling lacks pertinence and cannot reasonably allocate resources, resulting in resource waste or untimely handling of critical faults. Generating resource scheduling information according to the fault classification can optimize resource allocation. When a vehicle braking system fault (high priority) occurs, allocate redundant computing resources for quick processing; put the low-priority entertainment system fault into the asynchronous repair queue; allocate dynamic elastic resources for the medium-priority engine performance anomaly fault and adjust the processing strategy according to the actual situation, improving resource utilization and reducing the system operation cost to ensure the continuous and stable operation of the vehicle. Finally, dynamically adjust the in-vehicle terminal according to the fault resource scheduling information. Traditional in-vehicle terminals lack the ability to adaptively adjust in the face of faults, affecting the user experience and driving safety. Dynamically adjusting the in-vehicle terminal can automatically optimize the vehicle operation strategy according to the fault situation, enable local cached data and switch to the backup communication protocol when the vehicle network link is interrupted, and migrate non-critical tasks when the resources are overloaded to ensure the core functions, improving the user experience and driving safety, and ensuring the safe and stable operation of the vehicle in various situations.

[0026] In summary, through a series of closely connected steps such as obtaining multi-source heterogeneous data, dynamic priority classification, constructing a collaborative model, constructing a topology graph, extracting fault features, generating resource scheduling information, and dynamically adjusting the in-vehicle terminal, many problems in traditional vehicle diagnostic management are solved, and comprehensive, accurate, and efficient remote diagnosis and management of vehicles are realized.

[0027] In one embodiment, the step of dynamically prioritizing the multi-source heterogeneous data to obtain a multi-source data priority set includes: S201, extracting data type identifiers from the multi-source heterogeneous data, wherein the data type identifiers include sensor data, image data, communication protocol data, and log data; S202, assigning a basic priority weight to each data type identifier according to a preset rule base, wherein the preset rule base includes data real-time requirements, data magnitude, and security level; S203, obtaining a network bandwidth resource occupancy rate and a resource load value of a vehicle-mounted terminal for real-time monitoring, dynamically adjusting the basic priority weight according to the resource occupancy rate and the load value, and generating a dynamic priority weight; S204, sorting the multi-source heterogeneous data according to the dynamic priority weights to generate a multi-source data priority set, and dividing the multi-source data priority set into a real-time processing queue, a cache queue, and an asynchronous processing queue according to priority from high to low.

[0028] As described in the above steps S201-S204, the present invention extracts data type identifiers from multi-source heterogeneous data. In an actual vehicle-mounted environment, multi-source heterogeneous data is complex and diverse, and different types of data have completely different characteristics and processing requirements. If these data types cannot be accurately distinguished, data processing will inevitably fall into chaos, which will seriously affect the accuracy and efficiency of processing.

[0029] Taking a car equipped with an intelligent driving assistance system as an example, the data collected by its on-board terminal includes sensor data, image data, communication protocol data, log data and other types. Among them, sensor data is responsible for real-time perception of the vehicle's surrounding environment, image data is used to identify road signs and pedestrians, etc., communication protocol data ensures internal and external communication of the vehicle, and log data records system operation and user operation information. By extracting data type identifiers, the system can clarify the purpose of various types of data and then select the appropriate processing method. For example, a strong real-time algorithm is used for sensor data to quickly detect obstacles, and a special image recognition algorithm is used for image data to improve recognition accuracy, laying a solid foundation for subsequent processing, realizing data classification management and analysis, mining potential relationships, and improving vehicle operation monitoring and diagnosis capabilities.

[0030] Next, assign basic priority weights to each data type identifier according to the preset rule library. Since the importance and value of different types of data vary greatly in vehicle operation and fault diagnosis, the lack of a unified priority assignment standard will lead to chaotic processing priorities, resulting in phenomena such as delayed processing of important data and unreasonable resource allocation. Take an intelligent logistics transport vehicle as an example. The engine sensor data is crucial for real-time vehicle monitoring. The data volume is small and it is related to safety. According to the preset rule library, a higher basic priority weight should be assigned. For the log data of the in-vehicle entertainment system, the real-time requirement is low and the impact on safety is small, so a lower weight is assigned. In this way, when the vehicle has an abnormality, the system gives priority to processing the engine sensor data, diagnoses the fault in a timely manner, and at the same time reasonably allocates resources to process other data, ensuring that key data is processed first, optimizing resource utilization, and improving the timeliness and accuracy of fault diagnosis.

[0031] Then, obtain the network bandwidth resource occupancy rate and resource load value of the real-time monitoring in-vehicle terminal, and dynamically adjust the basic priority weight according to these real-time data to generate the dynamic priority weight. The vehicle operation environment is complex and changeable, and the network bandwidth resources and system resources are in a dynamic change at any time. If only relying on the fixed basic priority weight, when resources are scarce, high-priority data may not be processed in time due to insufficient resources, while low-priority data occupies too many resources, affecting the system performance. Take a network car during peak hours on urban roads as an example. At this time, the network bandwidth resource occupancy rate is high and the system resource load is large. Through real-time monitoring, the system finds that the positioning data and vehicle status monitoring data are crucial for ensuring driving safety and order service, while the in-vehicle advertisement push data is relatively less urgent. Therefore, the system dynamically adjusts the weight, increases the priority of key data, and reduces the resource allocation of advertisement push data, ensuring that key data can still be processed first under limited resources, and improving the system stability and the ability to handle emergencies.

[0032] Finally, sort the multi-source heterogeneous data according to the dynamic priority weight to generate a multi-source data priority set, and divide it into a real-time processing queue, a cache queue, and an asynchronous processing queue from high to low priority. When processing multi-source heterogeneous data, if there is no reasonable sorting and classification mechanism, the data processing will fall into a disorderly state, key data cannot be processed in time, and system resources are also difficult to make full use of. Take an autonomous driving test vehicle as an example. The sensor data related to autonomous driving decision-making and the communication protocol data critical to safety are given high priority and enter the real-time processing queue to ensure the driving safety of the vehicle. The log data of the vehicle operation status has a lower real-time requirement and is placed in the cache queue for temporary storage. The low-priority data such as the update data of the in-vehicle entertainment system enters the asynchronous processing queue. The system processes the data according to the priority order. The real-time processing queue ensures the immediate processing of key data, the cache queue avoids data loss, and the asynchronous processing queue processes low-priority data when resources are idle, making full use of system resources and improving the processing efficiency and stability.

[0033] In one embodiment, the step of constructing an edge-cloud collaborative multi-modal model according to the multi-source data priority set includes: S301. Extract real-time processing queue data from the multi-source data priority set and distribute the data to edge computing nodes; S302. Obtain the computing power distribution according to the edge computing nodes; S303. Generate intermediate feature vectors based on the processing results of the edge computing nodes and upload the intermediate feature vectors to the cloud; S304. Construct a multi-modal fusion model in the cloud, where the multi-modal fusion model includes a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch; S305. Input the intermediate feature vectors into the multi-modal fusion model to generate dynamic data information through cross-modal feature alignment and weight allocation strategies.

[0034] As described in the above steps S301 - S305, the present invention extracts real-time processing queue data from the multi-source data priority set and distributes it to edge computing nodes.

[0035] In scenarios such as intelligent transportation systems that involve a large amount of real-time data processing, if all data is uploaded to the cloud for processing, problems such as high network bandwidth pressure and high transmission latency will be faced, resulting in the inability to process real-time data in a timely manner, thereby affecting the system response speed and decision-making accuracy. Just like the logistics management system in a large industrial park, a large amount of real-time data generated by vehicles and robots, such as driving trajectories and working statuses, if directly uploaded to the cloud, emergencies such as vehicle speeding or robot failures cannot be monitored in a timely manner due to transmission latency.

[0036] By extracting real-time processing queue data from the multi-source data priority set and distributing it to edge computing nodes, data can be processed near the data source, reducing transmission time and bandwidth consumption, and improving real-time performance. Edge computing nodes can quickly pre-process real-time data, reduce the burden on the cloud, and quickly respond to emergencies locally, improving the reliability and stability of the system. Then, obtain the computing power distribution according to the edge computing nodes. Different edge computing nodes have different computing capabilities and resource statuses. If the computing power distribution is not understood, the allocation of data processing tasks will be unreasonable, resulting in uneven node loads and affecting the overall processing efficiency of the system. For example, in an urban intelligent security system, if a large number of computing tasks are assigned to edge nodes with weak computing capabilities, it will cause slow processing or even collapse, while other powerful nodes have idle resources. Obtaining the computing power distribution can provide a basis for task allocation, optimize resource allocation, assign appropriate tasks to appropriate nodes, avoid uneven loads, and can also dynamically adjust the task allocation strategy according to it, improving the processing efficiency and performance of the system.

[0037] Then, an intermediate feature vector is generated based on the processing results of the edge computing nodes and uploaded to the cloud. The data preliminarily processed by the edge computing nodes is often scattered and at a low level. Directly uploading it to the cloud will increase the cloud processing volume and complexity, and the processing result formats and standards of different nodes are inconsistent, which brings difficulties to unified processing. Taking the smart home system as an example, after the real-time data generated by the sensors in each room is preliminarily processed by the edge computing nodes, such as calculating the average temperature and humidity change trends, if the original processing results are directly uploaded, it will increase the processing burden on the cloud. Generating the intermediate feature vector can refine and integrate the edge processing results, transform the low-level information into high-level and representative features, reduce the data transmission volume and the cloud processing burden, and at the same time unify the format, which is convenient for cross-modal fusion analysis in the cloud and provides a more accurate and comprehensive basis for decision-making. A multi-modal fusion model including a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch is constructed in the cloud. In practical applications, the data modalities are diverse. Traditional single models can only process single modalities and cannot fully explore the potential relationships and complementary information between different modality data, which affects the model performance and accuracy. For example, in intelligent medical diagnosis, relying solely on medical record texts or imaging data for diagnosis is likely to overlook other important information, leading to misdiagnosis and missed diagnosis. For the constructed multi-modal fusion model, the convolutional neural network branch can process spatial structure data such as images and videos to extract spatial features; the temporal recurrent network branch processes time series data to capture dynamic changes; the graph attention network branch processes graph structure data to mine association relationships. Taking the intelligent transportation management system as an example, it fuses multi-modal data such as traffic camera video images, vehicle driving data, and road network topologies, can analyze more comprehensively and deeply, improve the model performance and accuracy, and provide a reliable basis for traffic management decision-making.

[0038] Finally, the intermediate feature vector is input into the multi-modal fusion model, and dynamic data information is generated through cross-modal feature alignment and weight allocation strategies. The feature representations and distributions of different modality data are different. When fusing, the feature alignment problem needs to be solved, and the importance of each modality data for decision-making is different, so weights need to be reasonably allocated. Otherwise, when different modality data are fused, they will interfere with each other, affecting the model performance and accuracy. Inputting it into the cloud multi-modal fusion model, different modality features are mapped to the same space through cross-modal feature alignment, and then weights are allocated according to the importance of each modality data for risk assessment. For example, the weight of financial statement data is high, and the weight of social network relationship data is low, so as to generate more accurate dynamic data information, such as credit risk scores, and provide reliable support for the risk decision-making of financial institutions.

[0039] In one embodiment, the step of constructing the vehicle data association topology graph according to the data association information and semantic information includes: S401. Extract the device identifier, data flow path, and timestamp sequence from the data association information; S402. Generate device function description tags and data dependency information according to the semantic information; S403. Map the device identifier to the function description tags to construct device - function nodes; S404. Generate data transmission edges between nodes according to the data flow path and the timestamp sequence, and mark the transmission frequency and delay parameters; S405. Construct a vehicle data association topology graph based on the device - function nodes, data transmission edges and data dependency information.

[0040] As described in the above steps S401 - S405, the present invention extracts device identifiers, data flow paths and timestamp sequences from the data association information. In a vehicle system, the data association information is complex and disorderly. Without extracting key elements, it is difficult to clarify the data interaction situation between various vehicle devices. For example, when troubleshooting vehicle faults, without a device identifier, it is impossible to determine the device to which the abnormal data belongs; without a data flow path, it will be difficult to trace the source and destination of the data; without a timestamp sequence, it is impossible to master the time sequence and frequency of data transmission. By extracting these key information, it provides basic data for constructing a topology graph. The device identifier can distinguish and manage different devices, the data flow path shows the data transmission direction, and the timestamp sequence records the transmission time information, which helps to grasp the operation law of the vehicle system. Taking an electric vehicle as an example, the identifiers of devices such as the battery management system and the motor controller can be extracted from the data association information, and the data transmission path between them and the specific time of data transmission can be understood, providing a basis for subsequent analysis and optimization.

[0041] Next, generate device function description tags and data dependency information according to the semantic information. Although the semantic information describes the meaning and use of data and devices, it is often vague and scattered. Without processing, it is difficult to obtain device functions and data dependencies. For example, the description of device functions in vehicle technical documents does not clearly define their data dependencies with other devices, which is not conducive to system analysis and optimization. Generating device function description tags and data dependency information can transform the semantic information into structured data. The device function description tags summarize the main functions of the device, and the data dependency information reveals the data associations between devices, helping to understand the working principle of the vehicle system and the data flow mechanism. In an electric vehicle, based on the semantic information, the function description tag "battery status monitoring and regulation" of the battery management system can be generated, and its data dependencies with devices such as the motor controller and the on - vehicle charger can be found, providing support for system optimization and fault diagnosis.

[0042] Then, map the device identifier to the function description label to construct a device - function node. In a vehicle system, the device identifier and the function description label are independent of each other. Without mapping, it is impossible to link the device identity with its function, and it is difficult to visually display the distribution of device functions. For example, only knowing the device identifier but not its specific function will cause difficulties in system analysis and management. By constructing a device - function node through mapping, the device identity and function can be integrated to form an intuitive node representation. Each node clearly shows the device identifier and the corresponding function, facilitating visual analysis to discover the functional collaboration relationships and potential problems among devices. In an intelligent connected vehicle, after mapping the device identifier of the electronic control unit to the function description label, nodes can be constructed to display the function distribution of each ECU, improving the efficiency of fault diagnosis. Generate data transfer edges between nodes according to the data flow path and timestamp sequence, and label the transfer frequency and delay parameters. When constructing a topology graph, merely determining the device - function nodes is not enough; it is also necessary to clarify the data transfer relationships between nodes. If data transfer edges are not generated and parameters are not labeled, the data transfer situation cannot be accurately described. For example, without knowing the transfer frequency and delay, it is impossible to evaluate the real - time performance and reliability of data, and it is difficult to evaluate the communication performance of the vehicle system. Generating data transfer edges and labeling parameters can completely describe the data transfer process. The data transfer edge shows the flow direction, the transfer frequency reflects periodicity, and the delay parameter reflects the time delay. In a vehicle equipped with an advanced driver assistance system, by analyzing the data flow path and timestamp sequence, generating the data transfer edge from the camera sensor to the central processor and labeling the transfer frequency and delay parameters helps to discover communication problems and optimize them.

[0043] Finally, construct a vehicle data association topology graph based on the device - function nodes, data transfer edges, and data dependency relationship information. After completing the previous steps, if the information is not integrated into a unified topology graph, it is impossible to comprehensively and visually display the data association and collaborative working methods among the devices in the vehicle system. For example, although the function of each device and the data transfer situation are understood, it is impossible to grasp the overall system structure and operation mechanism, which is not conducive to system optimization and fault diagnosis. Constructing a vehicle data association topology graph can visualize various data association information, enabling us to understand the system structure and operation mechanism as a whole. Through the topology graph, the functional collaboration, data transfer, and dependency relationships among devices can be clearly seen, helping to discover problems and optimization points. In a large commercial vehicle fleet management system, constructing a vehicle data association topology graph can display the system structure and data association of each vehicle. Integrating the fleet topology graph can help managers understand the operating status of vehicles, discover potential fault hazards, optimize vehicle configuration and scheduling, and improve operation efficiency and safety.

[0044] In one embodiment, the step of extracting cross - modal fault feature information from the vehicle data association topology graph includes: S501. Extract the node degree distribution, edge weight distribution, and subgraph clustering features from the vehicle data - associated topology graph; S502. Map the topology graph to a low - dimensional vector space based on the graph embedding algorithm to generate a graph - structure feature vector; S503. Combine the temporal features and image features in the dynamic data information, and perform feature fusion through the attention mechanism to obtain dynamic fusion feature data; S504. Obtain a preset fault mode library, and match the dynamic fusion feature data according to the preset fault mode library to filter out abnormal feature segments that deviate from the normal mode; S505. Calculate the similarity between the abnormal feature segments and historical fault cases to generate cross - modal fault feature information.

[0045] As described in the above steps S501 - S505, the present invention extracts the node degree distribution, edge weight distribution, and subgraph clustering features from the vehicle data - associated topology graph. The vehicle data - associated topology graph contains a large amount of complex information. Without extracting these key features, it is difficult to grasp the vehicle system structure and data association status. For example, during fault diagnosis, without understanding the node degree distribution, it is impossible to determine the devices whose connection tightness with the faulty device changes; without knowing the edge weight distribution, it is difficult to judge whether the importance and intensity of data transmission are abnormal; without extracting the subgraph clustering features, it is impossible to discover the impact of faults on specific functional modules or subsystems. Extracting these features can simplify and quantify the topology graph information. The node degree distribution can reflect the importance and connection tightness of devices, which helps to locate key nodes; the edge weight distribution reflects the importance and intensity of data transmission, helping to identify abnormal paths; the subgraph clustering features can discover associated subsystems, providing targeted information for fault diagnosis and improving efficiency and accuracy. Taking intelligent electric vehicles as an example, by analyzing these features, when the vehicle has a power shortage fault, it is possible to focus on checking the battery management system, the motor control system, and their data transmission conditions.

[0046] Next, map the topology graph to a low - dimensional vector space based on the graph embedding algorithm to generate a graph - structure feature vector. The vehicle data - associated topology graph is a complex graph structure, which is difficult to directly process by traditional methods, and its high dimension leads to high computational and storage costs, and low efficiency of feature matching and similarity calculation. The graph embedding algorithm can map the topology graph to a low - dimensional vector space. The generated graph - structure feature vector retains the main structure information, reduces the dimension, and reduces the computational and storage costs. At the same time, the low - dimensional vector space is convenient for fusion and comparison with other data, and mature algorithms can be used for analysis and processing. In the fault diagnosis system of intelligent connected vehicles, after mapping the high - dimensional topology graph to a low - dimensional vector space, it is possible to quickly compare the graph - structure feature vectors of the current vehicle and historical fault cases, improving the accuracy and efficiency of fault diagnosis.

[0047] Then, combine the temporal features and image features in the dynamic data information, and perform feature fusion through the attention mechanism to obtain dynamic fusion feature data. In vehicle fault diagnosis, the temporal features and image features in the dynamic data information are independent of each other. Without fusion, it is impossible to make full use of their complementary information, which limits the accuracy of fault diagnosis. Temporal features reflect the changing trend of vehicle parameters over time, and image features show the appearance and state of vehicle components. Considering them separately may ignore some fault information. By fusing these features through the attention mechanism, weights can be automatically assigned according to importance, highlighting important features and obtaining more comprehensive and accurate dynamic fusion feature data. In the fault diagnosis system of autonomous vehicles, by combining temporal features such as vehicle speed and acceleration and image features of on-vehicle cameras, after fusion using the attention mechanism, it can more accurately reflect the vehicle operating state, improve the accuracy and reliability of fault diagnosis, obtain a preset fault mode library, and match the dynamic fusion feature data according to this library to screen out abnormal feature segments that deviate from the normal mode. The dynamic fusion feature data contains a large amount of information, most of which is normal operation data. Without screening, it will increase the difficulty of calculation and analysis, and is easily interfered by normal data, making it difficult to discover fault features. The preset fault mode library provides a reference standard. Through matching, abnormal feature segments that may be related to faults can be quickly screened out, reducing the amount of data for subsequent analysis and improving the efficiency of fault diagnosis. In the fault diagnosis system of automobile engines, by matching the temporal features such as engine speed, torque, and temperature and the image features of internal components with the preset fault mode library, abnormal feature segments are screened out, providing pertinence for subsequent in-depth analysis.

[0048] Finally, calculate the similarity between the abnormal feature segments and historical fault cases to generate cross-modal fault feature information. The screened abnormal feature segments cannot directly determine the fault type and cause. Different faults may have similar abnormal features, and their manifestations are different due to various factors. By calculating the similarity with historical fault cases, historical experience and knowledge can be utilized to generate cross-modal fault feature information, comprehensively and accurately describing fault features and manifestations. By comparing the similarities, the most similar historical fault cases can be found, and the current fault type and cause can be inferred, providing targeted suggestions for fault diagnosis and repair. In the fault diagnosis system of large vehicle repair plants, based on the similarity calculation results between abnormal feature segments and historical fault cases, the fault cause can be quickly located and repaired, while continuously improving the historical fault case library.

[0049] In one embodiment, the step of generating fault resource scheduling information according to the fault classification data includes: S601. Obtain a preset resource allocation policy model, and input the fault classification data into the preset resource allocation policy model to obtain resource allocation information, where the resource allocation information includes fault level information, repair time-consuming information, and influence range parameter information; S602. Allocate redundant computing resources for high-priority faults, allocate dynamic elastic resources for medium-priority faults, and allocate asynchronous repair queues for low-priority faults according to the fault level information; S603. Obtain the real-time position information and network coverage status of the vehicle terminal, and generate a resource scheduling path plan; S604. Generate fault resource scheduling information according to the resource scheduling path plan and the resource allocation information, where the fault resource scheduling information includes a computing node switching instruction, a data transmission link switching instruction, and a service degradation instruction.

[0050] As described in the above steps S601-S604, the present invention obtains a preset resource allocation policy model, and inputs the fault classification data into it to obtain resource allocation information, which includes fault level information, repair time-consuming information, and influence range parameter information. In the vehicle fault handling scenario, the fault situations are complex and diverse. Without a scientific resource allocation strategy, it is difficult to allocate resources reasonably. The impact degrees, repair times, and affected ranges of different faults vary greatly. Blindly allocating resources will cause high-priority faults to be processed untimely, while low-priority faults occupy too many resources, affecting the operation efficiency and safety of the vehicle system. The preset resource allocation policy model provides a scientific basis for resource allocation. By inputting the fault classification data, it can accurately obtain various information about the faults, helping us clearly understand the specific conditions of each fault, so as to allocate resources targeted and improve the efficiency and quality of fault handling. For example, in the intelligent connected vehicle fault handling system, after inputting the classification data of engine faults and in-vehicle entertainment system faults, the model can output that the engine fault is of high priority, with a long repair time and affecting the power system; the in-vehicle entertainment system fault is of low priority, with a short repair time and a small influence range, which helps the maintenance personnel to give priority to handling engine faults.

[0051] Next, corresponding resources are allocated to faults of different priorities according to the fault level information, that is, redundant computing resources are allocated to high-priority faults, dynamic elastic resources are allocated to medium-priority faults, and asynchronous repair queues are allocated to low-priority faults. Faults of different levels have different resource requirements and urgencies. A unified resource allocation method cannot meet the requirements of high-priority faults for resource timeliness and stability, and will also cause waste or shortage of resources. If high-priority faults lack redundant computing resources, the processing time will be extended, affecting vehicle safety; if low-priority faults occupy too many real-time resources, it will waste resources and affect the handling of important faults. Through differential resource allocation, the needs of different faults can be fully met. Allocating redundant computing resources to high-priority faults can ensure uninterrupted processing and improve timeliness and reliability; allocating dynamic elastic resources to medium-priority faults can flexibly adjust resource allocation and avoid waste; allocating asynchronous repair queues to low-priority faults can use idle resources to handle faults and improve the overall resource utilization rate. Taking the management system of a large logistics fleet as an example, a fault in the truck braking system is a high priority, and redundant resources such as standby servers and computing nodes will be allocated; a fault in the tire pressure monitoring system is a medium priority, and elastic resources will be dynamically allocated; a fault in the in-vehicle radio is a low priority and will be put into an asynchronous repair queue.

[0052] Then, the real-time position information and network coverage status of the in-vehicle terminal are obtained, and then a resource scheduling path plan is generated. When scheduling fault resources, it is not enough to only consider the fault type and level. The vehicle position and network coverage also need to be considered. Without knowing the real-time position of the vehicle, the nearest maintenance resources cannot be allocated, which will increase the maintenance time and cost; poor network coverage will affect data transmission and remote diagnosis effects, resulting in low processing efficiency. Obtaining this information can generate a more reasonable resource scheduling path plan. Considering the vehicle position can select the nearest and fast-responsive maintenance resources to reduce costs; according to the network coverage status, a suitable data transmission method and path can be selected to ensure stable and fast data transmission and improve processing efficiency. For example, in an intelligent transportation system, when a taxi breaks down in the suburbs, after the system obtains its position and network status, it schedules nearby maintenance personnel to go there and selects alternative methods such as satellite communication to transmit data to quickly handle the fault.

[0053] Finally, fault resource scheduling information is generated based on the resource scheduling path planning and resource allocation information, including computing node switching instructions, data transmission link switching instructions, and service degradation instructions. During the fault handling process, to ensure the rational utilization and efficient scheduling of resources, it is necessary to convert the resource scheduling path planning and resource allocation information into specific operation instructions. The lack of clear instructions will cause confusion for maintenance personnel and the system in resource allocation and processing, resulting in low efficiency. For example, the absence of instructions when switching computing nodes or data transmission links will cause data loss or processing interruption; the absence of service degradation instructions when resources are insufficient will affect the system stability. Generating fault resource scheduling information can convert the planning and information into executable instructions. The computing node switching instructions can flexibly allocate computing resources and improve efficiency; the data transmission link switching instructions can ensure stable data transmission; the service degradation instructions can ensure the operation of critical services when resources are insufficient and ensure the system stability. Taking the fault handling system of intelligent electric vehicles as an example, when the battery management system fails, the system will issue corresponding instructions according to the situation, such as switching computing nodes, data transmission links, or performing service degradation, to ensure timely fault handling and safe vehicle operation.

[0054] In one embodiment, the step of dynamically adjusting the in-vehicle terminal according to the fault resource scheduling information includes: S701, obtaining the software service response delay according to the in-vehicle terminal; S702, obtaining the in-vehicle terminal status according to the software service response delay, where the in-vehicle terminal status includes resource overload, network link interruption, and critical service failure; If resource overload is detected, some tasks are migrated to adjacent edge nodes or the cloud according to the fault resource scheduling information; if network link interruption is detected, local cached data is enabled and the standby communication protocol is switched; If critical service failure is detected, the service degradation mode is triggered to give priority to ensuring the operation of core functions; S703, feeding back the adjusted operating status according to the software service response delay to the cloud.

[0055] As described in the above steps S701 - S703, the present invention obtains the software service response delay according to the in-vehicle terminal. In a complex in-vehicle environment, factors such as vehicle vibration, electromagnetic interference, and network instability during vehicle driving will cause fluctuations in the software service response time. If the response delay cannot be obtained in a timely and accurate manner, it is difficult to detect potential problems in the system. For example, for the navigation system of an intelligent connected vehicle, if the response delay increases due to network congestion or high processor load, the driver may miss navigation prompts. Obtaining the response delay can monitor the system operation status in real time, provide a basis for subsequent fault diagnosis and adjustment, and can also analyze the system performance change trend through long-term monitoring to provide a reference for optimization and upgrade.

[0056] Next, obtain the in-vehicle terminal status based on the software service response delay, including resource overload, network link interruption, and critical service failure, and take corresponding measures for different statuses. Just obtaining the response delay is not sufficient to comprehensively understand the in-vehicle terminal condition, and different problems require different solution strategies. When resource overload is detected, migrate some tasks to adjacent edge nodes or the cloud according to the fault resource scheduling information, which can relieve the burden on the in-vehicle terminal and ensure the normal operation of critical tasks. For example, an autonomous driving vehicle migrates non-real-time data analysis tasks. If a network link interruption is detected, enable local cached data and switch to an alternative communication protocol, which can maintain the basic functions of the system and avoid information loss and service interruption. For instance, when the network is interrupted during driving, enable the cached map and switch the communication protocol. When a critical service failure is detected, trigger the service degradation mode to prioritize the operation of core functions and ensure vehicle safety and basic operations. For example, when a core algorithm failure of autonomous driving occurs, switch to manual driving and maintain safety functions.

[0057] Finally, feedback the adjusted operating status based on the software service response delay to the cloud. If the adjusted status is not fed back, the cloud cannot timely understand the actual situation of the in-vehicle system, making it difficult to conduct effective management and optimization, and it is also not conducive to long-term performance analysis and fault prediction. Feeding back the operating status can achieve information interaction and collaborative work between the in-vehicle terminal and the cloud. The cloud can dynamically manage and optimize the vehicle network based on this. For example, adjust the resource allocation strategy according to resource usage and network conditions. At the same time, the cloud can analyze a large amount of operating status data, mine potential fault patterns and optimization points, and provide references for vehicle design and maintenance. For example, in an intelligent fleet management system, the cloud optimizes resource allocation and predicts faults based on vehicle feedback information.

[0058] As Figure 2 shown, the present invention also provides a remote diagnosis and management system based on an in-vehicle terminal, including: A first acquisition module 1, configured to acquire multi-source heterogeneous data collected in real time by the in-vehicle terminal; A classification module 2, configured to perform dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set; A construction module 3, configured to construct an edge-cloud collaborative multimodal model according to the multi-source data priority set, and obtain multiple dynamic data information according to the edge-cloud collaborative multimodal model; A second acquisition module 4, configured to acquire the data association information and semantic information of each dynamic data allocation information, and construct a vehicle data association topology map according to the data association information and semantic information; A first extraction module 5, configured to perform feature extraction on the vehicle data association topology map to obtain cross-modal fault feature information, and obtain diagnostic fault information according to the cross-modal fault feature information; The second extraction module 6 is used to extract features from the diagnostic fault information to obtain fault classification data, and generate fault resource scheduling information according to the fault classification data; The mobilization module 7 is used to dynamically adjust the in-vehicle terminal according to the fault resource scheduling information.

[0059] In one embodiment, the grading module 2 includes: An extraction unit for extracting data type identifiers from the multi-source heterogeneous data, where the data type identifiers include sensor data, image data, communication protocol data, and log data; An allocation unit for allocating basic priority weights to each data type identifier according to a preset rule library, where the preset rule library includes data real-time requirements, data volume, and security level; A first acquisition unit for acquiring the network bandwidth resource occupancy rate and resource load value of the in-vehicle terminal monitored in real time, dynamically adjusting the basic priority weights according to the resource occupancy rate and load value, and generating dynamic priority weights; A sorting unit for sorting the multi-source heterogeneous data according to the dynamic priority weights, generating a multi-source data priority set, and dividing the multi-source data priority set into a real-time processing queue, a cache queue, and an asynchronous processing queue in descending order of priority.

[0060] In one embodiment, the establishment module 3 includes: A second acquisition unit for extracting real-time processing queue data from the multi-source data priority set and distributing the data to edge computing nodes; A third acquisition unit for obtaining computing power distribution according to the edge computing nodes; A generation unit for generating an intermediate feature vector based on the processing results of the edge computing nodes and uploading the intermediate feature vector to the cloud; A construction unit for constructing a multi-modal fusion model in the cloud, where the multi-modal fusion model includes a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch; An input unit for inputting the intermediate feature vector into the multi-modal fusion model and generating dynamic data information through a cross-modal feature alignment and weight allocation strategy.

[0061] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, value library, or other medium provided in this application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0062] It should be noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, device, article or method. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of another identical element in the process, device, article or method comprising that element.

[0063] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent results or equivalent process transformations made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are equally included in the patent protection scope of the present invention.

Claims

1. A remote diagnosis management method based on an in-vehicle terminal, characterized in that Including: Obtain multi-source heterogeneous data collected in real time by the vehicle-mounted terminal; Perform dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set; Construct an edge-cloud collaborative multi-modal model according to the multi-source data priority set, and obtain multiple dynamic data information according to the edge-cloud collaborative multi-modal model; Obtain the data association information and semantic information of each dynamic data allocation information, and construct a vehicle data association topology graph according to the data association information and semantic information; Extract cross-modal fault feature information from the vehicle data association topology graph, and obtain diagnostic fault information according to the cross-modal fault feature information; Extract fault classification data from the diagnostic fault information, and generate fault resource scheduling information according to the fault classification data; Dynamically adjust the vehicle-mounted terminal according to the fault resource scheduling information.

2. The remote diagnosis management method based on an in-vehicle terminal according to claim 1, wherein, The step of performing dynamic priority classification on the multi-source heterogeneous data to obtain a multi-source data priority set includes: Extract data type identifiers from the multi-source heterogeneous data, where the data type identifiers include sensor data, image data, communication protocol data, and log data; Assign basic priority weights to each data type identifier according to a preset rule library, and the preset rule library includes data real-time requirements, data magnitude, and security level; Obtain the network bandwidth resource occupancy rate and resource load value of the vehicle-mounted terminal monitored in real time, and dynamically adjust the basic priority weights according to the resource occupancy rate and load value to generate dynamic priority weights; Sort the multi-source heterogeneous data according to the dynamic priority weights to generate a multi-source data priority set, and divide the multi-source data priority set into a real-time processing queue, a cache queue, and an asynchronous processing queue from high to low in priority.

3. The remote diagnosis management method based on a vehicle-mounted terminal according to claim 1, wherein The step of constructing an edge-cloud collaborative multi-modal model according to the multi-source data priority set includes: Extract real-time processing queue data from the multi-source data priority set, and distribute the data to edge computing nodes; Obtain the computing power distribution according to the edge computing nodes; Generate intermediate feature vectors based on the processing results of the edge computing nodes, and upload the intermediate feature vectors to the cloud; construct a multi-modal fusion model in the cloud, and the multi-modal fusion model includes a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch; Input the intermediate feature vectors into the multi-modal fusion model, and generate dynamic data information through cross-modal feature alignment and weight allocation strategies.

4. A remote diagnosis management method based on a vehicle-mounted terminal according to claim 1, characterized in that, The step of constructing a vehicle data association topology graph according to the data association information and semantic information includes: Extract device identifiers, data flow paths, and timestamp sequences from the data association information; Generate device function description tags and data dependency information according to the semantic information; Map the device identifiers and function description tags to construct device-function nodes; Generate data transmission edges between nodes according to the data flow paths and timestamp sequences, and mark the transmission frequency and delay parameters; construct a vehicle data association topology graph based on the device-function nodes, data transmission edges, and data dependency information.

5. The remote diagnosis management method based on a vehicle-mounted terminal according to claim 1, characterized in that The steps of extracting cross-modal fault feature information from the vehicle data association topology graph include: Extracting node degree distribution, edge weight distribution, and subgraph clustering features from the vehicle data association topology graph; Mapping the topology graph to a low-dimensional vector space based on a graph embedding algorithm to generate a graph structure feature vector; Combining the temporal features and image features in the dynamic data information, and performing feature fusion through an attention mechanism to obtain dynamic fusion feature data; Obtaining a preset fault mode library, and matching the dynamic fusion feature data according to the preset fault mode library to screen out abnormal feature segments deviating from the normal mode; Calculating the similarity between the abnormal feature segments and historical fault cases to generate cross-modal fault feature information.

6. The remote diagnosis management method based on a vehicle-mounted terminal according to claim 1, wherein The steps of generating fault resource scheduling information according to the fault classification data include: Obtaining a preset resource allocation policy model, and inputting the fault classification data into the preset resource allocation policy model to obtain resource allocation information, where the resource allocation information includes fault level information, repair time-consuming information, and influence range parameter information; Allocating redundant computing resources for high-priority faults, dynamic elastic resources for medium-priority faults, and asynchronous repair queues for low-priority faults according to the fault level information; Obtaining the real-time position information and network coverage status of the in-vehicle terminal to generate a resource scheduling path plan; Generating fault resource scheduling information according to the resource scheduling path plan and the resource allocation information, where the fault resource scheduling information includes computing node switching instructions, data transmission link switching instructions, and service degradation instructions.

7. A remote diagnosis management method based on a vehicle-mounted terminal according to claim 1, characterized in that, The steps of dynamically adjusting the in-vehicle terminal according to the fault resource scheduling information include: Obtaining software service response latency according to the in-vehicle terminal; Obtaining the in-vehicle terminal status according to the software service response latency, where the in-vehicle terminal status includes resource overload, network link interruption, and critical service failure; If resource overload is detected, some tasks are migrated to adjacent edge nodes or the cloud according to the fault resource scheduling information; if network link interruption is detected, local cached data is enabled and the standby communication protocol is switched; If critical service failure is detected, the service degradation mode is triggered to give priority to ensuring the operation of core functions; Feeding back the adjusted operating status to the cloud according to the software service response latency.

8. A remote diagnosis and management system based on an in-vehicle terminal, characterized in that, Including: A first acquisition module for acquiring multi-source heterogeneous data collected in real time by the in-vehicle terminal; A classification module for dynamically classifying the multi-source heterogeneous data into a multi-source data priority set; A construction module for constructing an edge-cloud collaborative multi-modal model according to the multi-source data priority set, and obtaining multiple dynamic data information according to the edge-cloud collaborative multi-modal model; A second acquisition module for acquiring the data association information and semantic information of each dynamic data allocation information, and constructing a vehicle data association topology graph according to the data association information and semantic information; A first extraction module for extracting cross-modal fault feature information from the vehicle data association topology graph, and obtaining diagnostic fault information according to the cross-modal fault feature information; A second extraction module, configured to extract features from the diagnostic fault information to obtain fault classification data, and generate fault resource scheduling information according to the fault classification data; A mobilization module, configured to dynamically adjust the in-vehicle terminal according to the fault resource scheduling information.

9. The remote diagnosis management system based on a vehicle-mounted terminal according to claim 8, wherein, The grading module includes: An extraction unit, configured to extract data type identifiers from the multi-source heterogeneous data, where the data type identifiers include sensor data, image data, communication protocol data, and log data; An allocation unit, configured to allocate basic priority weights for each data type identifier according to a preset rule library, where the preset rule library includes data real-time requirements, data volume, and security level; A first acquisition unit, configured to acquire the network bandwidth resource occupancy rate and resource load value of the in-vehicle terminal in real-time monitoring, and dynamically adjust the basic priority weights according to the resource occupancy rate and load value to generate dynamic priority weights; A sorting unit, configured to sort the multi-source heterogeneous data according to the dynamic priority weights to generate a multi-source data priority set, and divide the multi-source data priority set into a real-time processing queue, a cache queue, and an asynchronous processing queue in descending order of priority.

10. A remote diagnosis management system based on a vehicle-mounted terminal according to claim 8, characterized in that, The establishment module includes: A second acquisition unit, configured to extract real-time processing queue data from the multi-source data priority set and distribute the data to edge computing nodes; A third acquisition unit, configured to obtain computing power distribution according to the edge computing nodes; A generation unit, configured to generate an intermediate feature vector based on the processing results of the edge computing nodes and upload the intermediate feature vector to the cloud; A construction unit, configured to construct a multi-modal fusion model in the cloud, where the multi-modal fusion model includes a convolutional neural network branch, a temporal recurrent network branch, and a graph attention network branch; An input unit, configured to input the intermediate feature vector into the multi-modal fusion model to generate dynamic data information through a cross-modal feature alignment and weight allocation strategy.

Images