Automobile TARA analysis method and system based on multi-modal input and hybrid intelligence

Abstract

The invention provides an automobile TARA analysis method and system based on multi-modal input and hybrid intelligence, and the method comprises the steps: extracting text entity information and image objects from vehicle design data, carrying out the intelligent matching of the text entity information and image objects, and constructing a structured asset list, detailed attribute description and a network topology structure diagram; performing threat identification and attack path deduction on the identified assets, and retrieving most relevant information fragments; and calling a large language model to generate an analysis result based on the multi-stage structured prompt, and outputting a standardized TARA report after post-processing. According to the invention, through the improved target identification model, the detection precision and robustness of tiny icons, slender buses and shielding elements in a complex and intensive automobile framework diagram are obviously improved; according to the method, the accuracy and authority of large language model reasoning are improved through path deduction and most relevant information fragments; the subjective deviation of manual analysis is eliminated while the analysis time is greatly shortened, and the high consistency of analysis results is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile risk assessment, and in particular to an automobile TARA analysis method and system based on multi-modal input and hybrid intelligence. BACKGROUND

[0002] With the rapid development of automobile intelligence and networking, the software code volume and network attack surface of vehicles have increased dramatically, and network security problems have become increasingly serious. To address this challenge, the global automobile industry generally uses threat analysis and risk assessment (TARA) methods to identify and assess potential network security risks.

[0003] However, existing TARA analysis methods mainly rely on manual execution, usually completed by a team of experienced automobile network security experts, which has the following defects: 1) For a complex automobile system, a complete manual TARA analysis usually takes several weeks or even months, which is seriously lagging behind the rapid iteration development rhythm of current automobile products; 2) The TARA analysis process is highly dependent on the personal experience and knowledge reserve of experts. Due to the lack of completely unified detailed standards in the industry, different experts or teams may have significant differences in analyzing the same system, making it difficult to guarantee the objectivity and consistency of risk assessment; 3) A qualified automobile network security analyst needs to have a combination of knowledge in hardware, software, communication, cryptography, etc., which has a long training period and high labor cost, making it difficult for many enterprises to form or maintain a professional analysis team; 5) New attack techniques, vulnerabilities (such as 0-day vulnerabilities) and security regulations emerge in an endless stream, making it difficult for manual analysis teams to keep up with all the latest developments and apply them to existing analysis models, resulting in a time gap in the analysis results.

[0004] Therefore, there is an urgent need for a technical solution that can overcome the above-mentioned defects and achieve TARA analysis automation, intelligence and standardization, in order to improve analysis efficiency and quality and reduce dependence on professional personnel. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides an automobile TARA analysis method and system based on multi-modal input and hybrid intelligence, which further shortens the TARA analysis time and further improves the accuracy of the analysis.

[0006] In a first aspect, the present application provides an automobile TARA analysis method based on multi-modal input and hybrid intelligence, comprising the following steps: S1) Obtain vehicle design data and extract text entity information and image objects from the vehicle design data; S2), using a cross-modal information fusion algorithm that fuses spatial position, semantic similarity, and domain ontology knowledge to intelligently match the extracted text entity information and image objects, and constructing a structured asset list, detailed attribute description, and network topology structure diagram; S3), threat identification and attack path deduction are performed on the identified assets, and the most relevant information fragments are retrieved; S4), a multi-stage structured prompt is constructed, and a large language model is used to generate analysis results, and a standardized TARA report is output after post-processing.

[0007] As preferred, in step S1), text entity information is extracted from vehicle design data using optical character recognition technology OCR.

[0008] As preferred, in step S1), image objects are extracted from vehicle design data using an improved YOLOv8 target recognition model.

[0009] As preferred, in step S1), the improvement of the improved YOLOv8 target recognition model is that an attention mechanism and a multi-scale feature fusion module for small target recognition are introduced into the backbone network of the YOLOv8 network model to construct an improved YOLOv8 target recognition model. And the improved YOLOv8 target recognition model is trained through a special data set; the special data set is constructed through artificial annotation and the unique symbols and diversified drawing styles of ECUs, CANs, LINs, Ethernet buses, sensors, actuators, and connectors in the automobile architecture diagram.

[0010] As preferred, in step S1), the center coordinates and bounding box information of the entities detected by the improved YOLOv8 target recognition model, as well as the endpoint coordinates and line feature of the detected connection lines, are used to construct the physical and logical connection relationships between entities through geometric correlation rules, symbol semantic recognition, and connectivity analysis based on graph theory, and convert them into a network topology graph.

[0011] As preferred, in step S2), the cross-modal information fusion algorithm that fuses spatial position, semantic similarity, and domain ontology knowledge is used to intelligently match the text entity information and image objects, and construct a structured asset list, detailed attribute description, and network topology structure diagram; specifically including the following steps: S21), the text entities and graphic objects are preliminarily associated through the spatial overlap and distance between text blocks and graphic objects; S22), the semantic similarity between the text description and the graphic object class label is calculated using an embedding model; S23), correct the confidence match below the preset confidence threshold using the predefined ontology knowledge base in the automotive field, and form an entity-attribute-relation triple; S24), based on the entity-attribute-relation triple, construct a structured asset list, detailed attribute description, and network topology diagram of the analyzed vehicle system.

[0012] As preferred, in step S3), threat identification and attack path deduction are performed on the identified assets; specifically including the following steps: S31), the asset is taken as a keyword, and vector retrieval technology is used to match in the built-in threat knowledge base containing known vulnerability information and historical attack cases, to realize threat identification; S32), the identified threat is taken as an initial threat node, and graph neural network GNN is used to reason on the pre-built attack chain knowledge graph to deduce potential attack paths; S33), based on the identified threat and attack path, dynamic weighted retrieval enhanced RAG algorithm is used to retrieve the most relevant information segment as context.

[0013] As preferred, in step S33), for each piece of information retrieved, the dynamic weighted retrieval enhanced RAG algorithm calculates its comprehensive weight through the following rules: authority quantification rule, timeliness decay function, and comprehensive weighting algorithm.

[0014] As preferred, in step S4), a multi-stage structured prompt Prompt is constructed, and a large language model is used to generate analysis results, and a standardized TARA report is output after processing, specifically including the following steps: S41), a multi-stage structured prompt Prompt is constructed using the asset list and network topology diagram, the deduced attack path, the RAG algorithm to retrieve the most relevant information segment, and the explicit TARA analysis methodology; S42), based on the multi-stage structured prompt Prompt, a large language model is used for constrained reasoning, and analysis results are generated; S43), the analysis results returned by the large language model are post-processed to generate a standardized TARA report containing a visual attack path graph and a structured report table.

[0015] In a second aspect, the present application provides an automotive TARA analysis system based on multi-modal input and hybrid intelligence, comprising: A data uploading module for users to upload vehicle design data; A feature extraction module for extracting text entity information and image objects from vehicle design data; The fusion module uses a cross-modal information fusion algorithm that integrates spatial location, semantic similarity, and domain ontology knowledge to intelligently match extracted text entity information and image objects, and construct a structured asset list, detailed attribute descriptions, and network topology diagram. The threat identification and path deduction module is used to identify threats to identified assets, deduce attack paths, and retrieve the most relevant information fragments. The analysis module is used to construct multi-stage structured prompts and generate analysis results using a large language model, which are then post-processed to output a standardized TARA report.

[0016] The beneficial effects of this invention are as follows: 1. This invention significantly improves the detection accuracy and robustness of tiny icons, thin buses, and occluded elements in complex and dense automotive architecture diagrams through the improved YOLOv8 target recognition model, with a detection recall and accuracy improvement of more than 20%. 2. This invention intelligently matches text entity information and image objects to construct a structured asset list, detailed attribute descriptions, and network topology diagrams, automatically building structured assets containing detailed attributes and precise topology, thus bridging the semantic gap between text descriptions and graphics. 3. This invention achieves threat retrieval of assets through a threat knowledge base and discovers deeper, less obvious, and even cross-domain potential attack paths through graph neural networks, giving the invention a strong generalization ability to cope with new attack scenarios; and improves the accuracy and authority of subsequent large language model inference by retrieving the most relevant information fragments through a dynamic weighted RAG algorithm. 4. This invention shortens the traditional manual analysis cycle, which takes weeks or even months, to less than 40 minutes, greatly accelerating the product development and security verification process. At the same time, it eliminates the subjective bias of manual analysis, ensuring a high degree of consistency in the analysis results. The standard deviation of the results is reduced from 7.2% for manual analysis to 0.8% for this invention. Meanwhile, the threat coverage and analysis accuracy are both better than the average level of manual analysis. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1 like Figure 1 As shown, this embodiment provides a vehicle TARA analysis method based on multimodal input and hybrid intelligence, including the following steps: S1), obtaining vehicle design data, and extracting text entity information and image objects from the vehicle design data; In the embodiment, at least one format of vehicle design data uploaded by a user is received, and the format of the vehicle design data can be PDF, DOCX, PNG, or JPG. The vehicle design data includes a vehicle system architecture diagram, a function description text, and a component list.

[0019] In the embodiment, text entity information is extracted from the vehicle design data by an optical character recognition technology OCR.

[0020] In the embodiment, image objects are extracted from the vehicle design data by an improved YOLOv8 target recognition model.

[0021] The improvement of the improved YOLOv8 target recognition model is that the improved YOLOv8 target recognition model is constructed by introducing an attention mechanism for small target recognition and a multi-scale feature fusion module on the basis of the backbone network of the YOLOv8 network model. The detection accuracy and robustness of small icons, slender buses, and shielded elements in complex and dense automobile architecture diagrams are significantly improved. In the embodiment, the detection recall rate and accuracy of the improved YOLOv8 target recognition model on the automobile architecture diagram are improved by more than 20% compared with the general YOLOv8.

[0022] In addition, in the embodiment, the improved YOLOv8 target recognition model is trained by manually annotating and arranging tens of thousands of high-quality field-specific picture data sets for hundreds of specific symbols and diversified drawing styles of ECUs, CANs, LINs, Ethernet buses, sensors, actuators, connectors, and the like in the automobile architecture diagram.

[0023] In the embodiment, the center coordinates and the bounding box information of the entities detected by the improved YOLOv8 target recognition model, and the endpoint coordinates and line features of the detected connection lines are used to construct the physical and logical connection relationships between the entities by geometric correlation rules, symbol semantic recognition, and connectivity analysis based on graph theory, and the relationships are converted into a network topology diagram. The specific process is as follows: In the embodiment, the improved YOLOv8 target recognition model outputs two sets of collections: a node set N and a line set L, by judging whether a line is connected to a node . The two endpoint coordinates of the line are extracted , ; then the bounding box region of the node is obtained ; Then calculate the Euclidean distance from the end point to the center of the node bounding box, or determine whether the end point falls within the bounding box: let the node center be , and the connecting end point be , and the distance be: ; If , or the end point coordinates are within the bounding box area , it is determined that the physical connection exists, and is recorded as .

[0024] In this embodiment, for the logical verification of the symbol semantic recognition, this embodiment introduces semantic verification: Read the class label recognized by the improved YOLOv8 target recognition model, and the logical rule is: If is CAN_Bus, it allows multiple nodes connected to it as ECU; If is Wire, it usually only connects two end points; If is a connector, it must act as an intermediate node.

[0025] Connectivity analysis and topology generation based on graph theory; Finally, construct a mathematical network topology graph , where is the vertex set, consisting of the node set N and the line set L; is the edge set, derived from the line set L; if the line connects node and node , an edge is established in .

[0026] S2), using a cross-modal information fusion algorithm that fuses spatial position, semantic similarity, and domain ontology knowledge to intelligently match text entity information and image objects, and construct a structured asset list, detailed attribute description, and network topology structure diagram; specifically including the following steps: S21), preliminarily associate text entities and graphic objects through the spatial overlap and distance between text blocks and graphic objects; that is: In the formula, is the center coordinate of the text block; is the center coordinate of the graphic object; For the diagonal length of the graphic object; For the confidence score of the spatial position-based association between the text block and the graphic object.

[0027] S22), calculate the semantic similarity between the text description and the graphic object category label using the embedding model BERT-based model; that is: In the formula, denotes the semantic similarity score, is the feature vector output after the text description is encoded by the embedding model; is the feature vector output after the graphic object category label is encoded by the embedding model; denotes the vector dot product operation; denotes the vector norm; S23), correct the confidence match below the preset confidence threshold using the predefined ontology knowledge base in the automotive field to form entity-attribute-relation triples; In this embodiment, when the system detects that a certain graphic object cannot form a high-confidence match with any text box around it, all isolated text fragments within a certain radius around the graphic object are taken out; the pre-installed ontology library in the automotive field is called to calculate the Levenshtein distance of each candidate text with all standard terms in the ontology library.

[0028] S24), based on the entity-attribute-relation triples, construct the structured asset list, detailed attribute description, and network topology diagram of the analyzed vehicle system.

[0029] In this embodiment, the assets specifically include the following four categories: Hardware assets: refer to physical existing electronic components.

[0030] For example: remote vehicle terminal (T-BOX), vehicle information entertainment system host (IVI Head Unit), gateway (Gateway), engine control unit (EMS), millimeter wave radar, camera, etc.

[0031] Communication assets: refer to physical or logical channels for data transmission.

[0032] For example: CAN bus (power CAN, body CAN), Ethernet, LIN bus, Bluetooth connection (Bluetooth Link), Wi-Fi connection, USB interface, etc.

[0033] Software or system assets: refer to the environment running on hardware.

[0034] For example: Android operating system, QNX system, OTA update service, in-vehicle navigation app, firmware.

[0035] Data assets: refer to high-value information that is stored or transmitted.

[0036] For example: user privacy data (contacts), vehicle location information (GPS), digital car key (DigitalKey), vehicle control commands.

[0037] S3) Perform threat identification and attack path deduction on the identified assets, and retrieve the most relevant information fragments; specifically including the following steps: S31) Using assets (such as in-vehicle infotainment systems IVI) as keywords, the threat is identified by matching them in a built-in threat knowledge base containing known vulnerability information and historical attack cases through vector retrieval technology. S32) Using the identified threats as initial threat nodes, a graph neural network (GNN) is used to reason on a pre-built attack chain knowledge graph to deduce potential attack paths. The specific steps are as follows: A pre-built attack chain knowledge graph, the Attack Graph, is constructed by importing the common attack patterns CAPEC and ATT&CK matrices from the graph database Neo4j. Nodes: Attack techniques (such as Bluetooth code execution), asset types (such as IVI), and permission status (such as root privileges).

[0038] Edge: The sequential relationship of attack steps (e.g., exploiting vulnerabilities -> gaining privileges).

[0039] Reasoning steps: Subgraph Mapping: Maps the generated structured asset topology graph to the attack chain knowledge graph. Specifically, it aligns the IVI nodes in the asset graph with the IVI asset type nodes in the knowledge graph and uses the scanned Bluetooth vulnerability as the starting node for activation. State propagation (GNN Message Passing): Message passing is performed using a graph neural network (GNN), setting the initial threat node's state to Hinit=[1,0,...]. Path extraction and pruning: After k iterations, check the activation status of all nodes; if the activation value of a key asset node exceeds the threshold, it indicates that there is an attack path from the starting point to the ending point. The system uses a backtracking algorithm to trace back from the high-risk endpoint to the upstream node with the highest activation value, thereby connecting the specific attack path chain: Bluetooth -> IVI -> Gateway -> Power CAN -> Brakes.

[0040] Compared with traditional rule-based or search-based engines, the graph neural network GNN can discover deeper, more non-obvious, and even cross-domain potential attack paths by learning complex dependency and propagation patterns in graph structures.

[0041] S33), based on the identified threats and deduced attack paths, the dynamic weighted retrieval enhancement RAG algorithm retrieves the most relevant information fragments as the context; In this embodiment, for each piece of information retrieved, the dynamic weighted retrieval enhancement RAG algorithm calculates its comprehensive weight by the following rules: The authority quantification rule assigns different basic weights according to the information source. For example, national standards or industry regulations are assigned a weight of 0.8, vehicle enterprise internal design documents are assigned a weight of 0.6, academic papers are assigned a weight of 0.4, and ordinary technical blog articles are assigned a weight of 0.2.

[0042] The timeliness decay function adjusts the weight according to the information publishing time, and the newer the information, the higher the weight. For example, the weight value decays by 0.1 per year of publishing year, but to retain the basic value, the minimum original weight of 0.5 is retained.

[0043] The comprehensive weighting algorithm obtains the maximum weight by weighted summation of the authority weight and the timeliness weight, for example: comprehensive weight = authority weight 0.6 + timeliness weight 0.4. And preferentially select the knowledge fragment with the highest comprehensive weight.

[0044] S4), construct a multi-stage structured prompt Prompt, and use a large language model to generate an analysis result, and output a standardized TARA report after processing, including the following steps: S41), use the asset list and network topology structure diagram, deduced attack path, RAG algorithm to retrieve the most relevant information fragments, and clear TARA analysis methodology to construct a multi-stage structured prompt Prompt; S42), based on the multi-stage structured prompt Prompt, use a large language model for constrained reasoning, and generate an analysis result; S43), post-process the analysis result returned by the large language model to generate a standardized TARA report containing a visual attack path graph and a structured report table.

[0045] In this embodiment, the large language model is a Qwen-7B model or a Gemini model that has been fine-tuned for domain adaptation and optimized by combining Architectural Enhancement and Adversarial Retrieval Training techniques.

[0046] The structured report form lists each identified threat in tabular form, including but not limited to: threat description, damage scenario, attack path, risk level (high, medium, low), and specific security mitigation recommendations.

[0047] Embodiment 2 The embodiment provides a car TARA analysis system based on multi-modal input and hybrid intelligence, comprising: A data uploading module for users to upload vehicle design data; In the embodiment, the format of the vehicle design data can be PDF, DOCX, PNG, JPG. The vehicle design data includes vehicle system architecture diagram, function description text, and parts list.

[0048] A feature extraction module for extracting text entity information and image objects from the vehicle design data; In the embodiment, the text entity information is extracted from the vehicle design data by an optical character recognition technology OCR.

[0049] The image objects are extracted from the vehicle design data by an improved YOLOv8 target recognition model. The improvement of the improved YOLOv8 target recognition model is that the improved YOLOv8 target recognition model is constructed by introducing an attention mechanism for small target recognition and a multi-scale feature fusion module on the basis of the backbone network of the YOLOv8 network model. The detection accuracy and robustness of micro icons, slender buses, and occluded elements in complex and dense automobile architecture diagrams are significantly improved. In the embodiment, the detection recall rate and accuracy of the improved YOLOv8 target recognition model on the automobile architecture diagram are improved by more than 20% compared with the general YOLOv8.

[0050] In addition, the embodiment trains the improved YOLOv8 target recognition model through artificial annotation and arrangement of tens of thousands of high-quality field-specific picture data sets for hundreds of unique symbols and diversified drawing styles of ECUs, CANs, LINs, Ethernet buses, sensors, actuators, connectors, etc. in the automobile architecture diagram.

[0051] In the embodiment, the center coordinates and boundary box information of the detected entities, and the endpoint coordinates and line features of the detected connection lines are used to construct the physical and logical connection relationships between the entities by geometric correlation rules, symbol semantic recognition, and connectivity analysis based on graph theory, and the relationships are converted into a network topology graph.

[0052] The fusion module uses a cross-modal information fusion algorithm of fusion spatial position, semantic similarity and domain ontology knowledge to intelligently match the extracted text entity information and image objects, and constructs a structured asset list, detailed attribute description and network topology structure diagram. Specifically, the following steps are included: S21), preliminary association of text entities and graphic objects is performed by spatial overlap and distance between text blocks and graphic objects; S22), semantic similarity between text description and graphic object category label is calculated using a BERT-based embedding model; S23), a predefined ontology knowledge base in the automobile field is used to correct the confidence matching below the preset confidence threshold to form entity-attribute-relation triples; S24), based on the entity-attribute-relation triples, a structured asset list, detailed attribute description and network topology structure diagram of the analyzed vehicle system are constructed.

[0053] The threat identification and path deduction module is used for threat identification and attack path deduction of the identified assets, and retrieval of the most relevant information fragments. Specifically, the following steps are included: S31), the asset (such as an in-vehicle infotainment system IVI) is taken as a keyword, and vector retrieval technology is used to match in a built-in threat knowledge base containing known vulnerability information and historical attack cases, to achieve threat identification; S32), the identified threat is taken as an initial threat node, and a graph neural network GNN is used to reason on a pre-built attack chain knowledge graph to deduce potential attack paths; S33), based on the identified threat and deduced attack path, a dynamic weighted retrieval enhanced RAG algorithm is used to retrieve the most relevant information fragments as context.

[0054] The analysis module is used for constructing a multi-stage structured prompt Prompt, and generating analysis results using a large language model, and outputting a standardized TARA report after processing. Specifically: S41), a multi-stage structured prompt Prompt is constructed using the asset list and network topology structure diagram, the deduced attack path, the RAG algorithm retrieved most relevant information fragments, and the explicit TARA analysis methodology; S42), based on the multi-stage structured prompt Prompt, a large language model is used for constrained reasoning, and analysis results are generated; S43), the analysis results returned by the large language model are post-processed to generate a standardized TARA report containing a visual attack path diagram and a structured report table.

[0055] Embodiment 3 This example is for TARA analysis of an in-vehicle infotainment system (IVI), as follows: S1)、The user uploads a complex engineering JPG picture containing the architecture of the IVI system, as well as a PDF design document describing its functions and operating system; S2)、Using the improved YOLOv8 target recognition model, hundreds of graphic assets such as IVI host, Bluetooth module, Wi-Fi module, and CAN bus are accurately recognized from the JPG picture; The geometric information and semantic features of the detected entities and connection lines are analyzed and detected, intelligently identifying the internal connection of the IVI host and Bluetooth module, the complex physical and logical topology relationship of the IVI host connected to the CAN bus through the gateway, etc. Through optical character recognition technology OCR, key text information such as Android operating system and App download function is extracted from the PDF document; Then the text recognized by OCR (such as IVI infotainment system) is intelligently matched with the graphic objects recognized by the YOLOv8 target recognition model, and finally the complete structured asset list, detailed attributes, and accurate network topology graph of the IVI system are constructed.

[0056] S3)、The system uses vector retrieval technology embedding vector retrieval method, quickly matches a known high-risk vulnerability CVE-2024-XXXX in the built-in CVE, ATT&CK, etc. knowledge base, which allows attackers to remotely execute code through the Bluetooth service.

[0057] GNN receives the initial threat node of the compromised IVI system, through the attack chain knowledge graph pre-built in the Neo4j graph database, GNN not only discovers that the IVI is connected to the power CAN bus through the vehicle gateway, but also further infers that through this connection, the attacker may launch an attack on the brake ECU from the IVI system, deducing a high-risk, multi-step attack path: external attacker-Bluetooth module-IVI system (exploit CVE-2024-XXXX vulnerability)-vehicle gateway-power CAN bus-brake ECU. The reasoning ability of GNN is shown here to be superior to traditional path search.

[0058] For the above threats and attack paths, relevant information is retrieved from the knowledge base. For example, an authoritative technical analysis article about this CVE vulnerability (published in 2023, with high timeliness and medium authority weight) and a CAN bus security specification of the car enterprise (published in 2022, with high authority and higher timeliness weight) are retrieved. Through the dynamic weighting algorithm, these two pieces of information obtain the highest comprehensive weight due to their high authority and, or high timeliness, and are preferentially extracted as supplementary context for the large language model to make decisions, ensuring the reliability of the decision information.

[0059] S4), the identified asset information, the deduced attack path, and the high-quality, high-confidence knowledge fragments filtered by the dynamic weighting RAG are combined with the TARA standard steps of ISO / SAE 21434 to construct a multi-stage, highly structured prompt (Prompt). The prompt is sent to the Gemini large language model that has been fine-tuned for deep domain adaptation. The large language model is explicitly instructed to play the role of a senior automotive network security expert and strictly follow the TARA methodology embedded in the Prompt for logical reasoning. Finally, a TARA report for the IVI system is generated, and the report is output in the form of a table as shown in Table 1: The above embodiments and descriptions described in the specification are only to illustrate the principles and best embodiments of the present application, and the present application can have various changes and improvements without departing from the spirit and scope of the present application. These changes and improvements all fall within the scope of the claimed application.

Claims

1. A vehicle TARA analysis method based on multimodal input and hybrid intelligence, comprising the following steps: S1) Obtain vehicle design data and extract text entity information and image objects from the vehicle design data; S2) Intelligently match the extracted text entity information and image objects to construct a structured asset list, detailed attribute descriptions, and network topology diagram; S3) Perform threat identification and attack path deduction on the identified assets, and retrieve the most relevant information fragments; S4) Construct a multi-stage structured prompt and use a large language model to generate analysis results, and output a standardized TARA report after post-processing.

2. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 1, characterized in that: In step S1), optical character recognition (OCR) technology is used to extract text entity information from the vehicle design data.

3. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 1, characterized in that: In step S1), the improved YOLOv8 target recognition model is used to extract image objects from the vehicle design data.

4. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 3, characterized in that: In step S1), the improvement of the improved YOLOv8 target recognition model is as follows: an attention mechanism for small target recognition and a multi-scale feature fusion module are introduced into the backbone network of the YOLOv8 network model to construct the improved YOLOv8 target recognition model. The improved YOLOv8 target recognition model was trained using a dedicated dataset. This dedicated dataset was constructed using unique symbols and diverse drawing styles for ECUs, CAN, LIN, Ethernet buses, sensors, actuators, and connectors in automotive architecture diagrams, as well as through manual annotation.

5. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 3, characterized in that: In step S1), the center coordinates and bounding box information of the detected entity, as well as the endpoint coordinates and line type features of the detected connecting lines are used; By employing geometric association rules, symbolic semantic recognition, and graph theory-based connectivity analysis, the physical and logical connections between entities are constructed and transformed into a network topology graph.

6. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 1, characterized in that: In step S2), a cross-modal information fusion algorithm that integrates spatial location, semantic similarity, and domain ontology knowledge is used to intelligently match text entity information and image objects, constructing a structured asset list, detailed attribute descriptions, and a network topology diagram; specifically, the following steps are included: S21) Initial association between text entities and graphic objects is made by measuring the spatial overlap and distance between text blocks and graphic objects; S22) Calculate the semantic similarity between text descriptions and graphical object category labels using an embedding model; S23) Use a predefined automotive domain ontology knowledge base to correct confidence matches that are below a pre-set confidence threshold, forming entity-attribute-relationship triples; S24) Based on entity-attribute-relationship triples, construct a structured asset list, detailed attribute descriptions, and network topology diagram of the analyzed vehicle system.

7. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 6, characterized in that: In step S3), threat identification and attack path simulation are performed on the identified assets; specifically, the following steps are included: S31) Using assets as keywords, the threat identification is achieved by matching them in the built-in threat knowledge base containing known vulnerability information and historical attack cases through vector retrieval technology; S32) The identified threats are used as initial threat nodes, and graph neural networks (GNNs) are used to reason on a pre-built attack chain knowledge graph to deduce potential attack paths. S33) Based on the identified threats and the inferred attack paths, the most relevant information fragments are retrieved as context using the dynamically weighted retrieval enhanced generation RAG algorithm.

8. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 7, characterized in that: In step S33), for each retrieved piece of information, the Dynamic Weighted Retrieval Enhancement Generation (RAG) algorithm calculates its comprehensive weight according to the following rules: Authoritative quantification rules, time-sensitive decay function, and comprehensive weighted algorithm.

9. The vehicle TARA analysis method based on multimodal input and hybrid intelligence according to claim 8, characterized in that: In step S4), a multi-stage structured prompt is constructed, and analysis results are generated using a large language model. After post-processing, a standardized TARA report is output, which specifically includes the following steps: S41) Construct a multi-stage structured prompt using asset lists and network topology diagrams, deduced attack paths, RAG algorithm to retrieve the most relevant information fragments, and a clear TARA analysis methodology; S42) Based on multi-stage structured prompts, constrained reasoning is performed using a large language model, and analysis results are generated; S43) Post-process the analysis results returned by the large language model to generate a standardized TARA report that includes a visualized attack path diagram and a structured report table.

10. A vehicle TARA analysis system based on multimodal input and hybrid intelligence, characterized in that, include: The data upload module is used by users to upload vehicle design data; The feature extraction module is used to extract text entity information and image objects from vehicle design data; The fusion module uses a cross-modal information fusion algorithm that integrates spatial location, semantic similarity, and domain ontology knowledge to intelligently match extracted text entity information and image objects, and construct a structured asset list, detailed attribute descriptions, and network topology diagram. The threat identification and path deduction module is used to identify threats to identified assets, deduce attack paths, and retrieve the most relevant information fragments. The analysis module is used to construct multi-stage structured prompts and generate analysis results using a large language model, which are then post-processed to output a standardized TARA report.

Images