Cross-vendor federated inference method, device, equipment, medium, vehicle and system for vehicle model

Abstract

The application provides a cross-vendor vehicle model federated inference method, device, electronic equipment, computer readable storage medium, vehicle and system. The method comprises the following steps: a vehicle initiating node in a local vendor domain broadcasts a federated inference request in a vehicle ad hoc network, receives a processing identifier fed back by a vehicle responding node in a heterogeneous vendor domain, and determines a cross-vendor federated collaborative node; encapsulates driving environment data into a standardized vehicle sensor frame, and distributes the standardized vehicle sensor frame to each collaborative node through an encrypted vehicle communication link; each collaborative node extracts local feature information by using a locally deployed vehicle model inference engine, and the internal parameters are hidden from each vendor domain; uploads a standardized output package after injecting a differential disturbance sequence into the local feature information; and the vehicle initiating node performs federated fusion in the dimension of ciphertext to generate a global cognitive result, which helps to improve the multi-vehicle environment cognitive ability under a lower exposure risk.

Description

Technical Field

[0001] This application relates to the field of intelligent connected vehicles and in-vehicle artificial intelligence inference technology, and more specifically, to a cross-vendor in-vehicle model federated inference method, apparatus, electronic device, computer-readable storage medium, vehicle, and system. Background Technology

[0002] With the development of intelligent connected vehicles, vehicle-to-infrastructure (V2I) infrastructure, and in-vehicle AI perception technologies, the demand for vehicles to perceive surrounding traffic participants, road conditions, and dynamic obstacles in complex road environments is constantly increasing. Single-vehicle sensors are easily limited by factors such as field of view obstruction, detection distance, weather conditions, and the vehicle's own computing resources, making it difficult to obtain comprehensive environmental information in all driving scenarios. Therefore, how to leverage the collaborative capabilities between vehicles and between vehicles and networks to enhance environmental awareness has become an important technological direction for in-vehicle intelligent systems.

[0003] Existing vehicle-mounted environmental collaborative perception solutions typically employ a cloud-based centralized processing architecture. Vehicles upload collected visual, radar, or positioning data to a platform server. The platform server then performs format conversion and centralized modeling of the data uploaded from different vehicles using a unified interface, before calling pre-built perception models to generate road environment analysis results. Subsequently, the platform server distributes the road environment analysis results to vehicle terminals via cellular networks or roadside communication networks to assist vehicles in updating their local driving decision-making information.

[0004] However, this cloud-based centralized processing solution has significant technical drawbacks. Because vehicle-side data needs to be centrally uploaded to the platform server, the sensor data formats, model output structures, and communication link states of different vehicles vary considerably. This leads to issues such as high data conversion overhead, significant transmission latency, and insufficient participation of local vehicle perception capabilities during centralized processing. Furthermore, the raw perception data and model-related information from the vehicle-side are at high risk of exposure during centralized interaction, resulting in low adaptability of multi-vehicle collaborative perception in terms of privacy protection, model protection, and real-time response, making it difficult to meet the needs for multi-view environmental cognition in complex driving environments. Summary of the Invention

[0005] This application provides a cross-vendor in-vehicle model federated inference method, apparatus, electronic device, computer-readable storage medium, vehicle, and system to at least alleviate the aforementioned technical problems.

[0006] A cross-vendor in-vehicle model federated inference method includes: The initiating node of this vendor domain broadcasts a federated inference request within the in-vehicle ad hoc network, receives processing identifiers from the in-vehicle response nodes of heterogeneous vendor domains, and determines the cross-vendor federated collaboration node from the in-vehicle response nodes based on the processing identifiers. The vehicle-mounted initiating node encapsulates driving environment data into standardized vehicle-mounted sensor frames and distributes the standardized vehicle-mounted sensor frames to each of the cross-vendor federated collaborative nodes through an encrypted vehicle-mounted communication link. Each of the cross-vendor federated collaborative nodes uses a locally deployed vehicle model inference engine to perform feature extraction on the standardized vehicle sensor frames and generate local feature information; the internal parameters of the vehicle model inference engine are hidden from each vendor domain. Each of the cross-vendor federated collaborative nodes injects a differential perturbation sequence into the local feature information to obtain privacy-preserving data, and then encapsulates the privacy-preserving data into a standardized output package and uploads it to the vehicle initiation node; The in-vehicle initiating node performs encrypted-dimensional federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

[0007] Optionally, the initiating node of the local vendor domain broadcasts a federated inference request within the vehicular ad hoc network, receives processing identifiers from the responding nodes of heterogeneous vendor domains, and determines cross-vendor federated cooperation nodes from the responding nodes based on the processing identifiers, including: The initiating node broadcasts the federated inference request to the in-vehicle ad hoc network. The federated inference request carries a minimum threshold for the number of participating nodes and a federated privacy strength level. The responding node of the vehicle responds to the processing identifier based on the federated inference request. The vehicle-mounted initiating node receives the processing identifier, reads the historical federated execution credentials and vehicle network reputation score, and uses the historical federated execution credentials and vehicle network reputation score as the screening criteria for determining the cross-vendor federated collaboration node. The vehicle-initiating node marks the corresponding vehicle-response node as a cross-vendor federated collaboration node when the vehicle network reputation score reaches a preset score threshold and the historical federated execution credentials have no abnormal records within a preset time span. The total number of the determined cross-vendor federated collaboration nodes is then verified using the minimum number of participating nodes threshold.

[0008] Optionally, the vehicle-initiating node encapsulates driving environment data into standardized vehicle sensor frames and distributes these standardized vehicle sensor frames to each of the cross-vendor federated collaboration nodes via an encrypted vehicle communication link, including: The vehicle-mounted initiating node acquires driving environment data collected by local physical sensors and extracts the underlying device-specific structural features belonging to the hardware of the specified manufacturer from the driving environment data. The vehicle-mounted initiating node uses a cross-device universal visual space protocol to align the dimensions of the proprietary structural features of the underlying device, generating standardized vehicle-mounted sensor frames that eliminate hardware form differences. The vehicle-initiating node performs intermediate representation compatibility verification between the standardized vehicle sensor frame and the determined cross-vendor federated collaboration nodes. After the verification is successful, the standardized vehicle sensor frame is sent to each of the cross-vendor federated collaboration nodes through the encrypted vehicle communication link.

[0009] Optionally, each of the cross-vendor federated collaborative nodes utilizes a locally deployed in-vehicle model inference engine to perform feature extraction on the standardized in-vehicle sensor frames, generating local feature information, including: Each of the cross-vendor federated collaborative nodes activates the locally deployed vehicle model inference engine and inputs the standardized vehicle sensor frames into the computational flow graph containing the tensor mapping structure in the vehicle model inference engine; Each of the cross-vendor federated collaborative nodes performs feature extraction on the standardized vehicle sensor frames based on the computation flow graph in the vehicle-independent computing sandbox environment, generating local feature information representing road condition perception and prediction elements. During the feature extraction process, each of the cross-vendor federated collaborative nodes blocks the transmission path of the network topology and internal parameters of the vehicle model inference engine to the external vehicle communication port, so that the local feature information does not carry the internal parameters of the vehicle model inference engine, and maintains the cross-vendor hidden state of the internal parameters.

[0010] Optionally, each of the cross-vendor federated collaborative nodes injects a differential perturbation sequence into the local feature information to obtain privacy-preserving data, and then encapsulates the privacy-preserving data into a standardized output packet before uploading it to the vehicle initiation node, including: Each of the cross-vendor federated collaboration nodes extracts the federated privacy strength level from the federated inference request as the basis for determining the perturbation strength of the differential perturbation sequence; Each of the cross-vendor federated collaborative nodes determines the perturbation sequence generation rules for obfuscating the real distribution based on the federated privacy strength level, and generates the corresponding differential perturbation sequence; Each of the cross-vendor federated collaborative nodes superimposes the differential perturbation sequence onto the local feature information to obtain privacy-preserving data that masks the original road condition features; Each of the cross-vendor federated collaborative nodes uses an in-vehicle anti-tampering verification mechanism to digitally sign the privacy-protected data, enabling the privacy-protected data to carry proof of execution legitimacy, generating a standardized output package and uploading it to the in-vehicle initiating node.

[0011] Optionally, the vehicle-mounted initiating node performs federated fusion of the encrypted dimensions of each of the standardized output packets to generate a global cognitive result for the current driving environment, including: The vehicle-mounted initiating node extracts the ciphertext feature tensor from each of the standardized output packets; The vehicle-mounted initiating node performs anti-poisoning robust aggregation on each of the encrypted feature tensors, removing abnormal feature tensors that deviate from the distribution center interval, and retaining the encrypted feature tensors that have not been removed. The vehicle-mounted initiating node performs federated fusion of the preserved ciphertext feature tensor through the vehicle-mounted homomorphic processing unit to generate a global cognitive result containing multi-view road condition information. The vehicle-mounted initiating node compares the accuracy of the road condition perception of the global cognition result with the accuracy of the local reasoning of the single vehicle to obtain an accuracy improvement value. When the accuracy improvement value reaches the accuracy improvement requirement, the global cognition result is saved.

[0012] Optionally, the step of performing anti-poisoning robust aggregation on each of the ciphertext feature tensors, removing anomalous feature tensors that deviate from the distribution center interval, and retaining the ciphertext feature tensors that were not removed, includes: The vehicle-mounted initiating node calculates the vector distance metric between each of the encrypted feature tensors in the encrypted computation space carrying each of the encrypted feature tensors, and uses it as the distance judgment basis for identifying the abnormal feature tensors; The vehicle-mounted initiating node numerically sorts each of the encrypted feature tensors based on the vector distance metric, thereby obtaining a sorted sequence of encrypted feature tensors. The ciphertext feature tensor located at the sorting truncation ratio positions at both ends of the ciphertext feature tensor sorting sequence by the vehicle-mounted initiating node is determined as the range of feature tensors to be discarded that deviates from the distribution center interval. The vehicle-mounted initiating node determines the encrypted feature tensor located at the sorting truncation ratio position as an abnormal feature tensor that may have been maliciously tampered with and performs a data discard operation. The encrypted feature tensor that is not discarded is retained and passed to the subsequent encrypted dimension federated fusion stage.

[0013] Optionally, after generating the global cognitive result, the method further includes: The vehicle-mounted initiating node obtains the first environmental perception deviation record of the standardized output package submitted by each of the cross-vendor federated collaborative nodes in this federated processing cycle, compared with the global fusion benchmark corresponding to the global perception result. The vehicle-mounted initiating node appends and stores the first environmental perception deviation record to the corresponding cross-vendor federated collaboration node's vehicle-mounted historical deviation record library; The vehicle-initiating node dynamically downgrades the communication priority of the cross-vendor federated collaboration node in the next federated processing cycle or performs a removal operation on the cross-vendor federated collaboration node based on the continuous deviation change trend presented in the vehicle historical deviation record database. The vehicle-mounted initiating node synchronizes the communication priority change status of the cross-vendor federated collaboration node to the other vehicle nodes in the vehicle network via the vehicle network broadcast link, so that it can be used in the next round of federated processing cycle to determine the cross-vendor federated collaboration node.

[0014] Optionally, after generating the global cognitive result, the method further includes: The vehicle-mounted initiating node allocates basic computing resource scheduling credentials to the corresponding cross-vendor federated collaboration node based on the duration of the vehicle network connection maintained by the cross-vendor federated collaboration node. The vehicle-mounted initiating node extracts the data deviation metric value of the standardized output packet uploaded by the cross-vendor federated collaboration node compared with the average level of distributed collaboration. The average level of distributed collaboration is determined by the standardized output packet uploaded by each cross-vendor federated collaboration node in this federated processing cycle. The data deviation metric value is used as the basis for issuing additional computing resource scheduling credentials. When the data deviation metric falls within the target deviation range and the vehicle model accuracy level identifier of the cross-vendor federated collaboration node meets the target accuracy level condition, the vehicle initiating node issues the additional computing resource scheduling certificate to the corresponding cross-vendor federated collaboration node. The cross-vendor federated collaborative nodes exchange corresponding distributed data processing bandwidth in the vehicular ad hoc network based on the acquired basic computing resource scheduling credentials and the additional computing resource scheduling credentials.

[0015] A cross-vendor in-vehicle model federated inference device, comprising: The collaborative node determination module is used to enable the vehicle initiating node of the local vendor domain to broadcast federated inference requests within the vehicle ad hoc network, receive processing identifiers fed back by vehicle response nodes of heterogeneous vendor domains, and determine cross-vendor federated collaborative nodes from the vehicle response nodes based on the processing identifiers. The sensor frame distribution module is used to enable the vehicle initiating node to encapsulate driving environment data into standardized vehicle sensor frames, and distribute the standardized vehicle sensor frames to each of the cross-vendor federated collaboration nodes through an encrypted vehicle communication link. The local feature generation module is used to enable each of the cross-vendor federated collaborative nodes to perform feature extraction on the standardized vehicle sensor frames using the locally deployed vehicle model inference engine to generate local feature information; the internal parameters of the vehicle model inference engine are hidden from each vendor domain. The output packet generation module is used to enable each of the cross-vendor federated collaborative nodes to inject differential perturbation sequences into the local feature information to obtain privacy-preserving data, and to encapsulate the privacy-preserving data into a standardized output packet and upload it to the vehicle initiation node. The federated fusion module enables the vehicle-mounted initiating node to perform encrypted-dimensional federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

[0016] An electronic device includes a processor and a memory, the memory storing a computer program that, when executed by the processor, causes the processor to perform any of the aforementioned cross-vendor in-vehicle model federated inference methods.

[0017] A computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the aforementioned cross-vendor in-vehicle model federated inference methods.

[0018] A vehicle includes an onboard communication unit, an onboard computing unit, and an onboard storage unit. The onboard storage unit stores a computer program. When the computer program is executed by the onboard computing unit, the vehicle participates in an onboard ad hoc network through the onboard communication unit and executes any of the aforementioned cross-vendor onboard model federated inference methods.

[0019] A cross-vendor in-vehicle model federated inference system includes in-vehicle initiation nodes in the local vendor domain and in-vehicle response nodes in heterogeneous vendor domains. The initiating node is used to broadcast federated inference requests within the in-vehicle ad hoc network, receive processing identifiers fed back by the responding nodes, and determine cross-vendor federated collaboration nodes from the responding nodes based on the processing identifiers. The vehicle initiating node is also used to encapsulate driving environment data into standardized vehicle sensor frames, and distribute the standardized vehicle sensor frames to each of the cross-vendor federated collaboration nodes through an encrypted vehicle communication link. Each of the cross-vendor federated collaborative nodes is used to perform feature extraction on the standardized vehicle sensor frames using a locally deployed vehicle model inference engine to generate local feature information; the internal parameters of the vehicle model inference engine are hidden from each vendor domain. Each of the cross-vendor federated collaborative nodes is also used to inject differential perturbation sequences into the local feature information to obtain privacy-preserving data, and then encapsulate the privacy-preserving data into a standardized output package and upload it to the vehicle initiation node; The in-vehicle initiating node is also used to perform encrypted dimension federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

[0020] Technical advantages of the technical solution provided in this application This application's cross-vendor in-vehicle model federated inference method addresses the technical shortcomings of cloud-based centralized processing solutions in the background art, such as high data conversion overhead, large transmission latency, insufficient participation of local vehicle perception capabilities, and high risks of privacy and model-related information exposure. Instead, the initiating node in the local vendor's domain broadcasts a federated inference request within the in-vehicle ad hoc network, receives processing identifiers from responding nodes in heterogeneous vendor domains, and determines cross-vendor federated collaborative nodes based on these identifiers. Compared to traditional solutions that rely on a platform server to uniformly collect vehicle data, this application first forms a set of collaborative nodes within the vehicle network, enabling the computing power and environmental perception capabilities of surrounding vehicles to participate in the current driving environment processing. This helps reduce the single-point processing pressure on the centralized server and improves the flexibility of multi-vehicle collaborative participation.

[0021] This application encapsulates driving environment data into standardized vehicle sensor frames via an onboard initiating node and distributes them to various cross-vendor federated collaborative nodes through an encrypted onboard communication link. Compared to traditional solutions where data from different vehicles is directly uploaded and then centrally converted by a platform server, standardized vehicle sensor frames provide a unified input format for different nodes to receive and process the same driving environment data. The encrypted onboard communication link reduces the risk of data leakage and tampering during transmission. Therefore, this application can alleviate the problem of low processing adaptability caused by differences in vehicle data formats and insufficient communication security when different nodes participate in collaborative processing.

[0022] This application enables cross-vendor federated collaborative nodes to perform feature extraction on standardized vehicle sensor frames using locally deployed in-vehicle model inference engines, generating local feature information, and hiding the internal parameters of the in-vehicle model inference engine from each vendor domain. Compared to traditional centralized processing schemes where vehicle-side data and model-related information need to interact centrally around a platform server, this application retains feature extraction to be performed locally on the collaborative nodes. Each node uploads local feature information rather than the internal parameters of the in-vehicle model inference engine, which helps reduce the possibility of exposing internal model information during cross-domain interactions, while enabling the local inference capabilities of different vehicle nodes to be utilized.

[0023] This application further enables cross-vendor federated collaborative nodes to inject differential perturbation sequences into local feature information to obtain privacy-preserving data. This privacy-preserving data is then encapsulated into a standardized output package and uploaded to the vehicle-initiating node. Compared to traditional solutions that directly upload vehicle-collected data or undisturbed intermediate data, the differential perturbation sequence provides perturbation protection for local feature information, and the standardized output package provides a more unified data carrier format for subsequent fusion. This balances information fusion and privacy protection requirements during collaborative processing, reducing the risk of reverse inference of original road condition features.

[0024] This application involves the initiating node performing encrypted-dimensional federated fusion of standardized output packets to generate a global cognitive result specific to the current driving environment. Compared to traditional solutions that centrally process uploaded data in plaintext on the server side, this application fuses the output packets uploaded by each collaborating node at the encrypted level, enabling local features from multiple nodes to jointly participate in the generation of the global cognitive result with lower exposure risk. Therefore, this application addresses the issues of real-time performance, privacy protection, and insufficient utilization of multi-vehicle capabilities inherent in centralized collaborative perception in the prior art, providing a more suitable collaborative reasoning processing method for complex driving environments. This results in a cognitive result for the current driving environment possessing richer multi-perspective information sources and higher adaptability to collaborative processing. Attached Figure Description

[0025] Figure 1 Corresponding to the cross-vendor in-vehicle model federated inference method; Figure 2 Corresponding to cross-vendor in-vehicle model federated inference devices; Figure 3 Corresponding electronic devices; Figure 4 Corresponding computer-readable storage medium; Figure 5 Corresponding vehicle; Figure 6 Corresponding to cross-vendor in-vehicle model federated inference system. Detailed Implementation

[0026] The following combination Figures 1 to 6 The embodiments of this application will be described. Figure 1 Corresponding to the cross-vendor in-vehicle model federated inference method, Figure 2 Corresponding to cross-vendor in-vehicle model federated inference devices, Figure 3 Corresponding electronic devices, Figure 4 Corresponding computer-readable storage media, Figure 5 Corresponding vehicles, Figure 6 This corresponds to a cross-vendor in-vehicle model federated inference system. The same technical features in each embodiment use consistent names. The technical processing results from the previous technical processing stage serve as the input or judgment basis for the next technical processing stage, thus establishing a continuous data processing relationship between the in-vehicle initiating node, the cross-vendor federated collaboration node, standardized in-vehicle sensor frames, local feature information, privacy-preserving data, standardized output packets, and global cognitive results.

[0027] In one possible scenario, such as Figure 1As shown, a cross-vendor in-vehicle model federated inference method is provided, comprising: an in-vehicle initiating node in the local vendor domain broadcasting a federated inference request within an in-vehicle ad hoc network, receiving processing identifiers fed back by in-vehicle response nodes in heterogeneous vendor domains, and determining cross-vendor federated collaborative nodes from the in-vehicle response nodes based on the processing identifiers; the initiating node encapsulating driving environment data into standardized in-vehicle sensor frames, and distributing the standardized in-vehicle sensor frames to each of the cross-vendor federated collaborative nodes through an encrypted in-vehicle communication link; each of the cross-vendor federated collaborative nodes using a locally deployed in-vehicle model inference engine to perform feature extraction on the standardized in-vehicle sensor frames to generate local feature information; the internal parameters of the inference engine are hidden from each vendor domain; each of the cross-vendor federated collaborative nodes injects a differential perturbation sequence into the local feature information to obtain privacy-preserving data, encapsulates the privacy-preserving data into a standardized output packet, and uploads it to the initiating node; the initiating node performs encrypted-dimensional federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

[0028] In one possible scenario, the initiating node in the local vendor domain is an initiating vehicle-side intrusive computing node capable of sending collaborative inference messages and receiving collaborative inference responses within the intrusive ad hoc network. The responding node in the heterogeneous vendor domain is provided by vehicles from other vendor domains, capable of receiving federated inference requests and returning processing identifiers. The intrusive ad hoc network is a temporary inter-vehicle communication network formed by adjacent vehicle nodes according to vehicle-to-everything (V2X) communication protocols. The initiating node first generates a federated inference request, which includes at least a message type field, an initiating node address field, a minimum participating node threshold, a federated privacy strength level, and a driving environment data type field. The message type field enables the responding node to identify that the message belongs to a federated inference task; the initiating node address field enables the responding node to return a processing identifier; the minimum participating node threshold limits the lower limit of the number of nodes participating in collaborative inference; the federated privacy strength level is used to subsequently determine the intensity of the differential perturbation sequence; and the driving environment data type field enables the responding node to determine whether its locally deployed intrusive model inference engine can process subsequently received standardized intrusive sensor frames. The initiating node encapsulates the federated inference request into a broadcast message that can be transmitted within the vehicular ad hoc network and broadcasts the federated inference request within the vehicular ad hoc network. After parsing the federated inference request, the responding node writes its node address, heterogeneous vendor domain identifier, availability status of its locally deployed vehicular model inference engine, and standardized vehicular sensor frame receiving capability into a processing identifier, and feeds back the processing identifier to the initiating node. The initiating node parses the fields of the received processing identifier, locates the responding node using the node address in the processing identifier, confirms that the responding node belongs to a heterogeneous vendor domain using the heterogeneous vendor domain identifier, and determines whether the responding node can participate in this federated inference based on the availability status of its locally deployed vehicular model inference engine and standardized vehicular sensor frame receiving capability. Thus, it identifies cross-vendor federated collaborative nodes from among the responding nodes.

[0029] In one possible scenario, driving environment data refers to data collected by the vehicle using local physical sensors in the current driving environment. This data can include image data from cameras, target reflection data from millimeter-wave radar, point-like spatial data from lidar, near-range obstacle data from ultrasonic sensors, and vehicle positioning data. The onboard initiating node performs temporal and spatial alignment on the driving environment data to form multi-source driving environment data at the same acquisition time. The onboard initiating node further reads the underlying device-specific structural features related to the manufacturer's hardware from the multi-source driving environment data. These features can include image resolution, channel arrangement, sensor coordinate origin, sensor mounting orientation, pixel sampling ratio, radar range resolution, and point-like spatial data density. The vehicle-mounted initiating node utilizes a cross-device universal visual spatial protocol to perform size and coordinate alignment on the underlying device's proprietary structural features. This protocol defines the frame header, sensor payload, spatial coordinates, and time stamp fields for standardized vehicle-mounted sensor frames. The frame header records the frame type, the sensor payload carries the size-aligned driving environment data, the spatial coordinates record the spatial mapping relationship after alignment, and the time stamp associates the data acquisition time with the current driving environment. The vehicle-mounted initiating node combines these fields into a standardized vehicle-mounted sensor frame and distributes it to various cross-vendor federated collaborative nodes via an encrypted vehicle communication link. The encrypted vehicle communication link uses a session key to encrypt the transmission of the standardized vehicle-mounted sensor frame and includes an integrity check value in the transmission packet, enabling the cross-vendor federated collaborative nodes to verify that the standardized vehicle-mounted sensor frame has not been tampered with during transmission.

[0030] In one possible scenario, the vehicle-mounted model inference engine is an in-vehicle inference program deployed locally on a cross-vendor federated collaborative node. The in-vehicle model inference engine can include a sensor frame input interface, a tensor mapping structure, a road condition element encoding structure, and a local feature output interface. The sensor frame input interface reads the sensor payload field, spatial coordinate field, and time stamp field from standardized vehicle sensor frames and maps the sensor payload field to an input tensor. The tensor mapping structure performs dimensional rearrangement, scale normalization, and spatial location mapping on the input tensor, making it an intermediate tensor adapted to the computational flow graph of the locally deployed in-vehicle model inference engine. The road condition element encoding structure performs convolution mapping, attention weighting, or temporal encoding on the intermediate tensor to extract road condition perception and prediction elements such as road boundaries, obstacle distribution, lane passability areas, and dynamic traffic participant states. The local feature output interface organizes the road condition perception and prediction elements into local feature information. Local feature information is an intermediate representation formed by cross-vendor federated collaborative nodes after performing local inference on standardized vehicle sensor frames. Local feature information may include feature dimension descriptions, pre-encryption feature values, road condition perception prediction element category labels, and current driving environment association labels. Local feature information does not include the network topology, weight parameters, and bias parameters of the vehicle model inference engine. The cross-vendor federated collaborative nodes perform the above processing in an in-vehicle independent computing sandbox environment. This in-vehicle independent computing sandbox environment isolates the file access path, memory access path, and external vehicle communication port of the vehicle model inference engine, allowing local feature information to be uploaded through a controlled output path, while the internal parameters of the vehicle model inference engine cannot be output through the external vehicle communication port.

[0031] Preferably, in the specific technical implementation of this application, the vehicle-mounted model inference engine can adopt a computational flow graph formed by sequentially connecting a sensor frame input interface, a tensor mapping structure, a spatial neighborhood convolutional layer, a multi-scale convolutional layer, a cross-sensor channel fusion structure, a road condition element attention mapping structure, a temporal coding structure, and a local feature output interface. The sensor frame input interface is used to read standardized vehicle-mounted sensor frames; the tensor mapping structure is used to convert the standardized vehicle-mounted sensor frames into input tensors; the spatial neighborhood convolutional layer is used to perform local spatial weighted accumulation on the input tensors; the multi-scale convolutional layer is used to form multi-scale spatial response tensors within different sensing ranges; the cross-sensor channel fusion structure is used to perform weighted fusion of channels from different sensor payload types in the multi-scale spatial response tensors; the road condition element attention mapping structure is used to weight the fused tensor expression according to road condition perception and prediction elements; the temporal coding structure is used to perform temporal correlation of the weighted element results within adjacent federated processing cycles by combining a time stamp field; and the local feature output interface is used to organize the temporally correlated element expression into local feature information. The convolution kernel weights, channel fusion weights, feature template parameters, temporal coding weights, normalization adjustment parameters, and bias parameters in the above computational flow graph are all internal parameters of the vehicle model inference engine. The cross-vendor federated collaboration node only outputs local feature information and does not output the internal parameters of the vehicle model inference engine.

[0032] Preferably, when the sensor frame input interface in this application parses the standardized vehicle sensor frame, it first reads the sensor payload field, spatial coordinate field, and time stamp field from the standardized vehicle sensor frame, and then arranges the sensor payload field into an input tensor according to the spatial coordinate field. In the spatial plane of the input tensor, the row direction corresponds to the longitudinal sampling position of the vehicle in the vehicle coordinate expression, the column direction corresponds to the lateral sampling position of the vehicle in the vehicle coordinate expression, and the channel direction corresponds to sensor payload types such as camera image channel, radar reflection intensity channel, point spatial height channel, and vehicle positioning auxiliary channel; the tensor elements located at the intersection of the same row direction, the same column direction, and the same channel direction in the input tensor represent the numerical expression of the corresponding sensor payload type at the corresponding spatial position around the vehicle. The sensor frame input interface generates a validity mask tensor at the same time as forming the input tensor. The validity mask tensor has the same spatial sampling position as the input tensor, and each element in the validity mask tensor indicates whether there is a valid sensor payload at the spatial sampling position corresponding to the input tensor; the validity mask tensor then participates in the spatial neighborhood convolutional layer and the cross-sensor channel fusion structure, so that the influence of the hole sampling position or invalid sampling position on the local feature information is reduced.

[0033] Preferably, the mathematical implementation principle of the tensor mapping structure in this application involves spatial resampling, channel rearrangement, and scale normalization of the input tensor. Spatial resampling first determines the target sampling position of each sensor payload field in the vehicle coordinate representation based on the spatial coordinate field. Then, it reads the adjacent sampling points around the target sampling position and weights them according to the spatial distance between the adjacent sampling points and the target sampling position to form a vehicle coordinate tensor. The vehicle coordinate tensor then enters the channel rearrangement process. This process arranges the camera image channel, radar reflection intensity channel, point spatial height channel, and vehicle positioning assistance channel to fixed channel positions according to the pre-configured sensor payload type order, forming a channel-aligned tensor. The channel-aligned tensor then enters the scale normalization process. This process uses the channel mean parameter and channel scale parameter configured before deployment to unify the numerical range of different sensor payload types, forming a normalized input tensor. The normalized input tensor serves as the input to the spatial neighborhood convolutional layer, and the validity mask tensor simultaneously serves as the sampling validity basis for the spatial neighborhood convolutional layer.

[0034] Preferably, in this application, the spatial neighborhood convolutional layer extracts local spatial features through convolutional layers. At each target sampling location of the normalized input tensor, the spatial neighborhood convolutional layer reads the neighborhood tensor elements centered on the target sampling location and reads the convolutional kernel weights corresponding to the neighborhood tensor elements dimension by dimension. The spatial neighborhood convolutional layer multiplies the neighborhood tensor elements and convolutional kernel weights term by term, then performs spatial accumulation and channel accumulation on the term-by-term products to obtain a neighborhood weighted accumulated value. The spatial neighborhood convolutional layer superimposes the neighborhood weighted accumulated value with the bias parameter, and then performs channel normalization adjustment and bounded nonlinear mapping to form a spatial convolutional response tensor. Channel normalization adjustment is used to reduce the differences in sensor load value ranges among different cross-vendor federated cooperative nodes, and bounded nonlinear mapping is used to compress abnormally large response values ​​and retain positive road condition responses. Each tensor element in the spatial convolutional response tensor represents the local spatial response of the corresponding vehicle's surrounding spatial location under the corresponding convolutional kernel. The spatial convolutional response tensor is then input into a multi-scale convolutional layer.

[0035] Preferably, in this application, the multi-scale convolutional layer forms spatial representations with different receptive ranges through multiple parallel convolutional layers. The multi-scale convolutional layer reads tensor elements from the spatial convolutional response tensor within smaller, moderate, and larger neighborhoods, respectively. The smaller neighborhood is used to represent fine-grained road condition perception and prediction elements such as lane line edges and near-distance obstacle edges; the moderate neighborhood is used to represent local target contours such as vehicles, non-motorized vehicles, and pedestrians; and the larger neighborhood is used to represent regional road condition distributions in intersection areas, curve areas, and occluded areas. The multi-scale convolutional layer performs convolutional weighted accumulation, channel normalization adjustment, and bounded nonlinear mapping on the tensor elements within different neighborhoods, forming small-range, medium-range, and large-range spatial response tensors. The multi-scale convolutional layer then concatenates the small-range, medium-range, and large-range spatial response tensors according to the channel direction and reduces redundant channels through channel compression mapping to form a multi-scale spatial response tensor. The multi-scale spatial response tensor incorporates fine-grained edge representation, local target contour representation, and regional road condition distribution representation into the same tensor representation, and then serves as the input to the cross-sensor channel fusion structure.

[0036] Preferably, the cross-sensor channel fusion structure in this application is used to fuse responses of different sensor payload types in a multi-scale spatial response tensor. The cross-sensor channel fusion structure first calculates a channel reliability description based on the validity mask tensor. This channel reliability description represents the effective sampling ratio and spatial continuity of each sensor payload type in the current driving environment. The cross-sensor channel fusion structure then inputs the channel reliability description into a normalized weight allocation process to form cross-sensor channel fusion weights. The cross-sensor channel fusion structure then performs a channel-by-channel weighted summation of the response values ​​from the camera image channel, radar reflection intensity channel, point spatial height channel, and vehicle positioning assistance channel in the multi-scale spatial response tensor according to the cross-sensor channel fusion weights, forming a cross-sensor fusion tensor. Each tensor element in the cross-sensor fusion tensor represents the fused response of the corresponding vehicle's surrounding spatial location under the combined action of multiple sensor payload types. The cross-sensor fusion tensor is then input into a road condition element attention mapping structure.

[0037] Preferably, the mathematical implementation principle of the road condition element attention mapping structure in this application is to perform similarity matching between the spatial location description in the cross-sensor fusion tensor and the element template description corresponding to the road condition perception prediction elements, and then assign weights based on the similarity matching results. The road condition element attention mapping structure first extracts a spatial location description vector from each spatial sampling location of the cross-sensor fusion tensor. The elements of the spatial location description vector sequentially represent the multi-scale spatial response, cross-sensor fusion response, and the corresponding value of the validity mask tensor at that spatial sampling location. The road condition element attention mapping structure then reads the road condition element template vectors corresponding to road boundaries, obstacle distribution, lane passable areas, and dynamic traffic participant states from the internal parameters of the vehicle model inference engine. The road condition element attention mapping structure performs element-wise multiplication and summation of the spatial location description vector and each road condition element template vector to form an element matching score. The road condition element attention mapping structure then normalizes and assigns weights to the element matching scores to form road condition element attention weights. Finally, the road condition element attention mapping structure uses the road condition element attention weights to weight the cross-sensor fusion tensor to form an element enhancement tensor. The channels in the element enhancement tensor correspond to different road condition perception and prediction elements. Therefore, the element enhancement tensor can distinguish the spatial representation of road boundaries, obstacle distribution, lane passable areas, and dynamic traffic participant states in the same current driving environment.

[0038] Preferably, in this application, the temporal coding structure is used to combine the feature enhancement tensor with the time stamp field, so that local feature information reflects the dynamic changes in the current driving environment. The temporal coding structure first establishes a temporal cache tensor according to the time stamp field. The temporal direction of the temporal cache tensor corresponds to the adjacent federated processing cycle, and the spatial direction of the temporal cache tensor follows the vehicle's longitudinal and lateral sampling positions from the feature enhancement tensor. The channel direction of the temporal cache tensor corresponds to road condition perception and prediction elements such as road boundaries, obstacle distribution, lane passability areas, and dynamic traffic participant states. The temporal coding structure reads the vehicle's local motion information and performs motion compensation resampling on the feature enhancement tensor of the previous federated processing cycle based on the vehicle's local motion information to form a compensated historical feature tensor. The compensated historical feature tensor corresponds to the feature enhancement tensor of the current federated processing cycle under the same vehicle coordinate expression, so that the dynamic traffic participant states and static road structures have comparable spatial positions in the temporal direction. The temporal coding structure then performs temporal convolution weighted summation on the compensated historical feature tensor and the feature enhancement tensor of the current federated processing cycle to form a temporally correlated tensor. This temporally correlated tensor then enters the gating update process. The gating update process generates temporal gating weights based on the difference between the feature enhancement tensor of the current federated processing cycle and the compensated historical feature tensor, and uses these weights to adjust the fusion ratio between the historical state tensor and the feature enhancement tensor of the current federated processing cycle, forming a temporally encoded tensor. This temporally encoded tensor serves as the input to the local feature output interface.

[0039] Preferably, the local feature output interface in this application generates local feature information through feature classification mapping, spatial location mapping, and feature processing mapping. Feature classification mapping reads the channel responses corresponding to road boundaries, obstacle distribution, lane passable areas, and dynamic traffic participant states from the time-series coding tensor, and performs weighted accumulation and normalized probability mapping on the channel responses to form a road condition feature category description; spatial location mapping reads the spatial responses corresponding to the vehicle's longitudinal sampling position and vehicle's lateral sampling position from the time-series coding tensor, and performs weighted accumulation and coordinate offset mapping on the spatial responses to form a road condition feature spatial description; feature processing mapping combines the road condition feature category description, road condition feature spatial description, time stamp field, and current driving environment association marker into local feature information. Each element in the local feature information comes from the weighted mapping result of the temporal coding tensor. The local feature information does not carry convolution kernel weights, cross-sensor channel fusion weights, road condition element template vectors, temporal coding weights, channel mean parameters, channel scale parameters, and bias parameters. Therefore, the local feature information can be used for subsequent differential perturbation sequence injection and normalized output packet encapsulation, while maintaining the cross-vendor hidden state of the internal parameters of the vehicle model inference engine.

[0040] Preferably, the vehicle-mounted model inference engine in this application can be configured with parameters using driving environment samples before deployment. The driving environment samples include historical standardized vehicle-mounted sensor frames and corresponding road condition element annotation information. The road condition element annotation information includes road boundary annotations, obstacle distribution annotations, lane passable area annotations, and dynamic traffic participant status annotations. The vehicle-mounted model inference engine performs tensor mapping, spatial neighborhood convolution, multi-scale convolution, cross-sensor channel fusion, road condition element attention mapping, temporal encoding, and local feature output on the historical standardized vehicle-mounted sensor frames to form local feature information during the training phase. During the training phase, the local feature information and the road condition element annotation information are used to calculate the loss. The loss function can be composed of a weighted average of road condition element category loss, spatial location bias loss, temporal consistency loss, and cross-sensor consistency loss. The road condition element category loss is calculated based on the cross-entropy between the category probability distribution in the local feature information during training and the category labels in the road condition element annotation information. The spatial location bias loss is calculated based on the smoothed absolute deviation between the spatial location description in the local feature information during training and the spatial location labels in the road condition element annotation information. The temporal consistency loss is calculated based on the difference between the changes in the state of the same dynamic traffic participant under adjacent time stamp fields and the changes after compensation from the vehicle's local motion information. The cross-sensor consistency loss is calculated based on the differences in the fusion response of the camera image channel, radar reflection intensity channel, and point spatial height channel at the same spatial location around the vehicle. These loss functions are used to adjust the convolution kernel weights, cross-sensor channel fusion weights, road condition element template vectors, temporal encoding weights, channel mean parameters, channel scale parameters, and bias parameters. The adjusted internal parameters are stored in the local storage area of ​​the onboard model inference engine and remain hidden when participating in federated inference at cross-vendor federated collaborative nodes.

[0041] Preferably, the vehicle-independent computing sandbox environment in this application imposes operational boundary restrictions on the aforementioned computational flow graph. After standardized vehicle sensor frames enter the vehicle-independent computing sandbox environment, they sequentially pass through the sensor frame input interface, tensor mapping structure, spatial neighborhood convolutional layer, multi-scale convolutional layer, cross-sensor channel fusion structure, road condition element attention mapping structure, temporal coding structure, and local feature output interface; the local feature output interface only outputs local feature information to the external vehicle communication port. The vehicle-independent computing sandbox environment retains the convolution kernel weights, cross-sensor channel fusion weights, road condition element template vectors, temporal coding weights, channel mean parameters, channel scale parameters, and bias parameters in the internal parameter access area, and blocks the data transmission path from the internal parameter access area to the external vehicle communication port. Thus, each cross-vendor federated collaborative node can utilize its locally deployed vehicle model inference engine to perform feature extraction on the standardized vehicle sensor frames, generating local feature information representing road condition perception prediction elements, while ensuring that the local feature information does not carry the internal parameters of the vehicle model inference engine.

[0042] In one possible scenario, the differential perturbation sequence is a numerical perturbation sequence that corresponds one-to-one with the feature dimensions of the local feature information. The length of the differential perturbation sequence is consistent with the number of feature values ​​in the local feature information. Each perturbation value in the differential perturbation sequence is used to superimpose on the corresponding feature value in the local feature information. Cross-vendor federated collaborative nodes read the federated privacy strength level from the federated inference request and select the perturbation distribution type, perturbation amplitude range, and perturbation sampling method based on the federated privacy strength level. A higher federated privacy strength level results in a larger perturbation amplitude range, while a lower level results in a smaller range. The cross-vendor federated collaborative nodes generate the differential perturbation sequence according to the perturbation distribution type, perturbation amplitude range, and perturbation sampling method, and then superimpose the differential perturbation sequence dimension-by-dimensional onto the local feature information to form privacy-preserving data. The privacy-preserving data retains the statistical representation used for federated fusion in the local feature information while reducing the direct exposure of the local feature information to the original road condition features. Cross-vendor federated collaboration nodes perform digest calculations on privacy-preserving data. The resulting privacy-preserving data digest is used as the signature input, and then a vehicle-mounted anti-tampering verification mechanism is used to digitally sign the privacy-preserving data digest. The digital signature and the privacy-preserving data are jointly encapsulated into a standardized output package. The standardized output package may include a cross-vendor federated collaboration node identifier, a current driving environment association identifier, the privacy-preserving data, the digital signature, and proof of execution legitimacy. The cross-vendor federated collaboration nodes upload the standardized output package to the vehicle-mounted initiating node so that the vehicle-mounted initiating node can verify the source and integrity of the standardized output package before subsequent fusion.

[0043] In one possible scenario, after receiving standardized output packets uploaded by various cross-vendor federated collaboration nodes, the vehicle-mounted initiating node first reads the cross-vendor federated collaboration node markers and the current driving environment association markers from the standardized output packets. Based on the current driving environment association markers, standardized output packets belonging to the same current driving environment are grouped into the same federated processing cycle. The vehicle-mounted initiating node verifies the digital signature and execution legitimacy proof in the standardized output packets. Standardized output packets that pass verification enter the encrypted dimension federated fusion processing. Encrypted dimension federated fusion refers to the vehicle-mounted initiating node performing aggregation calculations on the encrypted feature tensors in the standardized output packets without restoring the original local feature information of each cross-vendor federated collaboration node. The encrypted feature tensor is a tensor representation formed by encrypting and encapsulating privacy-preserving data, and each tensor element of the encrypted feature tensor corresponds to a perturbed feature value in the privacy-preserving data. The vehicle-mounted initiating node performs anti-poisoning robust aggregation on the encrypted feature tensor, removing anomalous feature tensors that deviate from the distribution center interval. Then, the vehicle-mounted homomorphic processing unit performs encrypted weighting, encrypted summation, and encrypted normalization on the remaining encrypted feature tensors to generate a global cognitive result containing multi-view road condition information. The global cognitive result can include a comprehensive representation of road boundaries, obstacle distribution, lane passability areas, and dynamic traffic participant states in the current driving environment. The vehicle-mounted initiating node provides the global cognitive result to the vehicle's local driving environment cognitive process.

[0044] In one possible scenario, the initiating node of the local vendor domain broadcasts a federated inference request within the vehicular ad hoc network, receives processing identifiers from vehicular response nodes of heterogeneous vendor domains, and determines cross-vendor federated collaboration nodes from the vehicular response nodes based on the processing identifiers. This includes: the initiating node broadcasting the federated inference request to the vehicular ad hoc network, the federated inference request carrying a minimum participating node number threshold and a federated privacy strength level; the vehicular response nodes feeding back the processing identifiers based on the federated inference request; the initiating node receiving the processing identifiers, reading historical federated execution credentials and vehicle network reputation scores, and using the historical federated execution credentials and vehicle network reputation scores as the screening criteria for determining the cross-vendor federated collaboration nodes; and, under the condition that the vehicle network reputation score reaches a preset score threshold and the historical federated execution credentials have no abnormal records within a preset time span, the initiating node marking the corresponding vehicular response node as a cross-vendor federated collaboration node, and verifying the total number of determined cross-vendor federated collaboration nodes using the minimum participating node number threshold.

[0045] In one possible scenario, when determining cross-vendor federated collaboration nodes, a minimum threshold for the number of participating nodes is used to determine whether the number of cross-vendor federated collaboration nodes participating in this federated inference meets the input quantity requirements for federated fusion. This minimum threshold can be configured based on the complexity of the current driving environment, the vehicle density in the vehicular ad hoc network, and the local computing resources of the vehicular initiating node (e.g., configured to a minimum of 3 cross-vendor federated collaboration nodes). The federated privacy strength level is used to constrain the perturbation strength when each cross-vendor federated collaboration node generates a differential perturbation sequence. The federated privacy strength level can be configured using a graded labeling method (e.g., configured as a low, medium, or high level). Historical federated execution credentials are the execution records of vehicular response nodes in previous federated processing cycles. These credentials may include standardized output packet submission records, digital signature verification records, timeout return records, and abnormal return records. The vehicle network reputation score is the communication trustworthiness value of the vehicular response node in the vehicular ad hoc network. This score can be calculated jointly from the standardized output packet pass rate, communication stability, and abnormal records in the historical federated execution credentials. After reading the processing identifier, the vehicle-mounted initiating node first locates the vehicle-mounted response node using the processing identifier, and then reads the historical federated execution credentials and vehicle network reputation score corresponding to that vehicle-mounted response node. The vehicle-mounted initiating node compares the vehicle network reputation score with a preset scoring threshold and compares the number of abnormal records of historical federated execution credentials within a preset time span with the anomaly judgment criteria. When the vehicle network reputation score reaches the preset scoring threshold and there are no abnormal records of historical federated execution credentials within the preset time span, the vehicle-mounted initiating node marks the corresponding vehicle-mounted response node as a cross-vendor federated collaboration node. After completing the filtering of all processing identifiers, the vehicle-mounted initiating node counts the total number of cross-vendor federated collaboration nodes and performs a quantity verification by comparing the total number of cross-vendor federated collaboration nodes with the minimum number of participating nodes threshold. After the quantity verification passes, the node proceeds to the standardized vehicle-mounted sensor frame distribution stage.

[0046] In one possible scenario, the vehicle-initiating node encapsulates driving environment data into standardized vehicle sensor frames and distributes these standardized vehicle sensor frames to each of the cross-vendor federated collaborative nodes via an encrypted vehicle communication link. This includes: the vehicle-initiating node acquiring driving environment data collected by local physical sensors and extracting underlying device-specific structural features belonging to a specific vendor's hardware from the driving environment data; the vehicle-initiating node using a cross-device universal visual space protocol to align the dimensions of these underlying device-specific structural features, generating standardized vehicle sensor frames that eliminate hardware form factor differences; and the vehicle-initiating node performing intermediate representation compatibility verification between the standardized vehicle sensor frames and the determined cross-vendor federated collaborative nodes. Upon successful verification, the standardized vehicle sensor frames are sent to each of the cross-vendor federated collaborative nodes via the encrypted vehicle communication link.

[0047] In one possible scenario, when generating standardized vehicle-mounted sensor frames, the driving environment data collected by local physical sensors is first written to the raw sensor data buffer. The vehicle-mounted initiating node reads the driving environment data from the raw sensor data buffer and parses the underlying device-specific structural features according to the sensor type. For image-based driving environment data, the underlying device-specific structural features may include the original image width, original image height, pixel channel order, distortion parameters, and exposure time. For radar-based driving environment data, the underlying device-specific structural features may include distance resolution, angular resolution, velocity resolution, and sensor coordinate origin. For point-like spatial driving environment data, the underlying device-specific structural features may include the point coordinate axis direction, point intensity value range, and point cloud sampling density. The vehicle-mounted initiating node inputs the underlying device-specific structural features into a cross-device universal visual spatial protocol. The cross-device universal visual spatial protocol first transforms the spatial coordinates from different sources into vehicle coordinate expressions based on the vehicle's driving direction, lateral direction, and vertical direction, and then maps images or point-like spatial data of different sizes to a sensor payload field of a uniform size. Size alignment can be achieved through resampling, edge patching, cropping, or scaling. The aligned sensor payload field and spatial coordinate field together constitute the main content of the standardized automotive sensor frame. The initiating node then sends the frame header field of the standardized automotive sensor frame to the cross-vendor federated collaboration node for intermediate representation compatibility verification. The cross-vendor federated collaboration node determines whether its locally deployed automotive model inference engine can read the standardized automotive sensor frame based on the input dimension, channel arrangement, and spatial coordinate format in the frame header field, and returns the verification result to the initiating node. After successful verification, the initiating node sends the complete standardized automotive sensor frame through an encrypted automotive communication link, ensuring that the input data received by the cross-vendor federated collaboration node matches the input interface of the locally deployed automotive model inference engine.

[0048] In one possible scenario, each of the cross-vendor federated collaborative nodes utilizes its locally deployed in-vehicle model inference engine to perform feature extraction on the standardized in-vehicle sensor frames, generating local feature information. This includes: each of the cross-vendor federated collaborative nodes activating its locally deployed in-vehicle model inference engine and inputting the standardized in-vehicle sensor frames into a computational flow graph containing a tensor mapping structure within the in-vehicle model inference engine; each of the cross-vendor federated collaborative nodes performing feature extraction on the standardized in-vehicle sensor frames based on the computational flow graph in an in-vehicle independent computing sandbox environment, generating local feature information representing road condition perception and prediction elements; during the feature extraction process, each of the cross-vendor federated collaborative nodes blocks the transmission path of the network topology and internal parameters of the in-vehicle model inference engine to the external vehicle communication port, so that the local feature information does not carry the internal parameters of the in-vehicle model inference engine, maintaining the cross-vendor hidden state of the internal parameters.

[0049] In one possible scenario, during feature extraction, the cross-vendor federated collaborative node first performs integrity verification on the received standardized vehicle sensor frames. After successful integrity verification, the cross-vendor federated collaborative node activates the locally deployed vehicle model inference engine. The computational flow graph in the vehicle model inference engine consists of multiple tensor processing nodes connected in execution order. The tensor mapping structure in the computational flow graph is used to convert the standardized vehicle sensor frames into intermediate tensors suitable for local computation. The feature encoding structure in the computational flow graph is used to extract road condition perception prediction elements from the intermediate tensors. The output organization structure in the computational flow graph is used to organize the road condition perception prediction elements into local feature information. The rows of the intermediate tensors can correspond to the spatial location indices in the standardized vehicle sensor frames, and the columns of the intermediate tensors can correspond to the channel indices in the sensor payload field. The tensor elements at the intersection of rows and columns represent the sensor values ​​at the corresponding spatial location and channel. The vehicle-independent computation sandbox environment allocates an independent memory access area and a controlled output interface for the vehicle model inference engine. After the standardized vehicle sensor frames enter the vehicle-independent computation sandbox environment through the input interface, they can only complete tensor mapping, feature encoding, and output organization along the computational flow graph. Cross-vendor federated collaboration nodes extract local feature information at the output structure and intercept access requests for network topology files, weight parameter files, and bias parameter files to external vehicle communication ports within the vehicle-independent computing sandbox environment. This ensures that the local feature information only contains numerical representations and dimensional descriptions of road condition perception and prediction elements, excluding the internal parameters of the vehicle model inference engine. Through this processing method, different vendor domains can share local feature information to participate in federated fusion without exposing the internal parameters of their locally deployed vehicle model inference engines.

[0050] In one possible scenario, each of the cross-vendor federated collaborative nodes injects a differential perturbation sequence into the local feature information to obtain privacy-preserving data. The privacy-preserving data is then encapsulated into a standardized output package and uploaded to the vehicle-mounted initiating node. This process includes: each of the cross-vendor federated collaborative nodes extracting a federated privacy strength level from the federated inference request as a basis for determining the perturbation strength of the differential perturbation sequence; each of the cross-vendor federated collaborative nodes determining a perturbation sequence generation rule for obfuscating the true distribution based on the federated privacy strength level, and generating a corresponding differential perturbation sequence; each of the cross-vendor federated collaborative nodes superimposing the differential perturbation sequence onto the local feature information to obtain privacy-preserving data that masks the original road condition features; and each of the cross-vendor federated collaborative nodes digitally signing the privacy-preserving data using an vehicle-mounted anti-tamper verification mechanism, enabling the privacy-preserving data to carry proof of execution legitimacy, generating a standardized output package, and uploading it to the vehicle-mounted initiating node.

[0051] In one possible scenario, when generating privacy-preserving data, cross-vendor federated collaborative nodes first read the feature dimension descriptions from the local feature information and determine the sequence length of the differential perturbation sequence based on these descriptions. The nodes then extract the federated privacy strength level from the federated inference request and convert it into perturbation sequence generation rules. These rules can include perturbation value distribution, upper and lower limits of perturbation amplitude, and a perturbation sampling seed update method. The perturbation value distribution determines the distribution pattern of perturbation values ​​in the differential perturbation sequence, the upper and lower limits of perturbation amplitude determine the range of perturbation values, and the perturbation sampling seed update method ensures that the differential perturbation sequences generated in adjacent federated processing cycles are different. The cross-vendor federated collaborative nodes generate the differential perturbation sequence according to the perturbation sequence generation rules and then superimpose the differential perturbation sequence onto the local feature information dimension by dimension, resulting in privacy-preserving data. Each perturbated feature value in the privacy-preserving data still corresponds to a road condition perception prediction element dimension in the local feature information. Therefore, the privacy-preserving data can continue to participate in the federated fusion of the encrypted dimension. Simultaneously, the perturbated feature values ​​in the privacy-preserving data are perturbed relative to the original values ​​in the local feature information, thereby reducing the direct exposure of the original road condition features. Cross-vendor federated collaborative nodes perform irreversible digest computation on the privacy-preserving data and use an onboard anti-tampering verification mechanism to digitally sign the privacy-preserving data digest obtained from the irreversible digest computation. The digital signature, privacy-preserving data, and execution legitimacy proof together form a standardized output package. After the standardized output package is uploaded to the onboard initiating node, the onboard initiating node can determine whether the standardized output package was generated by the corresponding cross-vendor federated collaborative node based on the digital signature and execution legitimacy proof.

[0052] In one possible scenario, the vehicle-mounted initiating node performs encrypted-dimensional federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment. This includes: the vehicle-mounted initiating node extracting encrypted feature tensors from each of the standardized output packets; the vehicle-mounted initiating node performing anti-poisoning robust aggregation on each of the encrypted feature tensors, removing abnormal feature tensors that deviate from the distribution center interval, and retaining the encrypted feature tensors that were not removed; the vehicle-mounted initiating node performing encrypted-dimensional federated fusion on the retained encrypted feature tensors through an onboard homomorphic processing unit to generate a global cognitive result containing multi-view road condition information; the vehicle-mounted initiating node comparing the road condition perception accuracy of the global cognitive result with the accuracy of single-vehicle local inference to obtain an accuracy improvement value, and saving the global cognitive result when the accuracy improvement value reaches the accuracy improvement requirement.

[0053] In one possible scenario, during encrypted-dimensional federated fusion, the vehicle-mounted initiating node reads privacy-preserving data from each standardized output packet and converts the privacy-preserving data into encrypted feature tensors according to a unified tensor format. The dimensions of the encrypted feature tensors correspond to the feature dimensions of local feature information, and the tensor elements of the encrypted feature tensors are the encrypted, perturbed feature values. The vehicle-mounted initiating node first performs anti-poisoning robust aggregation on each encrypted feature tensor. In this application, anti-poisoning robust aggregation first measures the degree of distribution deviation between each encrypted feature tensor in the encrypted computation space, and then removes abnormal feature tensors that deviate from the distribution center interval from the federated fusion input. The vehicle-mounted homomorphic processing unit performs addition, weighting, and normalization operations on the retained encrypted feature tensors in the encrypted computation space. The encrypted fusion tensor obtained by the vehicle-mounted homomorphic processing unit is converted into a global cognitive result. The global perception result simultaneously includes local observations of the same standardized vehicle sensor frame from multiple cross-vendor federated collaborative nodes. Therefore, the multi-view road condition information in the global perception result originates from the locally deployed vehicle model inference engines of different vehicle nodes. The vehicle initiating node inputs the global perception result into the local road condition perception evaluation process to obtain the road condition perception accuracy of the global perception result; the vehicle initiating node also inputs the single-vehicle local inference result into the same local road condition perception evaluation process to obtain the single-vehicle local inference accuracy; the vehicle initiating node compares the road condition perception accuracy of the global perception result with the single-vehicle local inference accuracy to obtain the accuracy improvement value, and when the accuracy improvement value reaches the accuracy improvement requirement, the global perception result is saved to the local vehicle storage area for subsequent use in the current driving environment.

[0054] In one possible scenario, the step of performing anti-poisoning robust aggregation on each of the encrypted feature tensors, removing abnormal feature tensors that deviate from the distribution center interval, and retaining the encrypted feature tensors that were not removed, includes: the vehicle-mounted initiating node calculating the vector distance metric between each of the encrypted feature tensors in the encrypted computation space carrying each of the encrypted feature tensors, as the distance judgment basis for identifying the abnormal feature tensors; the vehicle-mounted initiating node numerically sorting each of the encrypted feature tensors based on each of the vector distance metric, obtaining a encrypted feature tensor sorting sequence; the vehicle-mounted initiating node locating the encrypted feature tensors at the sorting truncation ratio positions at both ends of the encrypted feature tensor sorting sequence, determining them as the range of feature tensors to be discarded that deviate from the distribution center interval; the vehicle-mounted initiating node determining the encrypted feature tensors at the sorting truncation ratio positions as abnormal feature tensors that may have been maliciously tampered with and performing a data discarding operation, retaining the encrypted feature tensors that were not discarded and passing them to the subsequent encrypted dimension federated fusion stage.

[0055] In one possible scenario, the computation process of robust aggregation against poisoning is as follows: The vehicle-mounted initiating node places each encrypted feature tensor within the same federated processing cycle into an encrypted computation space. This space is configured as a secure computing environment capable of performing difference calculations, accumulation calculations, and comparison calculations on the elements of the encrypted tensors. For any two encrypted feature tensors, the vehicle-mounted initiating node performs encrypted difference calculations at the same tensor dimension position. The dimensional difference results obtained from the encrypted difference calculations are accumulated to form a vector distance metric. This vector distance metric represents the degree of deviation between the two encrypted feature tensors in the representation of road condition perception prediction elements. The vehicle-mounted initiating node summarizes the vector distance metrics between each encrypted feature tensor and other encrypted feature tensors to form the distance judgment criteria for each encrypted feature tensor. Encrypted feature tensors with larger distance judgment criteria indicate a higher degree of deviation from other encrypted feature tensors. The vehicle-mounted initiating node numerically sorts each encrypted feature tensor based on distance judgment, resulting in a sorted sequence of encrypted feature tensors. The middle position of the sorted sequence corresponds to the distribution center interval, while the two ends correspond to tensor ranges deviating from the distribution center interval. The vehicle-mounted initiating node locates the ranges of feature tensors to be discarded at both ends of the sorted sequence according to a sorting truncation ratio (e.g., 10% at each end as an example), and identifies encrypted feature tensors at these truncation ratio positions as anomalous feature tensors that may have been maliciously tampered with. The vehicle-mounted initiating node performs a data discard operation on the anomalous feature tensors and passes the remaining encrypted feature tensors to the vehicle-mounted homomorphic processing unit for subsequent encrypted dimension federated fusion, thus preventing anomalous feature tensors deviating from the distribution center interval from participating in the generation of the global cognitive result.

[0056] In one possible scenario, after generating the global cognitive result, the method further includes: the vehicle-mounted initiating node acquiring a first environmental cognitive deviation record of the standardized output packets submitted by each of the cross-vendor federated collaborative nodes in the current federated processing cycle compared to the global fusion benchmark corresponding to the global cognitive result; the vehicle-mounted initiating node appending the first environmental cognitive deviation record to the vehicle-mounted historical deviation record library of the corresponding cross-vendor federated collaborative node; the vehicle-mounted initiating node dynamically downgrading the communication priority of the cross-vendor federated collaborative nodes in the next federated processing cycle, or performing a removal operation of the cross-vendor federated collaborative nodes, based on the continuous deviation change trend presented in the vehicle-mounted historical deviation record library; and the vehicle-mounted initiating node synchronizing the communication priority change status of the cross-vendor federated collaborative nodes to the other vehicle nodes in the vehicle network through the vehicle network broadcast link, for use when determining the cross-vendor federated collaborative nodes in the next federated processing cycle.

[0057] In one possible scenario, after generating the global cognitive result, the vehicle-mounted initiating node maps the global cognitive result to a global fusion benchmark. The global fusion benchmark is a reference representation after fusion of multiple nodes within the same federated processing cycle, used to measure the deviation between the standardized output packets submitted by each cross-vendor federated collaborative node and the global cognitive result. The vehicle-mounted initiating node reads the standardized output packets submitted by each cross-vendor federated collaborative node and calculates the deviation between the encrypted feature tensor or the corresponding pre-fusion representation in the standardized output packet and the global fusion benchmark, forming a first environmental cognitive deviation record. The first environmental cognitive deviation record may include a cross-vendor federated collaborative node marker, a federated processing cycle marker, a deviation direction description, a deviation magnitude description, and a digital signature verification result. The vehicle-mounted initiating node appends the first environmental cognitive deviation record to the corresponding cross-vendor federated collaborative node's vehicle-mounted historical deviation record library. The vehicle-mounted historical deviation record library is indexed according to the cross-vendor federated collaborative node marker, enabling the vehicle-mounted initiating node to read the deviation changes of the same cross-vendor federated collaborative node within multiple federated processing cycles. Based on the continuous deviation trend presented in the vehicle's historical deviation record database, the vehicle-initiating node lowers the communication priority of cross-vendor federated collaboration nodes that continuously deviate from the distribution center range in the next federated processing cycle, or removes cross-vendor federated collaboration nodes from the candidate node range. The communication priority change status includes the cross-vendor federated collaboration node marking, the downgraded communication priority, and the removal status. The vehicle-initiating node synchronizes the communication priority change status to other vehicle nodes in the vehicle network through the vehicle network broadcast link, so that other vehicle nodes can read the communication priority change status and complete the node selection when determining cross-vendor federated collaboration nodes in the next federated processing cycle.

[0058] In one possible scenario, after generating the global cognitive result, the method further includes: the vehicular initiating node allocating basic computing resource scheduling credentials to the corresponding cross-vendor federated collaborative node based on the vehicular network connection maintenance duration of the cross-vendor federated collaborative node; the vehicular initiating node extracting a data deviation metric value of the standardized output packet uploaded by the cross-vendor federated collaborative node compared to the distributed collaborative average level, wherein the distributed collaborative average level is determined by the standardized output packets uploaded by each cross-vendor federated collaborative node in the current federated processing cycle, and using the data deviation metric value as the basis for issuing additional computing resource scheduling credentials; the vehicular initiating node issuing the additional computing resource scheduling credentials to the corresponding cross-vendor federated collaborative node when the data deviation metric value falls within the target deviation range and the vehicular model accuracy level identifier of the cross-vendor federated collaborative node meets the target accuracy level condition; and the cross-vendor federated collaborative node exchanging the obtained basic computing resource scheduling credentials and the additional computing resource scheduling credentials for corresponding distributed data processing bandwidth in the vehicular ad hoc network.

[0059] In one possible scenario, after generating the global cognitive result, the vehicular initiating node reads the vehicular network connection maintenance duration of each cross-vendor federated collaborative node during the current federated processing cycle. The vehicular network connection maintenance duration can be calculated from the time point when the cross-vendor federated collaborative node completes its federated inference request response and the time point when it submits the standardized output packet. The vehicular network connection maintenance duration reflects the continuous ability of the cross-vendor federated collaborative node to maintain communication and complete data processing within the current federated processing cycle. Based on the vehicular network connection maintenance duration, the vehicular initiating node allocates basic computing resource scheduling credentials to the corresponding cross-vendor federated collaborative node. These credentials are used to request distributed data processing bandwidth in the vehicular ad hoc network and can record the cross-vendor federated collaborative node flag, the federated processing cycle flag, and the basic bandwidth scheduling quota. The vehicular initiating node further reads the standardized output packets uploaded by each cross-vendor federated collaborative node during the current federated processing cycle and forms a distributed collaborative average level for each standardized output packet. The distributed collaborative average level represents the average characteristic expression formed by multiple standardized output packets in the same current driving environment. The initiating node calculates the deviation between the standardized output packets uploaded by each cross-vendor federated collaboration node and the average level of distributed collaboration, obtaining a data deviation metric. This metric is used to determine whether the corresponding standardized output packets are both fusion-compatible and do not excessively deviate from the average level of distributed collaboration. If the data deviation metric falls within the target deviation range and the vehicle model accuracy level identifier of the cross-vendor federated collaboration node meets the target accuracy level condition, the initiating node issues an additional computing resource scheduling certificate to the corresponding cross-vendor federated collaboration node. The vehicle model accuracy level identifier can be formed by the offline evaluation results of the vehicle model inference engine, version verification results, and historical federated execution certificates. Upon receiving the basic computing resource scheduling certificate and the additional computing resource scheduling certificate, the cross-vendor federated collaboration node submits both certificates to the vehicle ad hoc network and exchanges the corresponding distributed data processing bandwidth based on the scheduling quota recorded in the certificate, ensuring that the transmission of standardized vehicle sensor frames and the upload of standardized output packets in subsequent federated processing cycles have the corresponding bandwidth scheduling basis.

[0060] In one possible scenario, such as Figure 2As shown, a cross-vendor in-vehicle model federated inference device is provided, comprising: a cooperative node determination module, used to enable in-vehicle initiating nodes in the same vendor domain to broadcast federated inference requests within an in-vehicle ad hoc network, receive processing identifiers fed back by in-vehicle response nodes in heterogeneous vendor domains, and determine cross-vendor federated cooperative nodes from the in-vehicle response nodes based on the processing identifiers; a sensor frame distribution module, used to enable the initiating nodes to encapsulate driving environment data into standardized in-vehicle sensor frames, and distribute the standardized in-vehicle sensor frames to each of the cross-vendor federated cooperative nodes through an encrypted in-vehicle communication link; and a local feature generation module, used to enable each of the cross-vendor federated cooperative nodes to generate local features. The vendor-fed collaborative nodes utilize their locally deployed in-vehicle model inference engines to extract features from the standardized in-vehicle sensor frames, generating local feature information. The internal parameters of the in-vehicle model inference engine are hidden from each vendor's domain. The output packet generation module enables each of the cross-vendor federated collaborative nodes to inject differential perturbation sequences into the local feature information to obtain privacy-preserving data. The privacy-preserving data is then encapsulated into a standardized output packet and uploaded to the in-vehicle initiating node. The federated fusion module enables the in-vehicle initiating node to perform encrypted-dimensional federated fusion on each of the standardized output packets, generating a global cognitive result for the current driving environment.

[0061] In one possible scenario, the specific function of the collaborative node determination module is not simply a summary of the method steps, but rather enables the vehicle-mounted initiating node in the local vendor domain to complete the construction of the federated inference request, broadcast of the federated inference request, reception of the processing identifier, resolution of the processing identifier, and screening of cross-vendor federated collaborative nodes. The collaborative node determination module enables the vehicle-mounted initiating node to broadcast the federated inference request to the vehicle ad hoc network. The federated inference request carries a minimum threshold for the number of participating nodes and a federated privacy strength level. The vehicle-mounted responding node returns a processing identifier based on the federated inference request. The collaborative node determination module enables the vehicle-mounted initiating node to receive the processing identifier, read historical federated execution credentials and vehicle network reputation scores, and use these historical federated execution credentials and vehicle network reputation scores as the screening criteria for cross-vendor federated collaborative nodes. Furthermore, the collaborative node determination module enables the vehicle-mounted initiating node to mark the corresponding vehicle-mounted responding node as a cross-vendor federated collaborative node when the vehicle network reputation score reaches a preset score threshold and there are no abnormal records in the historical federated execution credentials within a preset time span. The total number of determined cross-vendor federated collaborative nodes is then verified using the minimum threshold for the number of participating nodes. Through the functions of the above modules, the collaborative node determines the output of the cross-vendor federated collaborative node, which serves as the receiving object for the sensor frame distribution module to distribute standardized vehicle sensor frames.

[0062] In one possible scenario, the sensor frame distribution module's specific functions are to enable the vehicle-mounted initiating node to complete the following steps: acquiring and reading driving environment data, extracting underlying device-specific structural features, generating standardized vehicle-mounted sensor frames, verifying intermediate representation compatibility, and transmitting encrypted data. The sensor frame distribution module allows the vehicle-mounted initiating node to acquire driving environment data collected by local physical sensors and extract underlying device-specific structural features belonging to a specific manufacturer's hardware from the driving environment data. The module then enables the vehicle-mounted initiating node to use a cross-device universal visual space protocol to align the dimensions of these underlying device-specific structural features, generating standardized vehicle-mounted sensor frames that eliminate hardware form factor differences. Furthermore, the module enables the vehicle-mounted initiating node to perform intermediate representation compatibility verification between the standardized vehicle-mounted sensor frames and designated cross-vendor federated collaborative nodes. After successful verification, the standardized vehicle-mounted sensor frames are transmitted to each cross-vendor federated collaborative node via an encrypted vehicle communication link. The standardized vehicle-mounted sensor frames output by the sensor frame distribution module serve as input data for feature extraction by the local feature generation module.

[0063] In one possible scenario, the local feature generation module's specific function is to enable each cross-vendor federated collaborative node to locally complete the activation of the in-vehicle model inference engine, input of standardized in-vehicle sensor frames, execution of the computational flow graph, generation of local feature information, and hiding of internal parameters. The local feature generation module enables each cross-vendor federated collaborative node to activate its locally deployed in-vehicle model inference engine, inputting standardized in-vehicle sensor frames into the computational flow graph containing tensor mapping structures within the in-vehicle model inference engine. Within an in-vehicle independent computational sandbox environment, the local feature generation module enables each cross-vendor federated collaborative node to perform feature extraction on the standardized in-vehicle sensor frames based on the computational flow graph, generating local feature information representing road condition perception and prediction elements. During feature extraction, the local feature generation module enables each cross-vendor federated collaborative node to block the transmission path of the in-vehicle model inference engine's internal network topology and internal parameters to the external vehicle communication port, ensuring that the local feature information does not carry the in-vehicle model inference engine's internal parameters, maintaining the cross-vendor hiding state of the internal parameters. The local feature information output by the local feature generation module serves as input data for the output packet generation module to generate privacy-preserving data.

[0064] In one possible scenario, the output packet generation module's specific function is to enable each cross-vendor federated collaborative node to generate differential perturbation sequences based on the federated privacy strength level, and to protect local feature information through these sequences. Specifically, the module enables each node to extract the federated privacy strength level from the federated inference request, using this level as the basis for determining the perturbation strength of the differential perturbation sequences. It also enables each node to determine the perturbation sequence generation rules used to obfuscate the real distribution based on the federated privacy strength level, and to generate corresponding differential perturbation sequences. Furthermore, the module allows each node to superimpose the differential perturbation sequences onto the local feature information, obtaining privacy-protected data that masks the original road condition features. Finally, the module enables each node to digitally sign the privacy-protected data using an onboard anti-tamper verification mechanism, ensuring the data carries proof of execution legitimacy, generating a standardized output packet, and uploading it to the onboard initiating node. The standardized output packet from the module serves as input data for the federated fusion module to perform encrypted-dimensional federated fusion.

[0065] In one possible scenario, the specific function of the federated fusion module is to enable the vehicle-initiating node to perform encrypted feature tensor extraction, anti-poisoning robust aggregation, encrypted dimension federated fusion, global cognitive result generation, and road condition perception accuracy comparison on the standardized output packets. The federated fusion module enables the vehicle-initiating node to extract encrypted feature tensors from each standardized output packet; it also enables the vehicle-initiating node to perform anti-poisoning robust aggregation on each encrypted feature tensor, removing abnormal feature tensors that deviate from the distribution center interval and retaining the remaining encrypted feature tensors; the module further enables the vehicle-initiating node to perform encrypted dimension federated fusion on the retained encrypted feature tensors through the vehicle-mounted homomorphic processing unit, generating a global cognitive result containing multi-view road condition information; finally, the module also enables the vehicle-initiating node to numerically compare the road condition perception accuracy of the global cognitive result with the accuracy of single-vehicle local inference, obtaining an accuracy improvement value, and saving the global cognitive result when the accuracy improvement value meets the accuracy improvement requirements. When performing robust aggregation to prevent poisoning, the federated fusion module enables the vehicle-initiating node to calculate the vector distance metric between each encrypted feature tensor in the encrypted computation space carrying each encrypted feature tensor, as the distance judgment basis for identifying abnormal feature tensors. The federated fusion module enables the vehicle-initiating node to numerically sort each encrypted feature tensor based on each vector distance metric, resulting in a sorted sequence of encrypted feature tensors. The federated fusion module enables the vehicle-initiating node to locate the encrypted feature tensors at the sorting truncation ratio positions at both ends of the sorted sequence, identifying them as feature tensors to be discarded that deviate from the distribution center interval. The federated fusion module enables the vehicle-initiating node to determine the encrypted feature tensors at the sorting truncation ratio positions as abnormal feature tensors that may have been maliciously tampered with and to perform data discarding operations, while retaining the encrypted feature tensors that are not discarded and passing them to the subsequent encrypted dimension federated fusion stage.

[0066] In one possible scenario, the cross-vendor in-vehicle model federated inference device may further include a deviation record management module. This module enables the initiating node to obtain a first environmental cognition deviation record of the standardized output packets submitted by each cross-vendor federated collaborative node within the current federated processing cycle, compared to the global fusion benchmark corresponding to the global cognition result, after generating the global cognition result. The deviation record management module also enables the initiating node to append the first environmental cognition deviation record to the corresponding cross-vendor federated collaborative node's in-vehicle historical deviation record library. Furthermore, the module enables the initiating node to dynamically downgrade the communication priority of cross-vendor federated collaborative nodes in the next federated processing cycle, or to perform a removal operation, based on the continuous deviation change trend presented in the in-vehicle historical deviation record library. Finally, the module enables the initiating node to synchronize the communication priority change status of cross-vendor federated collaborative nodes to other vehicle nodes in the vehicle network via the vehicle network broadcast link, for use when determining cross-vendor federated collaborative nodes in the next federated processing cycle. The communication priority change status output by the deviation record management module serves as the input basis for the collaborative node determination module to screen cross-vendor collaborative nodes in the next round of federated processing.

[0067] In one possible scenario, the cross-vendor in-vehicle model federated inference device may further include a resource scheduling credential allocation module. This module enables the initiating node to allocate basic computing resource scheduling credentials to the corresponding cross-vendor federated collaborative nodes based on the duration of the cross-vendor federated collaborative node's in-vehicle network connection after generating the global cognitive result. The module also enables the initiating node to extract a data deviation metric from the standardized output packets uploaded by the cross-vendor federated collaborative nodes compared to the average distributed collaborative level. The average distributed collaborative level is determined by the standardized output packets uploaded by each cross-vendor federated collaborative node within the current federated processing cycle, and this data deviation metric serves as the basis for issuing additional computing resource scheduling credentials. Furthermore, the module enables the initiating node to issue additional computing resource scheduling credentials to the corresponding cross-vendor federated collaborative nodes when the data deviation metric falls within the target deviation range and the cross-vendor federated collaborative node's in-vehicle model accuracy level meets the target accuracy level conditions. Finally, the module enables the cross-vendor federated collaborative nodes to exchange the acquired basic and additional computing resource scheduling credentials for corresponding distributed data processing bandwidth within the in-vehicle ad hoc network. The basic computing resource scheduling credentials and additional computing resource scheduling credentials obtained by the resource scheduling credential allocation module are used for bandwidth scheduling of standardized vehicle sensor frame distribution and standardized output packet upload in the next round of federated processing cycle.

[0068] In one possible scenario, such as Figure 3As shown, an electronic device is provided, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the cross-vendor vehicle model federated inference method as described in any one of the claims.

[0069] In one possible scenario, the electronic device can be deployed on an in-vehicle computing platform within a vehicle, or on a test computing platform used to validate cross-vendor in-vehicle model federated inference methods. The computer program stored in the electronic device's memory includes a federated inference request generation segment, a processing identifier filtering segment, a standardized in-vehicle sensor frame generation segment, an encrypted in-vehicle communication link transmission segment, a local feature information reception segment, a differential perturbation sequence processing segment, a standardized output packet verification segment, a encrypted feature tensor fusion segment, a first environmental cognitive bias record management segment, and a computing resource scheduling credential allocation segment. The federated inference request generation program segment and the processing identifier filtering program segment are used to determine cross-vendor federated collaborative nodes; the standardized vehicle sensor frame generation program segment and the encrypted vehicle communication link transmission program segment are used to distribute standardized vehicle sensor frames; the local feature information receiving program segment, the differential perturbation sequence processing program segment, and the standardized output packet verification program segment are used to receive and verify the data returned by each cross-vendor federated collaborative node; the encrypted feature tensor fusion program segment is used to generate global cognitive results; the first environmental cognitive deviation record management program segment and the computing resource scheduling credential allocation program segment are used to maintain the data for subsequent federated processing cycles of cross-vendor federated collaborative nodes after the global cognitive results are generated.

[0070] In one possible scenario, such as Figure 4 The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the cross-vendor vehicle model federated inference method described in any one of the claims.

[0071] In one possible scenario, the computer-readable storage medium can be a non-volatile storage medium that can be read by an onboard computing unit. The computer program stored in the computer-readable storage medium includes executable instructions for executing a cross-vendor onboard model federated inference method. When the executable instructions are read and run, the onboard initiating node in the local vendor domain first broadcasts a federated inference request within the onboard ad hoc network, receives processing identifiers, and determines the processing of cross-vendor federated collaborative nodes. Then, it completes the generation and distribution of standardized onboard sensor frames, the generation of local feature information, the generation of privacy-preserving data, the uploading of standardized output packets, the federated fusion of encrypted dimensions, and the generation of global cognitive results. The computer-readable storage medium may also store cross-device general visual spatial protocol configuration data, preset scoring thresholds, preset time spans, sorting truncation ratios, accuracy improvement requirements, target deviation intervals, and target accuracy level conditions. This configuration data is read during the execution of the computer program and participates in the selection of cross-vendor federated collaborative nodes, the generation of standardized onboard sensor frames, the anti-poisoning robust aggregation, the saving of global cognitive results, and the issuance and judgment of additional computing resource scheduling credentials.

[0072] In one possible scenario, such as Figure 5 The present invention provides a vehicle including an in-vehicle communication unit, an in-vehicle computing unit, and an in-vehicle storage unit. The in-vehicle storage unit stores a computer program. When the computer program is executed by the in-vehicle computing unit, the vehicle participates in an in-vehicle ad hoc network through the in-vehicle communication unit and executes the cross-vendor in-vehicle model federated inference method described in any one of the claims of this application.

[0073] In one possible scenario, the vehicle's onboard communication unit (VPC) is used to access the onboard ad hoc network. When the vehicle acts as an onboard initiating node, it broadcasts federated inference requests, receives processing identifiers, sends standardized onboard sensor frames, and receives standardized output packets. When the vehicle acts as an onboard responding node, the VPC receives federated inference requests, feedback processing identifiers, receives standardized onboard sensor frames, and uploads standardized output packets. The onboard computing unit performs computational processing in the cross-vendor onboard model federated inference method. When the vehicle acts as an onboard initiating node, the VPC determines cross-vendor federated collaboration nodes, generates standardized onboard sensor frames, performs encrypted dimension federated fusion, generates global cognitive results, generates first environmental cognitive bias records, and allocates basic and additional computing resource scheduling credentials. When the vehicle acts as a cross-vendor federated collaboration node, the VPC runs a locally deployed onboard model inference engine, performs feature extraction on standardized onboard sensor frames, generates local feature information, and injects differential perturbation sequences into the local feature information to generate privacy-preserving data. The onboard storage unit stores computer programs, cross-device universal visual spatial protocol configuration data, historical federated execution credentials, vehicle network reputation scores, onboard historical deviation records, basic computing resource scheduling credentials, and additional computing resource scheduling credentials. Through data transfer between the onboard communication unit, onboard computing unit, and onboard storage unit, the vehicle can participate in federated inference in the onboard ad hoc network as an onboard initiating node or a cross-vendor federated collaborative node in different federated processing cycles.

[0074] In one possible scenario, such as Figure 6 The system provides a cross-vendor in-vehicle model federated inference system, including in-vehicle initiating nodes in the local vendor domain and in-vehicle responding nodes in heterogeneous vendor domains. The initiating nodes broadcast federated inference requests within the in-vehicle ad hoc network, receive processing identifiers from the responding nodes, and determine cross-vendor federated collaborative nodes based on these identifiers. The initiating nodes also encapsulate driving environment data into standardized in-vehicle sensor frames and distribute these frames to the cross-vendor federated collaborative nodes via an encrypted in-vehicle communication link. Each cross-vendor federated collaborative node uses a locally deployed in-vehicle model inference engine to perform feature extraction on the standardized in-vehicle sensor frames, generating local feature information. The internal parameters of the inference engine are hidden from each vendor domain. Each cross-vendor federated collaborative node injects differential perturbation sequences into the local feature information to obtain privacy-preserving data, encapsulates the privacy-preserving data into standardized output packets, and uploads them to the initiating nodes. The initiating nodes also perform encrypted-dimensional federated fusion on each standardized output packet to generate a global cognitive result for the current driving environment.

[0075] In one possible scenario, in a cross-vendor in-vehicle model federated inference system, initiating nodes within the same vendor's domain and responding nodes from heterogeneous vendor domains establish communication relationships through an in-vehicle ad hoc network. The initiating node first broadcasts a federated inference request, and the responding node generates a processing identifier based on the request and feeds it back to the initiating node. The initiating node then determines the cross-vendor federated collaboration node based on the processing identifier, historical federated execution credentials, and vehicle network reputation score. Once the cross-vendor federated collaboration node is determined, the initiating node encapsulates driving environment data into standardized in-vehicle sensor frames and distributes them to the cross-vendor federated collaboration node via an encrypted in-vehicle communication link. The cross-vendor federated collaboration node uses its locally deployed in-vehicle model inference engine to perform feature extraction on the standardized in-vehicle sensor frames, generating local feature information while hiding the internal parameters of the inference engine from each vendor domain. The cross-vendor federated collaboration node further injects differential perturbation sequences into the local feature information to obtain privacy-preserving data, encapsulates this privacy-preserving data into standardized output packets, and uploads them to the initiating node. The initiating node performs encrypted-dimensional federated fusion on each standardized output packet to generate a global cognitive result for the current driving environment. The data transmission order between the aforementioned nodes ensures that standardized vehicle sensor frames, local feature information, privacy-preserving data, standardized output packets, and global cognitive results are sequentially connected, preventing cross-vendor federated collaboration nodes from directly exposing the internal parameters of the locally deployed vehicle model inference engine.

[0076] In one possible scenario, in a specific application, when a vehicle from this vendor's domain approaches a heavily obstructed intersection area, the initiating node in this vendor's domain collects driving environment data through local physical sensors and discovers insufficient observation information for certain road areas from a single vehicle's perspective. The initiating node broadcasts a federated inference request within the vehicular ad hoc network, carrying a minimum threshold for the number of participating nodes and a federated privacy strength level. Upon receiving the federated inference request, the responding nodes from heterogeneous vendor domains generate a processing identifier based on the availability status of their locally deployed vehicular model inference engine, the standardized vehicular sensor frame receiving capability, and node addresses, and return the processing identifier to the initiating node. The initiating node combines the processing identifier, historical federated execution credentials, and vehicle network reputation score to determine the cross-vendor federated collaboration nodes, then encapsulates the driving environment data into standardized vehicular sensor frames and distributes them. The cross-vendor federated collaboration nodes perform feature extraction locally and generate local feature information. Each cross-vendor federated collaboration node injects a differential perturbation sequence into the local feature information to form privacy-preserving data, and then encapsulates the privacy-preserving data into a standardized output packet for uploading. The vehicle-initiating node performs anti-poisoning robust aggregation and federated fusion of encrypted dimensions on the encrypted feature tensor in the standardized output packet within the encrypted computation space, generating a global cognitive result. This global cognitive result integrates road condition perception and prediction elements from different vehicle perspectives, enabling the vehicle-initiating node to obtain multi-view road condition information without acquiring internal parameters of cross-vendor federated collaborative nodes.

[0077] In one possible scenario, the vehicle-mounted model inference engine in the above embodiments can be implemented using a convolutional structure, a temporal coding structure, or a multi-sensor fusion structure. Taking a vehicle-mounted model inference engine using a multi-sensor fusion structure as an example, the vehicle-mounted model inference engine first reads standardized vehicle-mounted sensor frames through a sensor frame input interface, and then converts the standardized vehicle-mounted sensor frames into intermediate tensors through a tensor mapping structure. After the intermediate tensors enter the spatial coding structure, the spatial coding structure performs local neighborhood mapping on the tensor elements corresponding to the road spatial locations in the intermediate tensors to obtain spatial coding results. After the spatial coding results enter the temporal coding structure, the temporal coding structure combines the time stamp field to temporally correlate the spatial coding results within adjacent federated processing cycles to obtain temporal coding results. After the temporal coding results enter the road condition element output structure, the road condition element output structure organizes the temporal coding results into local feature information. The spatial coding structure, temporal coding structure, and road condition element output structure all run in the vehicle-independent computing sandbox environment. The vehicle-independent computing sandbox environment only allows local feature information to be output from the controlled output interface. Therefore, the internal parameters of the vehicle model inference engine will not be uploaded to the vehicle initiating node as part of the standardized output package.

[0078] In one possible scenario, the cross-device universal visual spatial protocol in the above embodiments can be implemented through pre-configured field specifications. These field specifications include frame header field specifications, sensor payload field specifications, spatial coordinate field specifications, and time stamp field specifications. The frame header field specification defines the frame type, data length, and input dimension of the standardized vehicle sensor frame; the sensor payload field specification defines the storage order of driving environment data at a uniform size; the spatial coordinate field specification defines the mapping method of sensor data from the manufacturer's hardware coordinate representation to the vehicle body coordinate representation; and the time stamp field specification defines the correspondence between the standardized vehicle sensor frame and the current driving environment acquisition time. The vehicle initiating node converts the driving environment data into standardized vehicle sensor frames according to the field specifications, and the cross-manufacturer federated collaboration nodes read the standardized vehicle sensor frames according to the same field specifications, thereby reducing the impact of differences in hardware form factors from different manufacturers on the input interface of the vehicle model inference engine.

[0079] In one possible scenario, the encrypted vehicle communication link in the above embodiments can be implemented through session key negotiation, data encryption, and integrity verification. Before distributing standardized vehicle sensor frames, the vehicle initiating node and the cross-vendor federated collaboration node negotiate a session key and use the session key to encrypt the standardized vehicle sensor frames. The vehicle initiating node calculates an integrity verification value for the encrypted standardized vehicle sensor frames and sends the integrity verification value along with the encrypted standardized vehicle sensor frames. After receiving the encrypted standardized vehicle sensor frames, the cross-vendor federated collaboration node first decrypts them using the session key, then recalculates the integrity verification value and compares it with the received integrity verification value. If the comparison passes, the cross-vendor federated collaboration node inputs the decrypted standardized vehicle sensor frames into the locally deployed vehicle model inference engine; if the comparison fails, the cross-vendor federated collaboration node discards the standardized vehicle sensor frames and returns a reception anomaly record to the vehicle initiating node. The reception anomaly record can be used as part of the historical federated execution credentials for subsequent cross-vendor federated collaboration node screening.

[0080] In one possible scenario, the vehicle-mounted anti-tampering verification mechanism in the above embodiments can be implemented through irreversible digest computation and digital signature verification. Cross-vendor federated collaborative nodes perform irreversible digest computation on the privacy-protected data to obtain a privacy-protected data digest; each node then uses its signature key to digitally sign the privacy-protected data digest, establishing a verifiable correspondence between the privacy-protected data digest and the cross-vendor federated collaborative node's identifier. Upon receiving the standardized output packet, the vehicle-mounted initiating node first reads the privacy-protected data and digital signature from the standardized output packet, then re-performs irreversible digest computation on the privacy-protected data to obtain the digest to be verified; the vehicle-mounted initiating node uses the verification data corresponding to the cross-vendor federated collaborative node to verify the digital signature, and performs a consistency comparison between the digest to be verified and the privacy-protected data digest corresponding to the digital signature. Standardized output packets that pass verification enter the encrypted dimension of federated fusion; standardized output packets that fail verification do not participate in the encrypted dimension of federated fusion, and an anomaly return record is recorded in the historical federated execution credentials.

[0081] In one possible scenario, the global cognitive result in the above embodiments can be stored by the vehicle-initiating node as a fused perception representation of the current driving environment. The global cognitive result has a continuous data relationship with the first environmental cognitive deviation record, the vehicle-mounted historical deviation record library, the communication priority change status, the basic computing resource scheduling certificate, and the additional computing resource scheduling certificate: the global cognitive result is first converted into a global fusion benchmark; the global fusion benchmark is then used to calculate the deviation between itself and each standardized output packet to obtain the first environmental cognitive deviation record; this first environmental cognitive deviation record is written into the vehicle-mounted historical deviation record library; the continuous deviation change trend in the vehicle-mounted historical deviation record library is used to generate the communication priority change status; simultaneously, the standardized output packet is used to calculate the deviation between itself and the distributed collaborative average level to obtain a data deviation metric; this data deviation metric, together with the vehicle-mounted model accuracy level identifier, is used to determine whether to issue an additional computing resource scheduling certificate. Therefore, the global cognitive result formed in the current federated processing cycle is not only used for cognitive processing of the current driving environment but also participates in the selection of cross-vendor federated collaborative nodes and the distributed data processing bandwidth scheduling in subsequent federated processing cycles.

[0082] The above Figures 3-6 For an exemplary description, please refer to the above. Figure 1 This will not be elaborated upon here.

Claims

1. A cross-vendor in-vehicle model federated inference method, characterized in that, include: The initiating node of this vendor domain broadcasts a federated inference request within the in-vehicle ad hoc network, receives processing identifiers from the in-vehicle response nodes of heterogeneous vendor domains, and determines the cross-vendor federated collaboration node from the in-vehicle response nodes based on the processing identifiers. The vehicle-mounted initiating node encapsulates driving environment data into standardized vehicle-mounted sensor frames and distributes the standardized vehicle-mounted sensor frames to each of the cross-vendor federated collaborative nodes through an encrypted vehicle-mounted communication link. Each of the cross-vendor federated collaborative nodes uses a locally deployed vehicle model inference engine to perform feature extraction on the standardized vehicle sensor frames and generate local feature information; the internal parameters of the vehicle model inference engine are hidden from each vendor domain. Each of the cross-vendor federated collaborative nodes injects a differential perturbation sequence into the local feature information to obtain privacy-preserving data, and then encapsulates the privacy-preserving data into a standardized output package and uploads it to the vehicle initiation node; The in-vehicle initiating node performs encrypted-dimensional federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

2. The method according to claim 1, characterized in that, The initiating node in the local vendor domain broadcasts a federated inference request within the vehicular ad hoc network, receives processing identifiers from the responding nodes in heterogeneous vendor domains, and determines cross-vendor federated collaboration nodes from the responding nodes based on the processing identifiers, including: The initiating node broadcasts the federated inference request to the in-vehicle ad hoc network. The federated inference request carries a minimum threshold for the number of participating nodes and a federated privacy strength level. The responding node of the vehicle responds to the processing identifier based on the federated inference request. The vehicle-mounted initiating node receives the processing identifier, reads the historical federated execution credentials and vehicle network reputation score, and uses the historical federated execution credentials and vehicle network reputation score as the screening criteria for determining the cross-vendor federated collaboration node. The vehicle-initiating node marks the corresponding vehicle-response node as a cross-vendor federated collaboration node when the vehicle network reputation score reaches a preset score threshold and the historical federated execution credentials have no abnormal records within a preset time span. The total number of the determined cross-vendor federated collaboration nodes is then verified using the minimum number of participating nodes threshold.

3. The method according to claim 1, characterized in that, The vehicle-mounted initiating node encapsulates driving environment data into standardized vehicle-mounted sensor frames, and distributes these standardized vehicle-mounted sensor frames to each of the cross-vendor federated collaborative nodes via an encrypted vehicle communication link, including: The vehicle-mounted initiating node acquires driving environment data collected by local physical sensors and extracts the underlying device-specific structural features belonging to the hardware of the specified manufacturer from the driving environment data. The vehicle-mounted initiating node uses a cross-device universal visual space protocol to align the dimensions of the proprietary structural features of the underlying device, generating standardized vehicle-mounted sensor frames that eliminate hardware form differences. The vehicle-initiating node performs intermediate representation compatibility verification between the standardized vehicle sensor frame and the determined cross-vendor federated collaboration nodes. After the verification is successful, the standardized vehicle sensor frame is sent to each of the cross-vendor federated collaboration nodes through the encrypted vehicle communication link.

4. The method according to claim 1, characterized in that, Each of the aforementioned cross-vendor federated collaborative nodes utilizes a locally deployed in-vehicle model inference engine to perform feature extraction on the standardized in-vehicle sensor frames, generating local feature information, including: Each of the cross-vendor federated collaborative nodes activates the locally deployed vehicle model inference engine and inputs the standardized vehicle sensor frames into the computational flow graph containing the tensor mapping structure in the vehicle model inference engine; Each of the cross-vendor federated collaborative nodes performs feature extraction on the standardized vehicle sensor frames based on the computation flow graph in the vehicle-independent computing sandbox environment, generating local feature information representing road condition perception and prediction elements. During the feature extraction process, each of the cross-vendor federated collaborative nodes blocks the transmission path of the network topology and internal parameters of the vehicle model inference engine to the external vehicle communication port, so that the local feature information does not carry the internal parameters of the vehicle model inference engine, and maintains the cross-vendor hidden state of the internal parameters.

5. The method according to claim 1, characterized in that, Each of the cross-vendor federated collaborative nodes injects a differential perturbation sequence into the local feature information to obtain privacy-preserving data. This privacy-preserving data is then encapsulated into a standardized output packet and uploaded to the vehicle-mounted initiating node, including: Each of the cross-vendor federated collaboration nodes extracts the federated privacy strength level from the federated inference request as the basis for determining the perturbation strength of the differential perturbation sequence; Each of the cross-vendor federated collaborative nodes determines the perturbation sequence generation rules for obfuscating the real distribution based on the federated privacy strength level, and generates the corresponding differential perturbation sequence; Each of the cross-vendor federated collaborative nodes superimposes the differential perturbation sequence onto the local feature information to obtain privacy-preserving data that masks the original road condition features; Each of the cross-vendor federated collaborative nodes uses an in-vehicle anti-tampering verification mechanism to digitally sign the privacy-protected data, enabling the privacy-protected data to carry proof of execution legitimacy, generating a standardized output package and uploading it to the in-vehicle initiating node.

6. A cross-vendor in-vehicle model federated inference device, comprising: The collaborative node determination module is used to enable the vehicle initiating node of the local vendor domain to broadcast federated inference requests within the vehicle ad hoc network, receive processing identifiers fed back by vehicle response nodes of heterogeneous vendor domains, and determine cross-vendor federated collaborative nodes from the vehicle response nodes based on the processing identifiers. The sensor frame distribution module is used to enable the vehicle initiating node to encapsulate driving environment data into standardized vehicle sensor frames, and distribute the standardized vehicle sensor frames to each of the cross-vendor federated collaboration nodes through an encrypted vehicle communication link. The local feature generation module is used to enable each of the cross-vendor federated collaborative nodes to perform feature extraction on the standardized vehicle sensor frames using the locally deployed vehicle model inference engine to generate local feature information; the internal parameters of the vehicle model inference engine are hidden from each vendor domain. The output packet generation module is used to enable each of the cross-vendor federated collaborative nodes to inject differential perturbation sequences into the local feature information to obtain privacy-preserving data, and to encapsulate the privacy-preserving data into a standardized output packet and upload it to the vehicle initiation node. The federated fusion module enables the vehicle-mounted initiating node to perform encrypted-dimensional federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

7. An electronic device comprising a processor and a memory, the memory storing a computer program that, when executed by the processor, causes the processor to perform the cross-vendor in-vehicle model federated inference method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the cross-vendor in-vehicle model federated inference method according to any one of claims 1 to 5.

9. A vehicle comprising an onboard communication unit, an onboard computing unit, and an onboard storage unit, wherein the onboard storage unit stores a computer program, and when the computer program is executed by the onboard computing unit, the vehicle participates in an onboard ad hoc network through the onboard communication unit and executes the cross-vendor onboard model federated inference method according to any one of claims 1 to 5.

10. A cross-vendor in-vehicle model federated inference system, comprising in-vehicle initiation nodes in the local vendor domain and in-vehicle response nodes in heterogeneous vendor domains; The initiating node is used to broadcast federated inference requests within the in-vehicle ad hoc network, receive processing identifiers fed back by the responding nodes, and determine cross-vendor federated collaboration nodes from the responding nodes based on the processing identifiers. The vehicle initiating node is also used to encapsulate driving environment data into standardized vehicle sensor frames, and distribute the standardized vehicle sensor frames to each of the cross-vendor federated collaboration nodes through an encrypted vehicle communication link. Each of the cross-vendor federated collaborative nodes is used to perform feature extraction on the standardized vehicle sensor frames using a locally deployed vehicle model inference engine to generate local feature information; the internal parameters of the vehicle model inference engine are hidden from each vendor domain. Each of the cross-vendor federated collaborative nodes is also used to inject differential perturbation sequences into the local feature information to obtain privacy-preserving data, and then encapsulate the privacy-preserving data into a standardized output package and upload it to the vehicle initiation node; The in-vehicle initiating node is also used to perform encrypted dimension federated fusion on each of the standardized output packets to generate a global cognitive result for the current driving environment.

Images