Intelligent machine interoperation system supporting vehicle-road group collaboration

Abstract

The invention relates to an intelligent machine interoperation system supporting vehicle-road group collaboration, and belongs to the technical field of intelligent vehicle networks. The system is attached to a vehicle-mounted intelligent platform and a roadside intelligent platform through a middleware plug-and-play mode, and comprises a cooperative adaptation subsystem, a cooperative control subsystem and a resource arrangement subsystem, the collaborative adaptation subsystem provides data interfaces for different semantic transformations, constructs logic steps for tasks, and fuses collaborative data of different platforms; the cooperative control subsystem discovers a cooperative object, constructs a communication topological structure, and negotiates and constructs a communication time sequence and cooperative content in a cooperative process; and the resource arrangement subsystem senses general computing resources of the collaborative object in real time, allocates computing resources, unloads a task to a target computing node, and cooperates with a plurality of platforms to complete the collaborative task. The system provided by the invention solves the problem of data isomerism of different semantics, supports a dynamic adaptation collaboration process, intelligently optimizes the utilization of communication resources and computing resources, and guarantees the collaboration quality of vehicle-road groups.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle networking and intelligent vehicle technology, and in particular relates to an edge-end collaborative computing system for heterogeneous vehicle-mounted intelligence. Background Art

[0002] With the advancement of intelligent driving and vehicle-to-vehicle communication technology, the vehicle-road group collaboration between smart cars and intelligent roadside infrastructure, that is, the collaboration between a large number of vehicle-to-vehicle and roadside intelligence, has become an innovative trend in the field of vehicle networking and intelligent vehicle technology. Since both smart cars and intelligent roadside infrastructure are intelligent machines driven by artificial intelligence, the essence of vehicle-to-road group collaboration is the collaboration of intelligent machines driven by artificial intelligence. However, the independence and heterogeneity of various vehicle-to-vehicle and roadside intelligent platforms pose challenges to interoperability in vehicle-to-road group collaboration. On the one hand, a large number of existing vehicle-to-vehicle and roadside systems operate independently and do not have the ability to perceive and make decisions in vehicle-to-road group collaboration; on the other hand, there are differences in the internal interface design of vehicle-to-vehicle and roadside intelligent platforms of different brands and models, and the neural network structures and neural network parameters are different, resulting in interface and semantic heterogeneity between vehicle-to-road intelligent platforms. The independence and heterogeneity between various vehicle-to-vehicle and roadside intelligent platforms hinder the effective intelligent machine collaboration between vehicle-to-road groups.

[0003] In addition, the demand for communication and computing resources for complex vehicle-road group collaboration is growing. In the scenario of vehicle-road group collaboration, limited communication and computing resources will affect the quality of vehicle-road group collaboration. It is urgent to ensure the effectiveness of vehicle-road group collaboration through efficient communication and computing resource orchestration. However, communication resources have different standards such as LTE-V and NR-V2X, and computing resources are distributed in different locations such as terminals and edges. How to shield the standard and location diversity of underlying resources according to the needs of vehicle-road group collaboration and ensure the effectiveness of vehicle-road group collaboration through active resource adaptation is of great significance.

[0004] The invention patent application with application number CN202211077833.9 disclosed a distributed path collaborative computing method for low-latency requirements of the Internet of Vehicles on December 6, 2022. Based on the improved particle swarm optimization algorithm, the mapping relationship between the vehicle-mounted sub-services and the fog network nodes under the shortest mapping delay is solved, and the collaborative computing between the vehicle-mounted and the edge is realized, solving the high latency problem of data processing under the existing cloud computing model. This technical solution focuses on solving the latency optimization problem between the vehicle and the edge, but does not solve the problem of how multiple independent heterogeneous intelligent platforms can collaborate, that is, it does not solve the problem of inability to collaborate between different vehicle-mounted systems or between vehicle-mounted systems and roadside facilities due to inconsistent interfaces and semantics.

[0005] The invention patent application with application number CN2022107942770 disclosed on October 25, 2022 a DDQN-based end-edge-cloud collaborative scheduling method and system in the edge environment of the Internet of Vehicles. An end-edge-cloud collaborative service offloading architecture is deployed in the edge environment of the Internet of Vehicles, a service delay calculation model is established, and the task offloading strategy is determined. Based on this, the optimization problem is determined and converted into a Markov decision process, and the scheduling solution is solved using a deep reinforcement learning method. This technical solution focuses on improving the efficiency of computing task offloading, and does not propose a solution to the fundamental problem of the lack of collaborative capabilities between heterogeneous intelligent machines. In addition, in the edge environment of the Internet of Vehicles, the solution may not fully consider how to dynamically adapt resources between different on-board and roadside intelligent platforms to continuously ensure the quality of collaboration.

[0006] The invention patent application with application number CN202111513367X disclosed on March 8, 2022 a highly reliable task collaborative computing method in the Internet of Vehicles based on fog computing. In the case of dynamic vehicle movement, the reliability of the vehicle is considered from the aspect of link duration, and the total delay of the vehicle to perform the task is considered from the aspects of transmission delay and computing delay, and the task offloading strategy is determined from the perspective of collaborative vehicles and tasks. This technical solution uses the Hungarian algorithm for task allocation, is committed to improving the success rate of task execution, focuses on the collaborative processing of on-board computing tasks, and solves the problem of computing collaborative optimization in specific scenarios. However, this solution may not be applicable to or solve the essential collaborative work problems between different systems, such as differences in interfaces and protocols. In addition, this solution focuses on improving the success rate of task execution, and may not fully address the problem of how to maintain and optimize collaborative quality in a changing environment.

[0007] The invention patent application with application number CN2022112184871 disclosed a learning-based Internet of Vehicles multi-layer edge computing network resource collaboration method on April 4, 2023. By considering the task computing intensity and exploration preference of vehicle users, historical learning results are used to update the system cost, and on this basis, the optimal task unloading location is selected. This technical solution only considers resource allocation, and the resource allocation of this method is not for vehicle-road group collaborative tasks. The application scenario of this technology is relatively narrow, focusing only on the network resource collaboration method of edge computing, and has not been expanded to the collaboration of more intelligent driving applications. Similarly, this solution does not solve the problem of how multiple independent heterogeneous intelligent platforms can collaborate.

[0008] The existing technology has not yet solved the problems of independence and heterogeneity in the collaborative computing of vehicle-mounted and roadside intelligent platforms, so it is necessary to study technical solutions to this problem. Summary of the invention

[0009] In the current intelligent driving environment, vehicle-mounted and road-side intelligent systems of different brands and specifications are highly closed and heterogeneous, resulting in poor interoperability between multiple entities such as intelligent vehicle-mounted devices and intelligent road-side devices. Such systems often use private software interfaces, and data and functions are strictly limited within the closed system. Data is difficult to share, and functions cannot be coordinated, forming isolated data and capability islands. In order to break through this heterogeneous barrier and achieve deep collaboration, the present invention provides an intelligent machine interoperability system that supports vehicle-road group collaboration. It is attached to existing vehicle-mounted and road-side intelligent platforms in a plug-and-play manner, and endows various independent and heterogeneous vehicle-mounted and road-side intelligent platforms with vehicle-road group collaboration capabilities. At the same time, the quality of vehicle-road group collaboration is ensured through active adaptation of communication and computing resources to vehicle-road group collaboration.

[0010] The present invention provides an intelligent machine interoperation system that supports vehicle-road group collaboration, which is attached to the vehicle-mounted and roadside intelligent platforms in a plug-and-play manner with middleware. The intelligent machine interoperation system includes a collaborative adaptation subsystem, a collaborative control subsystem, and a resource orchestration subsystem. The collaborative adaptation subsystem includes a business collaborative adaptation module and a collaborative logic construction module. The collaborative control subsystem includes a collaborative object discovery module and a collaborative process control module. The resource orchestration subsystem includes a general computing resource orchestration module.

[0011] When the business system of intelligent platform A receives a collaborative task, in the interoperable system of intelligent platform A:

[0012] (1) The collaborative logic building module builds the logical steps for the task, including heterogeneous data input processing, collaborative object discovery, collaborative process control, and computing and communication resource allocation;

[0013] (2) The business collaboration adaptation module sets up data interfaces for different semantic conversions, which convert the heterogeneous perception data input by the intelligent platform into feature data of corresponding formats according to the collaborative tasks; the business collaboration adaptation module also generates data requests and platform capability requirements according to the collaborative tasks and sends them to the resource orchestration subsystem;

[0014] (3) The collaborative object discovery module calls the communication system of intelligent platform A through the resource orchestration subsystem, broadcasts messages to the outside, dynamically discovers other neighboring intelligent platforms within the communication range of intelligent platform A, builds A's collaborative object set and sends it to the resource orchestration subsystem. A's collaborative object set includes A and A's neighbors. The intelligent platforms exchange collaborative control information through messages. The collaborative control information includes the identity, location, speed, available computing resources of the platform, and supported application functions of the intelligent platform.

[0015] (4) The collaborative process control module establishes a collaborative operation process between A and its collaborative objects, and manages the communication topology, communication sequence, and collaborative content exchange of the collaborative objects;

[0016] (5) After receiving the collaborative control information, the business collaborative adaptation module sends a data request message to the collaborative object and receives collaborative content, which is obtained by filtering the perception data of the collaborative object; the business collaborative adaptation module integrates the collaborative content sent by different collaborative objects;

[0017] (6) The general computing resource orchestration module obtains the real-time communication resources and computing resources in the collaborative object set, with the goal of minimizing resource usage while ensuring service quality. It allocates a set of optimal computing resources to the collaborative task and places the collaborative task on the collaborative object with the lowest cost. After the collaborative object completes the task, it returns the calculation result to the business collaborative adaptation module, which performs data conversion and returns it to A's business system.

[0018] The advantages and positive effects of the present invention are:

[0019] (1) The current existing technology focuses on task collaborative computing methods, especially in terms of reliability, but it does not solve or may not be applicable to the essential collaborative work problems between different systems, such as differences in interfaces and protocols. The intelligent machine interoperability system of the present invention is attached to the existing vehicle-mounted and roadside intelligent platforms in a plug-and-play manner of middleware, giving various independent and heterogeneous vehicle-mounted and intelligent platforms the ability to collaborate with each other in the vehicle-road group. The intelligent machine interoperability system of the present invention solves the interface and semantic heterogeneity problems between systems through a unified framework and open protocol of the collaborative adaptation subsystem. It can receive inputs from vehicle-mounted and roadside intelligent platforms of different manufacturers and models, and identify and process heterogeneous interfaces and semantics. This collaborative adaptation function effectively shields the differences between various systems, thereby providing collaborative intermediate data for upper-level collaborative applications.

[0020] (2) The intelligent machine interoperability system of the present invention focuses on the collaborative work of multiple businesses, including collaborative object discovery, collaborative timing control, collaborative topology control and collaborative content control, etc. It is capable of managing collaborative computing objects and their processes. This control function is not only executed according to predetermined rules, but also can adapt to dynamic changes in the environment, ensuring the flexibility and adaptability of collaborative operations.

[0021] (3) The intelligent machine interoperability system of the present invention enables independent vehicle-mounted and roadside intelligent platforms to have collaborative capabilities, and through resource optimization, it can ensure the quality of collaboration during the collaborative process. The general computing resource orchestration module of the present invention performs fine orchestration and scheduling of computing resources and communication resources within the host system to optimize the efficiency and quality of vehicle-mounted and roadside collaborative task execution. It is not only limited to task offloading strategies, but can also dynamically adapt to environmental changes, intelligently optimize communication and computing resources, and ensure the quality of vehicle-road group collaboration. However, the existing technology focuses on task collaborative computing methods, does not solve the coupling of collaborative processes and resource optimization, and does not achieve the adaptation of the two.

[0022] (4) The intelligent machine interoperability system of the present invention provides a more comprehensive collaborative computing solution for the Internet of Vehicles, which not only solves the problem of collaborative computing interoperability between heterogeneous devices, but also provides a complete vehicle-road collaborative group computing framework from communication computing to collaborative control to business applications. The intelligent machine interoperability system of the present invention provides a unified framework that spans system heterogeneity, solves the problems of interactive interfaces and semantic inconsistencies of various vehicle-mounted and road-side intelligent platforms, and realizes group collaboration between various heterogeneous vehicle-mounted and road-side intelligent platforms. At the same time, it can also dynamically adapt collaborative tasks and resource status to ensure the quality and efficiency of the collaborative process, and realize the joint optimization of intelligence and resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the working scenario of the intelligent machine interoperation system (middleware) of the present invention;

[0024] Figure 2 It is a schematic diagram of the implementation structure of the intelligent machine interoperation system of the present invention;

[0025] Figure 3 is a schematic diagram of the structure of the collaborative process control module of the present invention;

[0026] Figure 4 It is a structural diagram of the general computing resource arrangement module of the present invention;

[0027] Figure 5 It is a timing diagram of the intelligent machine interoperation system of the present invention executing a collaborative task. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0029] The present invention provides an intelligent machine interoperability system that supports vehicle-road group collaboration, which aims to solve the limitations of independent cognitive decision-making between various vehicle-mounted and roadside intelligent platforms, as well as the interoperability problems caused by inconsistent interfaces and semantics. In addition, the existing technology only considers the problems of computing offloading and resource optimization in non-collaborative computing scenarios, and is unable to adapt to vehicle-road group collaborative computing tasks. The middleware system proposed in the present invention can actively adapt to different vehicle-road group collaborative computing tasks, and arrange communication and computing resources in real time according to the status of computing tasks and resources to ensure the service quality of the collaborative process.

[0030] In the current scenarios of autonomous driving and intelligent transportation, a large number of vehicle-mounted and road-side intelligent platforms do not have the ability to collaborate and only support independent operation, resulting in limited cognitive scope and low precision of the intelligent platform, and the decision cannot achieve global optimization. The fundamental role of the intelligent machine interoperation system of the present invention is to integrate the above-mentioned independently running heterogeneous intelligent platforms into an intelligent machine network. It is a software system that is deployed in the vehicle-mounted and road-side intelligent platforms in a plug-and-play manner with middleware. The role of the intelligent machine interoperation system provided by the present invention is reflected in three aspects: First, the middleware expands the original neural network model that runs independently in the vehicle-mounted and road-side intelligent platforms into a neural network model that can run collaboratively, which can shield the heterogeneity of hardware, operating system, data interface and data semantics in the operation of the neural network model; in addition, the middleware supports process control of collaborative applications, and controls the timing, topology and content of the collaboration through a unified collaborative process negotiation mechanism. Finally, the middleware supports dynamic adaptation of the collaborative process, orchestration of communication resources and computing resources, and guarantees the collaborative service quality.

[0031] like Figure 1 As shown, the working environment of the intelligent machine interoperability system supporting vehicle-road group collaboration, that is, the intelligent machine network middleware implemented by the present invention is demonstrated. Before the introduction of the middleware, the vehicle-mounted and roadside intelligent platforms included a business system, a computing system, and a communication system. An independently running neural network model has been deployed in the business system to implement services such as target detection, semantic segmentation, and trajectory planning. The computing system is the local CPU, GPU, and memory resources. The communication system supports communication protocols such as 4G, 5G, DSRC, LTE-V, and NR-V2X. After the introduction of the intelligent machine network middleware of the present invention, the middleware is a plug-and-play software. The middleware working on each vehicle-mounted and roadside intelligent platform is respectively connected to the business system, computing system, and communication system of the corresponding intelligent platform. By interacting with these systems, the independently running business in the original business system is expanded into a collaborative business, realizing target detection, semantic segmentation, and trajectory planning for vehicle-road group collaboration.

[0032] The intelligent machine interoperation system (middleware) of the embodiment of the present invention is mainly implemented as a supporting software for efficient vehicle-road group collaborative computing. The core function of the intelligent machine interoperation system is to promote communication and collaborative work between different intelligent traffic devices, and the core functions include collaborative adaptation, collaborative control and resource orchestration. Collaborative adaptation means that the middleware can receive inputs of different data semantics and data interfaces, and adapt to heterogeneous intelligent platforms. This adaptation function shields the semantic and interface differences between devices. Collaborative control means that the middleware can manage and control collaborative objects and processes, including collaborative object discovery, collaborative object selection, collaborative timing control, collaborative content control and collaborative topology control between intelligent platforms. This control can not only be based on preset rules, but also adapt to dynamically changing traffic environments, providing a negotiation mechanism for multiple collaborative intelligent platforms. Resource orchestration means that the middleware is also responsible for the effective orchestration and scheduling of the computing and communication resources of the intelligent platform. This includes analyzing task requirements, evaluating resource status, and dynamically allocating computing resources and communication resources to optimize task execution and ensure the service quality of collaborative tasks. The middleware of the present invention aims to improve the availability and efficiency of collaborative computing scenarios in the Internet of Vehicles, while ensuring the versatility of heterogeneous computing devices in Internet of Vehicles scenarios. Heterogeneous intelligent computing devices that interact through middleware can achieve collaborative work among multiple businesses.

[0033] like Figure 2 As shown, the intelligent machine interoperability system of the embodiment of the present invention includes three subsystems: a collaborative adaptation subsystem that adapts to a variety of intelligent traffic collaborative computing tasks and supports business expansion and logic construction; a collaborative control subsystem that supports collaborative object discovery and system process control; and a resource scheduling subsystem that supports the scheduling of computing resources and communication resources.

[0034] The collaborative adaptation subsystem plays a vital role in the collaborative computing of vehicle-road groups. It is the part that directly interacts with the vehicle-mounted / roadside business system to be coordinated, ensuring the heterogeneous semantic access from different vehicle-mounted and roadside businesses. Specifically, the collaborative adaptation subsystem mainly consists of two modules: the business collaborative adaptation module and the collaborative logic construction module. The business collaborative adaptation module provides an interface that can process different semantic inputs and adapt to the business systems of various intelligent platforms, such as Figure 2 As shown in the figure, the business collaboration adaptation module sets up a conversion interface for connecting and integrating various vehicle-mounted and roadside business systems. The business collaboration adaptation module is responsible for processing various input and output data, processing the heterogeneous data semantics input by each intelligent platform, and aligning them into a unified format, so that even smart devices of different brands and models can work together consistently. The main function of the collaborative logic construction module is to build, implement and optimize a complete set of collaborative operation logic based on specific collaborative businesses to adapt to complex and changing traffic environments and business needs.

[0035] The business collaborative adaptation module supports a variety of collaborative computing applications and accepts data input from the intelligent platform. The computing process in the business collaborative adaptation module is controlled by the collaborative logic construction module. According to the collaborative logic defined in the collaborative logic construction module, the data of different intelligent devices are integrated step by step. The input features used by different collaborative tasks will be different. According to the input requirements of the task, the business collaborative adaptation module will extract the corresponding feature data from the perception data. A data interface is set in the business collaborative adaptation module. For each collaborative task, the data interface aligns the input heterogeneous perception data through the corresponding neural network model, unifies it into a feature vector representation in a preset format, generates a data request according to the received collaborative computing task, and sends the data request and the required intelligent platform computing resource capacity requirements to the resource orchestration subsystem. The business collaborative adaptation module also initiates a data request message to the collaborative object according to the collaborative timing, requests other platforms to send collaborative content data, and integrates the collaborative content sent by different collaborative objects. After the collaborative calculation of the computing task is completed, the business collaborative adaptation module returns the calculation result to the business system of the intelligent device.

[0036] The collaborative logic building module can build collaborative logic. When the intelligent platform business system receives a computing task, the collaborative logic building module of the collaborative adaptation subsystem provides various logical steps of collaboration according to the task. For example, a typical vehicle-road group collaborative computing task includes heterogeneous data input, collaborative object discovery, collaborative process control, computing and communication.

[0037] The collaborative control subsystem is the core component of the intelligent machine interoperability system of the present invention, and its goal is to ensure effective collaborative operation within the system and with the external environment. The collaborative control subsystem mainly consists of two modules: the collaborative object discovery module and the collaborative process control module, which work together to achieve efficient execution of collaborative computing tasks and business logic of the intelligent transportation system.

[0038] The main task of the collaborative object discovery module is to discover and maintain objects for collaborative operations, which are the vehicle-mounted and roadside intelligent platforms that have installed the middleware of the present invention. The collaborative object discovery module can dynamically discover other collaborative objects within the communication range and exchange collaborative control information with each other by using a specific protocol. The collaborative object discovery module broadcasts messages to the outside, calls the communication system of the intelligent platform through the resource orchestration subsystem, broadcasts messages to the outside, and receives reply messages from other intelligent platforms within the communication range of this intelligent platform. The broadcast messages and received reply messages contain collaborative control information. The collaborative control information includes information such as the identity, location, speed, platform computing resources, and supported application functions of the intelligent platform. For the collaborative object discovery module of intelligent platform A, the neighboring intelligent platforms within the communication range of A are discovered, and a collaborative object set including intelligent platform A is constructed. The set includes A and the discovered neighbors of A, and the computing resources of each platform are recorded, and the set is sent to the computing resource orchestration module.

[0039] like Figure 3 As shown in the figure, the collaborative process control module is responsible for establishing an effective collaborative operation process between the discovered collaborative objects, including the collaborative process control interface, collaborative process negotiation interface, collaborative object state interface, and collaborative topology, timing and content control modules. The collaborative process control module manages every detail of the collaborative operation, including communication topology, message timing and content exchange, through a detailed control and scheduling mechanism.

[0040] The collaborative object status interface is the interface between the collaborative object discovery module and is responsible for passing the status of each collaborative object in the collaborative task. The collaborative object status includes the capability information and location status of the vehicle-mounted and roadside intelligent platforms participating in the collaboration, such as their location, speed, functional status, etc., as well as the collaborative application information they can support.

[0041] The collaborative topology control module is responsible for determining and managing the communication topology in the collaborative process. The collaborative topology control module's work includes determining which collaborative objects should communicate directly, how data should flow between collaborative objects, and how to configure communication paths to maximize efficiency and reliability. In a collaborative perception task, for example, topology control may involve maintaining a connection map of all intelligent platforms to ensure that all participating vehicles can receive the most effective perception data from surrounding vehicles.

[0042] The collaborative timing control module is responsible for ensuring the correct timing of messages and data in the collaborative process. During the communication process of collaborative tasks, the communication information needs to maintain order, which means that in multiple rounds of communication for a certain business, it is necessary to control different communication objects to participate in the communication at the corresponding time. In collaborative perception tasks, there will be several rounds of communication in the process. In each round of communication, different collaborative objects will participate. It is necessary to control the sending objects of messages at different times according to the defined logic to ensure that data can be correctly and orderly shared with all participants, and the processing results can be coordinated and fed back to the relevant collaborative objects.

[0043] The collaborative content control module is responsible for managing and optimizing the data content exchanged during the collaborative process. This includes not only screening and processing shared data to meet the needs of specific collaborative tasks, but also processing, compressing or converting data to improve transmission efficiency and accuracy. For example, in a collaborative perception task, content control may involve screening out the most relevant information from the perception data collected by multiple vehicles. Through collaborative content control, communication data can be compressed and processed to reduce communication resource overhead.

[0044] The collaborative process negotiation interface is an interface to the collaborative process control module on other devices. It provides a negotiation mechanism between collaborative objects, allowing them to negotiate the process of collaborative tasks. Through this interface, multiple middleware can negotiate the collaborative topology, collaborative timing, and collaborative content in the collaborative process.

[0045] The collaborative process control interface is the interface between the collaborative adaptation subsystem and the collaborative adaptation subsystem, which feeds back the negotiated collaborative topology, collaborative timing, and collaborative content to the collaborative adaptation subsystem.

[0046] The resource orchestration subsystem of the present invention plays a pivotal role in the intelligent machine interoperation system in the vehicle-road collaborative scenario. Its core goal is to optimize and schedule the computing resources and communication resources of the vehicle-mounted and roadside intelligent platforms to support the communication and computing resources required for collaborative applications and ensure the service quality of the business. The resource orchestration subsystem includes a general computing resource orchestration module, which implements the fine management of computing resources and communication resources and actively adapts the group intelligence collaborative process. Among them, computing resources refer to vehicle-mounted computing resources or roadside edge computing resources. For example, vehicle-mounted computing resources are computing resources deployed on the vehicle-mounted intelligent platform, including resources such as CPU, GPU, and memory, which provide low-latency computing support for vehicle-road collaborative group computing applications. The vehicle-mounted computing resources are connected to various sensor systems inside the vehicle to obtain environmental observations, map information, etc., and support the deployment of intelligent driving related functions. Communication resources refer to vehicle-mounted communication systems or roadside communication systems, which are an indispensable part of the intelligent transportation system. It supports communication between vehicles (V2V), vehicles and infrastructure (V2I), and even vehicles and pedestrians (V2P). Among them, LTE-V and NR-V2X are the most advanced communication technologies at present, which provide strong technical support for realizing efficient and safe road traffic. In the embodiment of the present invention, the LTE-V protocol is used to exchange cooperative control information and object transmission information, and the NR-V2X protocol is used to exchange data.

[0047] like Figure 4 As shown, the general computing resource scheduling module of the present invention provides a general computing resource status update interface, through which the real-time status of communication resources and computing resources can be queried. For computing tasks, the computing tasks forwarded by the collaborative adaptation subsystem will first be received through the computing resource allocator and the computing task placer, and according to the nature of the task, the service quality requirements of the task and the availability of resources, the resource allocation strategy will be generated through the pre-trained neural network model, and the decision to offload the task to the on-board computing resources or roadside computing resources will be made. This process involves intelligent matching of tasks and resources to ensure maximum computing efficiency.

[0048] The computing resource allocator is responsible for the specific scheduling of computing resources. It selects the computing resources such as CPU and GPU required for the current computing task from the set of this intelligent platform and the current neighboring intelligent platforms. It will dynamically adjust the resource allocation strategy according to the service quality requirements of the current computing task to adapt to the changing computing needs. The computing resource allocator uses an artificial intelligence model to allocate a set of optimal resource sets for the current Internet of Vehicles to the computing task, that is, the minimum resource occupancy while ensuring the quality of service, including the minimum CPU, GPU and other resources required by the current computing task.

[0049] The computing task placer needs to select an optimal computing node for the computing task. The selected strategy is output through a pre-trained neural network model. The optional nodes are all computing nodes within the current machine communication range, including the computing resources of on-board intelligent devices and the computing resources on roadside edge computing devices. The computing task placer will detect the load of all available nodes in the collaborative object set, and select the computing node with the lowest placement cost for placement among the nodes that meet the resource set decision of the computing resource allocator. The cost includes computing latency and power consumption. The output of the computing task placer includes the resource set information allocated by the computing resource allocator and the target computing node information decided by the computing task placer, and is passed to the computing resource management and control interface.

[0050] The computing resource management and control interface directly controls the computing resources of specific vehicles / roadside intelligent platforms, executes collaborative tasks and computing resource allocation decisions. The execution method is to create a container for the current computing task, and run the container at the corresponding location according to the allocated resource set and the placement node decided to complete the computing task. The target computing node of the computing task of intelligent platform A may be A or one of A's neighbors. After the target computing node calculates and obtains the final calculation result, it is returned to A's business collaborative adaptation module, and then fed back to A's business system.

[0051] For communication needs, the communication mode selector will select the appropriate communication protocol according to the communication needs of the task. Control messages are sent via LTE-V to ensure low latency, while data such as large amounts of sensory features are sent via NR-V2X to take advantage of its high bandwidth.

[0052] The communication resource allocator is a key component in the vehicle-mounted and roadside communication system. It is responsible for the reasonable allocation of limited communication resources, such as bandwidth and spectrum, to meet the needs of different communication tasks. The communication resource allocator formulates the optimal communication resource allocation strategy based on the characteristics and constraints of the communication task, such as latency and packet loss rate. At the same time, the communication resource allocator can also dynamically monitor the network status, quickly respond to emergencies, and adjust the communication resource allocation plan in time to ensure the quality of important communication tasks. For example, when network congestion is detected, the communication resource allocator will change the communication channel in time to ensure quality. In addition, the communication resource allocator also uses technologies such as artificial intelligence to perform intelligent analysis and decision-making based on historical data and real-time conditions, try to avoid communication conflicts, continuously optimize the communication resource allocation algorithm, and improve the performance and reliability of the entire communication network.

[0053] Finally, the communication resource management and control interface will select a specific communication channel for data transmission based on the previous communication mode selection decision and communication resource allocation decision, ensure effective communication within the system and between the system and the external environment, and complete the transmission of various types of data and control information by accurately executing the scheduling and allocation decisions of communication resources.

[0054] like Figure 5 As shown in the figure, taking a collaborative perception application as an example, (1) when the business side of a certain vehicle-mounted intelligent platform A receives a collaborative perception task, A's collaborative logic construction module constructs the logical steps of the task; (2) Intelligent platform A sends the acquired perception data to its own collaborative adaptation subsystem; A's business collaborative adaptation module uses the data interface to convert the format of the input perception data, unifies it into a specified format for semantic alignment, such as unifying it into a feature vector form with specified physical dimensions, generates a data request based on the collaborative perception task, and sends the data request and platform capability requirements to the collaborative control subsystem; the data request records which areas' perception data are needed; (3) The collaborative control subsystem broadcasts the collaborative task request message, and the message passes through the resource orchestration subsystem to generate a communication resource scheduling decision, which is executed by the communication resources. (4) The resource orchestration subsystem receives feedback messages from other intelligent platforms from the communication module and passes them to the collaborative control subsystem. Through the collaborative control information fed back by the neighboring intelligent platforms, it discovers the neighboring intelligent platforms within the communication range of A and obtains a set of collaborative objects. (5) After receiving the collaborative control information, A's business collaborative adaptation module also sends a data request message to the collaborative object through the resource orchestration subsystem. The collaborative object sends the collaborative content to A. A's business collaborative adaptation module performs feature fusion on the collaborative content (perception features) of other platforms. (6) The resource orchestration subsystem calculates the optimal computing resource set and target computing node based on the collaborative perception task, places the task on the target computing node, and coordinates multiple platforms for computing. After the calculation is completed, the target computing node returns the collaborative perception result to A's collaborative adaptation subsystem, and finally returns it to A's business side.

[0055] The system of the present invention uses modular software architecture design and artificial intelligence methods to process heterogeneous semantics between different devices, realizes a set of intelligent machine network middleware that supports the collaborative interconnection of heterogeneous vehicle-mounted computing platforms, supports the interconnection and intercommunication of vehicle-mounted intelligent platforms at all levels and roadside intelligent platforms, and integrates them into a networked collaborative system to share information and make collaborative decisions. In this process, distributed optimization of computing and communication resources supports and guarantees the group intelligence computing power resources and data interaction communication resources between multiple heterogeneous intelligent platforms.

[0056] Except for the technical features described in the specification, all of them are known technologies to those skilled in the art. The present invention omits the description of known components and known technologies to avoid redundancy and unnecessary limitation of the present invention. The implementation methods described in the above embodiments do not represent all implementation methods consistent with the present application. On the basis of the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative labor are still within the protection scope of the present invention.

Claims

1. An intelligent machine interoperation system supporting vehicle-road group collaboration, characterized in that: The system is deployed on the vehicle-mounted and roadside intelligent platforms in a middleware plug-and-play manner. The interoperable system on each intelligent platform includes a collaborative adaptation subsystem, a collaborative control subsystem and a resource orchestration subsystem; the collaborative adaptation subsystem includes a business collaborative adaptation module and a collaborative logic construction module; the collaborative control subsystem includes a collaborative object discovery module and a collaborative process control module; the resource orchestration subsystem includes a general computing resource orchestration module; When the business system of intelligent platform A receives a computing task, in the interoperable system of intelligent platform A: (1) The collaborative logic building module builds the logical steps for the task, including heterogeneous data input processing, collaborative object discovery, collaborative process control, and computing and communication resource allocation; (2) The business collaboration adaptation module sets up data interfaces for different semantic conversions, which convert the heterogeneous perception data input by the intelligent platform into feature data of corresponding formats according to the computing tasks; The business collaboration adaptation module also generates data requests and platform capability requirements based on computing tasks and sends them to the resource orchestration subsystem; (3) The collaborative object discovery module calls the communication system of intelligent platform A through the resource orchestration subsystem, broadcasts collaborative control information, dynamically discovers other neighboring intelligent platforms within the communication range of intelligent platform A, and constructs a collaborative object set of A and sends it to the resource orchestration subsystem. A's collaborative object set includes A and its neighbors. The intelligent platforms exchange collaborative control information through messages. The collaborative control information includes the identity, location, speed, available computing resources of the platform, and supported application functions of the intelligent platform. (4) The collaborative process control module establishes a collaborative operation process between A and its collaborative objects, and manages the communication topology, communication sequence, and collaborative content exchange of the collaborative objects; (5) After receiving the collaborative control information, the business collaborative adaptation module sends a data request message to the collaborative object and receives collaborative content, which is obtained by filtering the perception data of the collaborative object; the business collaborative adaptation module integrates the collaborative content sent by different collaborative objects; (6) The general computing resource orchestration module obtains the real-time communication resources and computing resources in the collaborative object set, with the goal of minimizing resource usage while ensuring service quality. It allocates a set of optimal computing resources to the computing task and places the computing task on the collaborative object with the lowest cost. After the collaborative object completes the computing task, it returns the calculation result to the business collaborative adaptation module, which performs data conversion and returns it to A's business system.

2. The intelligent machine interoperation system according to claim 1, characterized in that: The collaborative process control module includes a collaborative process control interface, a collaborative process negotiation interface, a collaborative object state interface, a collaborative topology control module, a collaborative timing control module and a collaborative content control module; The collaborative object status interface is connected to the collaborative object discovery module to pass in the status of each collaborative object, which includes the location, speed, available computing resources, and supported computing tasks of the intelligent platform. The collaboration topology control module is used to determine and manage the communication topology in the collaboration process, including: determining which collaborative objects communicate directly, the direction of data flow between collaborative objects, and configuring communication paths to maximize efficiency and reliability; The collaborative timing control module is used to manage the correct timing of message and data transmission among collaborative objects during the collaborative process; The collaborative content control module is used to manage and optimize the data exchanged during the collaborative process, including: screening and processing shared data, processing, compressing or converting data; The collaborative process negotiation interface is used to connect to the collaborative process control modules of other intelligent platforms and provide a negotiation mechanism between collaborative objects to negotiate the collaborative topology, collaborative timing and collaborative content in the collaborative process. The collaborative process control interface is used to connect the collaborative adaptation subsystem and feed back the negotiated collaborative topology, collaborative timing, and collaborative content to the collaborative logic building module.

3. The intelligent machine interoperation system according to claim 1, characterized in that: The general computing resource arrangement module includes a general computing resource status update interface, a computing resource allocator, a computing task placer, a computing resource management and control interface, a communication mode selector, a communication resource allocator and a communication resource management and control interface; The general computing resource status update interface is used to query the real-time status of the communication resources and computing resources of the collaborative object; The computing resource allocator uses a pre-trained neural network model to decide and output an optimal computing resource set from the collaborative object set based on the current computing tasks and service quality requirements, with the goal of minimizing resource usage while ensuring service quality. The computing task placer detects the load status of all collaborative objects and selects an optimal computing node from the collaborative objects of the optimal computing resource set through a pre-trained neural network model, so that the cost of placing the computing task on the node is the lowest, and the cost includes computing latency and power consumption; the computing task placer outputs the optimal computing resource set and the optimal computing node to the computing resource management and control interface; The computing resource management and control interface creates a container for the current computing task, sends a computing request to the corresponding collaborative object based on the optimal computing resource set and the optimal computing node, runs the container at the corresponding location, and completes the computing task; The communication mode selector selects a communication protocol according to the communication requirements of the computing task; wherein, the control message uses the LTE-V protocol and the data uses the NR-V2X protocol; The communication resource allocator allocates communication resources for different communication tasks, dynamically monitors network conditions, and adjusts the communication resource allocation plan in a timely manner; The communication resource management and control interface selects a communication channel for sending control information or data based on the selected communication protocol and communication resource allocation decision.

Images