Computing power registration method and device, medium and vehicle

Abstract

The embodiment of the invention relates to a computing power registration method and device, a medium and a vehicle, and the method comprises the steps: feeding back a first confirmation message to a slave computing power service under the condition that a node registration message sent by the slave computing power service is received; adding the computing power nodes deployed by the slave computing power services into a computing power node list; and feeding back a second acknowledgement message to the slave computing power service under the condition that the node capability message sent by the slave computing power service is received. According to the embodiment of the invention, the computing power service is deployed on each operating system of each computing power node, and the master computing power service and the slave computing power service are set, so that different operating systems and different computing power nodes can communicate through the computing power services, and a foundation is laid for cross-system model calling and cross-node model calling. Moreover, a computing power registration mechanism is updated in a matched manner, so that each computing power node can be found and registered, and a foundation is laid for master computing power service to master the distribution of the whole-vehicle computing power resources and to schedule the whole-vehicle computing power resources.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle technology, and more particularly to a computing power registration method, apparatus, medium, and vehicle. Background Technology

[0002] In current vehicles, each functional module (such as autonomous driving, smart cockpit, etc.) is typically handled by an independent Electronic Control Unit (ECU). Each ECU's Operating System (OS) has its own models pre-deployed. When the vehicle is running, the OS on each ECU calls the models deployed on its own OS. Each OS manages its own responsible models, and the OSs and ECUs do not interfere with each other.

[0003] However, this model deployment method is rather rigid. Therefore, there is an urgent need to introduce a new model deployment framework and update the computing power registration mechanism accordingly. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a computing power registration method, apparatus, medium, and vehicle.

[0005] A first aspect of this disclosure provides a computing power registration method applied to a vehicle-side model deployment framework. The vehicle-side model deployment framework includes multiple computing power nodes. Each computing power node has at least one operating system deployed on it, and each operating system has a computing power service deployed on it. At least one model is deployed on each operating system. Among all the computing power services deployed on the multiple computing power nodes, one computing power service is designated as a master computing power service, and the remaining computing power services are designated as slave computing power services. The method is executed by the master computing power service and includes: Upon receiving the node registration message sent by the computing power service, a first confirmation message is sent back to the computing power service; Add the computing node on which the computing power service is deployed to the computing node list; Upon receiving the node capability message sent by the computing power service, a second confirmation message is sent back to the computing power service.

[0006] A second aspect of this disclosure provides a computing power registration device applied to a vehicle-side model deployment framework. The vehicle-side model deployment framework includes multiple computing power nodes. Each computing power node has at least one operating system deployed on it, and each operating system has a computing power service deployed on it. At least one model is deployed on each operating system. Among all the computing power services deployed on the multiple computing power nodes, one computing power service is a primary computing power service, and the remaining computing power services are secondary computing power services. The device is integrated into the primary computing power service. The device includes: The first feedback module is used to send a first confirmation message to the computing power service upon receiving the node registration message sent by the computing power service. The first addition module is used to add the computing node on which the computing power service is deployed to the computing node list; The second feedback module is used to send a second confirmation message to the computing power service upon receiving the node capability message sent by the computing power service.

[0007] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the method of the first aspect described above.

[0008] A fourth aspect of this disclosure provides a vehicle including the computing power registration device described in the second aspect and / or the computer-readable storage medium described in the third aspect.

[0009] The technical solution provided in this disclosure has the following advantages compared with the prior art: In this embodiment, the vehicle-side model deployment framework includes multiple computing power nodes. Each computing power node has at least one operating system deployed on it, and each operating system has a computing power service deployed on it. Each operating system also has at least one model deployed on it. Among all the computing power services deployed on the multiple computing power nodes, one computing power service is designated as the master computing power service, and the remaining computing power services are designated as slave computing power services. Correspondingly, the computing power registration method includes: upon receiving a node registration message from a slave computing power service, sending a first confirmation message to the slave computing power service; adding the computing power node where the slave computing power service is deployed to the computing power node list; and upon receiving a node capability message from the slave computing power service, sending a second confirmation message to the slave computing power service. Therefore, by employing the above technical solution, deploying computing power services on each operating system of each computing power node, and setting up master and slave computing power services, communication between different operating systems and different computing power nodes can be achieved through computing power services, laying the foundation for cross-system model calls and cross-node model calls. Furthermore, the computing power registration mechanism has been updated to enable each computing power node to be discovered and registered, laying the foundation for the main computing power service to grasp the distribution of computing power resources of the whole vehicle and to schedule the computing power resources of the whole vehicle. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0011] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a computing power registration method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a vehicle-side model deployment framework provided in an embodiment of this disclosure; Figure 3 This is a logical schematic diagram illustrating the working process of an NN adapter provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram illustrating the interaction between the master computing power service and the slave computing power service during the computing power registration process provided in this embodiment of the disclosure; Figure 5 This is a flowchart of a computing power node during the computing power registration process provided in this embodiment of the disclosure; Figure 6 This is a flowchart of the main computing power node in the computing power registration process provided in this embodiment of the disclosure; Figure 7 This is a schematic diagram of the structure of a computing power registration device provided in an embodiment of this disclosure. Detailed Implementation

[0013] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0014] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0015] In current vehicles, each functional module (such as autonomous driving and smart cockpit) is typically handled by an independent Electronic Control Unit (ECU). Each ECU's Operating System (OS) pre-deploys the models required by its own OS. When the vehicle is running, the OS on each ECU calls the models deployed on its own OS. Each OS manages its own responsible models, and the OSes and ECUs operate independently. However, this model deployment method is relatively rigid, leading to some ECUs having excessive computing power while others lack sufficient computing power. This results in uneven utilization of the vehicle's computing resources. For example, some ECUs operate at low load most of the time, with a large amount of their computing resources idle; when some ECUs need to perform complex inference tasks, their local computing power is insufficient to support high-precision, low-latency model operation, leading to performance degradation or even functional limitations. Furthermore, the vehicle's computing resources cannot be dynamically expanded to more flexibly meet the needs of different application scenarios. Therefore, there is an urgent need to introduce a new model deployment framework and update the computing power registration mechanism accordingly. In view of this, this disclosure provides a computing power registration method, apparatus, medium, and vehicle. The following section will first provide a detailed explanation of the computing power registration method.

[0016] Figure 1 This is a flowchart of a computing power registration method provided in an embodiment of this disclosure, which can be executed by a vehicle-side model deployment framework. Figure 1 As shown, the method provided in this embodiment includes the following steps: S110. Upon receiving the node registration message sent by the computing power service, send a first confirmation message back to the computing power service.

[0017] S120: Add the computing nodes where the computing power service is deployed to the computing node list.

[0018] S130. Upon receiving the node capability message sent from the computing power service, send a second confirmation message back to the computing power service.

[0019] To more clearly illustrate the computing power registration method provided in this disclosure, the vehicle-side model deployment framework will first be described in detail below. For example, Figure 2 This is a schematic diagram of the structure of a vehicle-side model deployment framework provided in an embodiment of this disclosure, such as... Figure 2 As shown, the vehicle-side model deployment framework includes: multiple computing power nodes, each computing power node having at least one operating system deployed on it, each operating system having a computing power service deployed on it, and the operating system having a model deployed on it. Among all the computing power services deployed on the multiple computing power nodes, one computing power service is the master computing power service, and the remaining computing power services are slave computing power services.

[0020] Specifically, a computing node is an ECU deployed on a vehicle within a vehicle-side model deployment framework, capable of performing inference tasks.

[0021] In some examples, all computing nodes are embedded computing nodes, where embedded computing nodes refer to computing nodes integrated into the vehicle. The main computing service is configured within the embedded computing nodes.

[0022] In other examples, the multiple computing nodes in the vehicle-side model deployment framework include embedded computing nodes and pluggable computing nodes, where pluggable computing nodes refer to external computing nodes that can be hot-swapped via a computing interface (i.e., a physical interface).

[0023] Specifically, pluggable computing nodes can include AIBOX, but are not limited to these. The computing power service in a pluggable computing node is a slave computing power service. The node registration message is sent after the pluggable computing node is inserted into the computing power interface to dynamically connect to the vehicle-side model deployment framework.

[0024] Understandably, in related technologies, the computing nodes on a vehicle are fixed and cannot be dynamically added, thus preventing the supplementation of the vehicle's overall computing power. This disclosure, through a customized plug-and-play protocol, incorporates pluggable computing nodes into the vehicle's on-board model deployment framework via a network (such as TCP or IP), enabling dynamic expansion of the vehicle's computing power and providing more flexible support for different application scenarios.

[0025] Specifically, a computing power service is a service that runs on computing power nodes, exposes itself through standardized interfaces, and provides computing capabilities to applications. Different operating systems and different computing power nodes can communicate through the computing power service.

[0026] Optional, such as Figure 2As shown, the computing power service includes: a session management module, which is used to manage the entire lifecycle of the session to which the model uses messages, as well as the storage and querying of session data; The model management module is used to manage the entire lifecycle of models and recover from anomalies. The memory management module is used to manage the allocation of memory resources during the execution of inference tasks by the model. Optionally, it is also used to manage the sharing mechanism of the model during the execution of inference tasks. The scheduling and management module is used to manage and control the computing resources of the computing power nodes on which it is deployed. In the case that its computing power service is the main computing power service, it is also used to schedule the use of computing power resources of the computing power nodes on which the secondary computing power service is deployed. The protocol management module is used to communicate with other computing power services based on a preset network, and also to communicate with applications. NN adapters are used to load and adapt models to the hardware computing platform of computing nodes.

[0027] Specifically, the session management module is responsible for the full lifecycle management of sessions initiated by applications using model messages (such as the full lifecycle management of sessions in scenarios with concurrent access from multiple applications), supporting session creation, retrieval, binding, and destruction. Message isolation and context association are achieved through a unique session ID. It also provides a session state persistence mechanism, supporting the storage and retrieval of historical session data to meet functional safety and diagnostic traceability requirements.

[0028] Specifically, the model management module implements full lifecycle management of models, covering model loading and inference management, and also including model deployment, uninstallation, and version management. It supports unified access to multiple model formats such as ONNX, TensorRT, and TFLite, and adapts to hardware computing platforms through the NN adapter. It features model anomaly detection and fault-tolerant recovery mechanisms, supporting automatic reloading or switching to a backup model in scenarios such as inference failure, hardware anomalies, or memory overflows, ensuring service continuity and system robustness.

[0029] Specifically, the memory management module oversees the allocation and sharing of memory resources during the model inference process. This sharing mechanism enables zero-copy data transfer across systems and nodes.

[0030] Specifically, the scheduling management module is responsible for controlling the use of computing resources. Each computing service's scheduling management module can decide when to execute inference tasks based on factors such as the computing resource usage of the computing nodes on which the service is deployed, inference task priorities, and / or model priorities. It supports various scheduling strategies, including immediate execution, delayed execution, pause / resume, and degraded execution. Sub-computing services can forward model usage messages to the master computing service to initiate computing resource scheduling requests. The master computing service can then make scheduling decisions based on global computing resources, achieving optimal utilization and load balancing of computing resources.

[0031] Specifically, the protocol management module defines and implements a customized communication protocol stack. It enables communication between various computing power services based on a pre-defined network (such as a TCP / IP network), and also enables communication between computing power services and applications. The protocol system covers node registration protocols, model usage message protocols, and / or computing power scheduling protocols. For example, the protocol system includes: a node registration protocol (automatically registering computing power when a computing power node starts up); a model usage message protocol (the application initiates model usage messages); and a computing power scheduling protocol (negotiating resource scheduling between master and slave computing power services).

[0032] Specifically, the computing node integrates a hardware computing platform. The NN adapter, through hardware abstraction and interface standardization, shields the underlying differences of the hardware computing platform (such as NPU, GPU, and / or DSP, etc.) and provides a unified and portable neural network inference interface for upper-layer applications, thereby loading and adapting the model to the hardware computing platform of the computing node.

[0033] For example, Figure 3 This is a logical diagram illustrating the working process of an NN adapter provided in an embodiment of this disclosure. Figure 3As shown, the NN adapter may include QNN Adaptor, TensorRT Adaptor, and / or Other Vendor Adaptor (i.e., any other vendor adapter known to those skilled in the art). The parts of the neural network framework (NN Framework) corresponding to the QNN Adaptor include: PyTorch, TF-lite, ONNX, QCOM Neural Processing SDK, and QCOM AI Engine Direct API (including NEON, HMX, HVX, and OpenCL). PyTorch and TF-lite are model training and lightweight frameworks, ONNX is a unified intermediate representation format for models, QCOM Neural Processing SDK is a model deployment toolchain, and Qualcomm AI Engine Direct API is a hardware-level acceleration interface. The hardware (HW) corresponding to the QNN Adaptor includes CPU, HTP, cDSP, and GPU. Accordingly, the model deployment process is as follows: the model is trained in the cloud using PyTorch or TF-lite and exported to ONNX; the QCOM Neural Processing SDK converts and optimizes the model; and finally, the model is deployed on the computing nodes. The model usage process is as follows: The QNN Adaptor loads and initializes the model, and calls the QCOM AI Engine Direct API to map different operators of the model to the corresponding hardware computing platforms for inference. The components of the neural network framework corresponding to the TensorRT Adaptor include: PyTorch, TensorFlow, NVIDIA TensorRT SDK, and NVIDIA CUDA. PyTorch and TensorFlow are the model training frameworks, the NVIDIA TensorRT SDK is the model deployment toolchain, and NVIDIA CUDA is the parallel computing interface. The hardware corresponding to the TensorRT Adaptor includes GPUs and DLAs. Accordingly, the model deployment process is as follows: The model is trained in the cloud using PyTorch or TensorFlow and exported as ONNX or TorchScript. The NVIDIA TensorRT SDK transforms and optimizes the model, and finally, the model is deployed on the computing nodes. The model usage process is as follows: The QNN Adaptor loads and initializes the model, and calls NVIDIA CUDA to map different operators of the model to the corresponding hardware computing platforms for inference.The parts of the neural network framework corresponding to OtherVendor Adaptor include: Vendor NPU SDK and Vendor GPU SDK, which are toolchains corresponding to OtherVendor Adaptor and correspond to any other matching hardware computing power platform known to those skilled in the art (i.e., other NN HW).

[0034] It is understood that the computing power service provided in this disclosure abstracts and encapsulates the underlying hardware computing power platform, shields the differences between different hardware architectures, provides a unified and standardized interface for calling computing power resources to upper-layer applications, decouples application logic from the hardware computing power platform, and ensures that applications do not need to be aware of the type of underlying computing power resources when calling the model, thus providing technical support for realizing cross-system calling of the model.

[0035] Specifically, in some examples, at least one model is deployed on each operating system; in other examples, at least one model is deployed on some operating systems while no model is deployed on others. This disclosure does not limit this.

[0036] In some examples, the model usage messages received by the main computing power service include: a first type of model usage message, which refers to model usage messages sent by applications deployed on the computing power nodes on which the computing power service is located; and a second type of model usage message, which refers to model usage messages sent by applications not on computing power nodes. The model usage messages received from the computing power service include the first type of model usage messages. That is, the second type of model usage requests initiated by applications not on computing power nodes are sent directly to the main computing power service.

[0037] Specifically, non-computing nodes refer to ECUs that do not have the ability to perform inference tasks. They are usually mainly responsible for signal acquisition, simple control, communication relay, or executing fixed logic.

[0038] Specifically, for the main computing power service, it can receive model usage messages (i.e., the first type of model usage messages) sent by applications deployed on the same computing power node as it, and it can also receive model usage messages (i.e., the second type of model usage messages) sent by applications on non-computing power nodes; for the slave computing power service, it can receive model usage messages sent by applications deployed on the same computing power node as it.

[0039] It is understandable that the main computing power service can schedule the use of computing power resources on the computing power nodes where the secondary computing power service is deployed. Therefore, by setting the main computing power service to receive the second type of model usage messages, it is possible to quickly determine the computing power nodes that respond to the second type of model usage messages, which helps to improve the response speed to model usage messages.

[0040] In other examples, each computing power service can receive model usage messages from applications deployed on the same computing power node as it, and can also receive model usage messages from applications on non-computing power nodes.

[0041] Specifically, the node registration message is a message sent by the computing power service to the main computing power service when the computing power nodes it deploys are started, restored, or have a change in status. It is used to declare its own existence, report model deployment information, and request access to the vehicle-side model deployment framework.

[0042] Specifically, there are several ways to set the node registration message. The following is an explanation of a typical example.

[0043] Optionally, the node registration message includes a first message header and a first message body; The first message header includes the command type, session ID, role of the requesting node, byte length of the message body, and checksum of the entire message; The first message body includes node registration information, which includes: the node's role, the node's device type, the type of operating system deployed, publicly available services, the node's network address, the parent node's network address, and a model list, which includes the model ID and model type of the deployed models.

[0044] Specifically, the command type indicates the intent of this communication. For example, the command type can be represented by the command name and / or command ID.

[0045] For example, the definitions of various types of commands are shown in Table 1 below, but are not limited thereto. According to Table 1, the command type in the model usage message and the new model usage message can be represented by AI_MODEL_EXTERNA_CMD_REQUEST_REMOTE and / or 0x0007.

[0046] Table 1

[0047] Specifically, the session ID is used to identify a complete interaction process. It is empty for the first request and is returned after the request node is generated. Subsequent requests carry this ID to indicate that they belong to the same interaction process.

[0048] Specifically, the role of a request node represents its function within the vehicle-side model deployment framework. A request node refers to the computing power node that receives messages. For example, the role of a request node can be represented by a role name and / or a role ID.

[0049] For example, the various roles of the request node are defined as shown in Table 2 below, but are not limited thereto.

[0050] Table 2

[0051] Specifically, the byte length of the message body data refers to the byte length of the message body portion.

[0052] Specifically, the checksum of the entire message is used to verify whether data corruption has occurred during transmission. For example, the checksum may include a CRC checksum, but is not limited to this.

[0053] Specifically, the role of this node represents its role in the vehicle-side model deployment framework, where "this node" refers to the computing power node that sends messages. For example, the role of this node can be represented by a role name and / or role ID. The definitions of various roles for this node are shown in Table 2 above, but are not limited to these.

[0054] Specifically, device type refers to the type of computing node. For example, device type can be represented by device type name and / or device type ID, but is not limited to this.

[0055] For example, the definitions of various device types are shown in Table 3 below, but are not limited thereto.

[0056] Table 3

[0057] Specifically, the type of deployed operating system refers to the type of operating system running on the computing node. For example, the type of deployed operating system can be represented by the operating system name and / or operating system ID, but is not limited to this.

[0058] Specifically, "publicly available services" refers to whether a computing power node publicly provides services, allowing other computing power nodes to actively connect to it. For example, `true` can represent publicly available services, and `false` can represent not publicly available services, but this is not a limitation.

[0059] Specifically, the network address of this node refers to the network address of the computing power node that sent the message.

[0060] Specifically, the network address of the parent node refers to the network address of the computing power node that receives the message.

[0061] Specifically, the model list is used to describe the models deployed on the computing power nodes. The model list includes the model ID (i.e., the unique identifier of the model), model type (i.e. the type of model), and / or model name (i.e. the name of the model), etc. The model ID is shown in Table 4 below, and the model type is shown in Table 5 below, but is not limited to these.

[0062] Table 4

[0063] Table 5

[0064] For example, the node registration message is as follows: First message header: uses JSON format { "cmd":AI_MODEL_EXTERNAL_CMD_REGISTER, / / command type "session": "", / / session ID "role":MODEL_NODE_ROLE_SLAVER, / / The role requested for the node "length":492, / / Length of message body data in bytes "checksum": 3FFE / / The checksum of the entire message } First message body: { "nodeinfo": { "role": MODEL_NODE_ROLE_SLAVER, / / The role of this node "domain": "DHU", / / Device type of this node "os": "Linux", / / The type of operating system deployed "public": true, / / Publicly disclose service information "localaddress": "192.168.2.2:5051", / / The network address of this node "parentaddress": "192.168.2.1:5050", / / Parent node's network address "modelvobs": [ / / Model list] { "id": MODEL_LLM, / / The model ID of the first model "type": "MODEL_TYPE_SINGLE" / / The model type of the first model }, { "id": MODEL_ASR, / / The model ID of the second model "type": "MODEL_TYPE_SINGLE" / / The model type of the second model }, ], } Specifically, the node capability message is a message sent from the computing power service to the main computing power service after receiving the first confirmation message, used to report model capability information.

[0065] Specifically, there are several ways to configure node capability messages. The following is an explanation of a typical example.

[0066] Optionally, the node capability message includes a second message header and a second message body; The second message header includes the command type, session ID, role of the requesting node, byte length of the message body, and checksum of the entire message. The second message body includes node capability information, which includes: model index, model name, model type, model ID, priority, lifecycle management method, popularity index, context window size, input data volume, output data volume, and model functions.

[0067] Specifically, the model index is used to describe which model is in the model list.

[0068] Specifically, priority is used to describe the priority of a model. For example, the smaller the value, the higher the priority. A priority of 1 means that this is one of the highest priority models and its resources will be guaranteed first, but it is not limited to this.

[0069] Specifically, lifecycle management refers to the way a model's lifecycle is managed. For example, lifecycle management methods include Persistent (once the model is loaded, it remains in memory and is not easily unloaded), transient (loaded temporarily and released after use), and on-demand (loaded on demand), but are not limited to these.

[0070] Specifically, the popularity metric indicates the frequency of model usage. For example, a smaller popularity metric value indicates more frequent use, but this is not the only possibility.

[0071] Specifically, the context window size indicates how much historical data or time range the model needs to retain when performing inference tasks.

[0072] Specifically, the input data volume type represents the amount of input data required for each inference by the model.

[0073] Specifically, the output data volume refers to the amount of data output by the model.

[0074] Specifically, model functionality refers to the functions that a model possesses. For example, model functionality is shown in Table 6 below, but it is not limited to this.

[0075] Table 6

[0076] For example, a node capability message is as follows: Second message header: uses JSON format { "cmd":AI_MODEL_EXTERNAL_CMD_ABILITY, / / Command type "session": "", / / session ID "role":MODEL_NODE_ROLE_SLAVER, / / The role requested for the node "length":2145, / / Length of message body data in bytes "checksum": FJF2 / / Verification value of the entire message } Second message body: { "Model": { "item": [ { "index": "1", / / Model index "name": "esrgan", / / Model name "type": "single", / / Model type "id": "0x0001", / / Model ID "capacitypriority": "1", / / priority "lifecycle": "persistent", / / Lifecycle management method "temperature": "1", / / Heat index "contextwinsize": "4800", / / Context window size "input": "270000", / / Input data size "output": "1000", / / Output data size "capacity": { "name": "MODEL_CAPABILITY_SUPER_RESOLUTION" / / Model functionality } }, { "index": "2", / / Model index "name": "child_recognize", / / Model name "type": "single", / / Model type "id": "0x0002", / / Model ID "capacitypriority": "1", / / priority "lifecycle": "persistent", / / Lifecycle management method "temperature": "1", / / Heat index "contextwinsize": "4800", / / Context window size "input": "270000", / / Input data size "output": "1000", / / Output data size "capacity": { "name": "MODEL_CAPABILITY_FACE_RECOGNITION" / / Model functionality } }, { "index": "3", / / Model index "name": "TextGeneratedImages", / / Model name "type": "single", / / Model type "id": "0x0003", / / Model ID "capacitypriority": "1", / / priority "lifecycle": "persistent", / / Lifecycle management method "temperature": "1", / / Heat index "contextwinsize": "4800", / / Context window size "input": "270000", / / Input data size "output": "1000", / / Output data size "capacity": [ { "name": "MODEL_CAPABILITY_TEXT_2_IMAGE" / / Model functionality }, { "name": "MODEL_CAPABILITY_IMAGE_2_TEXT" } ] }, { "index": "4", / / Model index "name": "dms", / / Model name "type": "algorithm", / / Model type "id": "0x0004", / / Model ID "capacitypriority": "1", / / priority "lifecycle": "persistent", / / Lifecycle management method "temperature": "1", / / Heat index "contextwinsize": "4800", / / Context window size "input": "270000", / / Input data size "output": "1000", / / Output data size "capacity": { "name": "MODEL_CAPABILITY_DMS" / / Model functionality } }, { "index": "5", / / Model index "name": "LLM", / / Model name "type": "single", / / Model type "id": "0x0005", / / Model ID "capacitypriority": "1", / / priority "lifecycle": "persistent", / / Lifecycle management method "temperature": "1", / / Heat index "contextwinsize": "4800", / / Context window size "input": "270000", / / Input data size "output": "1000", / / Output data size "capacity": { "name": "MODEL_CAPABILITY_DIALOG" / / Model functionality } }, ] } } Specifically, the first confirmation message is used to inform the computing power service that the node registration information has been successfully received, and the second confirmation message is used to inform the computing power service that the node capability information has been successfully received.

[0077] Specifically, there are several ways to set the first and second confirmation messages. The following is an explanation of a typical example.

[0078] Optionally, both the first and second confirmation messages include a third message header, which includes the command type, session ID, byte length of the message body, checksum of the entire message, and status code.

[0079] For example, the first confirmation message is as follows: Third message header: uses JSON format { "cmd": AI_MODEL_EXTERNAL_CMD_REGISTER, / / command type "session": " ", / / Session ID, defaults to empty "errorcode":0, / / Status code "length":0 / / Length of the message body in bytes "checksum":0 / / Verification value for the entire message } Understandably, by setting up node registration messages, node capability messages, first confirmation messages, and second confirmation messages, including message headers and message bodies, message parsing is made easier. Reading the message header first reveals the purpose, which computing node should process it, and the amount of data, before processing the message body. Furthermore, by setting the roles of requesting nodes, access control, resource isolation, and priority scheduling can be implemented based on these roles. Setting session IDs distinguishes different interaction processes, avoids data confusion, and ensures each session is processed independently. Setting the byte length of the message body data helps to clearly define boundaries and prevent packet fragmentation. Finally, setting checksums ensures reliable data transmission and prevents accidental operations.

[0080] Optionally, node registration messages are periodically sent from the computing power service until a first confirmation message is received from the computing power service; and / or, node capability messages are periodically sent from the computing power service until a second confirmation message is received from the computing power service.

[0081] Understandably, the computing power service uses the "send node registration message - confirm receipt of the first confirmation message + retry if not received" mode and the "send node capability message - confirm receipt of the second confirmation message + retry if not received" mode to ensure that the registration process can still succeed in the event of abnormal situations such as main computing power service initialization delay, network jitter or temporary failure.

[0082] Optionally, the method further includes: summarizing the node registration information of all computing power nodes to obtain first summary information, and summarizing the node capability information of all computing power nodes to obtain second summary information; The first and second summary information are distributed to each computing power service.

[0083] Understandably, by setting the master computing service to distribute the first and second summary information to each slave computing service, information synchronization can be achieved between the master and slave computing services, providing reference information for cross-system model calls and cross-node model calls.

[0084] Optionally, when the application sends a model usage message, the first target computing power service that receives the model usage message forwards the model usage message to the second target computing power service if the preset forwarding conditions are met. The second target computing power service loads the target model and uses the target model to execute the inference task corresponding to the model usage message. The target model refers to the model indicated by the model usage message. The preset forwarding conditions include: the first target computing power service and the target model are deployed on different operating systems and / or the computing power resources on the computing power node where the first target computing power service is deployed do not meet the preset computing power conditions.

[0085] In one example, the preset forwarding conditions include: the first target computing power service and the target model are deployed on different operating systems. Correspondingly, the second target computing power service and the target model are deployed on the same operating system.

[0086] Specifically, when the computing power service (i.e., the first target computing power service) receives a model usage message, it can determine whether the target model and the first target computing power service are deployed on the same operating system based on the first summary information and / or the second summary information. Furthermore, when the target model and the first target computing power service are deployed on different operating systems, the second target computing power service can be located based on the first summary information and / or the second summary information.

[0087] Understandably, by configuring the first target computing service and the target model to be deployed on different operating systems, the model uses message forwarding to the second target computing service, which then completes the target model loading and inference tasks. This avoids cross-operating system compatibility barriers, ensures reliable response to requests, and improves the success rate and response efficiency of model calls. Furthermore, this cross-operating system model call mechanism enables model sharing and reuse across different operating systems, avoiding redundant model deployment and resource idleness, thereby significantly improving model utilization efficiency.

[0088] In other examples, the preset forwarding conditions include: the first target computing power service and the target model are deployed on different operating systems, and the computing power resources on the computing power node where the first target computing power service is deployed do not meet the preset computing power conditions. Correspondingly, the second target computing power service and the target model are deployed on the same operating system, and the computing power resources on the computing power node where the second target computing power service is deployed meet the preset computing power conditions.

[0089] Specifically, the computing power service can obtain the computing power resource information of the computing power nodes on which it is deployed. The computing power resource information includes at least one of the following: the used computing power resources, the remaining computing power resources, and the total computing power resources of the computing power node.

[0090] Specifically, there are various ways to set up the "first target computing power service detects whether the computing power node it is deployed on meets the preset computing power conditions". The following are typical examples, but they do not constitute a limitation of this disclosure.

[0091] Optionally, the first target computing power service detects whether the computing power node it is deployed on meets the preset computing power conditions, including: the first target computing power service obtains the used computing power resources of the computing power node it is deployed on, and determines the load level corresponding to the used computing power resources of the computing power node; if the load level corresponding to the used computing power resources of the computing power node is greater than the preset load level threshold, the first target computing power service determines that the computing power node does not meet the preset computing power conditions. Further optional, the method includes: if the load level corresponding to the used computing resources of the computing power node is less than or equal to a preset load level threshold, the first target computing power service determines the computing power resources required by the model, and calculates the sum of the computing power resources required by the model and the used computing power resources of the computing power node to obtain new used computing power resources, wherein the computing power resources required by the model refer to the computing power required to load the target model and the target model to execute the current inference task; the first target computing power service determines the load level corresponding to the new used computing power resources of the computing power node; if the load level corresponding to the new used computing power resources of the computing power node is greater than the preset load level threshold, the first target computing power service determines that the computing power node does not meet the preset computing power conditions; if the load level corresponding to the new used computing power resources of the computing power node is less than or equal to the preset load level threshold, the first target computing power service determines that the computing power node meets the preset computing power conditions.

[0092] Specifically, when the computing power service (i.e. the first target computing power service) receives the model usage message, it can determine whether the preset computing power conditions are met based on the computing power resource information of the computing power node on which it is deployed, and it can also determine whether the target model and the first target computing power service are deployed on the same operating system.

[0093] Optionally, the main computing power service is also used to receive computing power resource information of computing power nodes reported by each slave computing power service. The main computing power service is also used to summarize the computing power resource information of all computing power nodes to obtain summary computing power resource information, and then distribute the summary computing power resource information to each slave computing power service.

[0094] Specifically, for each slave computing service, it can report the computing resource information of the computing nodes it is deployed on to the master computing service through the protocol management module. The master computing service can summarize the computing resource information of each computing node to obtain summary computing resource information, and then distribute the summary computing resource information to each slave computing service through the protocol management module to achieve information sharing. Thus, when a computing service (i.e., the first target computing service) receives a model usage message, if the first target computing service and the target model are deployed on different operating systems and the computing resources on the computing nodes where the first target computing service is deployed do not meet the preset computing resource conditions, it can filter for a second target computing service based on the model deployment summary information and computing resource information. Of course, the master computing service can also be used to filter for a second target computing service based on the model deployment summary information and computing resource information, and then send the filtering results to the first target computing service.

[0095] Understandably, by configuring the model to be forwarded to the second target computing service if the first target computing service and the target model are deployed on different operating systems and the computing resources on the computing node where the first target computing service is deployed do not meet the preset computing power conditions, the second target computing service can complete the target model loading and inference task execution. This avoids cross-operating system compatibility barriers and prevents avalanche caused by insufficient computing power on the computing node, thus improving the success rate and response efficiency of model invocation. Furthermore, this cross-node model invocation mechanism enables model sharing and reuse across different operating systems, avoiding duplicate model deployment and resource idleness, thereby significantly improving model utilization efficiency. On the other hand, it enables unified scheduling of computing resources for the entire vehicle to achieve vehicle-wide computing power sharing, allowing high-computing-power nodes to take over inference tasks from other computing nodes during idle periods, thereby improving the overall utilization rate of vehicle computing resources.

[0096] In this embodiment, a computing service is deployed on each operating system of each computing node, and a master computing service and a slave computing service are set up. This enables communication between different operating systems and different computing nodes through the computing service, laying the foundation for cross-system model calls and cross-node model calls. Furthermore, the computing registration mechanism has been updated to enable each computing node to be discovered and registered, laying the foundation for the master computing service to understand the distribution of computing resources throughout the vehicle and to schedule these resources.

[0097] The computing power registration method provided in this disclosure will be described in detail below with reference to a specific example. Figure 4 This is a schematic diagram illustrating the interaction between the master computing power service and the slave computing power service during the computing power registration process, provided in an embodiment of this disclosure. Figure 5 This is a flowchart illustrating the computing power registration process from a computing power node, provided in an embodiment of this disclosure. Figure 6This is a flowchart of a main computing power node during the computing power registration process provided in this embodiment of the disclosure. Figures 4-6 As shown, for each slave computing service, after the computing node it is deployed on starts up, it periodically sends a node registration message to the master computing service until it receives a first confirmation message from the master computing service, and then sends a node capability message to the master computing service. For the master computing service, after the computing node it is deployed on starts up, it listens for node registration messages. When it receives a node registration message from a slave computing service, it sends a first confirmation message to the slave computing service and adds the computing node where the slave computing service is deployed to the computing node list; it receives the node capability message sent by the slave computing service and sends a second confirmation message to the slave computing service. It should be noted that if the computing power node on which the computing power service is deployed is a pluggable computing power node, before periodically sending node registration messages to the main computing power service until receiving the first confirmation message from the main computing power node, the process also includes: sending a node join broadcast message to the main computing power service after inserting the computing power interface until receiving the join confirmation message from the main computing power service. Accordingly, when the main computing power service receives the node join broadcast message from the pluggable computing power node, it parses the node information (including network address, etc.) of the pluggable computing power node from the node join broadcast message, performs authentication based on the node information of the pluggable computing power node, and then, after successful authentication, sends a join confirmation message (including network address, etc.) to the slave computing power service deployed on the pluggable computing power node.

[0098] Figure 7 This is a schematic diagram of a computing power registration device provided in an embodiment of this disclosure. This computing power registration device can be understood as the aforementioned vehicle-side model deployment framework or a functional module within the aforementioned vehicle-side model deployment framework. For example... Figure 7 As shown, the computing power registration device includes: The first feedback module 710 is used to send a first confirmation message to the computing power service upon receiving the node registration message sent by the computing power service. The first addition module 720 is used to add the computing node on which the computing power service is deployed to the computing node list; The second feedback module 730 is used to send a second confirmation message to the computing power service upon receiving the node capability message sent by the computing power service.

[0099] Optionally, the node registration message includes a first message header and a first message body; The first message header includes the command type, session ID, role of the requesting node, byte length of the message body information, and checksum of the entire message; The first message body includes: the role of this node, the device type of this node, the type of the operating system deployed, the publicly available services, the network address of this node, the network address of the parent node, and a model list, wherein the model list includes the model ID and model type of the deployed models.

[0100] Optionally, the node capability message includes a second message header and a second message body; The second message header includes the command type, session ID, role of the requesting node, byte length of the message body, and checksum of the entire message; The second message body includes: model index, model name, model type, model ID, priority, lifecycle management method, popularity index, context window size, input data volume, output data volume, and model function.

[0101] Optionally, both the first confirmation message and the second confirmation message include a third message header, which includes the command type, session ID, byte length of the message body information, checksum of the entire message, and status code.

[0102] Optionally, the device further includes: a first aggregation module, used to aggregate the node registration information of all the computing power nodes to obtain first aggregation information, and to aggregate the node capability information of all the computing power nodes to obtain second aggregation information; The first summary information and the second summary information are distributed to each of the computing power services.

[0103] Optionally, the plurality of computing nodes include: An embedded computing node, wherein the main computing service is configured in the embedded computing node; A pluggable computing node, wherein the computing service in the pluggable computing node is the slave computing service, and the node registration message is sent after the pluggable computing node is inserted into the computing interface.

[0104] Optionally, the node registration message is sent periodically by the slave computing power service until the slave computing power service receives the first confirmation message.

[0105] The apparatus provided in this embodiment can execute the methods of any of the above embodiments, and its execution method and beneficial effects are similar, so they will not be described again here.

[0106] This disclosure also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the methods of any of the above embodiments. The execution method and beneficial effects are similar, and will not be described again here.

[0107] This disclosure also provides a vehicle including the apparatus described in any of the above embodiments and / or the computer-readable storage medium described in any of the above embodiments, which will not be described again here.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A computing power registration method, characterized in that, An application is provided to a vehicle-side model deployment framework, which includes multiple computing power nodes. Each computing power node has at least one operating system deployed on it, and each operating system has a computing power service deployed on it. At least one model is deployed on each operating system. Of all the computing power services deployed on the multiple computing power nodes, one is a master computing power service, and the remaining computing power services are slave computing power services. The method is executed by the master computing power service and includes: Upon receiving the node registration message sent by the computing power service, a first confirmation message is sent back to the computing power service; Add the computing node on which the computing power service is deployed to the computing node list; Upon receiving the node capability message sent by the computing power service, a second confirmation message is sent back to the computing power service.

2. The method according to claim 1, characterized in that, The node registration message includes a first message header and a first message body; The first message header includes the command type, session ID, role of the requesting node, byte length of the message body information, and checksum of the entire message; The first message body includes: the role of this node, the device type of this node, the type of the operating system deployed, the publicly available services, the network address of this node, the network address of the parent node, and a model list, wherein the model list includes the model ID and model type of the deployed models.

3. The method according to claim 1, characterized in that, The node capability message includes a second message header and a second message body; The second message header includes the command type, session ID, role of the requesting node, byte length of the message body, and checksum of the entire message; The second message body includes: model index, model name, model type, model ID, priority, lifecycle management method, popularity index, context window size, input data volume, output data volume, and model function.

4. The method according to claim 1, characterized in that, Both the first confirmation message and the second confirmation message include a third message header, which includes the command type, session ID, byte length of the message body information, checksum of the entire message, and status code.

5. The method according to claim 1, characterized in that, in, The node registration message includes node registration information, the node capability message includes node capability information, and the method further includes: The node registration information of all the computing power nodes is summarized to obtain the first summary information, and the node capability information of all the computing power nodes is summarized to obtain the second summary information. The first summary information and the second summary information are distributed to each of the computing power services.

6. The method according to claim 1, characterized in that, The plurality of computing power nodes include: An embedded computing node, wherein the main computing service is configured in the embedded computing node; A pluggable computing node, wherein the computing service in the pluggable computing node is the slave computing service, and the node registration message is sent after the pluggable computing node is inserted into the computing interface.

7. The method according to claim 1, characterized in that, The node registration message is sent periodically by the slave computing power service until the slave computing power service receives the first confirmation message.

8. A computing power registration device, characterized in that, An application is provided for a vehicle-side model deployment framework, the framework comprising multiple computing power nodes, each computing power node deploying at least one operating system, each operating system deploying a computing power service, and each operating system deploying at least one model. Of all the computing power services deployed on the multiple computing power nodes, one is a primary computing power service, and the remaining computing power services are secondary computing power services. The device is integrated into the primary computing power service and includes: The first feedback module is used to send a first confirmation message to the computing power service upon receiving the node registration message sent by the computing power service. The first addition module is used to add the computing node on which the computing power service is deployed to the computing node list; The second feedback module is used to send a second confirmation message to the computing power service upon receiving the node capability message sent by the computing power service.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

10. A vehicle, characterized in that, include: The computing power registration device of claim 8 or the computer-readable storage medium of claim 9.

Images