Field computer peripheral loading method and operating system generating method
By loading the drivers of computing peripherals on the operating system platform, hardware combinations of different modalities are realized, which solves the problem of the inability to handle the underlying operation of the operating system in the prior art, and improves the flexibility and adaptability of the computer system.
Patent Information
- Application Number
- CN202411130500.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-03
AI Technical Summary
The existing technology cannot effectively solve the dynamic combination of infrastructure level and the flexible matching scheduling of different hardware, and cannot handle the underlying operations of the operating system, especially to realize customized application scenarios in container clusters.
By loading the drivers of computing peripherals (GPU, NPU, DSP, FPGA, etc.) on the operating system platform adapted to the on-site computer's processor system, use peripheral management components and communication components to obtain drivers from the network server, and in Hardware combinations of different modes are implemented in the operating system to match the dynamic combination of differentiated infrastructure levels in the field.
It realizes flexible dynamic combination of on-site computer peripherals and enhances computing capabilities, and improves the flexibility and adaptability of the computer system through graphical monitoring and dynamic unloading/loading of drivers.
Smart Images

Figure CN119065728A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of operating systems, and in particular to an on-site computer peripheral loading method and an operating system generation method. Background Art
[0002] Existing technologies are based on container clusters and achieve customized or scenario-specific application scenarios through operations at the application software level. They cannot solve the dynamic combination at the infrastructure level, the flexible matching and scheduling of different hardware, or the lower-level operations of the operating system. Summary of the invention
[0003] In view of this, the embodiment of the present application provides a method for loading peripherals of a field computer and a method for generating an operating system, which loads the driver of computing peripherals (GPU, NPU, DSP, FPGA, etc.) in the computing field on the operating system platform adapted by the system of the processor of the field computer, realizes hardware combinations of different modes in the operating system, and matches the dynamic combination of the differentiated infrastructure level on site. The embodiment of the present application also provides a field computer, an operating system generating device thereof, and a field computer system accordingly.
[0004] In a first aspect, an embodiment of the present application provides a method for loading computer peripherals on site, comprising: obtaining the modality of the on-site computer, the modality including the system of the processor of the on-site computer and the system of peripherals used for computing, the peripherals including at least one of the following: GPU, NPU, DSP, FPGA; installing an operating system generated based on the system of the processor on the on-site computer, the components of the operating system including at least a peripheral management component and a communication component; the peripheral management component obtains a driver corresponding to the system of the peripheral from a network server through the communication component, and installs and starts the driver to load the computing peripheral.
[0005] As described above, the drivers of computing peripherals (GPU, NPU, DSP, FPGA, etc.) are loaded on the operating system platform that has been adapted to the system of the processor of the on-site computer, and hardware combinations of different modes are implemented in the operating system to match the dynamic combination of the differentiated infrastructure level on site.
[0006] In a possible implementation manner of the first aspect, it also includes: the peripheral management component obtains computing power for the on-site computer from a network server.
[0007] From the above, computing power is obtained from the network server through the peripheral management component to enhance the computing power of the on-site computer.
[0008] In a possible implementation of the first aspect, when multiple field computers are connected to the network, it also includes: the peripheral management component of the second field computer obtains the driver of the peripheral installed on the second field computer from the first field computer through the communication component of the field computer, and installs and starts the driver; and / or the peripheral management component of the second field computer obtains the computing power of the peripheral used for computing of the first field computer through the communication component of the field computer.
[0009] From the above, when the network server is removed, a field computer obtains the driver program from other field computers through its peripheral management component used for computing, and further implements hardware combinations of different modes in the operating system to match the basis of field differentiation.
[0010] In a possible implementation of the first aspect, it also includes: the peripheral management component sends its peripheral status to the collaborative management platform or the network server, so that the collaborative management platform can graphically monitor the peripherals of the on-site computer, wherein when the peripheral management component sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
[0011] From the above, the peripherals of the on-site computers are graphically monitored through the collaborative management platform, so that the peripherals of the on-site computers can be flexibly and dynamically combined.
[0012] In a possible implementation of the first aspect, it also includes: when any peripheral of the on-site computer is removed, the peripheral management component is also used to uninstall the driver of the peripheral; and / or when the on-site computer installs a new peripheral, the peripheral management component of the on-site computer obtains the driver corresponding to the system of the peripheral from a network server or other on-site computers, and installs and starts the driver to load the peripheral.
[0013] As described above, by uninstalling the driver of the removed peripheral and loading the driver of the newly installed peripheral through the peripheral management component, the dynamic combination of the peripherals of the on-site computer can be performed more flexibly.
[0014] In a second aspect, an embodiment of the present application provides a method for generating an operating system for an on-site computer, wherein the operating system is generated based on the system of the processor of the on-site computer, and its components include at least a peripheral management component and a communication component. The peripheral management component obtains a driver for a peripheral used for computing installed on the on-site computer from a network server through the communication component, and starts the driver. The peripheral includes at least one of the following: GPU, NPU, DSP, FPGA; the method includes: the IDE of the on-site computer adapts the source code of the boot boot, kernel and RAM root directory system of the operating system according to the system of the processor of the on-site computer, and copies the adapted source code to a specified directory on the IDE; the IDE compiles the adapted source code, the source code of the peripheral management component and the source code of the communication component according to the system of the processor of the on-site computer and the configuration template of the operating system to obtain the installation program of the operating system, and compiles the source code of the peripherals of the on-site computer to obtain the corresponding driver, and the configuration template includes the configuration files of the basic components of the operating system, the peripheral management component and the communication component; the IDE uploads the installation program and the driver to the network server.
[0015] Based on the above, the operating system's related source code and driver's source code are adapted and compiled according to the processor system of the on-site computer, and the operating system's installation program, peripheral management components, and communication components are generated to implement hardware combinations of different modes in the operating system to match the dynamic combination of the differentiated infrastructure level on site.
[0016] In a third aspect, an embodiment of the present application provides a field computer, wherein an operating system of the field computer is generated based on the system of the processor of the field computer, and its components include at least a peripheral management component and a communication component; the communication component is used for the field computer to connect to a network server via a network, and the peripheral management component is used to obtain a driver program for a peripheral used for computing installed on the field computer from the network server, and to install and start the driver program, wherein the peripheral includes at least one of the following: GPU, NPU, DSP, FPGA.
[0017] As described above, computing peripherals (GPU, NPU, DSP, FPGA, etc.) are loaded on the operating system platform that has been adapted to the system of the on-site computer's processor, and hardware combinations of different modes are implemented in the operating system to match the dynamic combination of the differentiated infrastructure level on site.
[0018] In a possible implementation of the third aspect, the peripheral management component is further used to obtain computing power for the on-site computer from a network server.
[0019] From the above, computing power is obtained from the network server through the peripheral management component to enhance the computing power of the on-site computer.
[0020] In a possible implementation of the third aspect, when multiple field computers are connected to the network, the peripheral management component of the second field computer is also used to obtain the driver program of the peripheral installed on the second field computer from the first field computer through the communication component of the field computer, and start the driver program; and / or the peripheral management component of the second field computer is also used to obtain the computing power of the peripheral used for computing of the first field computer through the communication component of the field computer.
[0021] From the above, when the network server is removed, a field computer obtains the driver program from other field computers through its peripheral management component, and further implements hardware combinations of different modes in the operating system to match the basis of field differentiation.
[0022] In a possible implementation of the third aspect, the peripheral management component is also used to send its peripheral status to the collaborative management platform or the network server, so that the collaborative management platform can graphically monitor the peripherals of the on-site computer, wherein when the peripheral management component sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
[0023] From the above, the peripherals of the on-site computers are graphically monitored through the collaborative management platform, so that the peripherals of the on-site computers can be flexibly and dynamically combined.
[0024] In a possible implementation of the third aspect, it also includes: when any peripheral of the on-site computer is removed, the peripheral management component is also used to uninstall the driver of the peripheral; and / or when the on-site computer installs a new peripheral, the peripheral management component of the on-site computer obtains the driver corresponding to the system of the peripheral from a network server or other on-site computers, and installs and starts the driver to load the peripheral.
[0025] As described above, the peripheral management component is also used to uninstall the driver of the removed peripheral and load the newly installed peripheral, so as to more flexibly perform the dynamic combination of the peripherals of the on-site computer.
[0026] In a fourth aspect, an embodiment of the present application provides an on-site computer operating system generation device, wherein the operating system is generated based on the system of the processor of the on-site computer, and its components include a peripheral management component and a communication component, wherein the peripheral management component obtains a driver for a peripheral used for computing installed on the on-site computer from a network server through the communication component, and installs and starts the driver, wherein the peripheral includes at least one of the following: GPU, NPU, DSP, FPGA; the device includes: an underlying technical framework for adapting the boot, kernel and R of the operating system on the IDE of the on-site computer according to the system of the processor of the on-site computer The source code of the AM root directory system is obtained by compiling the adapted source code, the source code of the peripheral management component and the source code of the communication component on the IDE according to the processor system of the on-site computer and the configuration template of the operating system, obtaining the installation program of the operating system, and compiling the source code of the peripherals of the on-site computer to obtain the corresponding driver program, uploading the installation program and the driver program to the network server, wherein the configuration template includes the configuration files of the basic components of the operating system, the peripheral management component and the communication component.
[0027] In a fifth aspect, an embodiment of the present application provides a field computer system, comprising: a field computer and a network server connected via a network; the operating system of the field computer is generated based on the system of the processor of the computer, and its components include a peripheral management component and a communication component; the communication component is used for the field computer to connect to the network server, and the peripheral management component is used to obtain the driver program of the peripheral device used for computing installed on the field computer from the network server, and install and start the driver program, and the peripheral device includes at least one of the following: GPU, NPU, DSP, FPGA; the network server includes an application warehouse, which is used to provide each field computer with its peripheral drivers and applications.
[0028] As described above, drivers for specialized peripherals (GPU, NPU, DSP, FPGA, etc.) are loaded on the operating system platform that has been adapted to the system of the on-site computer's processor, and hardware combinations of different modes are implemented in the operating system to match the dynamic combination of the differentiated infrastructure level on site.
[0029] In a possible implementation of the fifth aspect, the network server also includes one of the following: a container warehouse, used to provide the on-site computer with an image of a container that carries its application; an artificial intelligence warehouse, used to provide the on-site computer with artificial intelligence algorithm programs and models; an NPU module, used to provide the on-site computer with artificial intelligence computing power based on a virtual NPU; a robot control node, used to provide the on-site computer with robot control functions based on a container; and a container management node, used to manage the containers on the computer system.
[0030] From the above, the container warehouse and container management node of the network server are used to support the container-based application deployment of the on-site computer. The artificial intelligence warehouse and NPU module are used to enhance the data processing capability of the on-site computer in terms of algorithm and computing power. The robot control node is used to enhance the robot control capability of the on-site computer.
[0031] In a possible implementation of the fifth aspect, it also includes: a collaborative management platform, which is used to display the status of peripherals of each on-site computer based on an image; the peripheral management component is also used to send the peripheral status of the on-site computer to the collaborative management platform or the network server, wherein when the peripheral management component sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
[0032] From the above, the peripherals of the on-site computers are graphically monitored through the collaborative management platform, so that the peripherals of the on-site computers can be flexibly and dynamically combined.
[0033] In a possible implementation of the fifth aspect, it also includes: a model training platform, which is used to obtain sample data from the corresponding on-site computer through the network server to train the artificial intelligence model involved in the on-site computer.
[0034] From the above, the artificial intelligence model of the on-site computer is trained through the model training platform, thereby enhancing the data processing capability of the on-site computer.
[0035] In a possible implementation of the fifth aspect, when multiple field computers are connected to the network, the peripheral management component of the second field computer is also used to obtain the driver program of the peripheral installed on the second field computer from the first field computer through the communication component of the field computer, and install and start the driver program; and / or the peripheral management component of the second field computer is also used to obtain the computing power of the peripheral used for computing of the first field computer through the communication component of the field computer.
[0036] From the above, when the network server is removed, a field computer obtains the driver program from other field computers through its peripheral management component, and further implements hardware combinations of different modes in the operating system to match the basis of field differentiation.
[0037] In a possible implementation of the fifth aspect, the peripheral management component is also used to uninstall the driver of the peripheral when any peripheral of the on-site computer is removed; and / or when the on-site computer installs a new peripheral, the peripheral management component of the on-site computer obtains the driver corresponding to the system of the peripheral from a network server or other on-site computers, and installs and starts the driver to load the peripheral.
[0038] As described above, the peripheral management component is also used to uninstall the driver of the removed peripheral and start the driver of the newly installed peripheral, so as to more flexibly perform the dynamic combination of the peripherals of the on-site computer.
[0039] In a sixth aspect, an embodiment of the present application provides a computing device, including:
[0040] bus;
[0041] A communication interface connected to the bus;
[0042] at least one processor connected to the bus; and
[0043] At least one memory is connected to the bus and stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor executes any implementation of the first aspect or the second aspect of the present application.
[0044] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a computer, the computer executes any implementation of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A schematic diagram of a flow chart of an embodiment of an on-site computer peripheral loading method of the present application;
[0046] Figure 2 A schematic diagram of a flow chart of an embodiment of a method for generating an on-site computer operating system of the present application;
[0047] Figure 3 A schematic diagram of the structure of an on-site computer embodiment of the present application;
[0048] Figure 4 A schematic diagram of the structure of an embodiment of a field computer operating system generating device of the present application;
[0049] Figure 5 This is a structural diagram of a first embodiment of an on-site computer system of the present application;
[0050] Figure 6 A schematic diagram of a specific scenario of an on-site computer system embodiment of the present application;
[0051] Figure 7 A schematic diagram of a computing device according to various embodiments of the present application. DETAILED DESCRIPTION
[0052] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0053] In the following description, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects, or to distinguish different embodiments, and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.
[0054] In the following description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps must be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.
[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0056] An embodiment of the present application provides a method for loading peripherals of an on-site computer and a method for generating an operating system. The loading method includes: obtaining the modality of the on-site computer, the modality including the system of the processor of the on-site computer and the system of peripherals used for computing, the peripherals including at least one of the following: GPU, NPU, DSP, FPGA; installing an operating system generated based on the system of the processor on the on-site computer, the components of the operating system including at least a peripheral management component and a communication component; the peripheral management component obtains a driver corresponding to the system of the peripheral from a network server through the communication component, and installs and starts the driver to load the computing peripheral.
[0057] The technical solution of the embodiment of the present application loads the drivers of peripherals (GPU, NPU, DSP, FPGA, etc.) in the computing field on an operating system platform that has been adapted to the system of the processor of the on-site computer, implements hardware groups of different modes in the operating system, and is suitable for use with various computer systems for on-site control.
[0058] The field computers in each embodiment of the present application are connected to a network server via a network, and each field computer and the network server form a field computer system. The network can be a wired network or a wireless network, and can be a TSN network and an AUTBUS network.
[0059] The following describes various embodiments of the present application in conjunction with the accompanying drawings.
[0060] First combine Figure 1 An embodiment of an on-site computer peripheral loading method of the present application is introduced.
[0061] Figure 1 The flowchart of an embodiment of an on-site computer peripheral loading method is shown, including steps S110 to S130.
[0062] S110: Acquire the modality of the on-site computer.
[0063] The modality of the field computer includes the system of the processor of the field computer and the system of the peripherals used for calculation.
[0064] Among them, the processor embodiment includes the type of processor (such as CPU, MCU, etc.), the architecture of the processor (ARM, x86, aarch64, mips64, etc.), and the distribution manufacturer of the processor (Intel, AMD, Phytium, longarch, etc.), that is, the processor of the on-site computer can be of multiple architectures and manufacturers.
[0065] The types of the peripherals used for computing include at least one of the following: GPU, NPU, DSP, FPGA, and the system includes types and distribution manufacturers, etc.
[0066] S120: Install an operating system on the on-site computer, and start a communication component and a peripheral management component.
[0067] The operating system of the field computer is generated based on the processor system of the field computer. Its components include at least a peripheral management component and a communication component. The currently installed operating system already includes the driver of the peripheral used for communication involved in the communication component.
[0068] In some embodiments, the operating system of the field computer includes: a boot boot of the field computer, a kernel of the operating system, a RAM root directory system (initdr), a communication component, and a peripheral management component, wherein the kernel of the operating system includes basic components necessary for the operating system itself, for example, a thread management component. The kernel of the operating system includes a kernel program for starting the field computer.
[0069] It should be emphasized that the operating system of each embodiment of the present application is the most streamlined version, and its kernel does not include peripheral drivers other than the peripheral drivers for communication components, so it does not include drivers for computing peripherals such as GPU, NPU, DSP, FPGA, etc. The operating system of each embodiment of the present application can only run on the processor corresponding to the processor system on which it is based, and cannot run on processors corresponding to other systems.
[0070] In some embodiments, when the operating system of the field computer is generated based on the processor system of the field computer, the source code of the general abstract boot boot, the kernel of the operating system, and the RAM root directory system of the operating system are adapted based on the processor system of the field computer and then compiled.
[0071] S130: The peripheral management component obtains the driver of the peripheral installed in the on-site computer from the network server, and starts the driver.
[0072] The peripherals of the on-site computer include peripherals used for calculation, and the network server is connected to the on-site computer via a communication component.
[0073] From the above, by loading the peripherals for computing in this step, a combination of various modes of processors of various systems and various modes of peripherals GPU / NPU / DSP / FPGA for computing of various systems is realized. For example, the mode of XX company CPU+XX company NPU, the mode of XX company CPU+XX company FPGA, the mode of XX company CPU+XX company DSP, and the mode of XX company CPU+XX company GPU are realized.
[0074] Among them, each field computer and the field computer and the network server are connected through the network, and the communication component of each field computer is connected to the network through its peripherals for communication. After the operating system is installed, the communication component and the peripheral management component have also been started. The communication component finds the network server through multicast and establishes a connection with it. The communication component supports the corresponding communication protocol and supports multiple communication protocols, such as wireless, TSN and AUTBUS.
[0075] In some embodiments, the peripheral management component of the on-site computer also obtains computing power for the on-site computer from the network server. For example, the computing power of artificial intelligence is obtained.
[0076] In some embodiments, when multiple field computers are connected to the network, such as a first field computer and a second field computer, the first field computer has obtained relevant drivers from the network server, and the peripheral management component of the second field computer obtains the driver of the peripheral installed in the second field computer from the first field computer through the communication component of the field computer, and installs and starts the driver.
[0077] In some embodiments, when multiple on-site computers are connected to the network, such as a first on-site computer and a second on-site computer, the peripheral management component of the second on-site computer obtains the computing power of the peripherals involved in computing of the first on-site computer through the communication component of the on-site computer, for example, the computing power of the NPU or GPU.
[0078] In some embodiments, the peripheral management component of the on-site computer sends its peripheral status to a network server or a collaborative management platform to facilitate the collaborative management platform to graphically monitor the peripherals of the on-site computer. When the peripheral management component of the on-site computer sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
[0079] In some embodiments, the on-site computer also obtains one of the following from the network server: an application of the on-site computer, an image of a container carrying the application, the application involving an artificial intelligence algorithm program or model, artificial intelligence computing power, computing power of robot control functions; and / or the container on the on-site computer is managed through the network server.
[0080] In some embodiments, when any peripheral of the on-site computer is removed, the peripheral management component of the on-site computer is further used to uninstall the driver of the peripheral on the on-site computer.
[0081] In some embodiments, when a new peripheral is installed on a field computer, its peripheral management component obtains a driver corresponding to the architecture of the peripheral from a network server or other field computers, and installs and starts the driver to load the peripheral.
[0082] In some embodiments, an application on an on-site computer sends sample data to a network server for training an artificial intelligence model associated with the on-site computer.
[0083] In summary, an embodiment of a method for loading peripherals of an on-site computer loads drivers for peripherals (GPU, NPU, DSP, FPGA, etc.) in the computing field on an operating system platform that has been adapted to the system of the processor of the on-site computer, implements hardware combinations of different modes in the operating system, and matches the dynamic combination of differentiated infrastructure levels on site.
[0084] Combine the following Figure 2 An embodiment of a method for generating an on-site computer operating system of the present application is introduced.
[0085] A method embodiment for generating an on-site computer operating system is used to generate an on-site computer operating system in an on-site computer peripheral loading method embodiment.
[0086] Figure 2 The flowchart of an embodiment of a method for generating an on-site computer operating system is shown, including steps S210 to S240.
[0087] This embodiment runs on the IDE of the on-site computer, which is also the IDE of the computer system.
[0088] S210: Acquire the configuration template of the operating system, the original source code of the boot, kernel and RAM root directory system, the driver source code of the peripherals of the field computing, and the source code of the peripheral management component and the communication component.
[0089] Among them, the original source code of the boot, kernel and RAM root directory system of the operating system is the general source code of the type of operating system of the on-site computer. The configuration template of the operating system includes the configuration files of the basic components required by the operating system and the communication components and peripheral management components. The source code of the basic components required by the operating system is included in the original source code of the kernel, and the source code of the peripheral management component and the communication component are the source codes developed according to the functions of the peripheral management component, the communication component and the peripheral management.
[0090] Among them, the peripherals for on-site computing include peripherals used for computing (GPU, NPU, DSP, FPGA, etc.).
[0091] S220: According to the processor system of the on-site computer, the source codes of the boot, kernel and RAM root directory system of the operating system are adapted, and the adapted source codes are copied to a designated directory on the IDE of the on-site computer.
[0092] Among them, the source code of the operating system's boot, kernel and RAM root directory system before adaptation is the general source code of the boot, kernel and RAM root directory system of the operating system type. After adaptation, it becomes the dedicated source code of the boot, kernel and RAM root directory system based on the system of the processor of the on-site computer.
[0093] S230: Compile the adapted source code, the source code of the peripheral management component, the source code of the communication component and the source code of the peripherals of the field computer according to the system and configuration template of the processor of the field computer to obtain the installation program of the operating system of the field computer and the corresponding driver.
[0094] The installation program includes the operating system's boot, kernel and RAM root directory system execution programs, as well as the peripheral management component and communication component execution programs. The operating system's basic component execution programs are in the operating system's kernel.
[0095] Among them, the driver includes drivers for various peripherals that have been installed on the on-site computer and may be activated in the future, including drivers for peripherals used for communication, drivers for peripherals in the computing field (GPU, NPU, DSP, FPGA, etc.), and drivers for peripherals in other fields.
[0096] The driver of the peripheral device used for communication is also incorporated into the software package of the driver of the operating system. The driver of the peripheral device used for communication is installed while the operating system is installed, so that the communication component is started in time.
[0097] S240: Upload the operating system installation program and the peripheral device driver to the network server.
[0098] The installation program and the driver are stored in an application software warehouse of a network server.
[0099] In summary, an embodiment of a method for generating an on-site computer operating system adapts and compiles the relevant source code of the operating system and the source code of the driver according to the system adaptation of the processor of the on-site computer, generates the operating system installation program, peripheral management components and communication components, implements hardware combinations of different modes in the operating system, and matches the dynamic combination of the differentiated infrastructure level on site.
[0100] Combine the following Figure 3 An on-site computer embodiment of the present application is introduced.
[0101] A computer of an on-site computer embodiment is installed with an operating system generated by an on-site computer operating system generation method embodiment, and executes an on-site computer peripheral loading method embodiment to load the peripheral.
[0102] Figure 3 The structure of an on-site computer embodiment of the present application is shown, and an operating system 300 is installed on the hardware of the on-site computer. The operating system 300 is generated based on the system of the processor of the on-site computer and includes a peripheral management component 310 and a communication component 320. For example, the hardware at least includes: a CPU and a peripheral for computing among NPU / GPU / DSP / FPGA.
[0103] The operating system 300 is installed on the on-site computer through steps S110 and S120 of an embodiment of an on-site computer peripheral loading method.
[0104] The communication component 320 is used for the on-site computer to connect to other on-site computers and / or network servers.
[0105] The peripheral management component 310 is used to obtain the driver of the peripheral installed on the on-site computer from the network server and start the driver. The computing peripherals of the on-site computer include at least one of the following: GPU, NPU, DSP, FPGA. For its working principle and advantages, please refer to step S130 of an embodiment of a method for loading on-site computer peripherals.
[0106] Specifically, the peripheral management component 310 is used to generate a unique hardware ID in the computer system for the peripherals of the on-site host, and to establish a database entry (such as hardware type, such as NPU, GPU, etc., and the corresponding relationship with the host) according to the host ID, and send it to the network server to save the corresponding database. When the on-site host adds or uninstalls the corresponding peripheral hardware, the corresponding message is transmitted to the network server through the communication component 320, so that the corresponding driver can be pulled from the database of the network server for installation or update through the type information of the unique hardware ID.
[0107] In some embodiments, when multiple field computers are connected to the network, such as a first field computer and a second field computer, the first field computer has obtained relevant drivers from the network server, and the peripheral management component 310 of the second field computer obtains the driver of the peripheral installed in the second field computer from the first field computer through the communication component of the field computer, and starts the driver.
[0108] In some embodiments, when multiple on-site computers are connected to the network, such as a first on-site computer and a second on-site computer, the peripheral management component 310 of the second on-site computer obtains the computing power of the peripherals involved in computing of the first on-site computer through the communication component of the on-site computer, for example, the computing power of the NPU or GPU.
[0109] For example, different edge-side NPU hardware is used in some modal field computers to run corresponding reasoning frameworks for neural network reasoning, such as the commonly used YOLO, RESNET, OPENPOSE, LSTM, etc. At the same time, the multi-modal field computers can also use distributed communication components to transmit the data to be inferred to the NPU modules of other modal field computers for reasoning, and then return the results.
[0110] In some embodiments, the peripheral management component 310 is also used to send its peripheral status to a network server or a collaborative management platform to facilitate the collaborative management platform to graphically monitor the peripherals of the on-site computer. When the peripheral management component of the on-site computer sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
[0111] In some embodiments, the peripheral management component 310 is further configured to uninstall the driver of any peripheral on the on-site computer when the peripheral is removed from the on-site computer.
[0112] In some embodiments, the peripheral management component 310 is also used to obtain computing power for the on-site computer from the network server. For example, the computing power of artificial intelligence is obtained.
[0113] In some embodiments, the on-site computer also obtains one of the following from the network server: an application of the on-site computer, an image of a container carrying the application, the application involving an artificial intelligence algorithm program or model, artificial intelligence computing power, computing power of robot control functions; and / or the container on the on-site computer is managed through the network server.
[0114] In some embodiments, an application on an on-site computer sends sample data to a network server for training an artificial intelligence model associated with the on-site computer.
[0115] Combine the following Figure 4 An embodiment of a device for generating an on-site computer operating system of the present application is introduced.
[0116] The device of an embodiment of a device for generating an operating system of an on-site computer executes the method of an embodiment of a method for generating an operating system of an on-site computer on the IDE of the on-site computer, and has all the advantages thereof.
[0117] Figure 4 The structure of an embodiment of an on-site computer operating system generation device is shown, including: an acquisition module 410, an underlying technical framework 420 and a customized construction framework 430.
[0118] The acquisition module 410 is used to acquire the configuration template of the operating system, the original source code of the boot boot, kernel and RAM root directory system, the driver source code of the peripherals, the source code of the peripheral management component and the communication component. Please refer to step S210 of an embodiment of a method for generating a computer operating system on site for its operation and principle.
[0119] The underlying technical framework 420 is used to adapt the source code of the boot, kernel and RAM root directory system of the operating system according to the processor system of the on-site computer, and copy the adapted source code to the specified directory on the IDE of the on-site computer. Its operation and principle can be referred to step S220 of an embodiment of a method for generating an on-site computer operating system.
[0120] The customized construction framework 430 is used to compile the adapted source code, the source code of the peripheral management component, the source code of the communication component and the source code of the peripherals of the on-site computer according to the system and configuration template of the processor of the on-site computer, and obtain the installation program of the operating system of the on-site computer and the corresponding driver; it is also used to upload the installation program of the operating system and the driver of the peripherals to the network server. Its operation and principle please refer to steps S230 and S240 of an embodiment of a method for generating an on-site computer operating system.
[0121] Combine the following Figures 5 and 6 An on-site computer system embodiment of the present application is introduced.
[0122] An on-site computer system embodiment includes an on-site computer described in an on-site computer embodiment and a network server.
[0123] Figure 5 The structure of a first embodiment of an on-site computer system is shown, including: an on-site computer 510, a network server 520, a model training platform 530 and a collaborative management platform 540 connected through a network.
[0124] The field computer 510 includes a multi-modal field computer 510. The operating system on each field computer 510 includes the boot boot of the field computer, the kernel of the operating system, the communication component and the peripheral management component. The kernel of the operating system includes the basic components of the operating system. Each mode is a combination of a processor of a system and a peripheral for computing, for example, XX company CPU + XX company GPU. Each field computer 510 has been described in detail in a field computer embodiment and will not be described in detail.
[0125] The network server 520 includes: a hardware ID management module 5210, an application software warehouse 5220, a container warehouse 5230, an artificial intelligence warehouse 5240, an NPU module 5250, a robot control node 5260 and a container management node 5270.
[0126] The hardware ID management module 5210 is used as the server of the peripheral management component of the field computer 510 to manage all the hardware ID information of all the field computers 510 in the same network. The application software warehouse 5220 is used to provide each field computer 510 with its peripheral drivers and applications.
[0127] The application software warehouse 5220 is used to provide drivers and applications for peripherals of each on-site computer 510. The container warehouse 5230 is used to provide the images of containers carrying its applications to the on-site computers 510. The artificial intelligence warehouse 5240 is used to provide the on-site computers 510 with the artificial intelligence algorithm programs and parameters of the artificial intelligence models used by them.
[0128] Among them, the NPU module 5250 is equipped with high-computing-power NPU hardware, and can realize the computing power splitting function through vNPU (virtual NPU) technology to provide artificial intelligence computing power to the on-site computer 510, realize artificial intelligence reasoning with large computing power, and feed back the reasoning results to the corresponding on-site computer 510.
[0129] The robot control node 5260 is used to provide the on-site computer 510 with a robot control function based on a container, and the control function includes one of the following: a general robot control function and a collaborative function of multiple on-site robots. The container management node 5270 is used to manage various containers on the computer system.
[0130] The collaborative management platform 540 is used to display the peripheral status of the on-site computer 510 of each mode through images based on a container method, including the status of the driver, wherein the peripheral management component of the on-site computer 510 sends its peripheral status to the network server 520 or the collaborative management platform, so that the collaborative management platform can graphically monitor the peripherals of the on-site computer, wherein when the peripheral management component of the on-site computer sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
[0131] The collaborative management platform 540 is also used to display the applications, containers, artificial intelligence algorithms and model information of the on-site computers 510 of each modality through images based on containers, and provide graphical operation options for adding, deleting, modifying and checking related content of the corresponding modality host.
[0132] Specifically, when a modal field computer 510 is connected to the same network, the network server 520 obtains relevant information and pushes it to the collaborative management platform 540. The collaborative management platform 540 updates and displays its monitoring lists, adds the content of the field computer 510, and displays the corresponding installable drivers and containers based on the graphically displayed hardware ID information of the field computer 510 for the construction of multi-modal field computers 510, while providing a search function for the search and deployment of applications and artificial intelligence algorithms and models. From the above, the collaborative management platform 540 displays the container information of the field computers 510 of each modality in the same network, and provides a graphical operation method for performing various operations on the container. It provides a function for quickly packaging customized container applications, and publishes container applications to the network server warehouse for agile release and upgrade of containers.
[0133] The model training platform 530 is used to train sample data obtained from the corresponding on-site computer 510 through the network server 520 to obtain an artificial intelligence model related to the on-site computer 510.
[0134] Specifically, when the image or algorithm data accessed by the peripherals (camera, radar, sensor, etc.) of a modal on-site computer 510 is transmitted to the network server 520, the corresponding data is pre-processed and compressed and transmitted to the model training platform 530 in the cloud. At the same time, the model data trained by the model training platform 530 is received and published to the network server 520 for on-demand installation and use of the on-site computer 510 of this modality.
[0135] In some embodiments, an on-site computer system embodiment also includes its IDE, which is also the IDE of each on-site computer 510, and is used to generate the operating system of the on-site computer 510 through the method described in an on-site computer operating system generation method embodiment, and upload its installation program and peripheral drivers to the application software warehouse 5220 of the network server 520 in an on-site computer system embodiment.
[0136] Figure 6 A specific scenario of an on-site computer system embodiment is shown. Nine scenarios are illustrated in the figure, and any scenario may be used in actual use.
[0137] Scenes 1 to 4 are car scenes, which are smart car scenes. They are equipped with on-site computers of modes 1 to 4, using CPU1 to 4 of company A (which can be replaced by other CPU hardware) as the main control processor, and are equipped with NPU1 to 4 hardware of company B for artificial intelligence computing acceleration. The operating system is AMP architecture, and the non-real-time part uses a customized system streamlined version to facilitate the expansion of different needs in the later stage. If it is a streamlined system of the smart car version, the relevant components with different control functions of the robot control node 5260 in the network server 520 are installed by default. The smart car is equipped with high-performance hardware such as laser radar and depth camera, which can realize applications such as robot motion control, remote control communication, map navigation, follow-up obstacle avoidance, automatic driving, and human feature recognition. The smart car is networked at the system level through the Wi-Fi network.
[0138] The communication components of scenarios 5 to 7 are networked using TSN, and the operating system includes public version components. The mode of scenario 5 is company C's CPU5 + company D's GPU for image processing, the mode of scenario 6 is company C's CPU6 + company E's DSP for related control, and the mode of scenario 7 is company C's CPU6 + company F's FGPA for related control.
[0139] The communication components of scenes 8 to 9 use AUTBUS+TSN networking, the CPU uses the company G's CPU or directly uses the MCU bare board for control, the actuator runs the robot control system to control each joint, the target operation is identified through the visual camera, the real-time system initiates the control command, and multiple robotic arm hardware are connected through TSN, which can receive external commands or preset operation processes for unified collaborative operation. The hardware of the actuator includes peripherals such as GPU, DSP or FPGA.
[0140] The present application also provides a computing device, Figure 7 Detailed introduction.
[0141] The computing device 700 includes a processor 710 , a memory 720 , a communication interface 730 , and a bus 740 .
[0142] It should be understood that the communication interface 730 in the computing device 700 shown in this figure can be used to communicate with other devices.
[0143] The processor 710 may be connected to a memory 720. The memory 720 may be used to store the program code and data. Therefore, the memory 720 may be a storage unit inside the processor 710, or an external storage unit independent of the processor 710, or a component including a storage unit inside the processor 710 and an external storage unit independent of the processor 710.
[0144] Optionally, the computing device 700 may further include a bus 740. The memory 720 and the communication interface 730 may be connected to the processor 710 via the bus 740. The bus 740 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 740 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure is represented by only one line, but it does not mean that there is only one bus or one type of bus.
[0145] It should be understood that in the embodiment of the present application, the processor 710 can adopt a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Or the processor 710 adopts one or more integrated circuits to execute related programs to implement the technical solutions provided in the embodiment of the present application.
[0146] The memory 720 may include a read-only memory and a random access memory, and provides instructions and data to the processor 710. A portion of the processor 710 may also include a nonvolatile random access memory. For example, the processor 710 may also store information on the device type.
[0147] When the computing device 700 is running, the processor 710 executes the computer-executable instructions in the memory 720 to perform the operation steps of each method embodiment.
[0148] It should be understood that the computing device 700 according to the embodiment of the present application can correspond to the corresponding subjects in the methods according to the embodiments of the present application, and the above-mentioned and other operations and / or functions of each module in the computing device 700 are respectively for implementing the corresponding processes of each method of the present embodiment, which will not be repeated here for the sake of brevity.
[0149] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0151] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0154] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0155] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it is used to execute the operating steps of each method embodiment.
[0156] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media.Computer-readable media can be computer-readable signal media or computer-readable storage media.Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above.More specific examples (non-exhaustive lists) of computer-readable storage media include, electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.In this document, computer-readable storage media can be any tangible medium containing or storing programs, which can be used by instruction execution systems, devices or devices or used in combination with them.
[0157] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0158] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0159] The computer program code for performing the operation of the present application can be written in one or more programming languages or a combination thereof, and the programming language includes an object-oriented programming language, such as Java, Small tale, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).
[0160] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, all of which belong to the scope of protection of the present application.
Claims
1. A method for loading computer peripherals on site, It is characterized in that include: Acquire the modality of the on-site computer, wherein the modality includes the system of the processor of the on-site computer and the system of the peripherals used for computing, wherein the peripherals include at least one of the following: GPU, NPU, DSP, FPGA; Installing an operating system generated based on the processor system on the on-site computer, wherein the components of the operating system include at least a peripheral management component and a communication component; The peripheral management component obtains the driver corresponding to the system of the peripheral from the network server through the communication component, and installs and starts the driver to load the computing peripheral.
2. The method according to claim 1, It is characterized in that Also includes: The peripheral management component obtains computing power for the on-site computer from the network server.
3. The method according to claim 1, It is characterized in that When multiple on-site computers are connected to the network, it also includes: The peripheral management component of the second on-site computer obtains the driver of the peripheral used for computing installed in the second on-site computer from the first on-site computer through the communication component of the on-site computer, and installs and starts the driver; and / or The peripheral management component of the second on-site computer obtains the computing power of the peripheral used for computing of the first on-site computer through the communication component of the on-site computer.
4. The method according to claim 1, It is characterized in that Also includes: The peripheral management component sends its peripheral status to the collaborative management platform or the network server to facilitate the collaborative management platform to graphically monitor the peripherals of the on-site computer. When the peripheral management component sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
5. The method according to claim 1, It is characterized in that Also includes: When any peripheral device of the on-site computer is removed, the peripheral device management component uninstalls the driver of the peripheral device; and / or When a new peripheral is installed in the on-site computer, the peripheral management component of the on-site computer obtains a driver corresponding to the architecture of the peripheral from a network server or other on-site computers, and installs and starts the driver to load the peripheral.
6. A method for generating an on-site computer operating system, It is characterized in that The operating system is generated based on the system of the processor of the on-site computer, and its components include at least a peripheral management component and a communication component. The peripheral management component obtains a driver for a peripheral used for computing installed on the on-site computer from a network server through the communication component, and starts the driver. The peripheral includes at least one of the following: GPU, NPU, DSP, FPGA; the method includes: The IDE of the on-site computer adapts the source code of the boot, kernel and RAM root directory system of the operating system according to the processor system of the on-site computer, and copies the adapted source code to a designated directory on the IDE; The IDE compiles the adapted source code, the source code of the peripheral management component and the source code of the communication component according to the processor system of the on-site computer and the configuration template of the operating system to obtain the installation program of the operating system, and compiles the source code of the peripherals of the on-site computer to obtain the corresponding driver; The IDE uploads the installation program and the driver to the network server.
7. An on-site computer, It is characterized in that An operating system is installed on the on-site computer, the operating system is generated based on the system of the processor of the on-site computer, and its components include at least a peripheral management component and a communication component; The communication component is used for the on-site computer to connect to the network server via the network; The peripheral management component is used to obtain the driver of the peripheral used for computing installed on the on-site computer from the network server, and install and start the driver, and the peripheral includes at least one of the following: GPU, NPU, DSP, FPGA.
8. A device for generating an on-site computer operating system, It is characterized in that The operating system is generated based on the system of the processor of the on-site computer, and its components include a peripheral management component and a communication component. The peripheral management component obtains a driver for a peripheral installed on the on-site computer for computing from a network server through the communication component, and installs and starts the driver. The peripherals of the on-site computer include at least one of the following: GPU, NPU, DSP, FPGA; the device includes: The underlying technical framework is used to adapt the source code of the boot, kernel and RAM root directory system of the operating system on the IDE of the on-site computer according to the processor system of the on-site computer, and copy the adapted source code to a specified directory on the IDE; A customized construction framework is used to compile the adapted source code, the source code of the peripheral management component and the source code of the communication component on the IDE according to the processor system of the on-site computer and the configuration template of the operating system, obtain the installation program of the operating system, and compile the source code of the peripherals of the on-site computer to obtain the corresponding driver program, and upload the installation program and the driver program to the network server.
9. An on-site computer system, It is characterized in that include: On-site computers and network servers connected via a network; The operating system of the on-site computer is generated based on the architecture of the processor of the computer, and its components include a peripheral management component and a communication component; The communication component is used for the on-site computer to connect to the network server, and the peripheral management component is used to obtain the driver of the peripheral used for computing installed on the on-site computer from the network server, and install and start the driver, and the peripheral includes at least one of the following: GPU, NPU, DSP, FPGA; The network server includes an application warehouse for providing each on-site computer with drivers and applications for its peripherals.
10. The computer system according to claim 9, It is characterized in that The network server also includes one of the following: A container repository, used to provide the on-site computer with an image of a container carrying its application; An artificial intelligence warehouse for providing artificial intelligence algorithm programs and models to the on-site computer; An NPU module, used to provide artificial intelligence computing power to the on-site computer based on a virtual NPU method; A robot control node, used for providing a robot control function to the on-site computer in a container manner; The container management node is used to manage the containers on the computer system.
11. The computer system according to claim 9, It is characterized in that Also includes: Collaborative management platform, used to display the status of peripherals of each on-site computer based on images; The peripheral management component is also used to send the peripheral status of the on-site computer to the collaborative management platform or the network server, wherein when the peripheral management component sends its peripheral status to the network server, the network server forwards the peripheral status to the collaborative management platform.
12. The computer system according to claim 9, It is characterized in that Also includes: The model training platform is used to obtain sample data from the corresponding on-site computer through the network server to train the artificial intelligence model involved in the on-site computer.
13. The computer system according to claim 9, It is characterized in that When multiple on-site computers are connected to the network, The peripheral management component of the second on-site computer is further used to obtain the driver of the peripheral installed in the second on-site computer from the first on-site computer through the communication component of the on-site computer, and install and start the driver; and / or The peripheral management component of the second on-site computer is further used to obtain the computing power of the peripheral used for computing of the first on-site computer through the communication component of the on-site computer.
14. The computer system according to claim 9, It is characterized in that The peripheral management component is also used to uninstall the driver of any peripheral of the on-site computer when the peripheral is removed; and / or when a new peripheral is installed on the on-site computer, the peripheral management component of the on-site computer obtains the driver corresponding to the system of the peripheral from the network server or other on-site computers, and installs and starts the driver to load the peripheral.
15. A computing device, It is characterized in that include, bus; A communication interface connected to the bus; at least one processor connected to the bus; as well as At least one memory is connected to the bus and stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor executes the method according to any one of claims 1 to 6.
16. A computer-readable storage medium, It is characterized in that Program instructions are stored thereon, and when the program instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 6.
