Vehicle cabin-driving fusion system, chip, method, medium and product

Abstract

The invention relates to a vehicle cabin-driving fusion system, a chip, a method, a medium and a product. The system comprises a system-level chip in a vehicle and cloud equipment. A cabin-driving fusion controller and first data for cabin-driving fusion control are integrated in the system-on-chip; the demand resource of the first data is matched with the resource of the system-on-chip; the cloud equipment comprises second data for cabin-driving fusion control; and the system-on-chip is used for acquiring the second data from the cloud equipment through the cabin-driving fusion controller, and realizing cabin control and intelligent driving control of the vehicle based on the first data and the second data. Partial data is stored in the cloud, so that the resource demand on a vehicle end chip is reduced, and cabin and driving fusion control of a single chip can be realized.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle's cabin-driver fusion system, chip, method, medium and product. Background Art

[0002] Cost reduction and computing power concentration are the main themes of the popularization of automotive intelligence. The so-called cockpit-driving fusion, as the name suggests, is to highly integrate the cockpit domain and the intelligent driving domain into a high-performance computing unit to achieve comprehensive integration of hardware, software and applications, so as to better support the deployment and update of new functions, improve user experience, shorten the development cycle and reduce vehicle costs.

[0003] Under the domain concentration architecture, although the intelligent driving domain and the cockpit domain have achieved intra-domain integration, the two domains are still independent of each other. In some related technologies, the intelligent driving domain and the cockpit domain can be integrated on an electronic board, but they cannot be integrated on a chip. Summary of the invention

[0004] One of the purposes of the present application is to provide a vehicle cabin-driver fusion system, chip, method, medium and product. Since part of the data in the cabin-driver fusion system is stored in the cloud, the resource demand for the vehicle-side chip is reduced, thereby realizing single-chip cabin-driver fusion control.

[0005] In order to achieve the above purpose, the technical solution adopted in this application is as follows:

[0006] In a first aspect, the present application provides a vehicle cabin-driver fusion system, the system comprising a system-level chip and a cloud device in the vehicle; the system-level chip integrates a cabin-driver fusion controller and first data for cabin-driver fusion control; the required resources of the first data are matched with the resources of the system-level chip; the cloud device comprises second data for cabin-driver fusion control; the system-level chip is used to: obtain the second data from the cloud device through the cabin-driver fusion controller, and based on the first data and the second data, realize cabin control and intelligent driving control of the vehicle.

[0007] It can be seen that in this implementation, the vehicle's cabin-driver fusion system includes a chip on the vehicle side and a cloud device. By storing part of the data in the cloud, the resource requirements of the vehicle-side chip are reduced, so that single-chip cabin-driver fusion control can be achieved. When performing cabin-driver fusion control, the second data on the cloud is first obtained, and then the vehicle's cockpit control and intelligent driving control are performed based on the first and second data on the vehicle side.

[0008] In this way, the problem in related technologies that single-chip cabin-driver integration cannot be achieved due to chip resource limitations is overcome.

[0009] In a possible implementation, the first data includes: first functional data for cockpit control; all functional data for intelligent driving control; the second data includes: second functional data for cockpit control; the first functional data is different from the second functional data.

[0010] It can be seen that in this implementation, the second function data of the cockpit control is stored in the cloud, and the other cockpit control data and all the data of the intelligent driving control are stored locally on the vehicle side. That is, by weakening the local vehicle-side cockpit and allocating computing power and resources to the intelligent driving control, the single-chip cockpit-driving fusion control can be achieved without affecting the real-time performance of the intelligent driving, which has the characteristics of good user experience.

[0011] In another implementation, all the functional data of the cockpit can be stored in the cloud; or part of the cockpit and intelligent driving data can be stored in the cloud; or the intelligent driving data can be stored in the cloud, etc. It can be configured according to actual needs, with the characteristics of flexible implementation.

[0012] In a possible implementation manner, when the second data includes second functional data for cockpit control, the second functional data includes one or more of the following: ecological configuration data for cockpit control, and a partial algorithm for cockpit control.

[0013] It is understandable that the second data may also include other data, which are not listed here one by one. For example, the second data may also include some algorithms and functions that can be shared by the cockpit and the intelligent driving.

[0014] It can be seen that in this implementation, the second data stored in the cloud includes the ecological configuration data of the cockpit control and part of the algorithm of the cockpit control. In this way, even if part of the data is stored in the cloud, it will not affect the normal use of the user; it has the characteristics of flexible implementation and good user experience.

[0015] In a possible implementation, the ecological configuration data includes at least one of the following: cockpit applications, voice data, video data, and weather data; some algorithms include at least one of the following: voice processing algorithm, multi-person dialogue algorithm, and scene recommendation algorithm.

[0016] It can be seen that in this implementation, the ecological configuration data may include one or more of the application, voice data, video data, and weather data, and the algorithm stored in the cloud may include one or more of the voice processing algorithm, multi-person dialogue algorithm, and scene recommendation algorithm. The specific ecological configuration data and algorithm can be configured according to actual needs, which is flexible and has high user experience.

[0017] In one possible implementation, when the cabin-driver fusion system includes a cockpit control subsystem and an intelligent driving control subsystem, data is exchanged between the cockpit control subsystem and the intelligent driving control subsystem through an intra-domain cross-process manner.

[0018] It can be seen that data interaction in a cross-process manner within the domain can reduce latency and has the characteristics of high interactive performance.

[0019] On the second aspect, the present application provides a system-level chip, which integrates a cabin-cockpit fusion controller and first data used for cabin-cockpit fusion control; the required resources of the first data are matched with the resources of the system-level chip; the system-level chip is used to: obtain second data used for cabin-cockpit fusion control from a cloud device through the cabin-cockpit fusion controller, and based on the first data and the second data, realize cabin control and intelligent driving control of the vehicle.

[0020] It can be seen that in this implementation, a system-level chip in the vehicle realizes the integrated control of the cockpit and the driver. By storing part of the data in the cloud, the resource demand for the vehicle-side chip is reduced, so that the integrated control of the cockpit and the driver can be realized on a single chip. When performing the integrated control of the cockpit and the driver, the second data in the cloud is first obtained, and then the cockpit control and intelligent driving control of the vehicle are performed based on the first data and the second data on the vehicle side.

[0021] In this way, the problem in related technologies that single-chip cockpit-driver fusion cannot be achieved due to chip resource limitations is overcome. Thus, the following are achieved: First, compared with multi-chip solutions, single chips have higher integration, use less materials, and can share a cooling system, which reduces the investment in domain control, chips, and wiring harnesses in hardware, and reduces hardware costs and supply chain management costs; secondly, compared with cross-domain implementation solutions, the first data in this application (such as detection data collected by sensors) can be shared between cockpits and intelligent driving, further reducing hardware costs such as sensors; secondly, the cockpit-driver fusion system has the characteristics of low system complexity and high functional integration.

[0022] In one possible implementation, the system-level chip includes a data acquisition layer, a service layer and a human-computer interface layer; the data acquisition layer is used to: acquire first data; and transmit the first data to the service layer; the service layer is used to: acquire second data from a cloud device; process the first data and the second data to obtain target output data; and render the target output data in real time and output it to the human-computer interface layer.

[0023] It can be seen that in this implementation, the chip is specifically divided into a data acquisition layer, a service layer, and a human-machine interface layer, so that the first data is shared through the data acquisition layer, the cabin-pilot fusion control is realized through the call of the service layer, and real-time rendering and interaction are realized based on the human-machine interface layer. It has the characteristics of clear, simple and reliable implementation logic.

[0024] In one possible implementation, the data acquisition layer includes a hardware perception layer and a data processing layer; the service layer includes a cockpit service and an intelligent driving service; and when the first data includes first detection data and second detection data, the hardware perception layer is used to: collect the vehicle's whole vehicle sensor data and acquire the vehicle's whole vehicle detection data; the data processing layer is used to: process the whole vehicle detection data to obtain the first detection data for the cockpit service and the second detection data for the intelligent driving service; the cockpit service is used to: call the first detection data, perform cockpit control based on the first detection data and the second data to obtain first output data, and render the first output data in real time and output it to the human-machine interface layer; the intelligent driving service is used to: call the second detection data, perform intelligent driving control based on the second detection data to obtain second output data, and render the second output data in real time and output it to the human-machine interface layer.

[0025] It can be seen that in this implementation, for the hardware perception layer, the vehicle's entire vehicle sensor data can be collected to obtain the entire vehicle detection data, thereby realizing the collection of the entire vehicle detection data and solving the problem of repeated detection in multiple domains; for the data processing layer, Keya organizes the detection data for different services, etc., to obtain detection data for different services; the convenience of using detection data is improved; for cockpit services and intelligent driving services, their respective detection data can be directly called, which has the characteristics of high calling efficiency, and the detection data is processed to obtain output data, which is output to the human-computer interface layer after real-time rendering.

[0026] In one possible implementation, when the cockpit service includes at least two first services, for each of the at least two first services, the data processing layer is also used to: determine the detection data required for each first service in the first detection data; and transmit the detection data required for each first service to each first service by subscription.

[0027] It can be seen that in this implementation, the data processing layer can also perform more fine-grained sorting when sorting the detection data. For example, it can sort out the detection data for each service in the cockpit service; of course, it can also sort out the detection data for each service in the intelligent driving service. In practice, the processing process of the data processing layer can be configured according to actual needs, which has the characteristics of flexible implementation and high user experience.

[0028] In one possible implementation, the system-level chip also includes a model processing layer; the model processing layer includes at least two models; the service layer also includes an intelligent processing layer; the data processing layer is also used to: process the whole vehicle detection data to obtain detection data for each model in at least two models; each model in the model processing layer is also used to: call the detection data of each model by subscription; and process the detection data to obtain model output data of each model, and transmit the model output data to the intelligent processing layer; the intelligent processing layer is used to: obtain the target data based on the model output data and perform real-time rendering and output it to the human-computer interface layer.

[0029] It can be seen that in this implementation, the data processing layer can also organize the detection data according to the model to obtain the detection data for each model; then the model directly calls its own detection data by subscription, which is highly efficient.

[0030] On the third aspect, the present application provides a vehicle control method, which is applied to a system-level chip on the vehicle side, on which a cabin-driver fusion controller and first data for cabin-driver fusion control are integrated; the required resources of the first data are matched with the resources of the system-level chip; the method includes: when the vehicle establishes a connection with a cloud device, sending a first request to the cloud device; the first request is used to request to obtain second data for cabin-driver fusion control; receiving the second data sent by the cloud device; and realizing vehicle cabin control and intelligent driving control based on the first data and the second data through the cabin-driver fusion controller.

[0031] It can be seen that in this implementation, when the cockpit control and intelligent driving control of the vehicle are performed, it can be realized through a system-level chip in the vehicle combined with the second data in the cloud. In this way, the problem that the single-chip cabin-driving fusion cannot be realized due to the limitation of chip resources in the related technology is overcome. Thus, the following are achieved: First, compared with the multi-chip solution, the single chip has higher integration, uses less materials, and can share a set of cooling systems, which reduces the investment in domain control, chips, and wiring harnesses in hardware, and reduces hardware costs and supply chain management costs; secondly, compared with the cross-domain implementation solution, the first data in this application (such as detection data collected by sensors) can realize the sharing of cockpit and intelligent driving, further reducing the hardware cost of sensors and the like; secondly, the cabin-driving fusion system has the characteristics of low system complexity and high functional integration.

[0032] In one possible implementation, the cabin control and intelligent driving control of the vehicle are implemented based on the first data and the second data through the cabin-driver fusion controller, including: implementing cabin control of the vehicle at least based on the second data; and / or implementing intelligent driving control of the vehicle based on the first data and the second data; the first data is detection data used for intelligent driving control.

[0033] It can be seen that in this implementation, cockpit control and intelligent driving control can be performed based on the first data and / or the second data. The specific implementation can be configured according to actual needs, and has the characteristics of flexible implementation.

[0034] In one possible implementation, when the first request is used to request ecological configuration data and the second data is new ecological configuration data, the vehicle cabin control is implemented at least based on the second data, including: determining target ecological configuration data based on the received new ecological configuration data; displaying the target ecological configuration data on the human-machine interface layer of the vehicle; receiving a first operation on the target ecological configuration data; in response to the first operation, obtaining feedback data corresponding to the first operation from the cloud; and rendering the feedback data and outputting it through the human-machine interface layer.

[0035] It can be seen that in this implementation, the ecological configuration data is responded to by interacting with the cloud; the ecological configuration data is stored in the cloud, and the vehicle side is not configured with specific applications and other content, only a list is displayed, and data interaction and feedback are achieved by interacting with the cloud; it has the characteristics of high user experience.

[0036] In one possible implementation, target ecological configuration data is determined based on the received new ecological configuration data, including: if there is no original ecological configuration data in the vehicle, the new ecological configuration data is determined as the target ecological configuration data; if there is original ecological configuration data in the vehicle, the versions of the original ecological configuration data and the new ecological configuration data are compared; and the ecological configuration data with the latest version between the original ecological configuration data and the new ecological configuration data is determined as the target ecological configuration data.

[0037] It can be seen that in this implementation, it can be determined that the response is made according to the new ecological configuration data, thereby improving accuracy.

[0038] In one possible implementation, intelligent driving control of a vehicle is implemented based on first data and second data, including: collecting vehicle sensor data of the vehicle through a hardware perception layer to obtain vehicle detection data of the vehicle; processing the vehicle detection data through a data processing layer to obtain first data for intelligent driving control; obtaining the first data for intelligent driving control in the data processing layer through subscription; and performing intelligent driving control of the vehicle based on the first data and the second data.

[0039] It can be seen that in this implementation, the intelligent driving control process can directly obtain the relevant first data by subscription, and then perform intelligent driving control in combination with the second data. It has the characteristics of simple implementation and reliability.

[0040] In one possible implementation, when subscribers include a cockpit service, an intelligent driving service, and N models, N is greater than 1, and the first data used for intelligent driving control is obtained in the data processing layer through subscription, including: the cockpit service obtains the first detection data through the first subscription; the intelligent driving service obtains the second detection data through the second subscription; each of the N models obtains the detection data corresponding to the model based on the subscription corresponding to the model.

[0041] It can be seen that in this implementation, the specific subscription process is explained, and detection data can be subscribed for different services and models; it has the characteristics of flexible, simple and efficient implementation.

[0042] In a fourth aspect, the present application further provides a storage medium having a computer program or instruction stored thereon, and when the computer program or instruction is executed by a processor, the control method provided in the third aspect above is implemented.

[0043] In a fifth aspect, the present application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the control method provided in the third aspect is implemented.

[0044] It should be noted that the technical effects of the fourth to fifth aspects can refer to the detailed description of the third aspect above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 An optional structural diagram of a vehicle cabin-driver fusion system provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a first optional structure of a system-level chip for a vehicle provided in an embodiment of the present application;

[0047] Figure 3 A second optional structural diagram of a system-level chip for a vehicle provided in an embodiment of the present application;

[0048] Figure 4 A third optional structural diagram of the system-level chip for a vehicle provided in an embodiment of the present application;

[0049] Figure 5 A fourth optional structural diagram of the system-level chip for a vehicle provided in an embodiment of the present application;

[0050] Figure 6 A fifth optional structural diagram of the system-on-chip for a vehicle provided in an embodiment of the present application;

[0051] Figure 7A first optional flow chart of a vehicle control method provided in an embodiment of the present application;

[0052] Figure 8 A second optional flow chart of the vehicle control method provided in the embodiment of the present application;

[0053] Fig. 9 A third optional flow chart of the vehicle control method provided in the embodiment of the present application;

[0054] Fig.10 A fourth optional flow chart of the vehicle control method provided in the embodiment of the present application;

[0055] Fig.11 An optional structural diagram of the cabin-pilot fusion system provided in an embodiment of the present application;

[0056] Fig.12 An optional flowchart of the cockpit entertainment ecosystem configuration and request process provided in an embodiment of the present application;

[0057] Fig.13 An optional flowchart of a process for processing an independent perception layer data stream provided in an embodiment of the present application;

[0058] Fig.14 An optional flowchart of the end-side large model working process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0060] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] In the following description, the terms "first\second\third" are used only as examples to distinguish different objects, and do not represent a specific order for the objects, nor do they have a limitation on the order of precedence. It is understandable that "first\second\third" can be interchanged with a specific order or order of precedence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0063] Embodiments of the present application provide a control system, method, storage medium, and program product for a vehicle.

[0064] Below, various embodiments of the vehicle cabin-driver fusion system, chip, method, medium and product provided in the embodiments of the present application are described.

[0065] In a first aspect, an embodiment of the present application provides a vehicle cabin-driver fusion system. Figure 1 As shown, the cabin-driver fusion system includes a system-level chip 10 in the vehicle and a cloud device 20.

[0066] The system-level chip 10 integrates a cabin-pilot fusion controller 101 and first data 10A for cabin-pilot fusion control; the required resources of the first data match the resources of the system-level chip 10; the cloud device 20 includes second data 10B for cabin-pilot fusion control;

[0067] The system-level chip 10 is used to obtain the second data 10B from the cloud device 20 through the cabin-drive fusion controller 101, and realize the cabin control and intelligent driving control of the vehicle based on the first data 10A and the second data 10B.

[0068] The vehicle here can be a new energy vehicle, a gasoline vehicle, etc., or a sedan, a commercial vehicle, an engineering vehicle, etc. The embodiment of the present application does not limit the type of vehicle, and can be configured according to actual needs.

[0069] The cockpit-driver fusion system refers to the integration of various functions of the car cockpit (such as instruments, controls, infotainment, etc.) with various intelligent driving functions, integrated into the same hardware platform, and relying on certain hardware and software implementation. That is, the cockpit-driver fusion system integrates at least cockpit domain control and intelligent driving pre-control. It is understandable that with the support of computing resources, the cockpit-driver fusion system can also integrate other functions, which will not be listed here one by one.

[0070] The system-on-chip 10 integrates a cabin-controller fusion controller 101 and first data 10A for cabin-controller fusion control. The required resources of the first data match the resources of the system-on-chip 10.

[0071] The cabin-pilot fusion controller 101 refers to a collection of electronic devices for cabin-pilot fusion control. The cabin-pilot fusion controller 101 is integrated on a system-level chip 10, that is, the electronic devices for cabin-pilot fusion control are mainly integrated on this system-level chip 10. The first data 10A for cabin-pilot fusion control is also stored in the storage unit of the system-level chip 10.

[0072] The first data 10A refers to the data stored on the system-level chip 10 in the cabin-cockpit fusion control. The embodiment of the present application does not limit the specific content of the first data, and can be configured according to actual needs. For example, the first data may be part of the data for cockpit control and all of the data for intelligent driving control. Of course, the first data may also be part of the data for intelligent driving control and all of the data for cockpit control; or the first data may also be part of the data for intelligent driving control and part of the data for cockpit control.

[0073] The embodiment of the present application does not limit the type of system-level chip, and can be determined according to the actual required resources of the cabin-driver fusion system in the vehicle. For example, it can be determined according to the data volume of the first data.

[0074] The cloud device 20 includes the second data 10B for cabin-driver fusion control. The embodiment of the present application does not limit the type, resources, and computing power of the cloud device 20, and can be configured according to actual needs.

[0075] The second data 10B refers to the data stored in the cloud device 20 in the cabin-driving fusion control. The embodiment of the present application does not limit the specific content of the second data, and can be configured according to actual needs. For example, the second data can be part of the data used for cockpit control. Of course, the second data can also be part of the data used for intelligent driving control; or the second data can also be part of the data used for intelligent driving control and part of the data used for cockpit control.

[0076] For the cabin-driver fusion system: the cabin-driver fusion controller 101 in the system-level chip 10 can obtain the second data 10B from the cloud device 20, and based on the first data 10A and the second data 10B, the vehicle's cabin control and intelligent driving control can be achieved.

[0077] It can be seen that the vehicle's cabin-driver fusion system includes a chip on the vehicle side and a cloud device. By storing some data in the cloud, the resource requirements of the vehicle-side chip are reduced, so that single-chip cabin-driver fusion control can be achieved. When performing cabin-driver fusion control, the second data on the cloud is first obtained, and then the vehicle's cockpit control and intelligent driving control are performed based on the first and second data on the vehicle side.

[0078] In this way, the problem in the related art that single-chip cabin-driver fusion cannot be achieved due to chip resource limitations is overcome. Thus, the following is achieved: First, compared with the multi-chip solution, the single chip in this embodiment of the present application has higher single-chip integration, uses less materials, and can share a set of cooling systems, which reduces the investment in domain control, chips, and wiring harnesses in hardware, and reduces hardware costs and supply chain management costs; secondly, compared with the cross-domain implementation solution, the first data in this application (such as detection data collected by sensors) can be shared between the cockpit and intelligent driving, further reducing the hardware costs of sensors, etc.; secondly, the cabin-driver fusion system has the characteristics of low system complexity and high functional integration.

[0079] Next, allocation of the first data 10A and the second data 10B will be described.

[0080] In a possible implementation, the first data 10A includes: first functional data for cockpit control; all functional data for intelligent driving control. The second data 10B includes: second functional data for cockpit control; the first functional data is different from the second functional data.

[0081] It can be seen that the second function data of cockpit control is stored in the cloud, and other cockpit control data and all intelligent driving control data are stored locally on the vehicle side. That is, by weakening the local vehicle-side cockpit control and allocating computing power and resources to intelligent driving control, the real-time performance of intelligent driving can be maintained while achieving single-chip cockpit-driving fusion control, which has the characteristics of good user experience.

[0082] In another possible implementation, the local vehicle-side intelligent driving control can be weakened, and computing power and resources can be allocated to cockpit control. The specific implementation will not be described in detail.

[0083] Next, the second data 10B stored in the cloud device 20 will be described.

[0084] In the case where the second data includes second functional data for cockpit control, the second functional data includes one or more of the following: ecological configuration data for cockpit control, and a partial algorithm for cockpit control.

[0085] It is understandable that the second data may also include other data, which are not listed here one by one. For example, the second data may also include some algorithms and functions that can be shared by the cockpit and the intelligent driving.

[0086] It can be seen that in this implementation, the second data stored in the cloud includes the ecological configuration data of the cockpit control and part of the algorithm of the cockpit control. In this way, even if part of the data is stored in the cloud, it will not affect the normal use of the user; it has the characteristics of flexible implementation and good user experience.

[0087] In a possible implementation, the ecological configuration data includes at least one of the following: cockpit applications, voice data, video data, and weather data. Some algorithms include at least one of the following: voice processing algorithm, multi-person conversation algorithm, and scene recommendation algorithm.

[0088] The applications in the cockpit may include: chat programs, music programs, video programs, etc.

[0089] The ecological configuration data here refers to the list data of the cockpit application, voice data, video data, and weather data. The specific application voice data, video data, and weather data program content are all stored in the cloud device. That is, no application is installed or loaded on the vehicle side. The user's configuration items are directly perceived through the (ecological configuration data) list data, and the corresponding cloud application is activated based on the configuration items to realize the control of the application, etc.

[0090] The ecological configuration data and some algorithms can be configured according to actual needs, and the embodiments of the present application do not impose any sole restrictions on the contents of the ecological configuration data and some algorithms.

[0091] It can be seen that in this implementation, the ecological configuration data may include one or more of the application, voice data, video data, and weather data, and the algorithm stored in the cloud may include one or more of the voice processing algorithm, multi-person dialogue algorithm, and scene recommendation algorithm. The specific ecological configuration data and algorithm can be configured according to actual needs, which is flexible and has high user experience.

[0092] Next, the data interaction method between the subsystems in the cabin-pilot fusion system is explained.

[0093] In a possible implementation, when the cabin-driver fusion system includes a cockpit control subsystem and an intelligent driving control subsystem, data is exchanged between the cockpit control subsystem and the intelligent driving control subsystem through an intra-domain cross-process manner.

[0094] The cockpit control subsystem and the intelligent driving control subsystem can build different processes and exchange data in a cross-process manner within the domain.

[0095] In the relevant technology summary, data interaction between the cockpit control subsystem and the intelligent driving control subsystem needs to be carried out across domains. For example, interaction needs to be carried out in a cross-chip manner, which results in a longer data interaction delay.

[0096] In the embodiments of the present application, data is exchanged between subsystems in the cabin-driver fusion system through a cross-process manner within the domain, which can reduce latency and has the characteristics of high interactive performance.

[0097] In a second aspect, an embodiment of the present application provides a system-on-chip.

[0098] refer to Figure 2 As shown, the system-level chip 10 integrates a cabin-pilot fusion controller 101 and first data 10A for cabin-pilot fusion control; the required resources of the first data 10A match the resources of the system-level chip 10.

[0099] The system-level chip 10 is used to obtain the second data 10B for cabin-cockpit fusion control from the cloud device through the cabin-cockpit fusion controller 101, and realize the vehicle's cabin control and intelligent driving control based on the first data 10A and the second data 10B.

[0100] The resources of the system-on-chip 10 may include, but are not limited to, one or more of storage resources, central processing unit (CPU) resources, and graphics processing unit (GPU) resources. Of course, the system-on-chip 10 may also include other resources, which are not listed here one by one.

[0101] The resource requirements of the first data 10A match the resources of the SoC 10, which means that the resources of the SoC 10 can meet the requirements of the first data 10A. That is, the resource amount of the SoC 10 is greater than or equal to the resource amount required by the first data 10A.

[0102] For the first data, second data, vehicle and other information, please refer to the description of the cabin-driver fusion system, which will not be repeated here.

[0103] It can be seen that in this implementation, a system-level chip in the vehicle realizes the integrated control of the cockpit and the driver. By storing part of the data in the cloud, the resource demand for the vehicle-side chip is reduced, so that the integrated control of the cockpit and the driver can be realized on a single chip. When performing the integrated control of the cockpit and the driver, the second data in the cloud is first obtained, and then the cockpit control and intelligent driving control of the vehicle are performed based on the first data and the second data on the vehicle side.

[0104] In this way, the problem in related technologies that single-chip cockpit-driver fusion cannot be achieved due to chip resource limitations is overcome. Thus, the following are achieved: First, compared with multi-chip solutions, single chips have higher integration, use less materials, and can share a cooling system, which reduces the investment in domain control, chips, and wiring harnesses in hardware, and reduces hardware costs and supply chain management costs; secondly, compared with cross-domain implementation solutions, the first data in this application (such as detection data collected by sensors) can be shared between cockpits and intelligent driving, further reducing hardware costs such as sensors; secondly, the cockpit-driver fusion system has the characteristics of low system complexity and high functional integration.

[0105] Next, the architecture of the system-on-chip 10 is described.

[0106] In some embodiments, reference Figure 3 As shown, the system-level chip 10 includes a data acquisition layer 1001, a service layer 1002 and a human-machine interface layer 1003. The data acquisition layer 1001 is used to: acquire first data; and transmit the first data to the service layer 1002; the service layer 1002 is used to: acquire second data 10B from a cloud device; process the first data 10A and the second data 10B to obtain target output data; and render the target output data in real time and output it to the human-machine interface layer 1003.

[0107] The data acquisition layer 1001 is used to acquire the first data 10A. The first data 10A is the vehicle-side data. The data acquisition layer 1001 is used to acquire the relevant data of the vehicle-side in the cabin-driver fusion system.

[0108] In a possible implementation, the first data 10A may be relevant data in the vehicle detection data. Alternatively, the first data 10A may be intermediate data generated by the cockpit control or the intelligent driving control.

[0109] The service layer 1002 integrates various services in the cabin-driving fusion system. The embodiment of the present application does not limit the specific service type and can be configured according to actual needs. For example, the service layer 1002 may include cabin services and intelligent driving services.

[0110] Human-machine interface layer 1003: used for human-machine output. Human-machine interface layer 1003 can render and output the data processed by service layer 1002. The output here can be text output or voice output, etc.

[0111] It can be seen that in the embodiment, the system-level chip is specifically divided into a data acquisition layer, a service layer and a human-machine interface layer, so that the first data is shared through the data acquisition layer, the cabin-pilot fusion control is realized through the call of the service layer, and real-time rendering and interaction are realized based on the human-machine interface layer. It has the characteristics of clear, simple and reliable implementation logic.

[0112] In some embodiments, reference Figure 4 As shown in the content, the data acquisition layer 1001 includes the hardware perception layer 10011 and the data processing layer 10012; the service layer 1002 includes the cockpit service 10021 and the intelligent driving service 10022; the first data 10A includes the first detection data 10A1 and the second detection data 10A2.

[0113] The hardware perception layer 10011 is used to collect the vehicle's entire vehicle sensor data and obtain the vehicle's entire vehicle detection data.

[0114] The vehicle detection data here may include but is not limited to: detection data from vehicle-mounted cameras, millimeter-wave radars, ultrasonic radars, lidars, intelligent lighting systems, and various information from controllers under the vehicle integration unit (VIU), such as vehicle speed, direction angle, and other information.

[0115] The data processing layer 10012 is used to process the vehicle detection data to obtain the first detection data for the cockpit service and the second detection data for the intelligent driving service.

[0116] The data processing layer 10012 pre-senses the type of detection data required for each service (cockpit service and intelligent driving service), and processes the vehicle detection data based on the needs for each detection data, thereby obtaining the first detection data for the cockpit service and the second detection data for the intelligent driving service.

[0117] The cockpit service 10021 is used to: call the first detection data, perform cockpit control based on the first detection data and the second data to obtain first output data, and render the first output data in real time and output it to the human-machine interface layer.

[0118] The embodiment of the present application does not limit the method of calling the first detection data, and it can be configured according to actual needs. Exemplarily, the cockpit service can call the first detection data in the data processing layer 10012 based on a subscription method. The data processing process of the cockpit service 10021 here is similar to the original one. The difference is that the method of obtaining the first inspection data and the method of obtaining the second data are different from the original ones. The first detection data can be directly obtained in the data processing layer 10012 by subscription, and the second data needs to be obtained from the cloud device. Another difference is that the cockpit service here needs to render the output data after obtaining it, and then output it in the human-computer interface layer.

[0119] The intelligent driving service 10022 is used to: call the second detection data, perform intelligent driving control based on the second detection data to obtain second output data, and render the second output data in real time and output it to the human-machine interface layer.

[0120] The embodiment of the present application does not limit the method of calling the second detection data, and it can be configured according to actual needs. Exemplarily, the cockpit service can call the second detection data in the data processing layer 10012 based on a subscription method. The data processing process of the intelligent driving service 10022 here is similar to the original one. The difference is that the method of obtaining the second inspection data and the method of obtaining the second data are different from the original. The second detection data can be obtained directly in the data processing layer 10012 by subscription, and the second data needs to be obtained from the cloud device. Another difference is that the intelligent driving service 10022 needs to perform rendering after obtaining the output data, and then output it at the human-computer interface layer.

[0121] It can be seen that in the embodiment, for the hardware perception layer, the vehicle's whole vehicle sensor data can be collected to obtain the whole vehicle detection data, thereby realizing the collection of the whole vehicle detection data and solving the problem of repeated detection in multiple domains; for the data processing layer, Keya organizes the detection data for different services, etc., to obtain detection data for different services; the convenience of using detection data is improved; for cockpit services and intelligent driving services, their respective detection data can be directly called, which has the characteristics of high calling efficiency, and the detection data is processed to obtain output data, which is output to the human-computer interface layer after real-time rendering.

[0122] In some embodiments, reference Figure 5 As shown in the content, when the cockpit service 10021 includes at least two first services 100211, for each first service 100211 of the at least two first services 100211, the data processing layer 10012 is also used to: determine the detection data 10A11 required for each first service in the first detection data 10A1; and transmit the detection data 10A11 required for each first service to each first service 100211 by subscription.

[0123] The embodiment of the present application does not limit the type of the cockpit service including the first service, and can be configured according to actual needs. For example, the cockpit service may include services for seat control, services for air conditioning control, services for applications, and the like.

[0124] It can be seen that in this implementation, the data processing layer can also perform more fine-grained sorting when sorting the detection data. For example, it can sort out the detection data for each service in the cockpit service; of course, it can also sort out the detection data for each service in the intelligent driving service. In practice, the processing process of the data processing layer can be configured according to actual needs, which has the characteristics of flexible implementation and high user experience.

[0125] In some embodiments, reference Figure 6As shown, the system-level chip 10 also includes a model processing layer 1004; the model processing layer 1004 includes at least two models 10041; the service layer 1002 also includes an intelligent processing layer 10023.

[0126] The data processing layer 10012 is also used to process the whole vehicle detection data to obtain detection data 10A12 for each model 10041 in at least two models 10041.

[0127] Here, the data processing layer 10012 knows in advance the type of detection data required for each model 10041 , and sorts the vehicle detection data according to the type of detection data required for each model 10041 to obtain the detection data 10A12 of each model 10041 .

[0128] Each model 10041 in the model processing layer 1004 is also used to: call the detection data of each model 10041 by subscription; and process the detection data to obtain the model output data of each model, and transmit the model output data to the intelligent processing layer 10023.

[0129] In advance, after configuring the subscribers including each model 10041 in the data processing layer 10012, each model 10041 in the model processing layer 1004 can call the detection data of each model 10041 based on the subscription.

[0130] The intelligent processing layer 10023 is used to obtain target data based on the model output data and then perform real-time rendering and output to the human-machine interface layer.

[0131] It can be seen that in this implementation, the data processing layer can also organize the detection data according to the model to obtain the detection data for each model; then the model directly calls its own detection data by subscription, which is highly efficient.

[0132] In a third aspect, an embodiment of the present application provides a vehicle control method, which is applied to a system-level chip on a vehicle side, wherein a cabin-driver fusion controller and first data for cabin-driver fusion control are integrated on the system-level chip. The required resources of the first data match the resources of the system-level chip.

[0133] refer to Figure 7 The method may include but is not limited to the following S701 to S703.

[0134] S701: When the vehicle is connected to the cloud device, the system-level chip sends a first request to the cloud device.

[0135] The first request is used to request to obtain second data for cabin-pilot fusion control.

[0136] The present application does not limit the method for establishing a connection between the vehicle and the cloud device, and can be configured according to actual needs. For example, when the vehicle is powered on, it automatically establishes a connection with the cloud device.

[0137] There is no specific limitation on the type of the request content of the first request, which can be configured according to actual needs.

[0138] Correspondingly, the cloud device receives the first request sent by the system-level chip, and feeds back second data to the system-level chip based on the first request.

[0139] S702. The system-level chip receives second data sent by the cloud device.

[0140] The system-on-chip receives the second data sent by the cloud device based on the connection relationship between the system-on-chip and the cloud device.

[0141] The second data here may be data sent once, or may be data sent multiple times.

[0142] S703, the system-level chip realizes the vehicle's cockpit control and intelligent driving control based on the first data and the second data through the cockpit-driving fusion controller.

[0143] Among them, cockpit control can be implemented based on the first data and / or the second data, and intelligent driving control can be implemented based on the first data and / or the second data.

[0144] It can be seen that in the embodiment, when the cockpit control and intelligent driving control of the vehicle are performed, it can be realized by combining a system-level chip in the vehicle with the second data in the cloud. In this way, the problem that the single-chip cabin-driving fusion cannot be realized due to the limitation of chip resources in the related technology is overcome. Thus, the following are achieved: First, compared with the multi-chip solution, the single chip has higher integration, uses less materials, and can share a set of cooling systems, which reduces the investment in domain control, chips, and wiring harnesses in hardware, and reduces hardware costs and supply chain management costs; secondly, compared with the cross-domain implementation solution, the first data in this application (such as detection data collected by sensors) can realize the sharing of cockpit and intelligent driving, further reducing the hardware cost of sensors, etc.; secondly, the cabin-driving fusion system has the characteristics of low system complexity and high functional integration.

[0145] Next, the process of the system-level chip in S703 realizing the vehicle cabin control and intelligent driving control based on the first data and the second data through the cabin-drive fusion controller is explained.

[0146] In some embodiments, the process may include but is not limited to the following S7031 and / or S7032.

[0147] S7031. The system-level chip controls the vehicle cabin at least based on the second data.

[0148] In a possible implementation, the system-level chip implements cockpit control of the vehicle based on the second data.

[0149] In another possible implementation, the system-level chip implements cockpit control of the vehicle based on the first data and the second data.

[0150] Which data of the first data and the second data are relied upon in the cockpit control can be determined based on the actual configuration and the specific functions of the cockpit control, and will not be listed one by one here.

[0151] S7032. The system-level chip realizes intelligent driving control of the vehicle based on the first data and the second data.

[0152] The first data is detection data used for intelligent driving control.

[0153] Of course, the system-level chip can also realize intelligent driving control of the vehicle based only on the first data.

[0154] It can be seen that in this implementation, cockpit control and intelligent driving control can be performed based on the first data and / or the second data. The specific implementation can be configured according to actual needs, and has the characteristics of flexible implementation.

[0155] Next, the process of the system-level chip in S7031 controlling the cockpit of the vehicle based on at least the second data is described.

[0156] In some embodiments, when the first request is used to request to obtain ecological configuration data and the second data is new ecological configuration data, the second data received by the system-level chip is the new ecological configuration data. Figure 8 As shown in the content, the process may include but is not limited to the following S70311 to S70315.

[0157] S70311. The system-level chip determines target ecological configuration data based on the received new ecological configuration data.

[0158] In a possible implementation, the system-level chip directly determines the received new ecological configuration data as the target ecological configuration data by default.

[0159] In another possible implementation, the system-level chip compares the received new ecological configuration data with the original ecological configuration data stored locally to obtain the target ecological configuration data.

[0160] S70312, the system-level chip displays the target ecological configuration data at the human-machine interface layer of the vehicle.

[0161] The target ecological configuration data here is a configuration list. The device itself does not limit the way to display the target ecological configuration data at the human-machine interface layer of the vehicle, and can be configured according to actual needs. For example, it can be displayed in text or voice.

[0162] S70313. The system-level chip receives a first operation for target ecological configuration data.

[0163] The type of the first operation is not limited here and can be configured according to actual needs. For example, the first operation can be playing music A.

[0164] S70314. In response to the first operation, the system-level chip obtains feedback data corresponding to the first operation from the cloud.

[0165] For example, in response to the first operation, the system-level chip obtains audio data corresponding to music A from the cloud.

[0166] S70315, the system-level chip renders the feedback data and outputs it through the human-machine interface layer.

[0167] For example, the system-level chip plays the audio data corresponding to music A through the human-computer interface layer, and renders the interface corresponding to music A in real time and displays it through the human-computer interface layer.

[0168] It can be seen that in this embodiment, the response to the ecological configuration data is performed by interacting with the cloud; the ecological configuration data is stored in the cloud, and the vehicle side is not configured with specific applications and other content, only a list is displayed, and data interaction and feedback are achieved by interacting with the cloud; it has the characteristics of high user experience.

[0169] Next, the process of determining the target ecological configuration data based on the received new ecological configuration data by the system-level chip in S70311 is described.

[0170] In some embodiments, the process may include but is not limited to the following S901 and S902.

[0171] S901: If there is no original ecological configuration data in the vehicle, the system-level chip determines the new ecological configuration data as the target ecological configuration data.

[0172] S902: If the vehicle has original ecological configuration data, the system-level chip compares the versions of the original ecological configuration data and the new ecological configuration data; and determines the ecological configuration data with the latest version between the original ecological configuration data and the new ecological configuration data as the target ecological configuration data.

[0173] It can be seen that in this embodiment, it can be determined to respond according to the new ecological configuration data, which improves accuracy.

[0174] Next, the process of realizing intelligent driving control of the vehicle by the system-level chip in S7032 based on the first data and the second data is explained.

[0175] In some embodiments, reference Fig. 9 As shown in the content, the process may include but is not limited to the following S70321 to S70324.

[0176] S70321, the system-level chip collects the vehicle's entire vehicle sensor data through the hardware perception layer to obtain the vehicle's entire vehicle detection data.

[0177] The implementation of S70321 can refer to the detailed description of the hardware perception layer in the system-level chip, which will not be described here.

[0178] S70322, the system-level chip processes the vehicle detection data through the data processing layer to obtain the first data for intelligent driving control.

[0179] The implementation of S70322 can refer to the detailed description of the data processing layer in the system-level chip, which will not be described in detail here.

[0180] S70323. The system-level chip obtains the first data for intelligent driving control in the data processing layer by subscription.

[0181] The implementation of S70323 can refer to the detailed description of the data processing layer and the service layer in the system-level chip, which will not be described here one by one.

[0182] S70324, the system-level chip performs intelligent driving control of the vehicle based on the first data and the second data.

[0183] It can be seen that in this embodiment, the intelligent driving control process can directly obtain the relevant first data by subscription, and then perform intelligent driving control in combination with the second data. It has the characteristics of simple implementation and reliability.

[0184] Next, the process of obtaining the first data for intelligent driving control in the data processing layer by the system-level chip in S70323 through subscription is explained.

[0185] When the subscribers include cockpit services, smart driving services, and N models, N is greater than 1. Fig.10 As shown in the content, the process may include but is not limited to the following S1001 to S1003.

[0186] S1001. The cockpit service obtains first detection data through a first subscription.

[0187] The first subscription corresponds to a subscription for a cockpit service, and the first subscription specifies the type of detection data required for the cockpit service.

[0188] S1002. The intelligent driving service obtains the second detection data through the second subscription.

[0189] The second subscription corresponds to the subscription of the intelligent driving service, and the second subscription specifies the type of detection data required by the intelligent driving service.

[0190] S1003. For each model among the N models, based on the subscription corresponding to the model, obtain the detection data corresponding to the model.

[0191] There is a subscription for each model, and the subscription for each model specifies the type of detection data required by the model.

[0192] It can be seen that in this embodiment, a specific subscription process is described, and detection data can be subscribed for different services and models; it has the characteristics of flexible, simple and efficient implementation.

[0193] Below, taking the cabin-driver fusion of a vehicle as an example, the cabin-driver fusion system provided by this embodiment of the present application is explained.

[0194] Cost reduction and computing power concentration are the main themes of the popularization of automotive intelligence. Cabin-driver fusion is to highly integrate the cockpit domain and intelligent driving domain into a high-performance computing unit to achieve comprehensive connection of hardware, software and applications, so as to better support the deployment and update of new functions, improve user experience, shorten the development cycle and reduce vehicle costs.

[0195] Under the domain concentration architecture, although the intelligent driving domain and the cockpit domain have achieved intra-domain integration, the two domains are still independent of each other, with two control boxes (Two Boxes) and two circuit boards (Two Boards). The cockpit-driver integration is committed to achieving one control box (One Box), one circuit board (One Board), and one chip (One Chip), among which One Chip is generally considered to be the "ultimate solution" for cockpit-driver integration, that is, running the intelligent driving domain and the intelligent cockpit domain on a system-on-chip (SoC) at the same time.

[0196] In related technologies, the cockpit and driving are still independent chips and independent systems, but they are integrated on a motherboard and have not yet reached the level of One Box, One Board, One Chip.

[0197] In this embodiment, a cabin-cockpit fusion system deployment solution based on a single-chip SOC is mainly explained. This solution enhances the real-time and reliability of cabin-cockpit data interaction, reduces data interaction delay, reduces the development and maintenance costs of multiple chips and multiple systems, and centrally processes perception data, which is more conducive to the rapid application of big data and artificial intelligence (AI) large models.

[0198] Next, the single-chip SOC cabin-driver fusion system is explained.

[0199] refer to Fig.11 The content shown includes the vehicle side 110 (equivalent to the above-mentioned vehicle) and the cloud side 120 (equivalent to the above-mentioned cloud device).

[0200] The vehicle side 110 may include: a human-machine interface (Human Machine Interface, HMI) 1101 (equivalent to the above-mentioned human-machine interface layer), a rendering engine 1102, basic core services 1103 (equivalent to the service layer), an AI intelligent brain decision layer 1104, (equivalent to the above-mentioned intelligent processing layer) AI perception layer 1105, a hardware perception data layer 1106 (equivalent to the above-mentioned hardware perception layer), a Linux basic system 1107 and a vehicle integration unit (Vehicle Integration Unit, VIU) 1108.

[0201] HMI1101 may include: vehicle equipment 11011 , air conditioning settings 11012 , local navigation 11013 , voice interaction 11014 , scene reconstruction 11015 , and cloud ecosystem interaction 11016 .

[0202] The rendering engine 1102 is mainly used to render the data of the basic core service layer through the graphics and image rendering engine and then display it in the HMI 1101 .

[0203] The basic core services 1103 include: intelligent driving service 11031, AI intelligent brain service 11032, instrument basic service 11033 and cockpit basic service 11034. The intelligent driving service 11031, instrument basic service 11033 and cockpit basic service 11034 respectively include: business module 1103A and diagnosis module 1103B.

[0204] The basic core service 1103 is used to receive data transmitted by the AI ​​intelligent brain decision layer 1104, process the data transmitted by the AI ​​intelligent brain decision layer 1104, and send it to the rendering engine 1102.

[0205] The AI ​​intelligent brain decision layer 1104 includes: scenario model 11041, scenario model N11042, large scenario model 11043 (model 110431, routing network 110432, feature extraction 110433) and terminal model synchronization service 11044.

[0206] The AI ​​intelligent brain decision layer 1104 is used to: receive data transmitted by the AI ​​perception layer 1105, perform decision processing based on the data transmitted by the AI ​​perception layer 1105, and transmit the processing results to the AI ​​intelligent brain service 11032 in the basic core service 1103.

[0207] The AI ​​perception layer 1105 includes: a data bus 11051 (equivalent to the above-mentioned data processing layer), AI1 preprocessing 11052, AI2 preprocessing 11053, and the like.

[0208] The AI ​​perception layer 1105 is used to: receive data sent by the hardware perception data layer 1106 through the data bus 11051, and transmit the data sent by the hardware perception data layer 1106 to the cockpit basic service 11034 in the basic core service 1103.

[0209] The AI ​​perception layer 1105 is also used to: receive data sent by the hardware perception data layer 1106 through the data bus 11051, separate and organize the data sent by the hardware perception data layer 1106 through AI1 preprocessing 11052 and AI2 preprocessing 11053, and then transmit them to the AI ​​intelligent brain decision layer 1104.

[0210] The hardware perception data layer 1106 includes: in-vehicle vision 11061, out-vehicle vision 11062, voice microphone (mic) 11063, ultrasonic radar 11064, temperature sensor 11065, vehicle to everything (V2X) 11066, hardware expansion socket 11067, etc.

[0211] The hardware perception data layer 1106 is used to transmit data to the data bus 11051 in the AI ​​perception layer 1105 .

[0212] The Linux base system 1107 is used to provide an operating system.

[0213] VIU 108 is used to transmit various vehicle data to the Linux base system 1107 via Ethernet, an interface (Service Provider Interface, SPI) provided by a service provider, or a universal serial data bus (Universal Asynchronous Receiver / Transmitter, UART).

[0214] The cloud 120 may include: a cloud ecological management platform 1201 , cloud services 1202 , and a cloud super brain 1203 .

[0215] The cloud-based ecological management platform 1201 includes: ecological management platform 12011, vehicle model 1 12012, vehicle model 2 12013, vehicle model N 12014 and ecological cloud-based open interface 12015.

[0216] The cloud ecosystem management platform 1201 is used to obtain ecosystem data from the third-party ecosystem suppliers 130 and the third-party information source suppliers 140. The cloud ecosystem management platform 1201 is also used to transmit hypertext data (HTML5, H5), streaming media, text, pictures and other data between the basic core services 1103 through the ecosystem cloud open interface 12015.

[0217] The cloud services 1202 include: cloud diagnosis 12021, big data collection 12022, remote control 12023, and mobile terminal over-the-air software upgrade (Firmware Over-The-Air, FOTA) and other service modules 12024.

[0218] The cloud service 1202 is used to communicate with the basic core service 1103 to store some data or functions of the vehicle side in the cloud, and obtain some data or functions in the cloud.

[0219] The cloud-based super brain 1203 includes: vehicle perception data collection 12031, feature extraction 12032, routing network 12033, scene model 12034, instruction execution 12035 and cloud-based model synchronization service 12036.

[0220] The cloud super brain 1203 is used to synchronously send data to cockpit, instrument, smart rack and other services through the cloud model synchronization service 12036 and the terminal model synchronization service 11044.

[0221] The vehicle perception data collection 12031 in the cloud super brain 1203 can also receive data transmitted by the data bus 11051 in the AI ​​perception layer 1105.

[0222] The instruction execution 12035 in the cloud super brain 1203 is also used to send instructions to the AI ​​intelligent brain service 11032 in the basic core service 1103.

[0223] HMI: Central control interactive interface, including basic settings interface, scene interaction, local voice, navigation and cloud ecological entertainment application HMI, mainly provides interface interactive drawing capabilities and provides a user-oriented window.

[0224] Intelligent driving service: intelligent driving related functional modules, including:

[0225] 1) Perception module: It obtains environmental information through sensors such as cameras and radars, such as obstacles (other cars, pedestrians, etc.), lane lines, etc. This is the top priority of the entire intelligent driving system, because the information provided by the perception module is the basis for other modules to make decisions.

[0226] 2) Positioning module: The vehicle’s position information is obtained through technologies such as the Global Positioning System (GPS) or Real-time kinematic (RTK). This usually involves the use of maps so that the vehicle can accurately know its position in the environment.

[0227] 3) Planning module: Based on the environmental information and vehicle status (such as speed, acceleration, etc.) provided by the perception and positioning modules, it plans an optimal path or speed and outputs it to the control module.

[0228] 4) Control module: Based on the output of the planning module, it controls the acceleration, deceleration, steering and other actions of the vehicle to achieve safe and efficient driving.

[0229] AI Brain: AI intelligent terminal model, providing various decision-making and instruction distribution, and providing humanized active interaction and recommendation functions.

[0230] Instrument basic services: instrument side HMI and business middleware service modules.

[0231] Basic cockpit services: The cockpit is the hub of human-vehicle interaction and reflects the level of intelligence of the entire vehicle. It has functions and services such as voice recognition, gesture recognition, high-definition display, active safety alarm, real-time navigation, online infotainment, emergency rescue, and remote diagnosis. In this solution, the local cockpit service is weakened, and resources such as computing power and memory are released to driving and AI. The entertainment ecosystem is fully deployed in the cloud, and local interactions are carried out through H5, Real Time Streaming Protocol (RTSP), etc., breaking the traditional application (application, APP) list method. The local interactive system focuses on AI active interaction, and renders the dialogue interaction interface in real time through interactive scenes. In a weak network environment, only basic functions such as vehicle control, voice, and navigation are retained locally.

[0232] AI intelligent brain decision-making layer: Deployment of large AI models on the end and other small models, providing users with accurate and considerate services through algorithms based on environmental perception data and system data.

[0233] AI perception layer: mainly performs preliminary pre-processing of the vehicle perception layer data.

[0234] Hardware perception layer: The main sensors include on-board cameras, millimeter-wave radars, ultrasonic radars, lidars, intelligent lighting systems, etc. The vehicle's own motion information is mainly obtained through speed sensors, angle sensors, inertial navigation systems and other components on the vehicle body. Through the independent perception layer, the system can obtain sufficient perception data, such as in-vehicle vision (optical), voice (acoustic), and chassis and body data such as steering wheels, brake pedals, accelerator pedals, gears, seat belts, etc., and use biometric technology (mainly face recognition and voice recognition in the cabin) to comprehensively judge the driver's (or other passengers') physiological state (portrait, face recognition, etc.) and behavioral state (driving behavior, voice, body behavior), and then push and actively interact according to specific scenarios.

[0235] The centralized hardware perception layer is responsible for the collection, organization and distribution of sensor data for the entire vehicle, solving the data delay and resource occupation caused by the need to transmit sensor data between different domains in the past.

[0236] Cloud Super Brain: A cloud-based supercomputing platform that uses big data uploaded by terminals and natural environment data (weather, earthquakes and other natural factors data) to perform calculations and inferences on a large cloud model. The calculation results are converted into instructions and sent to each terminal, which then performs corresponding execution actions.

[0237] Cloud ecosystem management platform: mainly responsible for the docking and management of the three-party ecosystem. Different from the traditional ecosystem access method, it configures different ecosystems according to different car models, breaking the tradition of reintegrating development versions when the ecosystem changes, saving more resources.

[0238] Cloud business platform: mainly includes cloud diagnosis platform, big data collection platform, remote control, remote authorization, over-the-air update (OTA) and other business platforms.

[0239] Hardware expansion interface: Universal hardware connector, convenient for hardware expansion, making hardware pluggable and functions upgradeable as needed.

[0240] Linux base system: A Linux-based cabin-driver fusion system that can integrate various vehicle sensors, instruments and driving instructions into a single software platform.

[0241] Integrate the functional modules of the cockpit and intelligent driving into a single-chip SOC to achieve data sharing and interaction, and use the computing power scheduling mechanism and real-time communication capabilities of the single-chip SOC to ensure the collaborative work between different functional modules. According to the real-time requirements of the driving function, optimize the system scheduling strategy to ensure that key tasks can be processed in a timely manner. In the early stage of the system, the container strategy can be used to isolate the cockpit and driving system. In the later stage, the impact of non-critical processes (cockpit entertainment processes) on system performance can be directly reduced through a complete resource management mechanism, thereby improving the response speed of real-time tasks.

[0242] The following describes the cockpit entertainment ecosystem configuration and request process. The process may include but is not limited to the following steps A1 to A6.

[0243] Step A1: Input vehicle model ecological requirements into the cloud vehicle model configuration management platform.

[0244] Step A2: The vehicle configuration management platform configures the third-party ecological access address according to the needs and synchronizes the configuration table to the cloud ecological access interface platform.

[0245] Step A3: The terminal system starts to establish a connection with the cloud, and after the connection is successful, requests the vehicle entertainment configuration table (equivalent to the above-mentioned ecological configuration data).

[0246] Step A4: The cloud ecosystem access interface platform returns the latest configuration table to the terminal. If it is the first request, it is directly saved locally. If there is already a configuration table locally, the configuration table version is compared. If the version number is greater than the local version, it is updated and saved.

[0247] Step A5: The user operates the vehicle terminal HMI or voice interaction to play music, videos and other entertainment ecosystems.

[0248] Step A6: The front end sends a play request to the cockpit service, the cockpit accesses the corresponding link address, and feeds back the corresponding data stream or H5 web page, etc., and HMI rendering.

[0249] refer to Fig.12 As shown in the content, the cockpit entertainment ecosystem configuration and request process may include but is not limited to the following S1201 to S1205.

[0250] S1201. The cloud ecosystem management platform configures the third-party entertainment ecosystem to the cloud ecosystem interface based on the vehicle model.

[0251] S1202. The cockpit basic service requests a network address from the cloud ecological interface.

[0252] S1203. The cloud ecological interface returns the configuration word to the cockpit basic service.

[0253] S1204. HMI sends an access request to the cockpit basic service.

[0254] S1205. The cockpit basic service returns data stream, H5, etc. to the HMI.

[0255] The following describes the processing process of the independent perception layer data stream. The process may include but is not limited to the following steps B1 to B6.

[0256] The independent perception layer is responsible for collecting and integrating the perception data of the entire vehicle, including the data of the controller mounted on the VIU, such as temperature, radar, lidar, Bluetooth key, etc.; the controller data directly mounted on the system side, such as camera, microphone, sensor (G-sensor), positioning (IMU+Gnss), etc.

[0257] Step B1: The controller / sensor data under the VIU is uploaded to the VIU via Ethernet or SPI or UART protocol.

[0258] Step B2: VIU is synchronized to the independent perception layer data bus service in the cabin-driver fusion system via Ethernet or other high-speed protocol channels.

[0259] The controller sensing data is accessed through the hardware expansion interface and synchronized to the cabin-pilot fusion system data bus as needed.

[0260] Step B3: The data bus service organizes and integrates the data and distributes it to other subscribers.

[0261] Step B4: The cockpit service obtains bus data such as DVR recording, voice, etc. in real time according to the usage scenario through subscription.

[0262] Step B5: The driving service obtains various perception data of the vehicle in real time by subscribing to the data bus, and realizes automatic driving and assisted driving functions based on positioning and decision control, among which part of the pre-processing work can be placed on the data bus for processing.

[0263] Step B6: The data bus desensitizes the data and uploads it to the cloud big data collection center according to needs.

[0264] refer to Fig.13 As shown in the content, the processing process of the independent perception layer data stream may include but is not limited to the following S1301 to S1309.

[0265] S1301, the sensors under the VIU transmit data transparently to the VIU.

[0266] S1302. VIU transparently transmits data to the data bus.

[0267] S1303. Other sensors transparently transmit data to the data bus.

[0268] S1304, the data bus performs data integration.

[0269] S1305. The cockpit subscribes to data from the data bus.

[0270] S1306. The data bus distributes data to the cockpit.

[0271] S1307, Zhijia subscribes to data from the data bus.

[0272] S1308, the data bus distributes data to the intelligent driving.

[0273] S1309, the data bus desensitizes and uploads the data to the cloud.

[0274] The working process of the large model on the end side is described below. The process may include but is not limited to the following steps C1 to C6.

[0275] The large model on the end refers to a large-scale deep learning model deployed on the device side without cloud computing. This model runs completely locally, does not require an Internet connection, and can be used even in flight mode. Therefore, it can well meet the user's privacy needs and the need for intelligent control of the car in a weak network environment. Because the local large model on the end consumes a lot of computing resources, the end + cloud deployment solution is generally adopted at present.

[0276] Step C1: The controller / sensor data under the VIU is uploaded to the VIU via Ethernet or SPI or UART protocol.

[0277] Step C2: VIU passes the data to the hardware perception layer.

[0278] Step C3: The hardware perception layer collects the whole vehicle domain perception data (VIU down-hook controller, cabin chip mainboard down-hook controller, hardware expansion interface down-hook controller, etc.), classifies and organizes them, and transmits them to the data bus.

[0279] Step C4: The data bus distributes the data to various services (cockpit service, driving service, etc.) through subscription channels.

[0280] Step C5: The data bus AI preprocessing layer performs preliminary preprocessing on the data.

[0281] The preprocessed data enters the local AI scene model, and the local model forms the final instructions through a series of feature extraction.

[0282] Step C6: The corresponding instruction is input to the execution layer, and the execution layer performs corresponding active interaction or vehicle control operations upon receiving the instruction.

[0283] refer to Fig.14As shown in the content, the end-side large model working process may include but is not limited to the following S1401 to S1408.

[0284] S1401, the sensor transparently transmits the VIU and sensor data to the VIU.

[0285] S1402, VIU transparently transmits sensor data to the hardware perception layer.

[0286] S1403, the hardware perception layer integrates the sensor data of the entire vehicle into the data bus.

[0287] S1404, the data bus pre-processes the data and transmits it to the AI ​​decision-making layer.

[0288] S1405, the AI ​​decision layer calculates the command response based on the local model and transmits the command to the AI ​​execution layer.

[0289] S1406. The cloud AI super brain big model responds to the cloud big model instructions and gives the instructions to the AI ​​execution layer.

[0290] S1407, the AI ​​decision-making layer synchronizes the cloud model with the cloud AI super brain model.

[0291] S1408. The data bus uploads the integer data to the cloud AI super brain model.

[0292] In a fourth aspect, an embodiment of the present application provides a storage medium, that is, a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the vehicle control method provided in the first aspect is implemented.

[0293] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the vehicle control method provided in the first aspect is implemented.

[0294] It should be noted here that the description of the above storage medium, device, apparatus and program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the storage medium, device, apparatus and program product embodiments of this application, please refer to the description of the method embodiment of this application for understanding.

[0295] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0296] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0297] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0298] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0299] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0300] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0301] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0302] The above description is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A vehicle cabin-driver fusion system, characterized in that: The system includes a system-on-chip in a vehicle and a cloud device; The system-level chip integrates a cabin-pilot fusion controller and first data for cabin-pilot fusion control; the required resources of the first data match the resources of the system-level chip; The cloud device includes second data for the cabin-pilot fusion control; The system-level chip is used to obtain the second data from the cloud device through the cabin-drive fusion controller, and realize cabin control and intelligent driving control of the vehicle based on the first data and the second data.

2. The cabin-pilot fusion system according to claim 1, characterized in that: The first data includes: first functional data for the cockpit control; all functional data for the intelligent driving control; The second data includes: second functional data for the cockpit control; the first functional data is different from the second functional data.

3. The cabin-driver fusion system according to claim 2, characterized in that: When the second data includes second function data for cockpit control, the second function data includes one or more of the following: Eco-configuration data for the cockpit control, and partial algorithms for the cockpit control.

4. The cabin-pilot fusion system according to claim 3, characterized in that: The ecological configuration data includes at least one of the following: the application program, voice data, video data, and weather data of the cockpit; The partial algorithms include at least one of the following: a speech processing algorithm, a multi-person dialogue algorithm, and a scene recommendation algorithm.

5. The cabin-driver fusion system according to any one of claims 1 to 4, characterized in that: In the case where the cabin-driver fusion system includes a cockpit control subsystem and an intelligent driving control subsystem, The cockpit control subsystem and the intelligent driving control subsystem exchange data through a cross-process manner within the domain.

6. A system-on-chip, characterized in that: The system-level chip integrates a cabin-pilot fusion controller and first data for cabin-pilot fusion control; the required resources of the first data match the resources of the system-level chip; The system-level chip is used to obtain second data used for the cabin-driver fusion control from a cloud device through the cabin-driver fusion controller, and realize cabin control and intelligent driving control of the vehicle based on the first data and the second data.

7. The system-on-chip according to claim 6, characterized in that: The system-level chip includes a data acquisition layer, a service layer and a human-machine interface layer; The data acquisition layer is used to: acquire the first data; and transmit the first data to the service layer; The service layer is used to: obtain the second data from the cloud device; process the first data and the second data to obtain target output data; and render the target output data in real time and output it to the human-computer interface layer.

8. The system-on-chip according to claim 7, characterized in that: In the case where the data acquisition layer includes a hardware perception layer and a data processing layer; the service layer includes a cockpit service and an intelligent driving service; and the first data includes first detection data and second detection data, The hardware perception layer is used to: collect the whole vehicle sensor data of the vehicle and obtain the whole vehicle detection data of the vehicle; The data processing layer is used to: process the whole vehicle detection data to obtain first detection data for the cockpit service and second detection data for the intelligent driving service; The cockpit service is used to: call the first detection data, perform cockpit control based on the first detection data and the second data to obtain first output data, and render the first output data in real time and then output it to the human-machine interface layer; The intelligent driving service is used to: call the second detection data, perform intelligent driving control based on the second detection data to obtain second output data, and render the second output data in real time and output it to the human-machine interface layer.

9. The system-on-chip according to claim 8, characterized in that: In the case where the cockpit service includes at least two first services, for each of the at least two first services, the data processing layer is further configured to: Determine the detection data required by each of the first services in the first detection data; and transmit the detection data required by each of the first services to each of the first services in a subscription manner.

10. The system-on-chip according to claim 8 or 9, characterized in that: The system-level chip also includes a model processing layer; the model processing layer includes at least two models; the service layer also includes an intelligent processing layer; The data processing layer is further used to: process the vehicle detection data to obtain detection data for each of the at least two models; Each of the models in the model processing layer is further used to: call the detection data of each of the models by subscription; process the detection data to obtain model output data of each of the models, and transmit the model output data to the intelligent processing layer; The intelligent processing layer is used to obtain target data based on the model output data, perform real-time rendering and output it to the human-machine interface layer.

11. A vehicle control method, characterized in that: The method is applied to a system-level chip on the vehicle side, wherein the system-level chip integrates a cabin-driver fusion controller and first data for cabin-driver fusion control; The required resources of the first data are matched with the resources of the system-level chip; the method comprises: When the vehicle is connected to the cloud device, a first request is sent to the cloud device; the first request is used to request to obtain second data for the cabin-driver fusion control; receiving the second data sent by the cloud device; The cabin control and intelligent driving control of the vehicle are realized through the cabin-drive fusion controller based on the first data and the second data.

12. The method according to claim 11, characterized in that The implementing the cockpit control and intelligent driving control of the vehicle based on the first data and the second data by the cockpit-drive fusion controller includes: Implementing cockpit control of the vehicle based at least on the second data; and / or, Intelligent driving control of the vehicle is achieved based on the first data and the second data; the first data is detection data used for the intelligent driving control.

13. The method according to claim 12, characterized in that In a case where the first request is used to request obtaining ecological configuration data and the second data is new ecological configuration data, the step of implementing the cabin control of the vehicle at least based on the second data includes: Determining target ecological configuration data based on the received new ecological configuration data; displaying target ecological configuration data at a human-machine interface layer of the vehicle; receiving a first operation for the target ecological configuration data; In response to the first operation, obtaining feedback data corresponding to the first operation from the cloud; The feedback data is rendered and then outputted through the human-machine interface layer.

14. The method according to claim 13, characterized in that The step of determining target ecological configuration data based on the received new ecological configuration data includes: If the vehicle does not have the original ecological configuration data, determining the new ecological configuration data as the target ecological configuration data; If original ecological configuration data exists in the vehicle, the versions of the original ecological configuration data and the new ecological configuration data are compared; and the ecological configuration data with the latest version between the original ecological configuration data and the new ecological configuration data is determined as the target ecological configuration data.

15. The method according to claim 12 or 13, characterized in that The intelligent driving control of the vehicle based on the first data and the second data includes: Collecting the whole vehicle sensor data of the vehicle through the hardware perception layer to obtain the whole vehicle detection data of the vehicle; Processing the vehicle detection data through a data processing layer to obtain first data for the intelligent driving control; Acquiring first data for the intelligent driving control in the data processing layer by subscription; Intelligent driving control is performed on the vehicle based on the first data and the second data.

16. The method according to claim 15, characterized in that In the case where the subscribers include a cockpit service, an intelligent driving service, and N models, where N is greater than 1, the obtaining of the first data for the intelligent driving control in the data processing layer by subscription includes: The cockpit service obtains first detection data through a first subscription; The intelligent driving service obtains second detection data through a second subscription; Each of the N models obtains detection data corresponding to the model based on a subscription corresponding to the model.

17. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or an instruction, and when the computer program or the instruction is executed by the processor, the method according to any one of claims 11 to 16 is implemented.

18. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and when the computer program or instructions are executed by a processor, the method according to any one of claims 11 to 16 is implemented.

Images