A method for processing camera data, a dual-system architecture, and a computer device

Abstract

The present invention relates to the technical field of data processing, and discloses a method for processing camera data, a dual-system architecture, and a computer device. The method is applied to a first operating system that runs on a system-on-chip. A virtualization system is deployed on the system-on-chip, and a camera data shared memory area and multiple operating systems are deployed in the virtualization system. A camera engine service, a camera driver service, and multiple first-type applications are deployed on the first operating system. The camera engine service and the camera driver service deployed on the first operating system respond to and process camera request information, obtain camera application data in response to the camera request information, and store the camera application data in the shared memory area of the system-on-chip. Thus, the sharing of camera application data is achieved through the shared memory area. The resources of the system-on-chip are reasonably allocated and scheduled to ensure the efficient operation of the camera driver service and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method for processing camera data, a dual-system architecture, and a computer device. Background Art

[0002] With the development of intelligent vehicles, intelligent vehicles in related technologies can provide users with rich multimedia and infotainment functions, as well as functions such as assisted driving and autonomous driving. Specifically, generally, intelligent vehicles are configured with two operating systems, Android (entertainment information system) and Linux (safety-critical system). The Android system has a wide range of application ecosystems and user-friendly interfaces, and is commonly used to configure rich multimedia and infotainment systems. The Linux system has good stability and customizability, so it is usually applied to high-reliability and high-real-time operating environments, and is mainly used in intelligent vehicles to run key applications such as assisted driving and autonomous driving. The on-vehicle camera system is of great significance in function modules such as autonomous driving, advanced driver assistance, and in-vehicle infotainment. In related technologies, the on-vehicle camera system usually adopts a single on-vehicle camera system solution and directly deploys the on-vehicle camera system on the Android system. Figure 1 The structural schematic diagram of a single Android system camera system is shown, as Figure 1 shown, the single Android system camera system configures a camera driver module 101, a camera hardware abstraction module 102, and a camera service module 103 on the entertainment information system. Multiple camera applications 104 with different functions can also be deployed on the Android system. Here, the entertainment information system 100 is the Android system.

[0003] However, for a dual-system intelligent vehicle with an Android system and a Linux system, the single-point on-vehicle camera system solution that directly deploys the on-vehicle camera system on the Android system involves resource sharing and data transmission between different function modules during the data processing process, and the data format and type need to be processed during the data sharing process, which will result in serious competition and consumption of storage, transmission, and memory resources, and cannot ensure the timeliness of data processing and transmission, as well as the safety and stability of the operation of the intelligent vehicle. Summary of the Invention

[0004] In view of this, the present invention provides a method for processing camera data, a dual-system architecture, and a computer device to solve the problems of the safety and stability of camera data processing.

[0005] In a first aspect, the present invention provides a method for processing camera data, which is applied to a first operating system. The first operating system runs on a system-on-chip, and a virtualization system is deployed on the system-on-chip. The virtualization system deploys a camera data shared memory area and multiple operating systems. Any one of the multiple operating systems can access the shared memory area. A camera engine service, a camera driver service, and multiple first-class applications are deployed on the first operating system. The method includes:

[0006] The camera engine service receives camera request information sent by a first-class application. The camera request information carries camera identification information and camera data information.

[0007] The camera engine service parses the camera request information to obtain a camera control instruction, and sends the camera control instruction to the camera driver service.

[0008] Based on the camera control instruction, the camera engine service creates a shared file in the camera data shared memory area. The shared file has a file descriptor corresponding to the camera control instruction.

[0009] Based on the camera control instruction, the camera driver service obtains the raw camera data and feeds it back to the camera engine service.

[0010] The camera engine service converts the raw camera data into camera application data corresponding to the camera request information, and stores the camera application data in the shared file.

[0011] The first-class application obtains the camera application data from the shared file based on the file descriptor.

[0012] In the camera data processing method of the present invention, a virtualization system is deployed on the system-on-chip. Based on the virtualization system, a first operating system and a second operating system are deployed, and a shared memory area is configured on the virtualization system. Through virtualization technology, the isolation of operating systems is realized, effectively ensuring the independent operation between multiple systems and ensuring the stability and reliability of the system. Further, the camera engine service and the camera driver service deployed on the first operating system respond to and process the camera request information, obtain the camera application data in response to the camera request information, and store the camera application data in the shared memory area of the system-on-chip. Thus, when the camera request information for the same camera is consistent, the sharing of camera application data can be achieved through the shared memory area. Reasonably allocate and schedule the resources of the system-on-chip, avoid resource competition, and ensure the efficient operation of camera-related services such as the camera engine service and the camera driver service.

[0013] In some alternative embodiments, before the camera engine service receives the camera request information sent by the first-class application, the method further includes:

[0014] In response to a startup request of a first operating system, power-on operations are performed on multiple cameras connected to a system-on-chip, and a communication connection between the system-on-chip and the multiple cameras is established through a camera driver service.

[0015] In some alternative embodiments, a camera engine service parses camera request information to obtain a camera control instruction, including:

[0016] When the camera engine service determines that a first type of application has camera data permission for camera identification information based on the camera request information, a virtual camera corresponding to the camera request information is created;

[0017] Based on the camera data information, a camera control instruction is generated;

[0018] Among them, the virtual camera is used to obtain camera application data corresponding to the camera request information from a shared memory area.

[0019] In the camera data processing method of the present invention, when the camera engine service determines that a first type of application has camera data permission for camera identification information based on the camera request information, the camera engine service creates a virtual camera corresponding to the camera request information, and obtains the camera application data corresponding to the camera request information from the shared memory area through the virtual camera. Thus, a corresponding virtual camera is created for each camera request information, quickly realizing efficient transmission of camera application data, effectively avoiding data delay, and significantly improving the user experience.

[0020] In some alternative embodiments, after storing the camera application data in a shared file, the method further includes:

[0021] The camera engine service sends the file descriptor of the shared file to the virtual camera corresponding to the camera request information.

[0022] In the camera data processing method of the present invention, the association between the camera request information and the camera application data is realized through the file descriptor. Thus, the first type of application can obtain the corresponding camera application data through the file descriptor. During the data transmission process, only the file descriptor corresponding to the camera request information needs to be sent, without frequently transmitting a large amount of camera raw data or camera application data, effectively reducing consumption such as memory and data transmission bandwidth. At the same time, the file descriptor is used by multiple functional modules such as the first type of application to obtain camera application data, effectively realizing sharing of camera image data between single systems or across systems, and reducing consumption of memory and memory bandwidth.

[0023] In some alternative embodiments, before the camera engine service receives the camera request information sent by the first type of application, the method further includes:

[0024] Construct and display a page including a list of camera hardware device information, so that the first type of application sends camera request information based on the list of camera hardware device information.

[0025] In some alternative embodiments, sending the camera control instruction to the camera driver service includes:

[0026] The camera driver service arbitrates multiple received camera request messages;

[0027] When there are two or more identical camera request messages among the multiple camera request messages, the identical camera request messages are combined and sent to the camera driver service.

[0028] In the camera data processing method of the present invention, the camera driver service arbitrates multiple received camera request messages. When there are two or more identical camera request messages among the multiple camera request messages, the identical camera request messages are combined and sent to the camera driver service. Thus, for the camera request messages that need to obtain the same camera application data, they are combined and processed, effectively improving the processing efficiency of the camera request messages, fundamentally realizing resource sharing, and significantly improving the utilization rate of resources such as memory and bandwidth.

[0029] In some alternative embodiments, the camera driver service obtains the raw camera data based on the camera control instruction and feeds it back to the camera engine service, including:

[0030] The camera driver service obtains the camera acquisition data from the target camera corresponding to the camera identification information based on the camera control instruction;

[0031] Deserialize the camera acquisition data to obtain the raw camera data;

[0032] Send the raw camera data to the camera engine service.

[0033] In some alternative embodiments, the camera data information includes at least one of the following: request frame format, frame rate, application identifier of the first type of application, shared data identifier, where the shared data identifier is used to indicate whether the camera request information applies the common data in the shared memory area.

[0034] In some alternative embodiments, the camera control instruction includes the camera identification information and at least one of the following: application identifier of the first type of application, application priority of the first type of application, device address of the target camera, device data of the target camera, device data length of the target camera, and instruction type of the target camera.

[0035] In a second aspect, the present invention provides a method for processing camera data, which is applied to a second operating system. A camera management service, a camera hardware abstraction service, and multiple second-type applications are deployed on the second operating system. The second operating system runs on a system-on-chip, and a virtualization system is deployed on the system-on-chip. The virtualization system deploys a camera data shared memory area, a first operating system, and a second operating system. A camera engine service and a camera driver service are deployed on the first operating system. The method includes:

[0036] The camera management service receives camera request information sent by a second-type application and sends the camera request information to the camera hardware abstraction service. The camera request information carries camera identification information and camera data information.

[0037] The camera hardware abstraction service sends the camera request information to the camera engine service of the second operating system, so that: the camera engine service parses the camera request information to obtain a camera control instruction, and based on the camera control instruction, creates a shared file in the camera data shared memory area, and sends the camera control instruction to the camera driver service. The camera driver service obtains raw camera data based on the camera control instruction and feeds it back to the camera engine service. The camera engine service converts the raw camera data into camera application data corresponding to the camera request information and stores the camera application data in the shared file. Among them, the shared file has a file descriptor corresponding to the camera control instruction.

[0038] The second-type application obtains the camera application data from the shared file based on the file descriptor.

[0039] In some optional implementation manners, before the camera management service receives the camera request information sent by the second-type application, the method further includes:

[0040] In response to a startup request of the second operating system, the camera hardware abstraction service establishes a communication connection between the second operating system and the camera engine service through the virtualization system.

[0041] The camera data processing method of the present invention deploys a virtualization system on a system-on-chip, deploys a first operating system and a second operating system based on the virtualization system, configures a shared memory area on the virtualization system, and realizes the isolation of operating systems through virtualization technology, effectively ensuring the independent operation between multiple systems and the stability and reliability of the system. Further, when a second type of application in the second operating system has a request to use camera data, the camera management service can receive the camera request information sent by the second type of application and send the camera request information to the camera hardware abstraction service. The camera request information carries camera identification information and camera data information. The camera hardware abstraction service sends the camera request information to the camera engine service of the second operating system. The camera engine service can respond to and process the camera request information with the camera driver service deployed on the first operating system to obtain camera application data in response to the camera request information, and store the camera application data in the shared memory area of the system-on-chip. Thus, when the camera request information for the same camera is consistent, the sharing of camera application data can be achieved through the shared memory area. Reasonably allocate and schedule the resources of the system-on-chip, avoid resource competition, and ensure the efficient operation of camera-related services such as the camera engine service and the camera driver service.

[0042] In a third aspect, the present invention provides a dual-system architecture for in-vehicle camera control. The dual-system architecture for in-vehicle camera control is deployed on a system-on-chip. A virtualization system is deployed on the system-on-chip, and a camera data shared memory area and multiple operating systems are deployed on the virtualization system;

[0043] The multiple operating systems include a first operating system and a second operating system;

[0044] A camera engine service, a camera driver service, and multiple first types of applications are deployed on the first operating system. The first operating system executes the camera data processing method of the first aspect or any corresponding implementation manner thereof based on the camera engine service, the camera driver service, and the multiple first types of applications;

[0045] A camera management service, a camera hardware abstraction service, and multiple second types of applications are deployed on the second operating system. The second operating system executes the camera data processing method of the second aspect or any corresponding implementation manner thereof based on the camera management service, the camera hardware abstraction service, and the multiple second types of applications.

[0046] In a fourth aspect, the present invention provides a computer device, including the dual-system architecture for in-vehicle camera control provided in the third aspect or any corresponding implementation manner thereof.

[0047] The camera data processing method of the present invention deploys a virtualization system on a system-on-chip, deploys a first operating system and a second operating system based on the virtualization system, and configures a shared memory area on the virtualization system. Through virtualization technology, the isolation of the operating systems is achieved, effectively ensuring the independent operation between multiple systems and the stability and reliability of the system. Further, the camera engine service and the camera driver service deployed on the first operating system respond to and process the camera request information to obtain camera application data in response to the camera request information, and store the camera application data in the shared memory area of the system-on-chip. Thus, when the camera request information for the same camera is consistent, the sharing of the camera application data can be achieved through the shared memory area. Reasonably allocate and schedule the resources of the system-on-chip, avoid resource competition, and ensure the efficient operation of camera-related services such as the camera engine service and the camera driver service. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 Shows the structural schematic diagram of a single Android system camera system;

[0050] Figure 2 Is the compositional structural schematic diagram of the dual-system architecture for in-vehicle camera control provided by the present invention;

[0051] Figure 3 Is the flowchart of the camera data processing method according to an embodiment of the present invention;

[0052] Figure 4 Is the compositional structural schematic diagram of the camera engine service provided by an embodiment of the present invention;

[0053] Figure 5 Shows the format schematic diagram of the camera request information in the camera data processing method according to an embodiment of the present invention;

[0054] Figure 6 Shows the format schematic diagram of the camera control instruction in the camera data processing method according to an embodiment of the present invention;

[0055] Figure 7 Is the flowchart of another camera data processing method according to an embodiment of the present invention;

[0056] Figure 8It is a schematic flowchart of another camera data processing method according to an embodiment of the present invention;

[0057] Figure 9 It shows a schematic diagram of the transmission process of camera data between the Android system and the Linux system;

[0058] Figure 10 It shows a schematic flowchart of modifying camera parameters of the camera data processing method according to an embodiment of the present invention;

[0059] Figure 11 It is a schematic hardware structure diagram of a computer device according to an embodiment of the present invention. Detailed implementation manners

[0060] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0061] To explain the solution in more detail, here first a simple explanation of the application scenario of the camera data processing method according to an embodiment of the present invention is given.

[0062] Data collected by on-vehicle cameras usually needs to be applied to two operating systems, Android (entertainment information system) and Linux (safety-critical system). The camera system is usually installed on the Android system. When the Linux system needs to use the data collected by the camera, it obtains the data collected by the camera by communicating with the Android system. The functions of the camera system for processing the data collected from the camera system are concentrated on the Android system, which greatly increases the risk of a single point of failure. If the Android system fails or crashes due to a malfunction of the functional modules deployed on the system, the key applications for controlling vehicle safety in the Linux system will also be unable to obtain camera data. Moreover, the Android system cannot simultaneously meet the requirements of entertainment information and safety-critical applications and does not conform to the standards related to automotive functional safety. This will have a greater impact on the security and stability of the Linux system in the vehicle. Further, serious resource competition is likely to occur between multiple functional modules of the Android system and the Linux system, resulting in performance bottlenecks in the Linux system and the Android system of the vehicle, and further leading to a higher risk of system instability and malfunction of functional modules. Even further, the image data collected by the same physical camera cannot be fully shared between different functional modules of the Android and Linux systems. When each functional module needs to obtain the image data collected by a camera with specific requirements, it needs to go through complex data processing and transmission processes, with a high complexity in software design and implementation. At the same time, it will also increase the consumption of the vehicle's CPU (Central Processing Unit), memory, and memory bandwidth resources. Thus, the real-time performance of the Linux system in obtaining image data from the camera is poor and cannot meet the real-time requirements of the driving assistance functional modules deployed on the Linux system for camera data. In addition, the camera hardware can only be controlled by the Android system, and the Linux system cannot obtain the control authority of the camera hardware. Therefore, the Linux system cannot perform logical optimization and parameter dynamic debugging on the camera based on parameters such as the environment.

[0063] Based on this, the present invention provides a method for processing camera data, a dual-system architecture for controlling a vehicle-mounted camera, and a computer device. Specifically, an embodiment of the present invention provides a dual-system architecture for controlling a vehicle-mounted camera based on a single System on Chip (SOC), which realizes functional isolation, resource optimization, and high stability of the camera system, enables efficient sharing of resources and data of the image data of the same physical camera between the Android system and the Linux system, and also enables sharing of the control authority of the same physical camera between the Android and Linux systems at both ends. Thus, the functional modules deployed in the Linux system and the Android system can both control the camera or obtain data from the camera according to actual needs, significantly improving the user experience.

[0064] Figure 2 It is a schematic diagram of the composition structure of the dual-system architecture for controlling a vehicle-mounted camera provided by the present invention.

[0065] Reference Figure 2 , the dual-system architecture for controlling a vehicle-mounted camera provided by an embodiment of the present invention is deployed on a System on Chip (SOC) 200. A virtualization system 201 is deployed on the System on Chip 200, and a camera data shared memory area 2011 (Tshare) and multiple operating systems ( Figure 2 are shown as a first operating system 202 and a second operating system 203 in the figure) are deployed on the virtualization system (Hypervisor) 201. The multiple operating systems may include a first operating system 202 and a second operating system 203.

[0066] Among them, a camera engine service 2021, a camera driver service 2022, and multiple first-type applications 2023 are deployed on the first operating system 202. The first operating system 202 executes the method for processing camera data applied to the first operating system provided by an embodiment of the present invention based on the camera engine service 2021, the camera driver service 2022, and the multiple first-type applications 2023.

[0067] A camera management service 2031, a camera hardware abstraction service 2032, and multiple second-type applications 2033 are deployed on the second operating system 203. The second operating system 203 executes the method for processing camera data applied to the second operating system provided by an embodiment of the present invention based on the camera management service 2031, the camera hardware abstraction service 2032, and the multiple second-type applications 2033.

[0068] It should be noted that Figure 2Only one type-one application 2023 and one type-two application 2033 are shown. In actual applications, the numbers of the type-one application 2023 and the type-two application 2033 can both be configured according to actual requirements, and no specific limitation is provided here.

[0069] Furthermore, the first operating system 202 may be a Linux system, and the second operating system 203 may be an Android system. The type-one applications 2023 configured in the Linux system usually may include applications or functional modules related to key vehicle safety functions such as autonomous driving applications, ADAS (Advanced Driving Assistance System), vehicle panoramic imaging applications, sentry mode, fatigue inspection, health detection, multi-modal object recognition, and scene image enhancement. The type-two applications 2033 configured in the Android system usually may include entertainment information applications such as intelligent voice applications and audio-video playback applications. It should be noted that the functional modules deployed in the first operating system 2023 in the embodiments of the present invention are the type-one applications 2023 described here, and the functional modules deployed in the second system 2033 are the type-two applications 2033 described here.

[0070] Thus, based on one SOC, the embodiments of the present invention utilize virtualization technology. First, a Type-1 (bare-metal type) virtualization system (Hypervisor) is deployed. Based on the virtualization system, the Android system and the Linux system are deployed and isolated from each other. The Type-1 Hypervisor is a virtualization technology that directly runs on physical hardware such as SOC and does not require a host operating system, so it has the characteristics of high performance and low resource overhead.

[0071] Furthermore, the main control capabilities such as the camera engine service and the camera driver service of the camera are all deployed on the Linux system. The camera driver service can utilize the characteristics of the Linux kernel mode and directly utilize the efficient virtualization interface, shared memory, and interrupt mechanism provided by the Hypervisor, effectively reducing the abstraction and context switching operations of the intermediate layer, significantly reducing the system overhead and latency of the Linux system and the Android system, thereby improving the overall performance of the Linux system and the Android system. And the boot of the Linux system has priority over the boot of the Android system. Therefore, deploying the main control capabilities of the camera such as the camera driver on the Linux system effectively ensures that both the Linux system and the Android system can obtain the relevant data of the camera device or control the camera at the first time after the system starts.

[0072] Further, multiple cameras 204 ( Figure 2 shown as one camera 204 in Figure 2 ) can be directly connected to the system-on-chip 200 through a hardware serial port (not shown in the figure), etc.

[0073] The dual-system architecture for in-vehicle camera control provided by the present invention adopts a method in which the camera driver service directly interacts with the camera hardware through the virtualization interface provided by the virtualization system, enabling the camera driver in the Linux system kernel mode to more efficiently manage and utilize hardware resources. Therefore, the dual-system architecture for in-vehicle camera control provided by the present invention is applicable to application scenarios of functional modules with high requirements for performance and response time, such as driving assistance and autonomous driving. Further, through virtualization technology, the influence of functional modules or applications such as entertainment information in the Android system on the camera system driver is isolated, effectively avoiding the impact of other application failures on the camera system and significantly improving the reliability of the camera system.

[0074] According to an embodiment of the present invention, an embodiment of a camera data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0075] In this embodiment, a camera data processing method is provided, which can be used in the above-mentioned first operating system. The first operating system runs on the system-on-chip, and a virtualization system (Hypervisor) is deployed on the system-on-chip. The virtualization system deploys a camera data shared memory area (Tshare) and multiple operating systems, and any one of the multiple operating systems can access the shared memory area. A camera engine service (Camera Engine), a camera driver service (Camera Driver), and multiple first-class applications are deployed on the first operating system. Figure 3 is a flowchart of the camera data processing method according to an embodiment of the present invention. As Figure 3 shown, the process includes the following steps:

[0076] Step S301, the camera engine service receives camera request information sent by the first-class application. The camera request information carries camera identification information and camera data information.

[0077] In some optional implementation manners, in order to more specifically describe the processing process of the camera engine service for the camera request information, the composition structure of the camera engine service is further divided. Figure 4 is a schematic diagram of the composition structure of the camera engine service provided by an embodiment of the present invention. Refer to Figure 4As shown in the figure, the camera engine service in the embodiment of the present invention may include: an image data management module 401, an image data conversion module 402, an image data transmission module 403, a virtual device management module 404, a virtual device request module 405, a device description module 406, a camera control module 407, an instruction encryption and decryption module 408, and a frame rate management module 409. It should be noted that in practical applications, the module division of the camera engine service can be determined according to actual needs, Figure 4 which is only for illustrative purposes and is not used to limit the camera engine module. The functions of each module of the camera engine service will be described in combination with the camera data processing method in the embodiment of the present invention and will not be described separately.

[0078] When a first type of application needs to use camera data, it can first obtain the information of the connected physical camera hardware device from the device description module of the camera engine service, select the camera identification information (camera ID) corresponding to the physical camera to be opened, and encapsulate the camera identification information and the camera data information into a camerarequest msg (camera request information), and send the camera request information to the device request management module of the camera engine service. Thus, the camera engine service receives the camera request information sent from the first type of application.

[0079] Figure 5 The figure shows a schematic diagram of the format of the camera request information in the camera data processing method according to the embodiment of the present invention. Refer to Figure 5 , in some optional embodiments, the camera request information may include camera identification information (cameraID) and camera data information. The camera data information is mainly used to characterize the requirements of the functional module for the camera data or the control parameters of the camera. Specifically, the camera data information includes a request frame format (request Format), a frame rate (FPS, Frame Per Second), an application identifier of the first type of application, and a shared (common) data identifier. The shared data identifier is used to characterize whether the camera request information applies to the common data in the shared memory area. The application identifier of the first type of application may be the IP (Internet Protocol) address, function description, or other application identifiers that can uniquely identify the first type of application. The application identifier of the first type of application here is mainly used to distinguish the requester who sends the camera request information.

[0080] Step S302, the camera engine service parses the camera request information to obtain a camera control instruction, and sends the camera control instruction to the camera driver service.

[0081] In some optional embodiments, step S302 may include:

[0082] In step S3021, when the camera engine service determines that a first type of application has the camera data permission for the camera identification information based on the camera request information, a virtual camera corresponding to the camera request information is created.

[0083] In some alternative embodiments, the device request management module of the camera engine service sends the camera request information to the instruction encryption and decryption module, and it can determine whether the first type of application that sends the camera request information has the access permission for the target camera according to the application identification of the first type of application.

[0084] When the first type of application has the access permission for the target camera, a virtual camera (Virtual Camera, VCamera) is created according to the camera identification information of the target camera. A virtual camera can be created based on each camera request information, and the virtual camera is used to obtain the camera application data corresponding to the camera request information from the shared memory area.

[0085] In step S3022, a camera control instruction is generated based on the camera data information, where the virtual camera is used to obtain the camera application data corresponding to the camera request information from the shared memory area.

[0086] In the camera data processing method of the present invention, when the camera engine service determines that a first type of application has the camera data permission for the camera identification information based on the camera request information, the camera engine service creates a virtual camera corresponding to the camera request information, and obtains the camera application data corresponding to the camera request information from the shared memory area through the virtual camera. Thus, a corresponding virtual camera is created for each camera request information, quickly realizing the efficient transmission of the camera application data, effectively avoiding data delay, and significantly improving the user experience.

[0087] In some alternative embodiments, the following operations can be performed to send the camera control instruction to the camera driver service in step S302:

[0088] In step S3023, the camera driver service arbitrates the received multiple camera request information.

[0089] In some alternative embodiments, the arbitration may include determining whether the multiple camera request information is the same.

[0090] In step S3023, when there are two or more identical camera request information among the multiple camera request information, the identical camera request information is merged and sent to the camera driver service.

[0091] If there are two or more identical camera request messages among multiple camera request messages, the camera application data obtained by multiple first-class application requests is the same. At this time, the identical camera request messages can be merged and sent to the camera driver service.

[0092] Here, the judgment of whether the camera request messages are the same mainly includes judging whether information such as the camera identifier, request frame format, frame rate, and shared data identifier is the same. When judging the application identifiers of multiple first-class applications, it is possible to judge whether the function descriptions of the first-class applications are consistent, and use the judgment result as one of the judgment conditions for whether the camera request messages are consistent.

[0093] In the camera data processing method of the present invention, the camera driver service arbitrates multiple received camera request messages. When there are two or more identical camera request messages among the multiple camera request messages, the identical camera request messages are merged and sent to the camera driver service. Thus, for camera request messages that need to obtain the same camera application data, merging processing is performed, effectively improving the processing efficiency of camera request messages, fundamentally realizing resource sharing, and significantly improving the utilization rate of resources such as memory and bandwidth.

[0094] Figure 6 FIG. shows a schematic diagram of the format of a camera control instruction in the camera data processing method according to an embodiment of the present invention. Refer to Figure 6 In some optional embodiments, the camera control instruction may include camera identifier information, application identifier, application priority, device address, device data, device data length, and instruction type.

[0095] Among them, the camera identifier information is information that the camera control instruction must contain. In addition, the camera control instruction further includes at least one of the following: application identifier, application priority, device address, device data, device data length, and instruction type. The application identifier is the application identifier of the first-class application, and the application priority is the application priority of the first-class application. The device address, device data, device data length, and instruction type are all relevant parameters of the target camera.

[0096] Here, the camera identifier information is mainly used to represent the identifier of the physical camera for which the camera request information of the first-class application requests to obtain data. It can be the ID (Identity document) of the pre-allocated camera, which can represent the uniqueness of the corresponding physical camera. The device data of the target camera mainly refers to the parameters involved in controlling the target camera, such as: exposure, etc. The device data length mainly refers to the number of bits occupied by the device data, such as: 32 bits, 64 bits. The instruction type mainly refers to camera data unidirectional transmission, bidirectional transmission, read / write instruction, erase instruction, etc.

[0097] Step S303: Based on the camera control instruction, the camera engine service creates a shared file in the camera data sharing memory area, and the shared file has a file descriptor corresponding to the camera control instruction.

[0098] For each camera control instruction, a shared file can be created, and a file descriptor that can represent the uniqueness of the shared file is assigned to the shared file corresponding to the camera control instruction.

[0099] Step S304: Based on the camera control instruction, the camera driver service acquires the raw camera data and feeds it back to the camera engine service.

[0100] In some optional embodiments, step S304 may include:

[0101] Step S3041: Based on the camera control instruction, the camera driver service acquires the camera acquisition data from the target camera corresponding to the camera identification information.

[0102] The camera driver service opens the camera corresponding to the camera identification in the camera request information and acquires the camera acquisition data before serialization.

[0103] Step S3042: Deserialize the camera acquisition data to obtain the raw camera data.

[0104] The camera driver module can deserialize the camera acquisition data to obtain the raw camera data. Here, the raw camera data is the image information data collected based on the current operating parameters such as the exposure of the camera. After the deserialization operation, the image information data is not processed in terms of frame rate, data format, etc.

[0105] Step S3043: Send the raw camera data to the camera engine service.

[0106] Step S305: The camera engine service converts the raw camera data into camera application data corresponding to the camera request information and stores the camera application data in the shared file.

[0107] The camera engine service can convert the raw camera data into camera application data corresponding to the camera request information based on requirements such as frame rate and data format in the camera request information, and store the camera application data in the shared file. Here, a file descriptor can be assigned to the camera application data corresponding to each camera request information.

[0108] In some alternative embodiments, after storing the camera application data in a shared file, the camera engine service may send the file descriptor of the shared file to a virtual camera corresponding to the camera request information.

[0109] In this way, the association between the camera request information and the camera application data is realized through the file descriptor. Thus, the first type of applications can obtain the corresponding camera application data through the file descriptor. During the data transmission process, only the file descriptor corresponding to the camera request information needs to be sent, without frequently transmitting a large amount of raw camera data or camera application data, effectively reducing the consumption of memory, data transmission bandwidth, etc. At the same time, the file descriptor is used by multiple functional modules such as the first type of applications to obtain the camera application data, effectively realizing the sharing of camera image data between single systems or across systems and reducing the consumption of memory and memory bandwidth.

[0110] Step S306, the first type of applications obtain the camera application data from the shared file based on the file descriptor.

[0111] In some alternative embodiments, step S3023 merges the same camera request information and sends it to the camera driver service as a single camera control instruction. Then, here, if the shared data identifiers selected by multiple first type of applications that issue the same camera request information all indicate the use of common data, then multiple first type of applications can all obtain the camera application data from the same shared file through the file descriptor. The image data conversion module of the camera engine module does not need to perform multiple conversions on the raw camera data, effectively reducing the consumption of corresponding resources such as memory and memory bandwidth. Thus, multiple camera request information can all obtain the camera application data from the same shared file based on the same file descriptor. The camera application data is image information data that meets the requirements of the camera request information for the number of frames, format, etc.

[0112] The camera data processing method of the present invention deploys a virtualization system on a system-on-chip, deploys a first operating system and a second operating system based on the virtualization system, and configures a shared memory area on the virtualization system. Through virtualization technology, the isolation of the operating systems is realized, effectively ensuring the independent operation between multiple systems and ensuring the stability and reliability of the system. Further, the camera engine service and the camera driver service deployed on the first operating system respond to and process the camera request information to obtain the camera application data in response to the camera request information, and store the camera application data in the shared memory area of the system-on-chip. Thus, when the camera request information for the same camera is consistent, the sharing of the camera application data can be realized through the shared memory area. Reasonably allocate and schedule the resources of the system-on-chip, avoid resource competition, and ensure the efficient operation of camera-related services such as the camera engine service and the camera driver service.

[0113] In this embodiment, a camera data processing method is provided, which can be used in the above-mentioned first operating system. The first operating system runs on a system-on-chip, and a virtualization system (Hypervisor) is deployed on the system-on-chip. The virtualization system deploys a camera data shared memory area (Tshare) and multiple operating systems, and a camera engine service (Camera Engine), a camera driver service (Camera Driver), and multiple first-class applications are deployed on the first operating system. Figure 7 It is a flowchart of the camera data processing method according to an embodiment of the present invention, as Figure 7 shown, the process includes the following steps:

[0114] Step S701, in response to a startup request of the first operating system, perform a power-on operation on multiple cameras connected to the system-on-chip, and establish a communication connection between the system-on-chip and the multiple cameras through the camera driver service.

[0115] Based on the above embodiment, the camera driver is deployed on the first operating system, and the first operating system can be Linux. The camera driver in the Linux kernel mode can directly utilize the efficient virtualization interface, shared memory, and interrupt mechanism provided by the Hypervisor, reducing the abstraction and context switching of the intermediate layer, reducing system overhead and latency, and thus improving the overall performance. This direct interaction method enables the kernel-mode driver to manage and use hardware resources more efficiently. And the boot of the Linux system takes precedence over the boot of the Android system.

[0116] Thus, during the startup process of the Linux system, in response to a startup request of the first operating system, a power-on operation can be performed on the physical hardware of multiple cameras connected to the system-on-chip, and a communication connection between the system-on-chip and the multiple cameras can be established through the camera driver service. Serialized image data transmitted from the camera end is obtained through the Serdesc (deserialization chip) built in the camera, and then the encoded data of the camera is deserialized. At the same time, direct hardware communication with the camera is established through the Camera Driver.

[0117] Step S702, construct and display a page including a list of camera hardware device information, so that the first-class applications send camera request information based on the list of camera hardware device information.

[0118] In some alternative embodiments, the device description module of the camera engine service may obtain the hardware device information of the connected cameras, and display the hardware device information in the form of a hardware device information list on the page of the Linux system. The user may select the Camera ID corresponding to the physical device of the camera to be turned on based on the page including the camera hardware device information list, and send the camera request information determined based on the camera hardware device information list to the camera driver service.

[0119] Step S703: The camera engine service receives the camera request information sent by the first type of application. The camera request information carries camera identification information and camera data information.

[0120] For details, please refer to Figure 3 Step S301 of the illustrated embodiment, which will not be elaborated here.

[0121] Step S704: The camera engine service parses the camera request information to obtain a camera control instruction, and sends the camera control instruction to the camera driver service.

[0122] For details, please refer to Figure 3 Step S302 of the illustrated embodiment, which will not be elaborated here.

[0123] Step S705: The camera engine service creates a shared file in the camera data shared memory area based on the camera control instruction. The shared file has a file descriptor corresponding to the camera control instruction.

[0124] For details, please refer to Figure 3 Step S303 of the illustrated embodiment, which will not be elaborated here.

[0125] Step S706: The camera driver service obtains the raw camera data based on the camera control instruction and feeds it back to the camera engine service.

[0126] For details, please refer to Figure 3 Step S304 of the illustrated embodiment, which will not be elaborated here.

[0127] Step S707: The camera engine service converts the raw camera data into camera application data corresponding to the camera request information, and stores the camera application data in the shared file.

[0128] For details, please refer to Figure 3 Step S305 of the illustrated embodiment, which will not be elaborated here.

[0129] Step S708: The first type of application obtains the camera application data from the shared file based on the file descriptor.

[0130] For details, please refer toFigure 3 Step S306 of the illustrated embodiment will not be elaborated here.

[0131] In this embodiment, a method for processing camera data is provided, which can be used for the above-mentioned second operating system. The camera management service (Camera Manager), the camera hardware abstraction service (Camera HAL), and multiple second-type applications are deployed on the second operating system. The second operating system runs on a system-on-chip, and a virtualization system is deployed on the system-on-chip. The virtualization system deploys a camera data shared memory area, a first operating system, and a second operating system. The camera engine service and the camera driver service are deployed on the first operating system. Figure 8 is a flowchart of the method for processing camera data according to an embodiment of the present invention, as Figure 8 shown, the process includes the following steps:

[0132] Step S801, the camera management service receives the camera request information sent by the second-type application and sends the camera request information to the camera hardware abstraction service. The camera request information carries camera identification information and camera data information.

[0133] In some optional implementation manners, when a second-type operation deployed on the second operating system needs to turn on the camera, a connection with the first operating system can be established through the camera management service, and then the hardware device information of the connected camera can be obtained through the device camera engine service of the first operating system, so as to select the Camera ID corresponding to the hardware device of the camera to be turned on. The second-type application encrypts information such as the Camera ID, image format, frame rate requirement, and requester alias and encapsulates it into a piece of camera request information, and sends the camera request information to the camera management service. The camera management service receives the camera request information sent by the second-type application and sends the camera request information to the camera hardware abstraction service. The camera request information sent by the second-type application here is similar to the camera request information sent by the first-type camera request information, which will not be elaborated here.

[0134] Step S802, the camera hardware abstraction service sends the camera request information to the camera engine service of the second operating system, so that: the camera engine service parses the camera request information to obtain a camera control instruction, and based on the camera control instruction, creates a shared file in the camera data shared memory area, and sends the camera control instruction to the camera driver service. The camera driver service obtains the raw camera data based on the camera control instruction and feeds it back to the camera engine service. The camera engine service converts the raw camera data into camera application data corresponding to the camera request information and stores the camera application data in the shared file. Among them, the shared file has a file descriptor corresponding to the camera control instruction.

[0135] After the camera hardware abstraction service receives the camera request information, it can directly send the camera request information to the camera engine service of the second operating system.

[0136] After the camera engine service of the second operating system receives the camera request information, referring to the Figures 2 - 7 processing and response process of the camera information received by the first type of application in the embodiment shown above, it responds to the camera request information received by the first type of application.

[0137] Step S803, the second type of application obtains the camera application data from the shared file based on the file descriptor.

[0138] The second type of application can also obtain the camera application data from the shared file based on the file descriptor. Here, the file identifier corresponds to the camera control instruction, and the camera control instruction is determined based on the camera request.

[0139] In some optional embodiments, before the camera management service receives the camera request information sent by the second type of application, in response to the startup request of the second operating system, the camera hardware abstraction service establishes a communication connection between the second operating system and the camera engine service through the virtualization system.

[0140] Specifically, the second operating system can be the Android system, and the first operating system can be the Linux system. When the Android system starts, the camera hardware service is started on the Android side. The camera hardware service is different from the traditional single Android camera hardware service module. Here, the camera hardware service directly establishes communication with the camera engine service of the Linux system through the virtualization communication provided by the virtualization system. In this way, there is no need to deploy a camera driver module and an image management module, a control management module, a communication module, and an encryption and decryption module of the camera hardware in the Android system.

[0141] Figure 9Shows a schematic diagram of the transmission process of camera data between the Android system and the Linux system.

[0142] As shown in the reference Figure 9 , the Linux system configures the Linux image processing buffer module. The Linux image processing buffer module can allocate space for camera application data in the shared memory area for the image Buffer (image memory), and obtain the Buffer-FD (file descriptor in the image memory) from the shared memory area. Information such as camera application data can be exchanged between the Linux system and the Android system through the Buffer-ID message (file descriptor message) containing the Buffer-FD (file descriptor in the image memory). The file descriptor message can include the Buffer-ID, data type, etc. The Linux image processing buffer module of the Linux system sends the file descriptor message to the Android system. The Android system Buffer module receives and processes the message, obtains the FD (file descriptor), and outputs the image data included in the camera application data. The image memory can refer to the shared file in the above Figures 2 - 7 illustrated embodiment.

[0143] In the camera data processing method of the present invention, a virtualization system is deployed on a system-on-chip. Based on the virtualization system, a first operating system and a second operating system are deployed, and a shared memory area is configured on the virtualization system. Through virtualization technology, the isolation of the operating systems is realized, effectively ensuring the independent operation between multiple systems and the stability and reliability of the system. Further, when a second type of application in the second operating system has a request to use camera data, the camera management service can receive the camera request information sent by the second type of application and send the camera request information to the camera hardware abstraction service. The camera request information carries camera identification information and camera data information. The camera hardware abstraction service sends the camera request information to the camera engine service of the second operating system. The camera engine service can respond to and process the camera request information with the camera driver service deployed on the first operating system to obtain camera application data in response to the camera request information, and store the camera application data in the shared memory area of the system-on-chip. Thus, when the camera request information for the same camera is consistent, the sharing of camera application data can be achieved through the shared memory area. Reasonably allocate and schedule the resources of the system-on-chip, avoid resource competition, and ensure the efficient operation of camera-related services such as the camera engine service and the camera driver service.

[0144] In some alternative embodiments, whether it is the camera data processing method applied to the first operating system or the camera data processing method applied to the second operating system, the camera request information can be camera request information used to represent a modification to the camera parameters. Specifically, Figure 10 FIG. shows a schematic diagram of the camera parameter modification process of the camera data processing method according to an embodiment of the present invention.

[0145] Referring to Figure 10 , the Linux system 1001 belongs to the first operating system in other descriptions, and the safety-critical system function module on the Linux system belongs to the first type of application in other descriptions. The I2C operation module 10012 can be the camera driver module in other descriptions. The physical camera 10013 with a data interface that truly conforms to the I2C protocol is the camera in other descriptions. The Android system 1002 belongs to the second operating system in other descriptions, and the entertainment information system function module 10021 belongs to the second type of application in other descriptions. The camera parameter modification process of the camera data processing method according to an embodiment of the present invention can include:

[0146] Step S1001, the safety-critical system function module sends out camera request information.

[0147] Specifically, when the safety-critical system function module 10011 needs to adjust the camera parameters, for example: needs to control the camera, obtain camera parameters, write camera parameters. The following operations can be performed: encapsulate and encrypt information such as Camera ID, requester alias, priority, device address, device data, device data length, instruction type, etc. into cmd msg (camera request information). And send the camera request information to the I2C operation module 10012 through the camera engine service, etc. The safety-critical system function module 10011 can directly send the camera request information to the camera engine service, decrypt the camera request information through the instruction encryption and decryption module of the camera engine service, and send it to the I2C operation module.

[0148] Step S1002, the entertainment information system function module sends out camera request information.

[0149] Specifically, when the entertainment information system function module 10021 needs to adjust the camera parameters, for example: needs to control the camera, obtain camera parameters, write camera parameters. The following operations can be performed: encapsulate and encrypt information such as Camera ID, requester alias, priority, device address, device data, device data length, instruction type, etc. into cmd msg (camera request information). And send the camera request information to the I2C operation module 10012 through the camera engine service, etc.

[0150] The entertainment information system function module 10021 can decrypt the camera request information through the I2C virtualization operation module 10022 to obtain the sent camera request information.

[0151] Step S1003, the I2C virtualization operation module sends the decrypted camera request information to the virtualized I2C device.

[0152] Specifically, the I2C virtualization operation module 10022 can send the decrypted camera request information to the I2C operation module of the Linux system sent by the virtual I2C device 10021. The I2C virtualization operation module can be the camera management service of the Android system, and the virtual I2C can be the hardware abstraction service of the Android system.

[0153] Step S1004, the virtualized I2C device sends the decrypted camera request information to the I2C operation module.

[0154] Step S1005, arbitrate and merge multiple camera request information to obtain camera control parameters.

[0155] The I2C operation module can be a camera driver module. The camera driver module determines whether the first type of application or the second type of application has the operation permission of the camera through the application identifier in the camera request information. If there are multiple camera request information sent by the first type of application or the second type of application, when the Camera ID is the same, arbitration is performed according to the application identifier, priority and other information of the first type of application or the second type of application to obtain "the adjusted lens parameters after arbitration and merging". At the same time, the physical camera 10013 that truly has a data interface conforming to the I2C protocol only responds to the most reasonable camera request information. The data of the most reasonable camera request information is written into the camera hardware device through the Camera Driver to complete the dynamic configuration effect of the camera hardware according to the environment. Here, the real I2C can be a camera that conforms to the I2C (communication protocol). The judgment basis of the most reasonable camera request information can be set according to the actual situation. For example, the camera request information sent by the application with the highest priority in the first type of application and the second type of application can be preferentially responded to.

[0156] The present invention provides a dual-system architecture for in-vehicle camera control. The dual-system architecture for in-vehicle camera control can refer to the above Figure 2 , the dual-system architecture for in-vehicle camera control is deployed on a system-on-chip. A virtualization system is deployed on the system-on-chip, and a camera data shared memory area and multiple operating systems are deployed on the virtualization system;

[0157] The multiple operating systems include a first operating system and a second operating system;

[0158] Deploy a camera engine service, a camera driver service, and multiple first - type applications on a first operating system. Based on the camera engine service, the camera driver service, and the multiple first - type applications, the first operating system executes the camera data processing method of the first aspect or any corresponding embodiment thereof.

[0159] Deploy a camera management service, a camera hardware abstraction service, and multiple second - type applications on a second operating system. Based on the camera management service, the camera hardware abstraction service, and the multiple second - type applications, the second operating system executes the camera data processing method of the second aspect or any corresponding embodiment thereof.

[0160] Please refer to Figure 11 , Figure 11 which is a schematic structural diagram of a computer device provided by an optional embodiment of the present invention. As Figure 11 shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high - speed interface and a low - speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (such as an array of servers, a set of blade servers, or a multi - processor system). Figure 11 In

[0161] FIG., a single processor 10 is taken as an example.

[0162] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above - mentioned embodiments.

[0163] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely provided relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] The memory 20 may include volatile memory, for example, random access memory; the memory may also include non-volatile memory, for example, flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memory.

[0165] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0166] The embodiments of the present invention further provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-transitory machine-readable storage medium and to be downloaded through a network and stored in a local storage medium, so that the methods described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0167] A part of the present invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operations of the computer. Those skilled in the art should understand that the forms of existence of computer program instructions in a computer-readable medium include but are not limited to source files, executable files, installation package files, etc. Correspondingly, the ways for computer program instructions to be executed by a computer include but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible by the computer.

[0168] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for processing camera data, characterized in that, Applied to a first operating system, the first operating system runs on a system-on-chip, a virtualization system is deployed on the system-on-chip, a camera data shared memory area and multiple operating systems are deployed in the virtualization system, a camera engine service, a camera driver service and multiple first-class applications are deployed on the first operating system, any one of the multiple operating systems can access the shared memory area, the first operating system is a Linux system, the second operating system is an Android system, a camera management service, a camera hardware abstraction service and multiple second-class applications are deployed on the second operating system; The method includes: The camera engine service receives camera request information sent by the first-class application, and the camera request information carries camera identification information and camera data information; The camera engine service parses the camera request information to obtain a camera control instruction, and sends the camera control instruction to the camera driver service; Based on the camera control instruction, the camera engine service creates a shared file in the camera data shared memory area, and the shared file has a file descriptor corresponding to the camera control instruction; Based on the camera control instruction, the camera driver service acquires camera raw data and feeds it back to the camera engine service; The camera engine service converts the camera raw data into camera application data corresponding to the camera request information, and stores the camera application data in the shared file; The first-class application obtains the camera application data from the shared file based on the file descriptor.

2. The camera data processing method according to claim 1, wherein Before the camera engine service receives the camera request information sent by the first-class application, the method further includes: In response to a startup request of the first operating system, perform a power-on operation on multiple cameras connected to the system-on-chip, and establish a communication connection between the system-on-chip and the multiple cameras through the camera driver service.

3. The camera data processing method according to claim 1, wherein The camera engine service parses the camera request information to obtain a camera control instruction, including: When the camera engine service determines based on the camera request information that the first-class application has the camera data permission for the camera identification information, create a virtual camera corresponding to the camera request information; Generate a camera control instruction based on the camera data information; Wherein, the virtual camera is used to obtain camera application data corresponding to the camera request information from the shared memory area.

4. The camera data processing method according to claim 3, wherein, After storing the camera application data in the shared file, the method further includes: The camera engine service sends the file descriptor of the shared file to the virtual camera corresponding to the camera request information.

5. The camera data processing method according to claim 1, wherein Before the camera engine service receives the camera request information sent by the first-class application, the method further includes: Construct and display a page including a list of camera hardware device information, so that the first-class application sends the camera request information based on the list of camera hardware device information.

6. The camera data processing method according to claim 1, wherein Sending the camera control instruction to the camera driver service includes: The camera driver service arbitrates multiple received camera request messages; When there are two or more identical camera request messages among the multiple camera request messages, the identical camera request messages are merged and sent to the camera driver service.

7. The camera data processing method according to claim 1, wherein The camera driver service obtaining camera raw data based on the camera control instruction and feeding it back to the camera engine service includes: The camera driver service obtains camera acquisition data from a target camera corresponding to the camera identification information based on the camera control instruction; Deserializing the camera acquisition data to obtain the camera raw data; Sending the camera raw data to the camera engine service.

8. The camera data processing method according to any one of claims 1-7, characterized in that, The camera data information includes at least one of the following: request frame format, frame rate, application identifier of the first type of application, shared data identifier, where the shared data identifier is used to indicate whether the camera request message applies the public data in the shared memory area.

9. The camera data processing method according to any one of claims 1-7, characterized in that The camera control instruction includes camera identification information and at least one of the following: application identifier of the first type of application, application priority of the first type of application, device address of the target camera, device data of the target camera, device data length of the target camera, and instruction type of the target camera.

10. A method for processing camera data, characterized in that, Applied to a second operating system, where a camera management service, a camera hardware abstraction service, and multiple second type of applications are deployed on the second operating system, the second operating system runs on a system-on-chip, a virtualization system is deployed on the system-on-chip, a camera data shared memory area, a first operating system, and the second operating system are deployed in the virtualization system, a camera engine service and a camera driver service are deployed on the first operating system, the first operating system is a Linux system, the second operating system is an Android system, the method includes: The camera management service receives the camera request message sent by the second type of application and sends the camera request message to the camera hardware abstraction service, and the camera request message carries camera identification information and camera data information; The camera hardware abstraction service sends the camera request message to the camera engine service of the first operating system, so that: the camera engine service parses the camera request message to obtain a camera control instruction, and based on the camera control instruction, creates a shared file in the camera data shared memory area, and sends the camera control instruction to the camera driver service, the camera driver service obtains camera raw data based on the camera control instruction and feeds it back to the camera engine service, the camera engine service converts the camera raw data into camera application data corresponding to the camera request message, and stores the camera application data in the shared file, where the shared file has a file descriptor corresponding to the camera control instruction; The second type of application obtains the camera application data from the shared file based on the file descriptor.

11. The camera data processing method according to claim 10, characterized in that, Before the camera management service receives the camera request information sent by the second type of application, the method further includes: In response to the startup request of the second operating system, the camera hardware abstraction service establishes a communication connection between the second operating system and the camera engine service through the virtualization system.

12. A dual-system architecture for in-vehicle camera control, characterized in that, The dual-system architecture for in-vehicle camera control is deployed on a system-on-chip. A virtualization system is deployed on the system-on-chip, and a camera data shared memory area and multiple operating systems are deployed on the virtualization system; The multiple operating systems include a first operating system and a second operating system; A camera engine service, a camera driver service, and multiple first types of applications are deployed on the first operating system. The first operating system executes the camera data processing method according to any one of claims 1-9 based on the camera engine service, the camera driver service, and the multiple first types of applications; A camera management service, a camera hardware abstraction service, and multiple second types of applications are deployed on the second operating system. The second operating system executes the camera data processing method according to claim 10 or 11 based on the camera management service, the camera hardware abstraction service, and the multiple second types of applications.

13. A computer device, characterized in that, including: The dual-system architecture for in-vehicle camera control according to claim 12.

Images