Method, device, storage medium and electronic device for sharing camera of multiple operating systems

Abstract

The present application relates to a kind of multi-operating system shared camera's method, device, storage medium and electronic equipment, comprising: first virtual machine module, second virtual machine module, virtual machine monitoring module, hardware resource module, physical camera, first operating system is installed in first virtual machine module, second operating system is installed in second virtual machine module, first operating system and second operating system run on same processor platform, virtual machine monitoring module distributes hardware resource module to second virtual machine module, to make physical camera utilize hardware resource module and carry out camera collection, after receiving camera data acquisition request, the physical address information of the video buffer area configured by first virtual machine module is transmitted to second virtual machine module by virtual machine monitoring module, to make second virtual machine module obtain corresponding camera data from physical camera and write in the video buffer area corresponding to physical address information.Can improve the camera efficiency of multi-operating system.

Description

Technical Field

[0001] This invention relates to the field of virtualization technology, and in particular to a method, apparatus, storage medium, and electronic device for sharing a camera across multiple operating systems. Background Technology

[0002] In the field of automotive intelligent cockpits, dual-operating system virtualization or multi-operating system virtualization technology based on Type I hypervisor has been widely used in mass production environments. Under this technical solution, taking a dual-operating system as an example, a single processor platform can simultaneously run a Linux virtual machine and an Android virtual machine. The Linux virtual machine hosts the instrument cluster system, while the Android virtual machine hosts the in-vehicle infotainment system. Both the Linux and Android virtual machines require cameras (video recording), such as for reversing cameras or surround-view cameras. However, due to limitations in hardware and software design, the same camera module cannot be directly used by both virtual machines simultaneously. This leads to a competition for camera resources when the two virtual machines in the dual-operating system have camera requirements, resulting in low camera efficiency. Furthermore, deploying two camera modules in the same location to meet the requirement of simultaneous acquisition of camera data by both operating systems significantly increases hardware costs, thereby reducing the economic efficiency and integration efficiency of the dual-operating system. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, storage medium and electronic device for sharing cameras across multiple operating systems.

[0004] Specifically, the present invention is achieved through the following technical solution: According to a first aspect of the present invention, a device for sharing a camera across multiple operating systems is provided. The device comprises: a first virtual machine module, a second virtual machine module, a virtual machine monitoring module, a hardware resource module, and a physical camera, wherein... The first virtual machine module has a first operating system installed, and the second virtual machine module has a second operating system installed. The first operating system and the second operating system run on the same processor platform. The virtual machine monitoring module is used to allocate the hardware resource module to the second virtual machine module so that the physical camera can use the hardware resource module to capture images. The second virtual machine module is used to obtain the second camera data corresponding to the second camera data acquisition request from the physical camera after receiving the second camera data acquisition request triggered by the second operating system; The first virtual machine module is used to, after receiving the first camera data acquisition request triggered by the first operating system, obtain the physical address information of the video buffer configured for the video data type according to the video data type corresponding to the first camera data acquisition request, and output the video data type and physical address information to the virtual machine monitoring module. The virtual machine monitoring module is also used to forward video data type and physical address information to the second virtual machine module; The second virtual machine module is also used to determine the corresponding target physical camera based on the video data type after receiving the video data type and physical address information, obtain the first camera data from the target physical camera that uses the hardware resource module to collect camera data, and store the first camera data in the video buffer corresponding to the physical address information.

[0005] Optionally, the first virtual machine module includes: a virtual video device, a camera interface, and a front-end virtual camera driver, wherein, The front-end virtual camera driver is used to create virtual video devices based on the virtual I / O mechanism, and to set up the corresponding camera interface and allocate a video buffer for each created virtual video device. The system uses a virtual video device to receive a first camera data acquisition request triggered by the first operating system. Based on the video data type of the virtual video device, it obtains the physical address information of the video buffer configured for the virtual video device. The physical address information and the video data type corresponding to the virtual video device are then output to the virtual machine monitoring module through the camera interface corresponding to the virtual video device.

[0006] Optionally, the camera interface is configured with a camera configuration file corresponding to the virtual video device, and the front-end virtual camera driver is also used to set up a hardware abstraction layer for the virtual video device, and register a camera instance and the camera configuration file corresponding to the camera instance in the hardware abstraction layer.

[0007] Optionally, the camera configuration file includes: maximum image buffer size, number of video and still image buffers, and supported resolution and frame rate range.

[0008] Optionally, the second virtual machine module includes: a backend virtual camera driver, a backend virtual camera forwarding service, and a hardware driver interface, wherein, The backend virtual camera driver is used to forward the received video data type and physical address information to the backend virtual camera forwarding service. The backend virtual camera forwarding service is used to determine the corresponding target physical camera based on the video data type, determine the corresponding target hardware driver interface based on the target physical camera, obtain the first camera data from the target physical camera connected to the target hardware driver interface, and store the first camera data in the video buffer corresponding to the received physical address information.

[0009] Optionally, the backend virtual camera forwarding service is further configured to, upon receiving a second camera data acquisition request triggered by the second operating system, determine the corresponding second physical camera based on the camera data type carried in the second camera data acquisition request, determine the corresponding second hardware driver interface based on the second physical camera, and acquire the second camera data from the target physical camera connected to the second hardware driver interface.

[0010] Optionally, the first virtual machine module includes an Android virtual machine, and the second virtual machine module includes a Linux virtual machine, wherein the Linux virtual machine is the backend and the Android virtual machine is the frontend.

[0011] The multi-operating system shared camera device in this technical solution includes: a first virtual machine module, a second virtual machine module, a virtual machine monitoring module, a hardware resource module, and a physical camera. The first virtual machine module contains a first operating system, and the second virtual machine module contains a second operating system; both operating systems run on the same processor platform. The virtual machine monitoring module allocates the hardware resource module to the second virtual machine module, enabling the physical camera to utilize the hardware resource module for image acquisition. The second virtual machine module, upon receiving a second camera data acquisition request triggered by the second operating system, acquires the second camera data corresponding to the second camera data acquisition request from the physical camera. The first virtual machine module... Upon receiving a first camera data acquisition request triggered by the first operating system, the system retrieves the physical address information of the video buffer configured for the video data type according to the video data type requested, and outputs the video data type and physical address information to the virtual machine monitoring module. The virtual machine monitoring module further forwards the video data type and physical address information to the second virtual machine module. The second virtual machine module, upon receiving the video data type and physical address information, determines the corresponding target physical camera based on the video data type, acquires the first camera data from the target physical camera using the hardware resource module for camera data acquisition, and stores the first camera data in the video buffer corresponding to the physical address information. In this way, by using the virtual machine monitoring module to forward the physical address information of the video buffer configured by the first virtual machine module for the first camera data acquisition request to the second virtual machine module, enabling the second virtual machine module to acquire camera data from the target physical camera using the hardware resource module and store it in the video buffer corresponding to the physical address information, multiple operating systems can share a physical camera, thereby improving the camera efficiency of multiple operating systems.

[0012] According to a second aspect of the present invention, a method for sharing a camera across multiple operating systems is provided, the method comprising: A virtual machine monitoring module that runs a first virtual machine module, a second virtual machine module that configures hardware resource modules, and a virtual machine module that allocates hardware resource modules to the second virtual machine module so that physical cameras can use hardware resource modules to capture images is provided on the same processor platform. The first virtual machine module has a first operating system installed in it, and the second virtual machine module has a second operating system installed in it. In response to the second virtual machine module receiving a second camera data acquisition request triggered by the second operating system, the second camera data corresponding to the second camera data acquisition request is acquired from the physical camera and displayed in the second virtual machine module; In response to the first virtual machine module receiving a first camera data acquisition request triggered by the first operating system, the physical address information of the video buffer configured for the video data type is obtained according to the video data type obtained in the first camera data acquisition request, and the video data type and physical address information are forwarded to the second virtual machine module through the virtual machine monitoring module. In response to the video data type and physical address information received by the second virtual machine module, the corresponding target physical camera is determined based on the video data type. First camera data is obtained from the target physical camera that uses the hardware resource module to collect camera data, and the first camera data is stored in the video buffer corresponding to the physical address information for display in the second virtual machine module.

[0013] According to a third aspect of the present invention, a storage medium is provided having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method for sharing a camera across multiple operating systems in any possible implementation of the first aspect.

[0014] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for sharing a camera across multiple operating systems in any possible implementation of the first aspect. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a device for sharing a camera across multiple operating systems, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of another device for sharing a camera across multiple operating systems, provided in an embodiment of the present invention; Figure 3 A flowchart illustrating a method for sharing a camera across multiple operating systems, provided in an embodiment of the present invention; Figure 4 This is another flowchart illustrating a method for sharing a camera across multiple operating systems, as provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In related technologies, running dual operating systems (Linux and Android virtual machines) simultaneously on a single processor platform and acquiring camera data through camera resource contention results in low efficiency for multiple operating systems acquiring camera data. For example, in a single operating system like Android, multiple applications (APPs) installed may simultaneously require camera data, but the camera (camera module) can only serve one application at a time. In such cases, a camera virtualization engine is introduced to aggregate and process the image parameter sets of multiple applications at the operating system level. Based on the image acquisition requests from multiple applications to the same target camera, and according to the image parameters contained in the corresponding image acquisition requests, the engine calculates the image parameter that maximizes the common part of each request parameter, and sends this maximized image parameter to the target camera. This controls the target camera to acquire image or video data according to this maximized image parameter. For example, both application A and application B request to use the same target camera. Application A requires an image resolution of 480P, while application B requires an image resolution of 720P. However, the highest resolution supported by the target camera is 1080P. The camera virtualization engine maximizes the common part of the image acquisition requests from application A and application B, obtaining 720P as the camera acquisition resolution to satisfy application B's high-resolution requirement. In this way, by reducing the resolution to 480P, application A's low-resolution requirement can also be satisfied, thus enabling multiple applications to share the same camera. However, this method is only suitable for single-operating system scenarios and is difficult to directly apply to the problem of sharing camera resources in a multi-operating system virtualization environment. For example, in related technologies, when an application requests access to a camera, the camera management component obtains camera information by querying the camera list in a single operating system. Based on the obtained camera information, it starts a virtual camera component, which then controls the physical camera through the camera interface. This allows the application to access the camera device through the camera interface, supporting the coexistence of physical and virtual cameras under a single operating system. This enables the application to flexibly access camera resources without being restricted to a single physical camera. However, this method still cannot solve the technical problem of multiple operating systems sharing cameras simultaneously in the context of automotive smart cockpits.

[0020] This embodiment aims to provide a method and apparatus for sharing a camera across multiple operating systems. By expanding the application scenario of camera sharing from sharing multiple applications under a single operating system to multiple operating systems, it enables secure and efficient sharing of the same camera resource between different virtual machines or operating systems.

[0021] See Figure 1 and Figure 2 This invention provides a device for sharing a camera across multiple operating systems. The device may include: a first virtual machine module 101, a second virtual machine module 102, a virtual machine monitoring module 103, a hardware resource module 104, and a physical camera 105. The first virtual machine module 101 has a first operating system installed, and the second virtual machine module 102 has a second operating system installed. The first operating system and the second operating system run on the same processor platform. The virtual machine monitoring module 103 is used to allocate the hardware resource module 104 (hardware resources) to the second virtual machine module 102 so that the physical camera 105 can use the hardware resource module 104 to perform camera capture. The second virtual machine module 102 is used to obtain the second camera data corresponding to the second camera data acquisition request from the physical camera 105 after receiving the second camera data acquisition request triggered by the second operating system; The first virtual machine module 101 is used to, after receiving the first camera data acquisition request triggered by the first operating system, obtain the physical address information of the video buffer configured for the video data type according to the video data type corresponding to the first camera data acquisition request, and output the video data type and physical address information to the virtual machine monitoring module 103. The virtual machine monitoring module 103 is also used to forward video data type and physical address information to the second virtual machine module 102; The second virtual machine module 102 is further configured to, after receiving the video data type and physical address information, determine the corresponding target physical camera based on the video data type, obtain the first camera data from the target physical camera that uses the hardware resource module 104 to collect camera data, and store the first camera data in the video buffer corresponding to the physical address information.

[0022] In this embodiment, multiple virtual machine modules are run on a single processor platform through a Type I Hypervisor. As an optional embodiment, the first virtual machine module includes an Android virtual machine, and the first operating system installed is the Android operating system. The second virtual machine module includes, but is not limited to, a Linux virtual machine, and the second operating system installed is the Linux operating system. The Linux virtual machine serves as the backend processing system for camera data, i.e., the Linux virtual machine is the backend Linux virtual machine, and the Android virtual machine serves as the frontend system for using camera data, i.e., the Android virtual machine is the frontend Android virtual machine.

[0023] In this embodiment, as an optional implementation, hardware resources are allocated to the Linux virtual machine via hardware pass-through. The Android virtual machine obtains camera data through a virtualization mechanism to achieve indirect access to the camera data. As an optional implementation, hardware resources include, but are not limited to: a camera module, a Mobile Industry Processor Interface (MIPI CSI), and an Image Signal Processor (ISP), to support different physical cameras.

[0024] In this embodiment, as an optional embodiment, the first virtual machine module includes: a virtual video device, a camera interface, and a front-end virtual camera driver, wherein, The front-end virtual camera driver is used to create virtual video devices based on the virtual I / O mechanism, and to set up the corresponding camera interface and allocate a video buffer for each created virtual video device. The system uses a virtual video device to receive a first camera data acquisition request triggered by the first operating system. Based on the video data type of the virtual video device, it obtains the physical address information of the video buffer configured for the virtual video device. The physical address information and the video data type corresponding to the virtual video device are then output to the virtual machine monitoring module through the camera interface corresponding to the virtual video device.

[0025] In this embodiment, the virtual video device is connected to the front-end virtual camera driver through a camera interface, and the front-end virtual camera driver is also connected to the Hypervisor.

[0026] In this embodiment, a virtual camera component is configured in the front-end Android virtual machine to enable the front-end Android virtual machine to support access to external or virtual cameras. The virtual camera component includes, but is not limited to, a virtual video device, a camera interface, and a front-end virtual camera driver connected to the camera interface. As an optional embodiment, the front-end virtual camera driver creates a virtual video device connected to the front-end virtual camera driver through the camera interface based on a virtual I / O mechanism, allocates a video buffer for the virtual video device, and sends the physical address information of the allocated video buffer to the back-end virtual camera driver through a Type I Hypervisor.

[0027] In this embodiment, the camera interface is configured with a camera configuration file corresponding to the virtual video device. As an optional embodiment, the front-end virtual camera driver sets up a hardware abstraction layer for the virtual video device, registers a camera instance and the camera configuration file corresponding to the camera instance in the hardware abstraction layer, so that by configuring the camera configuration file, the front-end Android virtual machine can identify and manage the virtual camera resources corresponding to the camera instance when it starts up, providing access capabilities for the camera interface, and subsequently, the camera instance can be called to display the acquired camera data according to the camera configuration file.

[0028] In this embodiment, as an optional implementation, the front-end virtual camera driver allocates an independent video buffer for each camera instance, and ensures that the video buffer can be efficiently accessed between different virtual machines through mechanisms such as memory sharing or direct memory access buffer (DMA-BUF, Data Memory Access Buffer), thereby achieving zero-copy data transmission.

[0029] In this embodiment, as an optional implementation, the camera configuration file is used to define key parameters of the camera, including but not limited to: maximum image buffer size, number of video and still image buffers, supported resolution and frame rate range. This allows the front-end Android virtual machine to initialize virtual camera resources according to the key camera parameters upon startup, and to acquire image or video data from the physical camera according to the configured key parameters. In this embodiment, the front-end virtual camera driver can dynamically adjust the number and size of the video buffer according to the virtual camera's acquisition needs to ensure the continuity and stability of image acquisition.

[0030] In this embodiment, as another optional embodiment, instance identifiers can be set for external cameras or virtual cameras in the hardware abstraction layer. This allows the front-end Android virtual machine to simultaneously identify internal cameras and external virtual cameras. When calling the camera interface, the corresponding camera resource can be selected according to the camera instance identifier, thereby enabling multi-instance management and access and supporting multiple instance cameras.

[0031] In this embodiment, as an optional embodiment, the virtual video device includes, but is not limited to: a camera service virtual device (hereinafter referred to as Camera Service) and a car service virtual device (hereinafter referred to as Car Service). The hardware abstraction layer set up for the camera service virtual device includes, but is not limited to, the camera hardware abstraction layer (CameraX HAL), and the hardware abstraction layer set up for the car service virtual device includes, but is not limited to, the enhanced vision system hardware abstraction layer (EVS HAL).

[0032] In this embodiment, as an optional implementation, a front-end virtual camera driver is implemented in the front-end Android virtual machine kernel. This front-end virtual camera driver creates a virtual video device based on a virtual I / O mechanism, allocates a video buffer for the camera interface, and maintains the video buffer state. For example, the physical address information of the video buffer and the video data type corresponding to the virtual video device are encapsulated into a virtio protocol data structure for subsequent transmission.

[0033] In this embodiment, the Hypervisor encapsulates the video data type and physical address information of the video buffer output by the front-end virtual camera driver and forwards them to the back-end virtual camera driver to establish a video buffer sharing channel between different virtual machines.

[0034] In this embodiment, as an optional embodiment, the second virtual machine module includes: a backend virtual camera driver, a backend virtual camera forwarding service, and a hardware driver interface, wherein... The backend virtual camera driver is used to forward the received video data type and physical address information to the backend virtual camera forwarding service. The backend virtual camera forwarding service is used to determine the corresponding target physical camera based on the video data type, determine the corresponding target hardware driver interface based on the target physical camera, obtain the first camera data from the target physical camera connected to the target hardware driver interface, and store the first camera data in the video buffer corresponding to the received physical address information.

[0035] In this embodiment, the backend virtual camera forwarding service is also used to, after receiving the second camera data acquisition request triggered by the second operating system, determine the corresponding second physical camera based on the camera data type carried in the second camera data acquisition request, determine the corresponding second hardware driver interface based on the second physical camera, and acquire the second camera data from the target physical camera connected to the second hardware driver interface.

[0036] In this embodiment, as an optional embodiment, the backend virtual camera forwarding service is in user mode. The user-mode backend virtual camera forwarding service receives the physical address information of the video buffer sent by the backend virtual camera driver. Based on the connection established in advance with the physical camera, it fills the camera data collected by the physical camera into the video buffer corresponding to the physical address information, thereby realizing the real-time video data acquisition of the frontend virtual machine.

[0037] In this embodiment, as an optional implementation, the backend virtual camera driver receives the physical address information of the video buffer sent by the frontend virtual camera driver through the Hypervisor. Based on the physical address information, it registers a character device in the kernel and sends the physical address information, i.e., the physical address information of the video buffer, to the user-mode backend virtual camera forwarding service through a pre-set file operation interface. The user-mode backend virtual camera forwarding service receives the physical address information of the video buffer sent by the backend virtual camera driver and, based on the pre-established connection with the physical camera device, fills the video buffer corresponding to the physical address information with the camera data acquired by the physical camera of the corresponding video data type, thereby realizing the real-time video data acquisition of the frontend virtual machine.

[0038] In this embodiment, in order to reduce the control link latency between the backend and the frontend, a real-time process is configured for the user-mode backend virtual display forwarding service, and the backend virtual camera forwarding service is scheduled and configured in real time. After the backend virtual display driver transmits information to the backend virtual camera forwarding service, the backend virtual camera forwarding service obtains camera data from the physical camera according to the configured real-time process, and transmits the obtained camera data to the video buffer corresponding to the physical address information, so as to improve the real-time performance and overall response speed of camera data transmission.

[0039] In this embodiment, the backend Linux virtual machine, Hypervisor, physical camera module, frontend virtual camera driver, backend virtual camera driver, and backend user-mode virtual camera forwarding service work together to realize the virtualization transmission and sharing of camera video data.

[0040] The following uses an Android virtual machine in a car's smart cockpit as the front-end Android virtual machine and a Linux virtual machine as the back-end Linux virtual machine as an example to explain the various parts involved in the multi-operating system camera sharing method of this embodiment.

[0041] Camera configuration for the front-end Android virtual machine: In this embodiment, configuration is performed in the front-end Android virtual machine to enable it to support access to external or virtual cameras. As an optional embodiment, the configuration includes: integrating the camera configuration file and camera service program into the hardware abstraction layer, and registering camera instances in the hardware abstraction layer, so that the Android system can identify and manage external or virtual camera resources at startup, providing access capabilities for the camera interface.

[0042] Camera parameter management: Define key camera parameters in the camera configuration file, including but not limited to: maximum image buffer size, number of video and still image buffers, and supported resolution and frame rate range. This allows the Android system to initialize camera resources based on these key parameters during startup, enabling the front-end Android virtual machine to acquire image or video data according to the configured parameters.

[0043] Multi-camera instance support: In the hardware abstraction layer, a separate instance identifier is provided for external or virtual cameras, so that the front-end Android virtual machine can recognize both internal and external virtual cameras at the same time. Thus, when calling the camera interface, the corresponding camera resource can be selected according to the instance identifier, realizing the management and access of multiple camera instances.

[0044] Creating a front-end virtual camera driver: Within the Android virtual machine kernel, a front-end virtual camera driver is implemented. This driver, based on virtual I / O mechanisms, creates a virtual video device, allocates a video buffer for the camera interface (camera instance), and ensures efficient access to the video buffer across different virtual machines through mechanisms such as shared memory or Direct Memory Access Buffer (DMA-BUF), achieving zero-copy data transfer while maintaining the video buffer's state. For example, the physical address information of the video buffer is encapsulated into a virtio protocol data structure for subsequent transmission. Furthermore, the number and size of the video buffers are dynamically adjusted according to camera acquisition requirements to ensure the continuity and stability of image acquisition.

[0045] Hypervisor forwarding: In this embodiment, the Hypervisor is responsible for receiving the physical address information of the video buffer sent by the front-end virtual camera driver and forwarding the physical address information to the back-end virtual camera driver, thereby realizing a buffer sharing channel between the front-end virtual machine and the back-end virtual machine. Through the Hypervisor's forwarding mechanism, different virtual machines can share the video data (camera data) of the physical camera, while avoiding conflicts caused by directly accessing the physical camera.

[0046] Backend virtual camera driver: A backend virtual camera driver is implemented in a backend Linux virtual machine. This backend virtual camera driver registers a lightweight character device, such as a miscellaneous device (misc), in the backend Linux virtual machine kernel for data interaction with the user-space backend virtual camera forwarding service. As an optional embodiment, the backend virtual camera driver receives control information and video buffer address information from the frontend virtual camera driver and provides a file operation interface, including read, write, control operations, and memory mapping functions. It maps the physical address information of the video buffer to the user-space backend virtual camera forwarding service, enabling the user-space backend virtual camera forwarding service to access and manipulate the video buffer memory, thereby achieving zero-copy data transmission and efficient virtual camera data transmission. As an optional embodiment, through ioctl functions or similar control interfaces, the user-space backend virtual camera forwarding service can obtain and configure the video buffer status and control information, achieving synchronous management of the frontend and backend buffers.

[0047] User-side backend virtual camera forwarding service: A user-space backend virtual camera forwarding service is implemented in the backend Linux virtual machine. This service establishes a communication channel with the kernel's backend virtual camera driver by enabling the character device registered by the backend virtual camera driver. Simultaneously, the video interface corresponding to the physical camera is started to obtain operational permissions for the physical camera.

[0048] In this embodiment, the backend virtual camera forwarding service parses the control information sent by the frontend virtual camera driver through the Hypervisor, and configures the physical camera's acquisition parameters based on the parsing results, including but not limited to resolution, frame rate, and buffer address. This allows the camera data acquired by the physical camera to be directly written to the video buffer of the frontend Android virtual machine, achieving transparent transmission of camera data from the physical camera to the frontend Android virtual machine. Thus, using the method of this embodiment, the frontend Android virtual machine can obtain real-time video acquisition data without directly accessing the physical camera, enabling secure and efficient sharing of single-camera resources in multi-virtual machine or multi-operating system environments.

[0049] In this embodiment, by employing zero-copy technology among the front-end virtual machine, the Hypervisor, and the back-end virtual machine, direct sharing of the end-to-end video buffer can be achieved. This eliminates redundant copying of camera data in memory, allowing both the front-end and back-end virtual machines to share the camera with extremely low memory and CPU consumption. This effectively improves camera efficiency across multiple operating systems, reduces memory bandwidth consumption, and thus lowers resource usage. Simultaneously, direct sharing significantly reduces data transmission latency and improves the virtual machine's response speed to camera data, thereby enhancing data transmission efficiency and overall display performance, achieving high performance and low latency. Furthermore, camera control commands can be separated from the camera data transmission channel, enabling independent management of control signals and data streams, reducing coupling, and improving stability and maintainability. Furthermore, in this embodiment, through the front-end virtual camera driver, the Hypervisor forwarding mechanism, and the user-space back-end virtual camera forwarding service, multiple virtual machines can securely and efficiently share single or multiple physical cameras with minimal resource consumption. This ensures that the front-end virtual machine obtains data collected by the physical camera in real time, enabling multiple operating systems to share single or multiple cameras. In addition, the method of this embodiment is applicable to camera resource sharing scenarios in multi-operating system environments, especially suitable for multi-system virtualization application scenarios of embedded systems such as automotive smart cockpits. It can be used in prototype verification and mass production environments, enabling secure and efficient sharing of single or multiple cameras between different virtual machines, supporting multi-instance management, thereby improving system resource utilization efficiency and display function stability, and enhancing system flexibility and scalability. For example, in a typical application scenario, a Type I hypervisor runs both a Linux virtual machine and an Android virtual machine on a single processor. The Linux virtual machine acts as the backend processing system, responsible for managing the physical camera and image acquisition, while the Android virtual machine acts as the frontend user system, requiring real-time acquisition of video data from the physical camera for use in in-vehicle infotainment (IVI) systems or driver assistance applications (such as reversing and surround view). Using the method in this embodiment, the real-time performance and stability of the physical camera data can be guaranteed, while also taking into account the access needs of different virtual machines, thus meeting the requirements for safe and efficient use of cameras in a multi-operating system environment.

[0050] Figure 3 This is a flowchart illustrating a method for sharing a camera across multiple operating systems, as provided in an embodiment of the present invention. Figure 3 As shown, the process includes: S101, A virtual machine monitoring module that runs a first virtual machine module, a second virtual machine module that configures hardware resource modules, and a virtual machine module that allocates hardware resource modules to the second virtual machine module so that physical cameras can use hardware resource modules to capture images, wherein the first virtual machine module has a first operating system installed and the second virtual machine module has a second operating system installed. In this embodiment, as an optional embodiment, the first virtual machine module includes an Android virtual machine, and the second virtual machine module includes a Linux virtual machine, wherein the Linux virtual machine is the backend and the Android virtual machine is the frontend.

[0051] S102. In response to the second virtual machine module receiving a second camera data acquisition request triggered by the second operating system, the second camera data corresponding to the second camera data acquisition request is acquired from the physical camera and displayed in the second virtual machine module. In this embodiment, as an optional embodiment, after receiving the second camera data acquisition request triggered by the second operating system, the backend virtual camera forwarding service determines the corresponding second physical camera based on the camera data type carried in the second camera data acquisition request, determines the corresponding second hardware driver interface based on the second physical camera, and acquires the second camera data from the target physical camera connected to the second hardware driver interface.

[0052] S103. In response to the first virtual machine module receiving a first camera data acquisition request triggered by the first operating system, the first virtual machine module acquires the physical address information of the video buffer configured for the video data type according to the video data type acquired in the first camera data acquisition request, and forwards the video data type and physical address information to the second virtual machine module through the virtual machine monitoring module. In this embodiment, as an optional embodiment, the first virtual machine module includes: a virtual video device, a camera interface, and a front-end virtual camera driver. In response to the first virtual machine module receiving a first camera data acquisition request triggered by the first operating system, it acquires the physical address information of the video buffer configured for the video data type according to the video data type corresponding to the first camera data acquisition request, and forwards the video data type and physical address information to the second virtual machine module through the virtual machine monitoring module, including: The front-end virtual camera driver creates virtual video devices based on the virtual I / O mechanism, and sets up a corresponding camera interface and allocates a video buffer for each created virtual video device. In response to the virtual video device receiving a first camera data acquisition request triggered by the first operating system, the system obtains the physical address information of the video buffer configured for the virtual video device based on the video data type of the virtual video device, and outputs the physical address information and the video data type corresponding to the virtual video device to the virtual machine monitoring module through the camera interface corresponding to the virtual video device, so that the video data type and physical address information can be forwarded to the second virtual machine module through the virtual machine monitoring module.

[0053] In this embodiment, as an optional embodiment, the camera interface is configured with a camera configuration file corresponding to the virtual video device. The front-end virtual camera driver is also used to set up a hardware abstraction layer for the virtual video device and register a camera instance and the camera configuration file corresponding to the camera instance in the hardware abstraction layer.

[0054] In this embodiment, as an optional embodiment, the camera configuration file includes: maximum image buffer size, number of video and still image buffers, and supported resolution and frame rate range.

[0055] S104. In response to the video data type and physical address information received by the second virtual machine module, determine the corresponding target physical camera based on the video data type, obtain the first camera data from the target physical camera that uses the hardware resource module to collect camera data, and store the first camera data in the video buffer corresponding to the physical address information for display in the second virtual machine module.

[0056] In this embodiment, as an optional embodiment, the second virtual machine module includes: a backend virtual camera driver, a backend virtual camera forwarding service, and a hardware driver interface. Responding to the video data type and physical address information received by the second virtual machine module, it determines the corresponding target physical camera based on the video data type, obtains first camera data from the target physical camera that uses the hardware resource module to collect camera data, and stores the first camera data in the video buffer corresponding to the physical address information, including: In response to the backend virtual camera driver receiving video data type and physical address information, it forwards the information to the backend virtual camera forwarding service. The backend virtual camera forwarding service determines the corresponding target physical camera based on the video data type, determines the corresponding target hardware driver interface based on the target physical camera, obtains the first camera data from the target physical camera connected to the target hardware driver interface, and stores the first camera data in the video buffer corresponding to the physical address information based on the received physical address information.

[0057] Figure 4 This is another flowchart illustrating a method for sharing a camera across multiple operating systems, as provided in an embodiment of the present invention. Figure 4As shown, this process includes the execution flow of the front-end application and the execution flow of the back-end forwarding service program. The execution flow of the front-end application includes: S201. The front-end application launches the virtual camera device; In this embodiment, as an optional embodiment, the front-end application is a front-end Android virtual machine. The first operating system installed in the front-end Android virtual machine starts the virtual camera device and notifies the front-end virtual machine driver, i.e., the front-end virtual camera driver.

[0058] S202. Obtain the device attribute information of the virtual camera device; In this embodiment, as an optional implementation, device attribute information is obtained through a front-end virtual machine driver. The device attribute information refers to key parameters of the virtual camera, including but not limited to: maximum image buffer size, number of video and still image buffers, and supported resolution and frame rate range.

[0059] S203. Set the frame parameters of the virtual camera device according to the device attribute information; In this embodiment, as an optional embodiment, the frame parameters include, but are not limited to, frame format and frame rate parameters, and the set frame parameters are output to the front-end virtual machine driver.

[0060] S204. Based on the virtual camera device, request frame buffer memory mapping; In this embodiment, as an optional embodiment, the frame buffer memory mapping includes, but is not limited to: mapping the video buffer to physical address information, and outputting the frame buffer memory mapping to the front-end virtual machine driver.

[0061] S205, Start data acquisition; In this embodiment, data collection is initiated by queuing, which means sending a data collection request to the front-end virtual machine driver. The front-end virtual machine driver then forwards the data collection request to the back-end virtual machine driver through the virtual machine monitoring module (Hypervisor).

[0062] S206. Process the camera data based on the camera data forwarded by the front-end virtual machine driver.

[0063] In this embodiment, image processing is performed by dequeuing, and after the dequeueing process is completed, the image is re-enqueued.

[0064] The execution flow of the backend forwarding service program includes: S301, The backend forwarding service program starts the physical camera device; In this embodiment, as an optional embodiment, the backend forwarding service program is a backend virtual camera forwarding service, which is set in a backend Linux virtual machine. A second operating system is also installed in the backend Linux virtual machine. The physical camera device is associated with the second operating system. After the backend Linux virtual machine starts, the physical camera device is started.

[0065] S302. Obtain the device attribute information of the physical camera device; In this embodiment, as an optional implementation, device attribute information is obtained through a backend virtual driver, i.e., a backend virtual camera driver. The device attribute information refers to key parameters of the physical camera, including but not limited to: maximum image buffer size, number of video and still image buffers, and supported resolution and frame rate range.

[0066] S303. Set the frame parameters of the physical camera device according to the device attribute information; In this embodiment, as an optional embodiment, the frame parameters include, but are not limited to, frame format and frame rate parameters, and the set frame parameters are output to the front-end virtual machine driver.

[0067] S304. Based on the frame parameters of the physical camera device, request frame buffer memory mapping; In this embodiment, as an optional embodiment, the frame buffer memory mapping includes, but is not limited to, the mapping between the video buffer and physical address information.

[0068] S305. Start data collection based on the data collection request sent by the backend virtual machine driver; In this embodiment, the acquisition request is forwarded from the front-end virtual machine driver to the back-end virtual driver through the virtual machine monitoring module, and the acquisition is started by queuing, so as to realize the sharing of the acquired camera data by multiple operating systems.

[0069] S306. After the collection is completed, leave the team.

[0070] In this embodiment, the collected camera data is stored in the video buffer, and the backend virtual driver is notified to perform the dequeue process. After the dequeue process is completed, the data is re-enqueued.

[0071] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method for sharing a camera across multiple operating systems in any of the above possible implementations.

[0072] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0073] Based on the same inventive concept, see [link to inventive concept] Figure 5 This invention also provides an electronic device, including a memory 101 (e.g., non-volatile memory), a processor 102, and a computer program stored on the memory 101 and executable on the processor 102. When the processor 102 executes the program, it implements the steps of the multi-operating system shared camera method described in any of the above possible implementations, which can be equivalent to the aforementioned multi-operating system shared camera device. Of course, the processor can also be used to process other data or perform calculations. This electronic device can be a PC, server, terminal, or other similar device.

[0074] like Figure 5 As shown, the electronic device may also include: memory 103, network interface 104, and internal bus 105. In addition to these components, other hardware may also be included, which will not be described in detail here.

[0075] It should be noted that the above-mentioned multi-operating system shared camera device can be implemented by software. As a logical device, it is formed by the processor 102 of the electronic device in which it is located reading the computer program instructions stored in the non-volatile memory into the memory 103 for execution.

[0076] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0077] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by special-purpose logic circuitry—such as FPGA (Field Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit), and the device can also be implemented as special-purpose logic circuitry.

[0078] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0079] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0080] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0081] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0082] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0084] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A device for sharing camera footage across multiple operating systems, characterized in that, include: The system comprises a first virtual machine module, a second virtual machine module, a virtual machine monitoring module, a hardware resource module, and a physical camera. The first virtual machine module has a first operating system installed, and the second virtual machine module has a second operating system installed. The first operating system and the second operating system run on the same processor platform. The virtual machine monitoring module is used to allocate the hardware resource module to the second virtual machine module so that the physical camera can use the hardware resource module to capture images. The second virtual machine module is used to obtain the second camera data corresponding to the second camera data acquisition request from the physical camera after receiving the second camera data acquisition request triggered by the second operating system; The first virtual machine module is used to, after receiving the first camera data acquisition request triggered by the first operating system, obtain the physical address information of the video buffer configured for the video data type according to the video data type corresponding to the first camera data acquisition request, and output the video data type and physical address information to the virtual machine monitoring module. The virtual machine monitoring module is also used to forward video data type and physical address information to the second virtual machine module; The second virtual machine module is also used to determine the corresponding target physical camera based on the video data type after receiving the video data type and physical address information, obtain the first camera data from the target physical camera that uses the hardware resource module to collect camera data, and store the first camera data in the video buffer corresponding to the physical address information.

2. The device for sharing a camera across multiple operating systems according to claim 1, characterized in that, The first virtual machine module includes: a virtual video device, a camera interface, and a front-end virtual camera driver, wherein, The front-end virtual camera driver is used to create virtual video devices based on the virtual I / O mechanism, and to set up the corresponding camera interface and allocate a video buffer for each created virtual video device. The system uses a virtual video device to receive a first camera data acquisition request triggered by the first operating system. Based on the video data type of the virtual video device, it obtains the physical address information of the video buffer configured for the virtual video device. The physical address information and the video data type corresponding to the virtual video device are then output to the virtual machine monitoring module through the camera interface corresponding to the virtual video device.

3. The device for sharing a camera across multiple operating systems according to claim 2, characterized in that, The camera interface is configured with a camera configuration file corresponding to the virtual video device. The front-end virtual camera driver is also used to set up a hardware abstraction layer for the virtual video device and register camera instances and camera configuration files corresponding to those camera instances in the hardware abstraction layer.

4. The device for sharing a camera across multiple operating systems according to claim 3, characterized in that, The camera configuration file includes: maximum image buffer size, number of video and still image buffers, and supported resolution and frame rate range.

5. The apparatus for sharing a camera across multiple operating systems according to any one of claims 1 to 4, characterized in that, The second virtual machine module includes: a backend virtual camera driver, a backend virtual camera forwarding service, and a hardware driver interface, wherein, The backend virtual camera driver is used to forward the received video data type and physical address information to the backend virtual camera forwarding service. The backend virtual camera forwarding service is used to determine the corresponding target physical camera based on the video data type, determine the corresponding target hardware driver interface based on the target physical camera, obtain the first camera data from the target physical camera connected to the target hardware driver interface, and store the first camera data in the video buffer corresponding to the received physical address information.

6. The device for sharing a camera across multiple operating systems according to claim 5, characterized in that, The backend virtual camera forwarding service is also used to, after receiving a second camera data acquisition request triggered by the second operating system, determine the corresponding second physical camera based on the camera data type carried in the second camera data acquisition request, determine the corresponding second hardware driver interface based on the second physical camera, and acquire the second camera data from the target physical camera connected to the second hardware driver interface.

7. The apparatus for sharing a camera across multiple operating systems according to any one of claims 1 to 4, characterized in that, The first virtual machine module includes an Android virtual machine, and the second virtual machine module includes a Linux virtual machine, wherein the Linux virtual machine is the backend and the Android virtual machine is the frontend.

8. A method for sharing camera footage across multiple operating systems, characterized in that, The method for sharing cameras across multiple operating systems includes: A virtual machine monitoring module that runs a first virtual machine module, a second virtual machine module that configures hardware resource modules, and a virtual machine module that allocates hardware resource modules to the second virtual machine module so that physical cameras can use hardware resource modules to capture images is provided on the same processor platform. The first virtual machine module has a first operating system installed in it, and the second virtual machine module has a second operating system installed in it. In response to the second virtual machine module receiving a second camera data acquisition request triggered by the second operating system, the second camera data corresponding to the second camera data acquisition request is acquired from the physical camera and displayed in the second virtual machine module; In response to the first virtual machine module receiving a first camera data acquisition request triggered by the first operating system, the physical address information of the video buffer configured for the video data type is obtained according to the video data type obtained in the first camera data acquisition request, and the video data type and physical address information are forwarded to the second virtual machine module through the virtual machine monitoring module. In response to the video data type and physical address information received by the second virtual machine module, the corresponding target physical camera is determined based on the video data type. First camera data is obtained from the target physical camera that uses the hardware resource module to collect camera data, and the first camera data is stored in the video buffer corresponding to the physical address information for display in the second virtual machine module.

9. A storage medium, characterized in that, The storage medium stores a program or instructions, which are executed by a processor to implement the steps of the method for sharing a camera across multiple operating systems as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for sharing a camera across multiple operating systems as described in any one of claims 1 to 7.

Images