Vehicle-mounted panoramic image implementation method and device based on virtual Camera and electronic equipment

Abstract

The invention discloses a virtual Camera-based vehicle-mounted panoramic image implementation method and device and electronic equipment, and relates to the field of auxiliary driving, and the method comprises the steps: collecting image data around a vehicle; image data around the vehicle is uploaded to an AIS Server unit through Camera; splicing the image data around the vehicle through a preset algorithm in the security domain; generating a panoramic video; acquiring a virtual Camera video stream according to the panoramic video; the virtual Camera video stream is transmitted back to the AIS Server unit through a Reprocess flow, and the AIS Server unit is used for sending the virtual Camera video stream back to the AIS Server unit; the virtual Camera video stream is copied to an output buffer area of an application request through an FAWPlug Proc module; the UserDone Proc module is called to send the panoramic video stream to the panoramic image application of the security domain and the entertainment domain respectively, the video after panoramic stitching is fused into the existing video channels of the security domain and the entertainment domain through the processed video virtual Camera, the application can use the panoramic image video like using other real Camera, and the use experience of the user is improved. And meanwhile, the stability of the existing frame is furthest ensured.

Description

Technical Field

[0001] This application relates to the field of assisted driving, and in particular to a method for realizing in-vehicle panoramic images based on a virtual camera, a system for realizing in-vehicle panoramic images based on a virtual camera, an electronic device, and a storage medium. Background Technology

[0002] Currently, automobiles are developing towards greater safety and intelligence. The use of cameras around the vehicle to create a panoramic image greatly enhances driving safety and convenience, making panoramic imaging an indispensable part of modern smart cars. Unlike mobile phones, car infotainment systems are generally divided into a safety domain and an entertainment domain. The safety domain requires stability and speed, while the entertainment domain is responsible for rich human-computer interaction and effects. Panoramic imaging utilizes both the safety and entertainment domains. The panoramic image from the safety domain is used before the entertainment domain is activated or in case of malfunction; when the entertainment domain is functioning normally, the panoramic image from the entertainment domain is used.

[0003] In addition to capturing images with a camera, panoramic imaging also requires specialized panoramic imaging algorithms to stitch the images together. There are several ways to integrate these algorithms, listed below:

[0004] Using a dedicated external controller (HAD) to handle the integration and stitching of panoramic image algorithms, the stitched complete video image is then transmitted to the vehicle's infotainment system. This solution requires an additional external controller (HAD), which increases costs. This solution is generally used in high-end vehicles.

[0005] To meet the scenario where both the safety domain and the entertainment domain have panoramic imaging, some models integrate panoramic imaging algorithms in the safety domain and the entertainment domain respectively. This approach will cause the same algorithm to run twice on the vehicle's infotainment system, resulting in excessive consumption of computing power for the vehicle's infotainment system. At the same time, it is also difficult to guarantee the consistency of the two systems.

[0006] Some models only integrate panoramic algorithms in the safety domain or entertainment domain. Integration in the safety domain is difficult to achieve rich effects due to system limitations, so integration in the entertainment domain is the only option, but it cannot cover the panoramic imaging quick start scenario.

[0007] For example, a Chinese patent, titled "Synchronous Positioning and Real-Time 3D Path Generation Method for Panoramic Images of Driving Environment," application number CN119888145B, discloses a method for acquiring multi-view images under low-light conditions, adjusting image brightness to enhance contrast, forming a panoramic image, marking feature points in the panoramic image, using the feature point location information to calculate the vehicle's driving direction and speed, predicting the vehicle's position, updating the feature point distribution, and constructing and correcting a 3D path map. This solution does not address the resource allocation issues in the safety and entertainment domains. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide a method for realizing in-vehicle panoramic images based on a virtual camera, a system for realizing in-vehicle panoramic images based on a virtual camera, an electronic device and a storage medium, aiming to solve the technical problems of excessive computing power consumption and inability to cover the fast-start scenario of panoramic images in the prior art.

[0009] This invention provides the following solution: a method for realizing in-vehicle panoramic imaging based on a virtual camera, comprising the following steps:

[0010] Collect image data around the vehicle; upload the image data around the vehicle to the AIS Server unit via the Camera;

[0011] Within the safe domain, image data around the vehicle is stitched together using a preset algorithm to generate a panoramic video.

[0012] Obtain the virtual camera video stream based on the panoramic video;

[0013] The virtual camera video stream is sent back to the AIS Server unit via the Reprocess process;

[0014] The FAWPlugin Proc module copies the virtual Camera video stream to the output buffer requested by the application.

[0015] The UserDone Proc module is invoked to send panoramic video streams to panoramic imaging applications in the security and entertainment domains, respectively.

[0016] Furthermore, including:

[0017] The preset algorithms include:

[0018] A panoramic image algorithm is run in a secure domain to acquire the multi-channel image data and perform stitching processing to generate a panoramic video.

[0019] Furthermore, including:

[0020] AIS Server includes:

[0021] The AisResourceManager unit and the AisProcChain Manager unit; the AisResourceManager unit manages hardware resources, and the AisProcChain Manager unit manages the image processing flow.

[0022] Furthermore, including:

[0023] The virtual camera video stream includes:

[0024] Virtual camera video streams can be accessed simultaneously by applications in both the security and entertainment domains through the standard camera interface.

[0025] Furthermore, including:

[0026] The FAWPlugin module is used to copy virtual camera input data to multiple application output buffers.

[0027] According to a second aspect of the present invention, an in-vehicle panoramic imaging device based on a virtual camera is provided, comprising:

[0028] The system includes a capture module, a video stitching module, a video stream acquisition module, a video stream transmission module, a copy module, and a calling module.

[0029] The acquisition module is used to acquire image data around the vehicle; and uploads the image data around the vehicle to the AIS Server unit via the Camera.

[0030] The video stitching module is used to stitch together image data around the vehicle in a safe domain using a preset algorithm to generate a panoramic video, wherein the preset algorithm is a panoramic image algorithm.

[0031] The video stream acquisition module is used to acquire a virtual camera video stream based on the panoramic video;

[0032] The video stream return module is used to return the virtual camera video stream to the AIS Server unit through the Reprocess process;

[0033] The copy module is used to copy the virtual Camera video stream to the output buffer of the application request via the FAWPlugin Proc module.

[0034] The calling module is used to call the UserDone Proc module to send the panoramic video stream to the panoramic image application in the security domain and entertainment domain, respectively.

[0035] Furthermore, including:

[0036] The AIS Server includes: the ais_server unit, the AisResourceManager unit, and the AisProcChainManager unit;

[0037] The AisResourceManager unit is used to manage sensor, CSIPHY, and IFE hardware resources;

[0038] The AisProcChainManager unit integrates the FAWPlugin processing module and the UserDone Proc module;

[0039] The FAWPlugin processing module is used to copy virtual camera input data to a multi-application output buffer;

[0040] The UserDone Proc module is used to distribute panoramic video streams to target applications.

[0041] Furthermore, including:

[0042] The AisProcChainManager unit also includes an ISP processing module and a GPU acceleration module, which are used to preprocess the raw image data to improve the stitching efficiency and output image quality of the panoramic image algorithm.

[0043] According to three aspects of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0044] The memory stores a computer program, which, when executed by a processor, causes the processor to perform steps of a method for implementing a virtual camera-based panoramic image of a vehicle.

[0045] According to four aspects of the present invention, a computer-readable storage medium is provided that stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a method for implementing a virtual camera-based in-vehicle panoramic image.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] This application integrates a panoramic algorithm into the security domain and transmits the stitched video stream from the security domain to the entertainment domain via the video channels of the security and entertainment domains. This enables both the security and entertainment domains to simultaneously have panoramic images. The video processed by the panoramic algorithm is virtualized into a single camera. Combined with the AIS_SERVER framework, the stitched panoramic video is integrated into the existing video channels of the security and entertainment domains. Applications can use the panoramic video without distinction, just like using any other real camera, while maximizing the stability of the existing framework.

[0048] This application encapsulates panoramic video streams into a standard format using virtual camera technology, seamlessly adapting to existing camera interface protocols in the security domain (QNX) and entertainment domain (Android). This eliminates the need to reconstruct the original system framework, minimizing compatibility risks associated with cross-domain data transmission and ensuring long-term stable system operation.

[0049] This application utilizes the FAWPlugin module and Sync mechanism of the AIS Server to keep the time error between the panoramic video streams acquired in the security domain and the entertainment domain within an extremely low range, completely solving the problems of image delay and misalignment caused by independent processing of the two systems, and ensuring the consistency and reliability of the image data.

[0050] This application is developed based on the Qualcomm SA8155 chip and AIS Server framework, and is compatible with mainstream hardware architectures in the automotive field. At the same time, the FAWPlugin module adopts an extensible plug-in design, which can flexibly adjust the data processing strategy according to the system configuration of different vehicle models, and has cross-brand and cross-series adaptability.

[0051] This application adopts a standardized interface design for the virtual camera, which facilitates the subsequent integration of new functional modules such as image enhancement and intelligent obstacle recognition without the need to reconstruct the existing video transmission link. This reduces the cost of technology iteration, supports the continuous upgrading and evolution of system functions, and allows the application layer to call the virtual camera without any difference through the standard interface. There is no need to develop separate adaptation logic for the security domain and entertainment domain, which unifies the development standard, shortens the product development cycle, and improves the development efficiency. Attached Figure Description

[0052] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a method for implementing in-vehicle panoramic imaging based on a virtual camera, provided by one or more embodiments of the present invention.

[0054] Figure 2 This is a structural diagram of a vehicle-mounted panoramic imaging system based on a virtual camera, provided by one or more embodiments of the present invention.

[0055] Figure 3 This is a general block diagram of a vehicle-mounted panoramic imaging system based on a virtual camera, according to a specific embodiment of the present invention.

[0056] Figure 4 This is a flowchart of the virtual camera implementation process of the AIS_SERVER framework in a specific embodiment of the present invention.

[0057] Figure 5 This is a flowchart of the Reprocess process return according to a specific embodiment of the present invention.

[0058] Figure 6 This is a block diagram of an electronic device for implementing a virtual camera-based panoramic image in a vehicle, provided by one or more embodiments of the present invention. Detailed Implementation

[0059] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Figure 1 This is a flowchart of a method for implementing in-vehicle panoramic imaging based on a virtual camera, provided by one or more embodiments of the present invention.

[0061] like Figure 1 As shown, it includes the following steps:

[0062] Step S1: Acquire image data around the vehicle; upload the image data around the vehicle to the AISServer unit via Camera.

[0063] Specifically, after the front, rear, left, and right cameras outside the vehicle capture images, the four streams are uploaded to ais_server via the Camera driver.

[0064] Step S2: Within the safe domain, image data around the vehicle is stitched together using a preset algorithm to generate a panoramic video;

[0065] Specifically, the panoramic imagery is extracted from these four streams in ais_server and then stitched together.

[0066] Step S3: Obtain the virtual camera video stream based on the panoramic video;

[0067] Specifically, the stitched panoramic video is transmitted back to the ais_server as a virtual camera. The panoramic image applications in the security domain and entertainment domain can then obtain the stitched panoramic video from the ais_server.

[0068] Step S4: The virtual camera video stream is sent back to the AIS Server unit via the Reprocess process;

[0069] Specifically, the panoramic video generated after processing by the security domain algorithm, which is a complete video stream with multi-channel image stitching, is packaged into a virtual Camera video stream that conforms to the vehicle system standard. This allows the system to recognize it as a video output from a real camera, rather than a specially formatted processing result, laying the foundation for subsequent cross-domain sharing.

[0070] The standard link in the AIS Server used for secondary processing and re-injection of image data does not require the construction of a new transmission channel. It reuses the existing system framework, ensures the stability of data transmission, and matches the native processing logic of the AIS Server for camera data.

[0071] By encapsulating and transmitting the data back, the panoramic video is integrated into the camera data management system of the AIS Server, enabling the processing results of the security domain to be uniformly scheduled by the AIS Server. Subsequently, it can be directly distributed to applications in the security domain and entertainment domain (just like calling a regular camera), realizing the core requirement of processing once and sharing across both domains.

[0072] Step S5: Copy the virtual camera video stream to the output buffer requested by the application using the FAWPlugin Proc module;

[0073] Specifically, in AIS Server, the core function of the FAWPlugin module is to distribute and adapt virtual camera data. Its working logic can be briefly summarized as follows:

[0074] Data reception: Receives panoramic video streams encapsulated by a virtual camera as input data sources;

[0075] Multi-buffer mapping: Identify the output buffer requests of each panoramic application in the security and entertainment domains and establish data mapping relationships;

[0076] Efficient replication: Simultaneously replicates the same virtual camera input data to multiple corresponding output buffers, ensuring that dual-domain applications can simultaneously acquire complete panoramic video streams;

[0077] Compatibility and Adaptation: Through standardized data format conversion, it adapts to the interface requirements of video data for different domain applications, enabling seamless cross-domain data transfer.

[0078] This mechanism avoids redundant data processing and ensures real-time acquisition of panoramic video by dual-domain applications, making it a key link in achieving resource sharing between the security domain and the entertainment domain.

[0079] Step S6: Call the UserDone Proc module to send the panoramic video stream to the panoramic image application in the security domain and entertainment domain respectively.

[0080] Specifically, the UserDone module is a key unit in the AIS Server responsible for the final delivery of video streams. Its core function is to accurately distribute the virtual Camera panoramic video stream, which has been copied and processed by the FAWPlugin module, to the panoramic image applications in the security domain and entertainment domain respectively. It is a bridge connecting video stream processing and terminal display.

[0081] After the FAWPlugin module completes copying the virtual Camera video stream to the dual-domain application's dedicated output buffer, the UserDone module triggers a data delivery command to extract the panoramic video stream from the corresponding output buffers of the two domains. Then, through the cross-domain data transmission channel within the vehicle system, it sends the panoramic video stream to the safety-level panoramic application (used for parking and low-speed safety assistance) in the safety domain (such as the QNX system) and the interactive panoramic application (used for visualization enhancement and user interaction) in the entertainment domain (such as the Android system), ultimately achieving simultaneous display of panoramic images across both domains.

[0082] Furthermore, including:

[0083] The preset algorithms include:

[0084] A panoramic image algorithm is run in a secure domain to acquire the multi-channel image data and perform stitching processing to generate a panoramic video.

[0085] Furthermore, including:

[0086] AIS Server includes:

[0087] The AisResourceManager unit and the AisProcChain Manager unit; the AisResourceManager unit manages hardware resources, and the AisProcChain Manager unit manages the image processing flow.

[0088] Furthermore, including:

[0089] The virtual camera video stream includes:

[0090] Virtual camera video streams can be accessed simultaneously by applications in both the security and entertainment domains through the standard camera interface.

[0091] Furthermore, including:

[0092] The FAWPlugin module is used to copy virtual camera input data to multiple application output buffers.

[0093] Specifically, by integrating a panoramic algorithm into the security domain, the video stream stitched in the security domain is transmitted to the entertainment domain via the video channels of the security and entertainment domains. This allows both the security and entertainment domains to simultaneously have panoramic images. The video processed by the panoramic algorithm is then virtually converted into a single camera. Combined with the AIS_SERVER framework, the stitched panoramic video is integrated into the existing video channels of the security and entertainment domains. Applications can then use the panoramic video without distinction, just like using any other real camera, while maximizing the stability of the existing framework.

[0094] Figure 2 This is a structural diagram of a vehicle-mounted panoramic imaging system based on a virtual camera, provided by one or more embodiments of the present invention.

[0095] like Figure 2 As shown, it includes:

[0096] The system includes a capture module, a video stitching module, a video stream acquisition module, a video stream transmission module, a copy module, and a calling module.

[0097] The acquisition module is used to acquire image data around the vehicle; and uploads the image data around the vehicle to the AIS Server unit via the Camera.

[0098] The video stitching module is used to stitch together image data around the vehicle in a safe domain using a preset algorithm to generate a panoramic video, wherein the preset algorithm is a panoramic image algorithm.

[0099] The video stream acquisition module is used to acquire a virtual camera video stream based on the panoramic video;

[0100] The video stream return module is used to return the virtual camera video stream to the AIS Server unit through the Reprocess process;

[0101] The copy module is used to copy the virtual Camera video stream to the output buffer of the application request via the FAWPlugin Proc module.

[0102] The calling module is used to call the UserDone Proc module to send the panoramic video stream to the panoramic image application in the security domain and entertainment domain, respectively.

[0103] Specifically, this hardware and software module integration solution is designed to address the efficient sharing of in-vehicle panoramic images between the security and entertainment domains and reduce wasted computing power.

[0104] Data acquisition layer: multiple cameras

[0105] It forms the basis of image sources. By deploying cameras in positions such as the front, rear, left, and right of the vehicle, it simultaneously collects raw image data of the vehicle's surrounding environment, providing multi-view materials for subsequent panoramic stitching.

[0106] Data transmission layer: Camera driver module

[0107] It acts as a bridge, standardizing the raw image data collected by the camera according to the vehicle system protocol before uploading it to the core processing unit AIS Server, ensuring that the data can be recognized and accessed by subsequent modules.

[0108] Core processing layer: AIS Server

[0109] It is the central hub of the device, which coordinates data scheduling and distribution through four internal sub-modules:

[0110] AisResourceManager: Manages hardware resources (such as cameras and image processing chips) to ensure that resources are allocated on demand;

[0111] AisProcChainManager: Oversees the image processing workflow and coordinates the task order of each module;

[0112] FAWPlugin module: A key adaptation module responsible for copying the virtual camera video stream to the dedicated buffer of the security domain / entertainment domain application, realizing "one source, multiple transmissions";

[0113] UserDone module: Completes the final data distribution, pushing the processed panoramic video stream to the dual-domain application module.

[0114] Algorithm Processing Layer: Panoramic Image Algorithm Module

[0115] Focusing on image fusion, because it is deployed in the security domain, the system area in the vehicle that prioritizes driving safety, can quickly start and respond, such as when a panoramic image is urgently needed when parking, and can avoid competing with entertainment domain functions for computing power. It uses a stitching algorithm to fuse multiple original images into a complete 360° panoramic video.

[0116] Virtualization layer: Virtual Camera generation and management module.

[0117] The stitched panoramic video is disguised as a video stream from a standard physical camera, i.e., a virtual camera, and then sent back to the AIS Server. This way, there is no need to modify the existing camera calling framework of the vehicle system, and applications in the security domain and entertainment domain can obtain panoramic images just like calling a regular camera.

[0118] Application Display Layer: Dual-Domain Panoramic Image Application Module

[0119] Directly facing users, safety domain applications (such as parking assistance interfaces) ensure the display of images in driving safety scenarios, while entertainment domain applications (such as panoramic interfaces with interactive effects) enhance the user experience. Both use the standard Camera interface to call virtual Camera video streams, achieving simultaneous use of the same panoramic source in both domains.

[0120] Furthermore, including:

[0121] The AIS Server includes: the ais_server unit, the AisResourceManager unit, and the AisProcChainManager unit;

[0122] The AisResourceManager unit is used to manage sensor, CSIPHY, and IFE hardware resources;

[0123] The AisProcChainManager unit integrates the FAWPlugin processing module and the UserDone Proc module;

[0124] The FAWPlugin processing module is used to copy virtual camera input data to a multi-application output buffer;

[0125] The UserDone Proc module is used to distribute panoramic video streams to target applications.

[0126] Furthermore, including:

[0127] The AisProcChainManager unit also includes an ISP processing module and a GPU acceleration module, which are used to preprocess the raw image data to improve the stitching efficiency and output image quality of the panoramic image algorithm.

[0128] It is worth noting that although only some basic functional modules are disclosed in this embodiment, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules does not mean that the scope of protection of the claims of this invention is limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0130] Figure 3 This is a general block diagram of a vehicle-mounted panoramic imaging system based on a virtual camera, according to a specific embodiment of the present invention.

[0131] like Figure 3 As shown, after the front, rear, left, and right cameras outside the vehicle capture images, the four streams are uploaded to ais_server through the Camera driver. The panoramic image is extracted from these four streams in ais_server, and the video is stitched together. Then, the stitched panoramic video is sent back to ais_server as a virtual camera. The panoramic image applications in the security domain and entertainment domain can obtain the stitched panoramic video in ais_server.

[0132] The core of the above solution lies in the implementation of the virtual camera, which requires an introduction to the Qualcomm AIS_SERVER framework:

[0133] The Qualcomm AIS_SERVER framework can be divided into three parts:

[0134] 1. ais_engine manages business processes uniformly (unrelated to virtual cameras).

[0135] 2. AisResourceManager is responsible for driver-related control, including sensors, phy, ife, etc., which are mainly hardware-related.

[0136] 3. AisProcChainManager is responsible for image processing, including pure hardware components such as ISP and GPU, as well as software components such as frameCopy and UserDone.

[0137] Based on the AIS_SERVER framework, the virtual camera implementation idea is as follows:

[0138] 1. Open the real camera and enable the hardware driver for the panoramic imaging service.

[0139] 2. For panoramic imaging services that acquire image data from the actual camera output, please refer to [link / reference]. Figure 4 It serves as the InputBuffer for the virtual Camera and is returned to the ais_server via the Reprocess process. (See also...) Figure 5 This enables data transfer from the real CameraOutputBuffer to the virtual CameraInputBuffer.

[0140] 3. Add a new FAWPlugin Proc module in ais_server to copy the image data passed in by the virtual camera in the previous step to the output buffer passed in by the camera client, thereby realizing the data transmission from the virtual camera input buffer to multiple virtual camera output buffers.

[0141] 4. In ais_server, call the UserDone Proc module to pass the OutputBuffer to the panoramic image application.

[0142] The video stream stitched together in the security domain is transmitted to the entertainment domain through the video channels of the security domain and the entertainment domain to solve the scenario where both the security domain and the entertainment domain have panoramic images at the same time.

[0143] By virtualizing the video processed by the panoramic algorithm into a single camera, the panoramic stitched video can be integrated into the existing video channels of the security and entertainment domains. Applications can use panoramic video without distinction, just like using other real cameras, while maximizing the stability of the existing framework.

[0144] Figure 6 This is a block diagram of an electronic device for implementing a virtual camera-based panoramic image in a vehicle, provided by one or more embodiments of the present invention.

[0145] like Figure 6 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0146] The memory stores a computer program that, when executed by the processor, causes the processor to perform steps of a method for implementing in-vehicle panoramic imaging based on a virtual camera.

[0147] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform steps of a method for implementing a virtual camera-based in-vehicle panoramic image.

[0148] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0149] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for realizing in-vehicle panoramic images based on a virtual camera, characterized in that, Includes the following steps: Collect image data around the vehicle; upload the image data around the vehicle to the AIS Server unit via the Camera; Within the safe domain, image data surrounding the vehicle is stitched together using a preset algorithm to generate a panoramic video. Based on the panoramic video, obtain a virtual camera video stream; The virtual camera video stream is then sent back to the AIS Server unit via the Reprocess process. The virtual camera video stream is copied to the output buffer requested by the application via the FAWPlugin Proc module. The UserDone Proc module is invoked to send the panoramic video stream to panoramic image applications in the security domain and entertainment domain, respectively.

2. A method for realizing in-vehicle panoramic images based on a virtual camera according to claim 1, characterized in that, The preset algorithm includes: A panoramic image algorithm is run in a secure domain to acquire the multi-channel image data and perform stitching processing to generate a panoramic video.

3. The method for realizing in-vehicle panoramic images based on a virtual camera according to claim 1, characterized in that, The AIS Server includes: The AisResourceManager unit and the AisProcChain Manager unit; the AisResourceManager unit manages hardware resources, and the AisProcChain Manager unit manages the image processing flow.

4. The method for realizing in-vehicle panoramic images based on a virtual camera according to claim 1, characterized in that, The virtual camera video stream includes: The virtual camera video stream can be accessed simultaneously by applications in both the security and entertainment domains through the standard camera interface.

5. The method for realizing in-vehicle panoramic imaging based on a virtual camera according to claim 1, characterized in that, The FAWPlugin module is used to copy virtual camera input data to multiple application output buffers.

6. A device for realizing in-vehicle panoramic imaging based on a virtual camera, characterized in that, include: The system includes a capture module, a video stitching module, a video stream acquisition module, a video stream transmission module, a copy module, and a calling module. The acquisition module is used to acquire image data around the vehicle; and uploads the image data around the vehicle to the AIS Server unit via the Camera. The video stitching module is used to stitch together image data around the vehicle in a safe domain using a preset algorithm; Generate panoramic video, wherein the preset algorithm is a panoramic image algorithm; The video stream acquisition module is used to acquire a virtual camera video stream based on the panoramic video; The video stream return module is used to return the virtual Camera video stream to the AISServer unit through the Reprocess process; The copy module is used to copy the virtual Camera video stream to the output buffer of the application request via the FAWPlugin Proc module. The calling module is used to call the UserDone Proc module to send the panoramic video stream to the panoramic image application in the security domain and entertainment domain, respectively.

7. The in-vehicle panoramic imaging device based on a virtual camera according to claim 6, characterized in that, The AIS Server includes: the ais_server unit, the AisResourceManager unit, and the AisProcChainManager unit; The AisResourceManager unit is used to manage sensor, CSIPHY, and IFE hardware resources; The AisProcChainManager unit integrates the FAWPlugin processing module and the UserDone Proc module. The FAWPlugin processing module is used to copy virtual camera input data to the multi-application output buffer; The UserDone Proc module is used to distribute panoramic video streams to target applications.

8. The in-vehicle panoramic imaging device based on a virtual camera according to claim 7, characterized in that, The AisProcChainManager unit also includes an ISP processing module and a GPU acceleration module, which are used to preprocess the raw image data to improve the stitching efficiency and output image quality of the panoramic image algorithm.

9. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method for implementing a virtual camera-based panoramic image of a vehicle as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method for implementing a virtual camera-based panoramic image of a vehicle as described in any one of claims 1-7.

Images