Method and system for expanding vehicle-mounted camera application connection number and electronic equipment

Abstract

The invention discloses a method and system for expanding the application connection number of vehicle-mounted cameras and electronic equipment, and relates to the field of vehicle-mounted camera shooting, and the method comprises the steps that one side of a security domain QNX comprises a plurality of physical cameras; wherein the physical camera corresponds to a vehicle-mounted camera entity; acquiring at least one path of physical camera image data from one side of the security domain QNX, and transmitting the physical camera image data to an image processing unit on one side of the security domain QNX; the image processing unit on one side of the security domain QNX is used for splicing the received physical camera image data into one path of camera data as input data of a software camera; one side of the entertainment domain Android acquires image data from a software camera on one side of the security domain QNX through an HAB communication link; and an image distribution unit on one side of the entertainment domain Android cuts the image data from the software camera on one side of the security domain QNX, restores the image data of the physical camera, and distributes the image data to the application on one side of the entertainment domain Android.

Description

Technical Field

[0001] This application relates to the field of vehicle-mounted cameras, and more particularly to methods, systems, electronic devices, storage media, and vehicle-mounted camera platforms for expanding the number of connected vehicle-mounted camera applications. Background Technology

[0002] With the rapid development of intelligent vehicle technology, the demand for cameras in in-vehicle infotainment systems is gradually increasing. However, due to limitations in the number of camera connection channels, current in-vehicle infotainment systems only support a fixed number of applications accessing the cameras simultaneously. To meet the needs of intelligent vehicles for environmental perception and interactive entertainment functions in various application scenarios, a method to expand the number of camera connections is needed.

[0003] The main solutions currently available on the market for this problem are as follows:

[0004] External hardware is used to stitch data from multiple cameras into a single stream before transmitting it to the vehicle's infotainment platform.

[0005] This solution optimizes camera data input and indirectly avoids the camera access restrictions of the vehicle's infotainment platform. However, its disadvantages include high hardware costs, significant latency, and poor real-time performance.

[0006] Configure an application whitelist. Control the number of applications that can connect from the camera's data output end. This solution limits the number of camera applications that can access the camera at the same time, impacting user experience. Summary of the Invention

[0007] The purpose of this invention is to provide a method, system, electronic device, storage medium, and vehicle camera platform for expanding the number of connected vehicle camera applications, thereby solving at least one of a number of technical problems.

[0008] 1. Difficulty in expanding camera data hardware links; 2. Excessive hardware consumption due to dual-system image data sharing.

[0009] This invention provides the following solution:

[0010] According to a first aspect of the present invention, a method for expanding the number of connected vehicle camera applications is provided. The vehicle system includes a security domain (QNX) and an entertainment domain (Android), which are connected via a HAB communication link, comprising:

[0011] The QNX security domain includes multiple physical cameras; these physical cameras correspond to the physical vehicle-mounted cameras.

[0012] At least one physical camera image data is acquired from the QNX side of the security domain and transmitted to the image processing unit on the QNX side of the security domain.

[0013] The image processing unit on the QNX side of the security domain stitches the received physical camera image data into one camera data stream, which is used as the input data for the software camera.

[0014] The Android side of the entertainment domain acquires image data from the software camera on the QNX side of the security domain via the HAB communication link;

[0015] The image distribution unit on the Android side of the entertainment domain segments the image data from the software camera on the QNX side of the security domain, restores the physical camera image data, and distributes it to the application on the Android side of the entertainment domain.

[0016] Further methods to expand the number of connected vehicle camera applications include:

[0017] The software camera includes a first software camera and a second software camera;

[0018] The second software camera acquires image data from the software camera on the QNX side of the security domain via the HAB communication link;

[0019] The first software camera connects to the camera application on the QNX side of the security domain.

[0020] Furthermore, including:

[0021] The QNX security domain's AIS driver module adds a software camera to the Qualcomm native AIS driver framework;

[0022] The software camera link and the physical camera link are independent of each other;

[0023] The QNX security domain's image processing unit acquires image data from at least one physical camera via the AIS driver module;

[0024] The physical camera image data, after being stitched and encapsulated, is transmitted back to the software camera of the AIS driver module as the input source of the software camera.

[0025] The Android image distribution unit in the entertainment domain receives image data transmitted from the second software camera via the HAB communication link;

[0026] The input image data is segmented and distributed to multiple entertainment-domain Android camera applications.

[0027] The security domain camera application acquires image data through the first software camera, enabling multiple applications in the security domain and entertainment domain to simultaneously access all physical camera resources.

[0028] Furthermore, including:

[0029] The software camera uses the encapsulated image data output by the image processing unit as its data source, and transmits it to the application via the Qualcomm platform image quality processor (IPE Prot).

[0030] Furthermore, including:

[0031] The image distribution unit is a lightweight module with preset image distribution functions, including transmitting image data to the Android camera application via CameraHAL and cameraframework.

[0032] Furthermore, including:

[0033] The AIS driver module reuses the communication interface of Qualcomm's native AIS driver framework to maintain the consistency of system interfaces.

[0034] Furthermore, including:

[0035] The physical cameras establish a data connection with the AIS driver module through the CSI interface, supporting simultaneous data input from at least four physical cameras.

[0036] According to a second aspect of the present invention, a system for expanding the number of connected vehicle camera applications is provided, which implements a method for expanding the number of connected vehicle camera applications. The system for expanding the number of connected vehicle camera applications includes:

[0037] The AIS driver module on the QNX side of the security domain includes the native AIS driver framework and the newly added software camera link;

[0038] The image processing unit on the QNX side of the security domain is used for image data acquisition, stitching, encapsulation, and transmission.

[0039] The Android-side image distribution unit in the entertainment domain is used to receive image data transmitted across domains and distribute it to multiple applications.

[0040] The HAB communication link is used for image data transmission between the security domain and the entertainment domain.

[0041] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0042] The memory stores a computer program that, when executed by a processor, causes the processor to perform steps such as methods for increasing the number of connections in an in-vehicle camera application.

[0043] According to a fourth aspect of the present invention, a computer-readable storage medium is provided storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform steps such as a method for expanding the number of connections in an in-vehicle camera application.

[0044] According to a fifth aspect of the present invention, a vehicle-mounted camera platform is provided, comprising:

[0045] Electronic devices for implementing steps such as methods to expand the number of connections in an automotive camera application;

[0046] The processor runs programs, and when the programs are running, they execute steps such as methods for increasing the number of connections in an automotive camera application based on data output from electronic devices.

[0047] Storage medium used to store programs that, when running, perform steps such as expanding the number of connections for an application of an in-vehicle camera in response to data output from an electronic device.

[0048] The above solution achieves the following beneficial technical effects:

[0049] This application circumvents Qualcomm's platform restrictions on the number of cameras an application can access by decoupling from the physical camera.

[0050] This application maximizes system stability and interface uniformity by reusing the communication interface of Qualcomm's native AIS driver framework.

[0051] This application, through an image processing unit and an image distribution unit, can complete the splicing, encapsulation, and distribution of multiple image data, and can serve as an external input source for image data. Attached Figure Description

[0052] Figure 1 This is a flowchart of a method for expanding the number of connected vehicle camera applications provided by one or more embodiments of the present invention.

[0053] Figure 2 This is a structural diagram of a system for expanding the number of connected vehicle cameras provided in one or more embodiments of the present invention.

[0054] Figure 3 This is a schematic diagram of a Qualcomm automotive platform AIS framework provided in a specific embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram of an extended camera connection process provided in a specific embodiment of the present invention.

[0056] Figure 5 This is a block diagram of an electronic device structure for a method to increase the number of connected vehicle cameras provided in one or more embodiments of the present invention. Detailed Implementation

[0057] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Figure 1 This is a flowchart of a method for expanding the number of connected vehicle camera applications provided by one or more embodiments of the present invention.

[0059] like Figure 1 The method shown here extends the number of connected vehicle camera applications. The vehicle system includes a security domain (QNX) and an entertainment domain (Android), which are connected via a HAB communication link, including:

[0060] Step S1: The QNX security domain includes multiple physical cameras; where each physical camera corresponds to an in-vehicle camera entity.

[0061] Step S2: Obtain at least one physical camera image data from the QNX side of the security domain and transmit it to the image processing unit on the QNX side of the security domain;

[0062] Step S3: The image processing unit on the QNX side of the security domain stitches the received physical camera image data into one camera data stream, which is used as the input data for the software camera.

[0063] Step S4: The Android side of the entertainment domain acquires image data from the software camera on the QNX side of the security domain via the HAB communication link;

[0064] In step S5, the image distribution unit on the Android side of the entertainment domain segments the image data from the software camera on the QNX side of the security domain, restores the physical camera image data, and distributes it to the application on the Android side of the entertainment domain.

[0065] In this embodiment, the method for expanding the number of connected vehicle camera applications includes:

[0066] The software camera includes a first software camera and a second software camera;

[0067] The second software camera acquires image data from the software camera on the QNX side of the security domain via the HAB communication link;

[0068] The first software camera connects to the camera application on the QNX side of the security domain.

[0069] In this embodiment, it includes:

[0070] The QNX security domain's AIS driver module adds a software camera to the Qualcomm native AIS driver framework;

[0071] The software camera link and the physical camera link are independent of each other;

[0072] The QNX security domain's image processing unit acquires image data from at least one physical camera via the AIS driver module;

[0073] The physical camera image data, after being stitched and encapsulated, is transmitted back to the software camera of the AIS driver module as the input source of the software camera.

[0074] The Android image distribution unit in the entertainment domain receives image data transmitted from the second software camera via the HAB communication link;

[0075] The input image data is segmented and distributed to multiple entertainment-domain Android camera applications.

[0076] The security domain camera application acquires image data through the first software camera, enabling multiple applications in the security domain and entertainment domain to simultaneously access all physical camera resources.

[0077] In this embodiment, it includes:

[0078] The software camera uses the encapsulated image data output by the image processing unit as its data source, and transmits it to the application via the Qualcomm platform image quality processor (IPE Prot).

[0079] In this embodiment, it includes:

[0080] The image distribution unit is a lightweight module with preset image distribution functions, including transmitting image data to the Android camera application via CameraHAL and cameraframework.

[0081] In this embodiment, it includes:

[0082] The AIS driver module reuses the communication interface of Qualcomm's native AIS driver framework to maintain the consistency of system interfaces.

[0083] In this embodiment, it includes:

[0084] The physical cameras establish a data connection with the AIS driver module through the CSI interface, supporting simultaneous data input from at least four physical cameras.

[0085] Figure 2 This is a structural diagram of a system for expanding the number of connected vehicle cameras provided in one or more embodiments of the present invention.

[0086] like Figure 2 The system shown is for expanding the number of connected vehicle camera applications. It is used to implement a method for expanding the number of connected vehicle camera applications. The system includes:

[0087] The AIS driver module on the QNX side of the security domain includes the native AIS driver framework and the newly added software camera link;

[0088] The image processing unit on the QNX side of the security domain is used for image data acquisition, stitching, encapsulation, and transmission.

[0089] The Android-side image distribution unit in the entertainment domain is used to receive image data transmitted across domains and distribute it to multiple applications.

[0090] The HAB communication link is used for image data transmission between the security domain and the entertainment domain.

[0091] It is worth noting that although this system / device only discloses the above-mentioned modules / units, it does not mean that this system / device is limited to the above-mentioned basic functional modules. On the contrary, what this invention intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. It cannot be assumed that the scope of protection of the claims of this invention is limited to the above-disclosed basic functional modules just because this embodiment only discloses a few basic functional modules.

[0092] In one specific embodiment, such as Figure 3 As shown, a method for expanding the number of connected vehicle camera applications based on the Qualcomm Automotive Platform AIS framework is disclosed:

[0093] Security Domain (QNX) AIS driver module;

[0094] The AIS driver module relies on Qualcomm's native AIS driver framework, on which a software camera communication link is added. This software camera link uses external input as its data source, and after passing through the Qualcomm platform image quality processor (IPEProt), the image data is transmitted to the application. The software camera link is independent of the physical camera link, thus circumventing the limitations imposed by Qualcomm's native driver architecture on the number of cameras that can be accessed.

[0095] Secure Domain (QNX) image processing unit;

[0096] A new image processing unit has been added to the security domain (QNX) for the acquisition, encapsulation, and distribution of image data. The image processing unit acquires image data from any physical camera through the AIS driver module, performs image stitching and encapsulation within the module using CPU and GPU resources, and then sends the stitched image data back to the software camera within the AIS driver module as the input source.

[0097] Android Entertainment Image Distribution Unit;

[0098] To reduce resource consumption caused by image data transmission between the security domain (QNX) and the entertainment domain (Android), an image distribution unit is additionally integrated into the entertainment domain (Android). This module only supports image distribution. When multiple entertainment domain applications need to access the same camera, they can obtain image data from the security domain (QNX) AIS driver module, transmit it to the entertainment domain (Android) via HAB communication, and then the image distribution unit transmits the input image data to multiple camera applications, thereby reducing the system load caused by multiple applications accessing the camera across systems.

[0099] In another specific embodiment, such as Figure 4 As shown, the process for increasing the number of connected cameras is publicly available:

[0100] If the security domain camera application and entertainment domain camera application 1 and entertainment domain camera application 2 need to access the resources of 4 physical cameras simultaneously, the required operations are as follows:

[0101] Process 1: Obtain image data from 4 physical cameras from the AIS driver module and transmit it to the image processing unit;

[0102] Process 2: The image processing unit stitches the data from the four physical cameras into one camera data stream and transmits it as the input source to software camera 1 and software camera 2;

[0103] Step 3: The security domain camera application opens the software camera 1 to acquire image data;

[0104] Process 4: The image data from the software camera 2 is transmitted to the Android image distribution unit via the HAB communication link;

[0105] Process 5: The image distribution unit copies the image data into two paths, which are then transmitted through CameraHAL and cameraframework to Application 1 and Application 2.

[0106] Figure 5 This is a block diagram of an electronic device structure for a method to increase the number of connected vehicle cameras provided in one or more embodiments of the present invention.

[0107] like Figure 5 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0108] The memory stores a computer program that, when executed by a processor, causes the processor to perform steps of a method for expanding the number of connections in an in-vehicle camera application.

[0109] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform steps of a method for expanding the number of connections in an in-vehicle camera application.

[0110] This application also provides an imaging platform, including:

[0111] Electronic devices, steps for implementing methods to expand the number of connections in an automotive camera application;

[0112] The processor runs a program that, when running, executes steps of a method to expand the number of connections for an in-vehicle camera application from data output by the electronic device.

[0113] Storage medium for storing programs that, when running, execute steps of a method to expand the number of connections for an in-vehicle camera application in response to data output from an electronic device.

[0114] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0115] The electronic device comprises a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory. The operating system can be any one or more computer operating systems that control the electronic device through processes, such as Linux, Unix, Android, iOS, or Windows. Furthermore, in this embodiment of the invention, the electronic device can be a smartphone, tablet computer, or other handheld device, or a desktop computer, portable computer, or other electronic device; there is no particular limitation in this embodiment.

[0116] In this embodiment of the invention, the executing entity for electronic device control can be an electronic device itself, or a functional module within an electronic device capable of calling and executing a program. The electronic device can obtain the firmware corresponding to the storage medium. This firmware is provided by the supplier, and different storage media may have the same or different firmware; no limitation is made here. After obtaining the firmware corresponding to the storage medium, the electronic device can write this firmware into the storage medium; specifically, it burns the firmware corresponding to the storage medium into the storage medium. The process of burning the firmware into the storage medium can be implemented using existing technology, and will not be elaborated upon in this embodiment of the invention.

[0117] Electronic devices can also obtain reset commands corresponding to the storage media. The reset commands corresponding to the storage media are provided by the supplier. The reset commands corresponding to different storage media can be the same or different, and no restrictions are imposed here.

[0118] At this time, the storage medium of the electronic device is a storage medium on which the corresponding firmware has been written. The electronic device can respond to the reset command corresponding to the storage medium on which the corresponding firmware has been written, thereby resetting the storage medium on which the corresponding firmware has been written according to the reset command. The process of resetting the storage medium according to the reset command can be implemented by existing technology and will not be described in detail in this embodiment of the invention.

[0119] For ease of description, the above devices are described separately by function as various units and modules. Of course, in implementing this application, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0120] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0121] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0122] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for expanding the number of connected vehicle camera applications, wherein the vehicle system includes a security domain (QNX) and an entertainment domain (Android), which are connected via a HAB communication link, characterized in that, The method for increasing the number of connections for vehicle-mounted camera applications includes: The QNX security domain includes multiple physical cameras; these physical cameras correspond to the physical vehicle-mounted cameras. At least one physical camera image data is acquired from the QNX side of the security domain and transmitted to the image processing unit on the QNX side of the security domain. The image processing unit on the QNX side of the security domain stitches the received physical camera image data into one camera data stream, which is used as the input data for the software camera. The Android side of the entertainment domain acquires image data from the software camera on the QNX side of the security domain via the HAB communication link; The image distribution unit on the Android side of the entertainment domain segments the image data from the software camera on the QNX side of the security domain, restores the physical camera image data, and distributes it to the application on the Android side of the entertainment domain.

2. The method for expanding the number of connected vehicle-mounted cameras according to claim 1, characterized in that, The method for increasing the number of connections for vehicle-mounted camera applications includes: The software camera includes a first software camera and a second software camera; The second software camera acquires image data from the software camera on the QNX side of the security domain via the HAB communication link; The first software camera connects to the camera application on the QNX side of the security domain.

3. The method for expanding the number of connected vehicle cameras according to claim 2, characterized in that, include: The QNX security domain's AIS driver module adds a software camera to the pre-set native driver framework; The software camera link and the physical camera link are independent of each other; The QNX security domain's image processing unit acquires image data from at least one physical camera via the AIS driver module; The physical camera image data, after being stitched and encapsulated, is transmitted back to the software camera of the AIS driver module as the input source of the software camera. The Android image distribution unit in the entertainment domain receives image data transmitted from the second software camera via the HAB communication link; The input image data is segmented and distributed to multiple entertainment-domain Android camera applications. The security domain camera application acquires image data through the first software camera, enabling multiple applications in the security domain and entertainment domain to simultaneously access all physical camera resources.

4. The method for increasing the number of connected vehicle cameras according to claim 3, characterized in that, include: The software camera uses the encapsulated image data output by the image processing unit as its data source, and transmits it to the application end via the image quality processor.

5. The method for increasing the number of connected vehicle cameras according to claim 3, characterized in that, include: The image distribution unit is a lightweight module with preset image distribution functions, including transmitting image data to the Android camera application via CameraHAL and cameraframework.

6. The method for expanding the number of connected vehicle cameras according to claim 3, characterized in that, include: The AIS driver module reuses the communication interface of the preset native driver framework to maintain the consistency of system interfaces.

7. The method for expanding the number of connected vehicle cameras according to claim 3, characterized in that, include: The physical cameras establish a data connection with the AIS driver module through the CSI interface, supporting simultaneous data input from at least four physical cameras.

8. A system for expanding the number of connected vehicle camera applications, used to implement the method for expanding the number of connected vehicle camera applications as described in any one of claims 1-7, characterized in that, The system for increasing the number of connections for the extended vehicle-mounted camera application includes: The AIS driver module on the QNX side of the security domain includes the native AIS driver framework and the newly added software camera link; The image processing unit on the QNX side of the security domain is used for image data acquisition, stitching, encapsulation, and transmission. The Android-side image distribution unit in the entertainment domain is used to receive image data transmitted across domains and distribute it to multiple applications. The HAB communication link is used for image data transmission between the security domain and the entertainment domain.

9. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method for increasing the number of connections in an in-vehicle camera application as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method for increasing the number of connections in an in-vehicle camera application as described in any one of claims 1 to 7.

Images