Vehicle-mounted multi-camera control method and device, terminal equipment and storage medium

Abstract

The invention provides a vehicle-mounted multi-camera control method and device, terminal equipment and a storage medium. The method comprises the following steps: receiving an operation request of opening a target camera by a target application program; wherein the target application program is configured with an access priority for the target camera, and the target camera allocates a corresponding resource value to the target application program; judging whether the access priority of the target application program is higher than the access priority of the application program currently occupying the target camera or not; judging whether the sum of the resource value allocated by the target application program and the resource values of all the current application programs of which the cameras are opened does not exceed the maximum value of the vehicle-mounted system resources or not; and if so, opening the target camera in response to the operation request. Accurate access control of the target camera can be realized according to the access priority of the target application program and the allocated resource value, and a complex camera equipment access scene in the automobile field is met.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle camera control, and in particular to a vehicle-mounted multi-camera control method, device, terminal equipment and storage medium. Background Art

[0002] With the rapid development of new energy vehicles, more and more cameras are connected to the vehicle, especially after entering the EA3 electrical architecture, the Ethernet access, the central control host display screen is not only the interaction of the multimedia cockpit domain system, but also accompanied by the intelligent driving domain and the central computing unit (CCU) domain related interaction. As a result, the central control screen application has an increasing demand for the reading of vehicle camera data.

[0003] In the in-vehicle Android system, the same camera device can be opened by multiple clients, but by default it can only be opened by one client program. When one client program opens it, other client programs will return an error message that the resource is occupied and the opening fails. Therefore, the system has a priority strategy for clients using the same camera device (i.e., the same camera ID). When a low-priority process opens the camera device, if a high-priority process comes at this time to seize and open the camera device resources, the low-priority process will be forced to release the camera resources and receive a notification of being seized and exited. Conversely, the low-priority process cannot interrupt the high-priority process. However, the process priority of most third-party Android applications is set by the system. For example, the process priority of the foreground program (including third-party applications) is higher than the process priority of the background program, resulting in the background client program being unable to seize the camera device resources being used by the foreground client program, which cannot meet the complex camera device access scenarios in the automotive field. In addition, the system can only allocate different resource values ​​for different camera IDs, and cannot allocate resources for clients. For the combination of multiple cameras and multiple clients in the vehicle, it cannot guarantee that the client with high priority can occupy more resources, resulting in poor accuracy and real-time performance of the client obtaining camera data. Summary of the invention

[0004] The main purpose of the present invention is to provide a vehicle-mounted multi-camera control method, device, terminal equipment and storage medium, aiming to solve the technical problems that existing vehicles cannot meet the complex camera device access scenarios in the automotive field and the accuracy and real-time performance of the client in obtaining camera data is poor.

[0005] In a first aspect, the present invention provides a vehicle-mounted multi-camera control method, comprising:

[0006] Receive an operation request from a target application to open a target camera; wherein the target application is configured with an access priority to the target camera, and the target camera allocates a corresponding resource value to the target application;

[0007] Determine whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera;

[0008] Determine whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources;

[0009] If both are true, then in response to the operation request, the target camera is turned on.

[0010] In a specific embodiment, it also includes:

[0011] Aggregating multiple target applications corresponding to the same target camera;

[0012] A configuration file is determined, where the configuration file is used to indicate that for the same target camera, different access priorities are configured for different target applications and different resource values ​​are allocated according to the importance of different target applications accessing the target camera.

[0013] In a specific embodiment, before determining whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, the method further includes:

[0014] Obtain the access priority and allocated resource value of the target application to the target camera.

[0015] In a specific embodiment, before determining whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, the method further includes:

[0016] Determine an application currently occupying the target camera;

[0017] Get the access priority of the application currently occupying the target camera;

[0018] Before determining whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources, the method further includes:

[0019] Get the maximum value of vehicle system resources;

[0020] Identify all applications that currently have cameras open;

[0021] The sum of the resource value allocated by the target application and the resource values ​​of all applications that have currently opened cameras is calculated.

[0022] In a specific embodiment, the target application has an access requirement to open multiple target cameras; the method further includes:

[0023] Receiving multiple operation requests from the target application to respectively open multiple target cameras;

[0024] Determine whether the access priority of the target application to each of the target cameras is higher than the access priority of the application currently occupying each of the target cameras;

[0025] Determine whether the sum of the resource value allocated by each target camera to the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources;

[0026] If both are true, then in response to the corresponding operation request, the corresponding target camera is opened.

[0027] In a second aspect, the present invention provides a vehicle-mounted multi-camera control device, comprising:

[0028] A camera management module, configured to receive an operation request from a target application to open a target camera; wherein the target application is configured with an access priority to the target camera, and the target camera allocates a corresponding resource value to the target application;

[0029] A camera arbitration module, used to determine whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera; and used to determine whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened cameras does not exceed the maximum value of the vehicle system resources;

[0030] The camera service module is used to open the target camera in response to the operation request when receiving the judgment result sent by the camera arbitration module.

[0031] In a specific embodiment, it also includes a camera configuration module, and the camera configuration module is used to:

[0032] Aggregating multiple target applications corresponding to the same target camera;

[0033] A configuration file is determined, where the configuration file is used to indicate that for the same target camera, different access priorities are configured for different target applications and different resource values ​​are allocated according to the importance of different target applications accessing the target camera.

[0034] In a specific embodiment, the target application has an access requirement to open a plurality of the target cameras;

[0035] The camera management module is further used to receive multiple operation requests from the target application to open multiple target cameras respectively; wherein the target application is configured with an access priority for each of the target cameras, and each of the target cameras is allocated a corresponding resource value to the target application;

[0036] The camera arbitration module is further used to determine whether the access priority of the target application to each of the target cameras is higher than the access priority of the application currently occupying each of the target cameras; and further used to determine whether the sum of the resource value allocated by each of the target cameras to the target application and the resource values ​​of all the applications that have currently opened the cameras does not exceed the maximum value of the vehicle system resources;

[0037] The camera service module is further configured to, upon receiving a judgment result sent by the camera arbitration module, respond to the corresponding operation request and open the corresponding target camera.

[0038] In a third aspect, the present invention provides a terminal device comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the vehicle-mounted multi-camera control method as described in the first aspect is implemented.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the vehicle-mounted multi-camera control method as described in the first aspect.

[0040] Compared with the prior art, the beneficial effect of the present invention lies in that: for the same target camera, different target applications are configured with different access priorities and allocated different resource values. By judging whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, and judging whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources, accurate access control of the target camera can be achieved according to the access priority and allocated resource value of the target application, satisfying the complex camera device access scenarios in the automotive field, and improving the accuracy and real-time performance of high-priority target applications in important scenarios in obtaining target camera data. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flowchart of a vehicle-mounted multi-camera control method provided by an embodiment of the present invention;

[0042] Figure 2 It is a timing interaction diagram of a vehicle-mounted multi-camera control method provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of the architecture of a vehicle-mounted multi-camera control device provided by an embodiment of the present invention;

[0044] Figure 4 It is a structural diagram of a terminal device provided by an embodiment of the present invention.

[0045] in:

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0051] See also Figure 1 , Figure 1 The present invention is a flowchart of a method for controlling multiple vehicle cameras according to an embodiment of the present invention.

[0052] A vehicle-mounted multi-camera control method according to an embodiment of the present invention comprises the following steps:

[0053] S100: Receive an operation request from a target application to open a target camera.

[0054] The target application is configured with an access priority to the target camera, and the target camera allocates a corresponding resource value to the target application.

[0055] In the embodiment of the present invention, there are multiple vehicle-mounted cameras, and there are also multiple applications that call the cameras, and each application has one or more corresponding target cameras. All cameras are classified according to the functions of the vehicle-mounted cameras, and can be divided into intelligent driving domain cameras, central computing unit (CCU) domain cameras, and cockpit domain cameras. Among them, the intelligent driving domain camera can be used to realize functions such as parking and pilot assisted driving, and can specifically include front and rear cameras, side view cameras, etc.; the CCU domain camera can be used to realize functions such as panoramic monitoring (Around View Monitor, AVM), etc., and can specifically include surround view cameras, etc.; the cockpit domain camera can be used to realize in-vehicle occupant monitoring, such as driver monitoring (Driver Monitoring System, DMS), passenger monitoring (Occupancy Monitoring System, OMS), driving record ((Digital Video Recorder, DVR)) and other functions, and can specifically include built-in camera for taking pictures, etc.

[0056] For example, see Figure 2 , Figure 2 It is a timing interaction diagram of a vehicle-mounted multi-camera control method provided by an embodiment of the present invention.

[0057] There are three target applications, namely target application 1, target application 2, and target application 3. All cameras are classified according to their functions, and can be divided into intelligent driving domain target cameras 1 to 6, CCU domain target cameras 7 to 11, and cockpit domain target cameras 12 to 18. Target application 1 sends an operation request to open target camera 1, target application 2 sends an operation request to open target camera 1, and target application 3 sends an operation request to open target camera 8.

[0058] The existing vehicle system must know the process priority and the priority of the camera device access rights when the process is running. Specifically, it is determined by the priority (oom_score) of the caller process when the client opens the camera (open Camera). The process priority value (abbreviated as adj) is viewed through / proc / process_pid / oom_score_adj. It is impossible to determine the priority relationship of each client's access rights to the camera device in advance, and the access efficiency is low. In addition, the system can only allocate different resource values ​​for different camera IDs, and cannot allocate resources for clients. For the combination of multiple cameras and multiple clients on the vehicle, it cannot guarantee that clients with high priority can occupy more resources, resulting in poor accuracy and real-time performance of the client in obtaining camera data.

[0059] To solve the above problem, in a specific embodiment, before step S100, the method further includes the following configuration steps:

[0060] S510, summarizing multiple target applications corresponding to the same target camera;

[0061] S520. Determine a configuration file, where the configuration file is used to indicate that for the same target camera, different access priorities are configured for different target applications and different resource values ​​are allocated according to the importance of different target applications accessing the target camera.

[0062] Among them, the resource values ​​allocated by the target camera to the target application include resources such as computing power and memory.

[0063] In this embodiment, the same target camera can be called and opened by multiple corresponding target applications. For the same target camera, different access priorities and resource values ​​are configured for different applications according to the importance of the target application accessing the target camera, which is applicable to the multi-scenario multiplexing requirements of the automotive industry camera. For example, for important target applications, a higher access priority can be set and more resource values ​​can be allocated. In addition, for special target applications, access can be denied by configuring the priority. For example, for important target cameras containing sensitive information, this application can be set separately to be inaccessible.

[0064] During configuration, all vehicle-mounted cameras are first aggregated, and then multiple target applications corresponding to the same target camera are aggregated one by one. Different access priorities and resource values ​​are configured for different applications to form a configuration file for subsequent reading.

[0065] In this way, according to the importance of the target application, the target application of each target camera resource can be prioritized and resource allocated, and the access priority and resources of each client to the target camera can be configured according to product and system requirements. If necessary, the access priority and resources of the client can be dynamically adjusted at runtime to realize the complex camera reuse scenarios in the automotive industry, so that all client resources can be reasonably allocated, and the accuracy and real-time performance of high-priority target applications in important scenarios to obtain target camera data can be improved. In addition, through the configuration file, the priority relationship of each target application's access rights to the target camera can be determined in advance to improve access efficiency.

[0066] See also Figure 3 , Figure 3 1 is a schematic diagram of the structure of a vehicle-mounted multi-camera control device provided by an embodiment of the present invention. A vehicle-mounted multi-camera control method provided by an embodiment of the present invention can be applied to a vehicle-mounted multi-camera control device. In a specific embodiment, the vehicle-mounted multi-camera control device adds a camera configuration module (CameraConfig) 103, and the camera configuration module 103 implements the above steps S510 to S520.

[0067] The camera configuration module 103 reads the configuration file when the vehicle-mounted system is running, summarizes the target applications corresponding to each vehicle-mounted target camera, and configures the access priority and allocated resource value of each target application to the target camera in the configuration file for each target camera. When the vehicle-mounted multi-camera control device is initialized during operation, the camera configuration module 103 parses the configuration file into the memory of the vehicle-mounted multi-camera control device, so as to subsequently obtain the access priority and allocated resource value of any target application to the target camera.

[0068] In a specific embodiment, before the step S200 determines whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, the following steps are also included:

[0069] S610: Obtain the access priority and allocated resource value of the target application to the target camera.

[0070] When a target application needs to access a target camera, the access priority and allocated resource value of the target application are first obtained from the memory of the vehicle-mounted multi-camera control device.

[0071] Furthermore, before the step S200 determines whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, the following steps are also included:

[0072] S620, determining an application currently occupying the target camera;

[0073] S630: Obtain the access priority of the application currently occupying the target camera.

[0074] When the target application needs to access the target camera, it is also necessary to determine the application currently occupying the target camera and obtain the access priority of the application currently occupying the target camera from the memory of the on-board multi-camera control device.

[0075] S200, determining whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera.

[0076] In this embodiment, when different target applications need to access the same target camera, priority arbitration is performed on the target applications with access requests.

[0077] Specifically, if it is determined that the target application's access priority to the target camera is higher than the access priority of the application currently occupying the target camera, the target application will have the opportunity to preempt and open the target camera, and the application currently occupying the target camera will be forced to release the target camera resources and exit, and will receive a preemption notification; if it is determined that the target application's access priority to the target camera is lower than the access priority of the application currently occupying the target camera, the target application will not be able to open the target camera, and the application currently occupying the target camera will continue to occupy the target camera.

[0078] Furthermore, the process priority of most existing third-party Android applications is set by the system. For example, the process priority of the foreground program is higher than the process priority of the background program. This cannot meet the scenario where the background client program preempts the camera device resources, and cannot meet the complex camera device access scenarios in the automotive field.

[0079] In this embodiment, the target application and the application currently occupying the target camera can be not only a foreground client program (including a third-party application) but also a background client program. In this way, the rule that the process priority of the foreground client program is higher than the process priority of the background client program can be bypassed, and the scenario of the background client program occupying the target camera device resources can be realized, meeting the complex camera device access scenarios in the automotive field.

[0080] In a specific embodiment, before determining in step S300 whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources, the following steps are further included:

[0081] S710, obtaining the maximum value of the vehicle system resources;

[0082] S720, determining all currently opened camera applications;

[0083] S730: Calculate the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications.

[0084] Since the vehicle system has a fixed upper limit for all camera access resources, when the upper limit exceeds the maximum value, the camera will fail to be opened. In this embodiment, for subsequent resource arbitration, the maximum value of the vehicle system resources is first obtained, then all currently opened camera applications are determined, and then the sum of the resource value allocated by the target application and the resource value of all currently opened camera applications is calculated. It can be understood that all currently opened cameras are other currently occupied cameras.

[0085] S300: Determine whether the sum of the resource value allocated by the target application and the resource values ​​of all currently turned on cameras does not exceed the maximum value of the vehicle system resources.

[0086] After determining that the target application's access priority to the target camera is higher than the access priority of the application currently occupying the target camera, it is necessary to accumulate resources for all applications that have opened cameras to determine whether the sum of the resource value allocated by the target camera to the target application and the resource value of all currently opened cameras does not exceed the maximum value of the vehicle system resources:

[0087] If it is determined that the sum of the resource value allocated by the target camera to the target application and the resource values ​​of all currently opened cameras does not exceed the maximum value of the vehicle system resources, the target application can open the target camera;

[0088] If it is determined that the sum of the resource value allocated by the target camera to the target application and the resource value of all currently opened cameras exceeds the maximum value of the vehicle system resources, it means that the resources of the device being used at this time are already tight and the new target application cannot apply to open successfully, then an error return will be made to the target application that is about to open the target camera.

[0089] S400: If yes, in response to the operation request, turn on the target camera.

[0090] After determining that the target application's access priority to the target camera is higher than the access priority of the application currently occupying the target camera, and determining that the sum of the resource value allocated by the target camera to the target application and the resource values ​​of all currently turned on cameras does not exceed the maximum value of the vehicle system resources, in response to the target application's operation request to open the target camera, turn on the target camera.

[0091] See also Figure 3 In a specific embodiment, the vehicle-mounted multi-camera control device adds a camera arbitration module (CameraArbitration) 102, and the camera arbitration module 102 implements the above steps S610~S630, S710~S730, S200~S300.

[0092] Specifically, the camera arbitration module 102 implements mutual exclusion and interruption control of the target camera's application according to the target application's access priority and the allocated resource value. During operation, the vehicle-mounted multi-camera control device initializes the camera configuration module 103 to parse the configuration file into the memory, and the target application calls the camera management module (CameraManager) 101 standard interface, and then calls the newly added camera arbitration module 102. The camera arbitration module 102 obtains the access priority of the application currently occupying the target camera and the access priority of the application currently occupying the target camera from the memory, and performs priority arbitration. Then, based on the resource value allocated by the target application, combined with the maximum value of the vehicle-mounted system resources, the resource value allocated by the target application and the sum of the resource values ​​of all currently open camera applications, it is determined whether the total application resource of the opened camera resources does not exceed the maximum value:

[0093] If it is a high priority and the total resource value does not exceed the maximum value of the vehicle system, the application that previously occupied the target camera is interrupted, and the operation request result of the target application is returned to obtain the opening qualification, and the camera service module (CameraService) 104 can be called;

[0094] If it is not a high priority or the total resource value exceeds the maximum value of the vehicle system, the target application's operation request result is directly returned as failure to open, the target camera is busy, and an error code is returned.

[0095] For example, see Figure 2 and Figure 3The camera management module 101 receives the operation request of the target application 1 to open the target camera 1. The camera arbitration module 102 performs priority arbitration and resource allocation to determine whether it is successful. Since the above two opening conditions are not met, the target camera 1 fails to be opened. The camera arbitration module 102 returns the request result, calls back QnError, and returns an error code.

[0096] The camera management module 101 receives the operation request from the target application 2 to open the target camera 1, and the camera arbitration module 102 determines whether the priority arbitration and resource allocation are successful. Since the above two opening conditions are met, the target application 2 obtains the qualification to open the target camera 1, calls the camera service module 104, and calls the camera abstract layer (CameraHal) 106 through the camera provider module (CameraProvider) 105 to open the target camera 1.

[0097] The target application 3 sends an operation request to open the target camera 8. The camera arbitration module 102 performs priority arbitration and resource allocation to determine whether it is successful. Since the above two opening conditions are not met, the target camera 1 fails to open. The camera arbitration module 102 returns the request result, calls back QnError, and returns an error code.

[0098] In a specific embodiment, the target application has an access requirement to open multiple target cameras; the method further comprises the following steps:

[0099] S810, receiving multiple operation requests from the target application to respectively open multiple target cameras;

[0100] S820, determining whether the access priority of the target application to each of the target cameras is higher than the access priority of the application currently occupying each of the target cameras;

[0101] S830, determining whether the sum of the resource value allocated by each target camera to the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources;

[0102] S840: If both are true, in response to the corresponding operation request, open the corresponding target camera.

[0103] In this embodiment, for the same target application that has access requirements for multiple target cameras, the target application's access rights to each target camera can be controlled individually, and the target application's access priority can be configured according to product and system requirements. It is not limited to foreground and background processes, and can meet more complex camera reuse scenarios in the automotive industry.

[0104] Specifically, multiple operation requests from a target application to open multiple target cameras are received, and arbitration is performed separately for each operation request:

[0105] Determine whether the target application's access priority to each target camera is higher than the access priority of the application currently occupying each target camera. If so, proceed to the next step of determination. If not, return the result of opening failure.

[0106] Determine whether the sum of the resource value allocated by each target camera to the target application and the resource value of all currently opened camera applications does not exceed the maximum value of the vehicle system resources. If so, open the corresponding target camera in response to the corresponding operation request. If not, return the result of failed opening.

[0107] It can be understood that, for the same target application, the access priority of the target application to each target camera can be the same or different, and the resource value allocated by each target camera to the target application can be the same or different.

[0108] Through the above steps, a target application that meets the two conditions can open the corresponding target camera, while a target application that does not meet the two conditions cannot open the corresponding target camera. That is, when the same target application has access requirements to multiple target cameras, it can eventually open all the target cameras, or only open some of the target cameras, or it may not be able to open all the target cameras.

[0109] Similarly, other target applications can also achieve precise access control of their multiple target cameras through the above steps, so that in multi-client multi-camera scenarios, important application scenarios such as reversing images can be given priority in resource allocation, and high-priority target applications in important scenarios can obtain their target camera data with accuracy and real-time performance.

[0110] Compared with the prior art, a vehicle-mounted multi-camera control method according to an embodiment of the present invention has the following beneficial effects:

[0111] For the same target camera, different target applications are configured with different access priorities and allocated different resource values. By judging whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, and judging whether the sum of the resource value allocated by the target application and the resource value of all currently opened camera applications does not exceed the maximum value of the vehicle system resources, accurate access control of the target camera can be achieved according to the access priority and allocated resource value of the target application, meeting the complex camera device access scenarios in the automotive field, and improving the accuracy and real-time performance of high-priority target applications in important scenarios in obtaining target camera data.

[0112] In addition, the vehicle-mounted multi-camera control method of an embodiment of the present invention can be transplanted to all current Android versions and all industries using the Android system, with a very high reuse rate.

[0113] See also Figure 3 , Figure 3 It is a structural schematic diagram of a vehicle-mounted multi-camera control device provided by one embodiment of the present invention.

[0114] A vehicle-mounted multi-camera control device according to an embodiment of the present invention includes:

[0115] The camera management module 101 is used to receive an operation request from a target application to open a target camera; wherein the target application is configured with an access priority to the target camera, and the target camera allocates a corresponding resource value to the target application;

[0116] The camera arbitration module 102 is used to determine whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera; and to determine whether the sum of the resource value allocated by the target application and the resource value of all currently opened cameras does not exceed the maximum value of the vehicle system resources;

[0117] The camera service module 104 is used to open the target camera in response to the operation request when receiving the judgment result sent by the camera arbitration module 102.

[0118] In the embodiment of the present invention, there are multiple vehicle-mounted cameras, and there are also multiple applications that call the cameras, and each application has one or more corresponding target cameras. All cameras are classified according to the functions of the vehicle-mounted cameras, and can be divided into intelligent driving domain cameras, central computing unit (CCU) domain cameras, and cockpit domain cameras. Among them, the intelligent driving domain camera can be used to realize functions such as parking and pilot assisted driving, and can specifically include front and rear cameras, side view cameras, etc.; the CCU domain camera can be used to realize functions such as panoramic monitoring (Around View Monitor, AVM), etc., and can specifically include surround view cameras, etc.; the cockpit domain camera can be used to realize in-vehicle occupant monitoring, such as driver monitoring (Driver Monitoring System, DMS), passenger monitoring (Occupancy Monitoring System, OMS), driving record ((Digital Video Recorder, DVR)) and other functions, and can specifically include built-in camera for taking pictures, etc.

[0119] For example, see Figure 2 , Figure 2 It is a timing interaction diagram of a vehicle-mounted multi-camera control method provided by an embodiment of the present invention.

[0120] There are three target applications, namely target application 1, target application 2, and target application 3. All cameras are classified according to their functions, and can be divided into intelligent driving domain target cameras 1 to 6, CCU domain target cameras 7 to 11, and cockpit domain target cameras 12 to 18. Target application 1 sends an operation request to open target camera 1, target application 2 sends an operation request to open target camera 1, and target application 3 sends an operation request to open target camera 8.

[0121] The existing vehicle system must know the process priority and the priority of the camera device access rights when the process is running. Specifically, it is determined by the priority (oom_score) of the caller process when the client opens the camera (open Camera). The process priority value (abbreviated as adj) is viewed through / proc / process_pid / oom_score_adj. It is impossible to determine the priority relationship of each client's access rights to the camera device in advance, and the access efficiency is low. In addition, the system can only allocate different resource values ​​for different camera IDs, and cannot allocate resources for clients. For the combination of multiple cameras and multiple clients on the vehicle, it cannot guarantee that clients with high priority can occupy more resources, resulting in poor accuracy and real-time performance of the client in obtaining camera data.

[0122] To solve the above problems, in the vehicle-mounted multi-camera control device of the vehicle-mounted system, a camera configuration module (CameraConfig) 103 and a camera arbitration module (CameraArbitration) 102 are newly added on top of the camera service module (CameraService) 104 .

[0123] In a specific embodiment, the camera configuration module 103 is used to:

[0124] Aggregating multiple target applications corresponding to the same target camera;

[0125] A configuration file is determined, where the configuration file is used to indicate that for the same target camera, different access priorities are configured for different target applications and different resource values ​​are allocated according to the importance of different target applications accessing the target camera.

[0126] Among them, the resource values ​​allocated by the target camera to the target application include resources such as computing power and memory.

[0127] In this embodiment, the same target camera can be called and opened by multiple corresponding target applications. For the same target camera, different access priorities and resource values ​​are configured for different applications according to the importance of the target application accessing the target camera, which is applicable to the multi-scenario multiplexing requirements of the automotive industry camera. For example, for important target applications, a higher access priority can be set and more resource values ​​can be allocated. In addition, for special target applications, access can be denied by configuring the priority. For example, for important target cameras containing sensitive information, this application can be set separately to be inaccessible.

[0128] During configuration, all vehicle-mounted cameras are first aggregated, and then multiple target applications corresponding to the same target camera are aggregated one by one. Different access priorities and resource values ​​are configured for different applications to form a configuration file for subsequent reading.

[0129] In this way, according to the importance of the target application, the target application of each target camera resource can be prioritized and resource allocated, and the access priority and resources of each client to the target camera can be configured according to product and system requirements. If necessary, the access priority and resources of the client can be dynamically adjusted at runtime to realize the complex camera reuse scenarios in the automotive industry, so that all client resources can be reasonably allocated, and the accuracy and real-time performance of high-priority target applications in important scenarios to obtain target camera data can be improved. In addition, through the configuration file, the priority relationship of each target application's access rights to the target camera can be determined in advance to improve access efficiency.

[0130] In a specific embodiment, the camera configuration module 103 reads the configuration file when the vehicle-mounted system is running, summarizes the target applications corresponding to each vehicle-mounted target camera, and configures the access priority and allocated resource value of each target application to the target camera in the configuration file for each target camera. When the vehicle-mounted multi-camera control device is initialized during operation, the camera configuration module 103 parses the configuration file into the memory of the vehicle-mounted multi-camera control device so as to subsequently obtain the access priority and allocated resource value of any target application to the target camera.

[0131] In a specific embodiment, the camera arbitration module 102 is further used for:

[0132] Obtain the access priority and allocated resource value of the target application to the target camera.

[0133] When a target application needs to access a target camera, the camera arbitration module 102 first obtains the access priority and allocated resource value of the target application from the memory of the vehicle-mounted multi-camera control device.

[0134] Furthermore, the camera arbitration module 102 is also used for:

[0135] Determine an application currently occupying the target camera;

[0136] Gets the access priority of the application currently occupying the target camera.

[0137] When the target application needs to access the target camera, the camera arbitration module 102 also needs to determine the application currently occupying the target camera and obtain the access priority of the application currently occupying the target camera from the memory of the vehicle-mounted multi-camera control device.

[0138] When different target applications need to access the same target camera, the camera arbitration module 102 performs priority arbitration on the target applications with access requests.

[0139] Specifically, if the camera arbitration module 102 determines that the target application's access priority to the target camera is higher than the access priority of the application currently occupying the target camera, the target application has the opportunity to preempt and open the target camera, and the application currently occupying the target camera will be forced to release the target camera resources and exit, and receive a preemption notification; if it is determined that the target application's access priority to the target camera is lower than the access priority of the application currently occupying the target camera, the target application cannot open the target camera, and the application currently occupying the target camera continues to occupy the target camera.

[0140] Furthermore, the process priority of most existing third-party Android applications is set by the system. For example, the process priority of the foreground program is higher than the process priority of the background program. This cannot meet the scenario where the background client program preempts the camera device resources, and cannot meet the complex camera device access scenarios in the automotive field.

[0141] In this embodiment, the target application and the application currently occupying the target camera can be not only a foreground client program (including a third-party application) but also a background client program. In this way, the rule that the process priority of the foreground client program is higher than the process priority of the background client program can be bypassed, and the scenario of the background client program occupying the target camera device resources can be realized, meeting the complex camera device access scenarios in the automotive field.

[0142] In a specific embodiment, the camera arbitration module 102 is further used for:

[0143] Get the maximum value of vehicle system resources;

[0144] Identify all applications that currently have cameras open;

[0145] The sum of the resource value allocated by the target application and the resource values ​​of all applications that have currently opened cameras is calculated.

[0146] Since the vehicle-mounted system has a fixed upper limit value for all camera access resources, when the upper limit value exceeds the maximum value, the camera opening will fail. In this embodiment, in order to perform subsequent resource arbitration, the camera arbitration module 102 first obtains the maximum value of the vehicle-mounted system resources, then determines all currently opened camera applications, and then calculates the sum of the resource value allocated by the target application and the resource value of all currently opened camera applications. It can be understood that all currently opened cameras are other currently occupied cameras.

[0147] After determining that the target application has a higher access priority to the target camera than the application currently occupying the target camera, the camera arbitration module 102 needs to accumulate resources for all applications that have opened cameras to determine whether the sum of the resource value allocated by the target camera to the target application and the resource value of all currently opened cameras does not exceed the maximum value of the vehicle system resources:

[0148] If it is determined that the sum of the resource value allocated by the target camera to the target application and the resource values ​​of all currently opened cameras does not exceed the maximum value of the vehicle system resources, the target application can open the target camera;

[0149] If it is determined that the sum of the resource value allocated by the target camera to the target application and the resource value of all currently opened cameras exceeds the maximum value of the vehicle system resources, it means that the resources of the device being used at this time are already tight and the new target application cannot apply to open successfully, then an error return will be made to the target application that is about to open the target camera.

[0150] After the camera arbitration module 102 determines that the access priority of the target application to the target camera is higher than the access priority of the application currently occupying the target camera, and determines that the sum of the resource value allocated by the target camera to the target application and the resource value of all currently opened cameras does not exceed the maximum value of the vehicle system resources, the camera service module 104, upon receiving the judgment result sent by the camera arbitration module 102, responds to the operation request of the target application to open the target camera, calls the camera abstraction layer (CameraHal) 106 through the camera provider module (CameraProvider) 105, and opens the target camera.

[0151] Therefore, the camera arbitration module 102 is specifically used to realize mutual exclusion and interruption control of the target camera application according to the access priority and allocated resource value of the target application. During operation, the vehicle-mounted multi-camera control device initializes the camera configuration module 103 to parse the configuration file into the memory, and the target application calls the camera management module (CameraManager) 101 standard interface, and then calls the newly added camera arbitration module 102. The camera arbitration module 102 obtains the access priority of the application currently occupying the target camera and the access priority of the application currently occupying the target camera from the memory, and performs priority arbitration. Then, according to the resource value allocated by the target application, combined with the maximum value of the vehicle-mounted system resources, the resource value allocated by the target application and the sum of the resource values ​​of all currently opened camera applications, it is determined whether the total application resource of the opened camera resources does not exceed the maximum value:

[0152] If it is a high priority and the total resource value does not exceed the maximum value of the vehicle system, the application that previously occupied the target camera is interrupted, and the result of the operation request of the target application is returned to obtain the opening qualification, and the camera service module 104 can be called;

[0153] If it is not a high priority or the total resource value exceeds the maximum value of the vehicle system, the target application's operation request result is directly returned as failure to open, the target camera is busy, and an error code is returned.

[0154] For example, see Figure 2 and Figure 3The camera management module 101 receives the operation request of the target application 1 to open the target camera 1. The camera arbitration module 102 performs priority arbitration and resource allocation to determine whether it is successful. Since the above two opening conditions are not met, the target camera 1 fails to be opened. The camera arbitration module 102 returns the request result, calls back QnError, and returns an error code.

[0155] The camera management module 101 receives the operation request from the target application 2 to open the target camera 1, and the camera arbitration module 102 determines whether the priority arbitration and resource allocation are successful. Since the above two opening conditions are met, the target application 2 obtains the qualification to open the target camera 1, calls the camera service module 104, and calls the camera abstract layer 106 through the camera provider module (CameraProvider) 105 to open the target camera 1.

[0156] The target application 3 sends an operation request to open the target camera 8. The camera arbitration module 102 performs priority arbitration and resource allocation to determine whether it is successful. Since the above two opening conditions are not met, the target camera 1 fails to open. The camera arbitration module 102 returns the request result, calls back QnError, and returns an error code.

[0157] In a specific embodiment, the target application has an access requirement to open multiple target cameras.

[0158] The camera management module 101 is further used to receive multiple operation requests from the target application to open multiple target cameras respectively; wherein the target application is configured with an access priority for each of the target cameras, and each of the target cameras is allocated a corresponding resource value to the target application;

[0159] The camera arbitration module 102 is further used to determine whether the access priority of the target application to each of the target cameras is higher than the access priority of the application currently occupying each of the target cameras; and further used to determine whether the sum of the resource value allocated by each of the target cameras to the target application and the resource value of all the applications that have currently opened the cameras does not exceed the maximum value of the vehicle system resources;

[0160] The camera service module 104 is further configured to, upon receiving a judgment result sent by the camera arbitration module 102 , respond to the corresponding operation request and open the corresponding target camera.

[0161] In this embodiment, for the same target application that has access requirements for multiple target cameras, the target application's access rights to each target camera can be controlled individually, and the target application's access priority can be configured according to product and system requirements. It is not limited to foreground and background processes, and can meet more complex camera reuse scenarios in the automotive industry.

[0162] Specifically, the camera management module 101 receives multiple operation requests from the target application to respectively open multiple target cameras, and the camera arbitration module 102 performs arbitration for each operation request separately:

[0163] Determine whether the target application's access priority to each target camera is higher than the access priority of the application currently occupying each target camera. If so, proceed to the next step of determination. If not, return the result of opening failure.

[0164] Determine whether the sum of the resource value allocated by each target camera to the target application and the resource value of all currently opened camera applications does not exceed the maximum value of the vehicle system resources. If so, open the corresponding target camera in response to the corresponding operation request. If not, return the result of failed opening.

[0165] It can be understood that, for the same target application, the access priority of the target application to each target camera can be the same or different, and the resource value allocated by each target camera to the target application can be the same or different.

[0166] Through the above steps, a target application that meets the two conditions can open the corresponding target camera, while a target application that does not meet the two conditions cannot open the corresponding target camera. That is, when the same target application has access requirements to multiple target cameras, it can eventually open all the target cameras, or only open some of the target cameras, or it may not be able to open all the target cameras.

[0167] Similarly, other target applications can also achieve precise access control of their multiple target cameras through the above steps, so that in multi-client multi-camera scenarios, important application scenarios such as reversing images can be given priority in resource allocation, and high-priority target applications in important scenarios can obtain their target camera data with accuracy and real-time performance.

[0168] Compared with the prior art, a vehicle-mounted multi-camera control device according to an embodiment of the present invention has the following beneficial effects:

[0169] The camera configuration module 103 configures different access priorities and allocates different resource values ​​for different target applications for the same target camera. The camera arbitration module 102 determines whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, and determines whether the sum of the resource value allocated by the target application and the resource value of all currently open camera applications does not exceed the maximum value of the vehicle system resources. It can achieve precise access control of the target camera according to the access priority and allocated resource value of the target application, meet the complex camera device access scenarios in the automotive field, and improve the accuracy and real-time performance of high-priority target applications in important scenarios in obtaining target camera data.

[0170] In addition, a vehicle-mounted multi-camera control device according to an embodiment of the present invention is applicable to all current Android versions and all industries using the Android system, and has a very high reuse rate.

[0171] See also Figure 4 , Figure 4 It is a structural diagram of a terminal device provided by an embodiment of the present invention.

[0172] The terminal device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in one of the vehicle-mounted multi-camera control method embodiments described above are implemented, for example: Figure 1 Alternatively, the processor implements the functions of each module in the above-mentioned device embodiments when executing the computer program.

[0173] Exemplarily, the computer program may be divided into one or more modules, and the one or more modules are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device. For example, the computer program may be divided into several modules, and the specific functions of each module have been described in detail in a vehicle-mounted multi-camera control method provided in any of the above embodiments, and the specific functions of the device will not be repeated here.

[0174] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the schematic diagram is only an example of a terminal device and does not constitute a limitation on a terminal device. The terminal device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0175] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect the various parts of the entire terminal device.

[0176] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the vehicle-mounted multi-camera control method by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0177] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle-mounted multi-camera control method is implemented.

[0178] If the module integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0179] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A vehicle-mounted multi-camera control method, characterized in that: include: Receive an operation request from a target application to open a target camera; wherein the target application is configured with an access priority to the target camera, and the target camera allocates a corresponding resource value to the target application; Determine whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera; Determine whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources; If both are true, then in response to the operation request, the target camera is turned on.

2. A vehicle-mounted multi-camera control method according to claim 1, characterized in that: Also includes: Aggregating multiple target applications corresponding to the same target camera; A configuration file is determined, where the configuration file is used to indicate that for the same target camera, different access priorities are configured for different target applications and different resource values ​​are allocated according to the importance of different target applications accessing the target camera.

3. A vehicle-mounted multi-camera control method according to claim 1, characterized in that: Before determining whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, the method further includes: Obtain the access priority and allocated resource value of the target application to the target camera.

4. The vehicle-mounted multi-camera control method according to claim 1, characterized in that: Before determining whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera, the method further includes: Determine an application currently occupying the target camera; Gets the access priority of the application currently occupying the target camera.

5. The vehicle-mounted multi-camera control method according to claim 1, characterized in that: Before determining whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources, the method further includes: Get the maximum value of vehicle system resources; Identify all applications that currently have cameras open; The sum of the resource value allocated by the target application and the resource values ​​of all applications that currently have cameras turned on is calculated.

6. A vehicle-mounted multi-camera control method according to any one of claims 1 to 5, characterized in that: The target application has an access requirement to open a plurality of the target cameras; the method further comprises: Receiving multiple operation requests from the target application to respectively open multiple target cameras; Determine whether the access priority of the target application to each of the target cameras is higher than the access priority of the application currently occupying each of the target cameras; Determine whether the sum of the resource value allocated by each target camera to the target application and the resource values ​​of all currently opened camera applications does not exceed the maximum value of the vehicle system resources; If both are true, then in response to the corresponding operation request, the corresponding target camera is opened.

7. The vehicle-mounted multi-camera control method according to claim 1, characterized in that: The target cameras include at least an intelligent driving domain target camera, a CCU domain target camera and a cockpit domain target camera.

8. A vehicle-mounted multi-camera control device, characterized in that: include: A camera management module, configured to receive an operation request from a target application to open a target camera; wherein the target application is configured with an access priority to the target camera, and the target camera allocates a corresponding resource value to the target application; A camera arbitration module, used to determine whether the access priority of the target application is higher than the access priority of the application currently occupying the target camera; and used to determine whether the sum of the resource value allocated by the target application and the resource values ​​of all currently opened cameras does not exceed the maximum value of the vehicle system resources; The camera service module is used to open the target camera in response to the operation request when receiving the judgment result sent by the camera arbitration module.

9. The vehicle-mounted multi-camera control device according to claim 8, characterized in that: It also includes a camera configuration module, which is used to: Aggregating multiple target applications corresponding to the same target camera; A configuration file is determined, where the configuration file is used to indicate that for the same target camera, different access priorities are configured for different target applications and different resource values ​​are allocated according to the importance of different target applications accessing the target camera.

10. The vehicle-mounted multi-camera control device according to claim 8, characterized in that: The camera arbitration module is further used to obtain the access priority and allocated resource value of the target application to the target camera.

11. The vehicle-mounted multi-camera control device according to claim 8, characterized in that: The camera arbitration module is further used to: determine the application currently occupying the target camera; and obtain the access priority of the application currently occupying the target camera.

12. The vehicle-mounted multi-camera control device according to claim 8, characterized in that: The camera arbitration module is also used to: obtain the maximum value of the vehicle system resources; determine all the applications that currently have the camera turned on; and calculate the sum of the resource value allocated by the target application and the resource value of all the applications that currently have the camera turned on.

13. The vehicle-mounted multi-camera control device according to any one of claims 8 to 12, characterized in that: The target application has an access requirement to open a plurality of the target cameras; The camera management module is further used to receive multiple operation requests from the target application to open multiple target cameras respectively; wherein the target application is configured with an access priority for each of the target cameras, and each of the target cameras is allocated a corresponding resource value to the target application; The camera arbitration module is further used to determine whether the access priority of the target application to each of the target cameras is higher than the access priority of the application currently occupying each of the target cameras; and further used to determine whether the sum of the resource value allocated by each of the target cameras to the target application and the resource values ​​of all the applications that have currently opened the cameras does not exceed the maximum value of the vehicle system resources; The camera service module is further configured to, upon receiving a judgment result sent by the camera arbitration module, respond to the corresponding operation request and open the corresponding target camera.

14. A terminal device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a vehicle-mounted multi-camera control method as described in any one of claims 1 to 7 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a vehicle-mounted multi-camera control method as described in any one of claims 1 to 7.

Images