Decoding resource inter-domain flow transfer control method, device and equipment and readable storage medium
By using the domain controller to isolate hardware domains and unload/load decoding resource drivers, and establishing inter-domain communication, the problem of low utilization of decoding resources fixed in a certain domain is solved. This enables the flow and reuse of decoding resources between different domains, improving hardware resource utilization and reducing CPU load.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SIENGINE TECH CO LTD
- Filing Date
- 2026-03-25
- Publication Date
- 2026-06-26
AI Technical Summary
In hardware platforms, accessing decoding resources within a fixed domain results in low utilization. The inability to flow between different domains leads to increased CPU resource consumption and the inability to decode high-resolution videos.
Hardware domain isolation is achieved through the domain controller, generating target domains and other domains. During system startup, the decoding resource driver is controlled to be unloaded and loaded, establishing inter-domain communication channels to enable the flow and reuse of decoding resources between different domains.
It improves the utilization of hardware decoding resources, avoids additional CPU resource consumption, and supports the decoding of high-resolution videos.
Smart Images
Figure CN121940540B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of hardware platform resource management technology, specifically to a method, apparatus, device, and readable storage medium for controlling the flow of decoding resource domains. Background Technology
[0002] In modern hardware platforms (such as automotive chip systems), domain isolation technology is typically used (e.g., Figure 1 Hardware resources are managed using domains A and B to improve system security and stability. The hardware platform's decoding resources are typically pre-configured within a specific domain via the Domain Control Manager (DCM). For example, by default, the decoding resource VPU (Video Processing Unit) can only be accessed within domain B. It should be understood that if the number of decoding resources on the hardware platform (e.g., one decoding hardware instance) is less than the number of domains (i.e., two domains, A and B), the decoding resources are usually fixed in domain B because domain B is the domain with the most frequent decoding scenarios.
[0003] It is worth noting that domain A requires decoding during the boot process and no longer needs decoding afterward. However, with the hardware platforms powered on simultaneously and decoding resources limited, domain B boots up slower than domain A. This means that when domain A has a need for decoding, domain B's decoding may not be ready. Consequently, domain A cannot use domain B's decoding resources through shared memory during the initial boot phase and can only use the CPU (Central Processing Unit) for decoding. This not only consumes additional CPU resources but also limits the decoding of low-resolution videos.
[0004] It is evident that how to enable the flow of decoding resources between different domains to improve the utilization rate of decoding resources is an urgent problem to be solved, so as to effectively avoid the consumption of additional CPU resources and support the decoding of high-resolution video. Summary of the Invention
[0005] This application provides a method, apparatus, device, and readable storage medium for controlling the flow of decoding resources between domains, which can solve the technical problem of low utilization of decoding resources caused by the fact that decoding resources can only be accessed within a fixed domain in the prior art.
[0006] In a first aspect, embodiments of this application provide a method for controlling the flow of decoding resource domains, comprising the following steps:
[0007] Hardware domain isolation is performed on the hardware platform through the domain controller manager to generate a target domain that starts up slower than other domains. The scope of the target decoding resources is not restricted.
[0008] When other domains are in the system startup state, the control controls the operation of the target decoding resource to decode the first playback video by the pre-configured first decoding resource driver corresponding to the other domains. After the target decoding resource is used up, the first decoding resource driver is unloaded.
[0009] When the target domain is in the system startup state, it controls the target domain to perform handshake communication with other domains so that the target domain can obtain target information corresponding to other domains. Based on the target information, it controls the pre-configured second decoding resource corresponding to the target domain to drive the operation of the target decoding resource to decode the second playback video, thereby realizing the flow of the target decoding resource. The target information includes the usage status of the decoding resource.
[0010] In conjunction with the first aspect, in one implementation, the control of a pre-configured first decoding resource corresponding to other domains drives the target decoding resource to decode the first playback video, including:
[0011] Start the pre-configured first control program corresponding to other domains, so as to load the first decoding resource driver through the first control program;
[0012] The first decoding resource driver is controlled to register with the domain controller so that only the first decoding resource driver can operate on the target decoding resource after registration. The second decoding resource driver is allowed to preempt registration.
[0013] The first decoding resource drives the operation of the target decoding resource to decode the first playback video and sends target information to the target domain.
[0014] In conjunction with the first aspect, in one implementation, the control target domain performs handshake communication with other domains to enable the target domain to obtain target information corresponding to other domains, and controls the pre-configured second decoding resource corresponding to the target domain to drive the operation of the target decoding resource to decode the second playback video according to the target information, including:
[0015] Start the pre-configured second control program corresponding to the target domain;
[0016] The second control program establishes communication with the first control program through inter-domain communication technology in order to obtain target information from the first control program;
[0017] When the target information detects that the usage status of the decoding resource is "used up" or "used abnormally", the second control program loads the second decoding resource driver.
[0018] Control the second decoding resource driver to register with the domain controller manager so that only the second decoding resource driver can operate on the target decoding resource after registration;
[0019] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0020] In conjunction with the first aspect, in one implementation, the target information further includes the unplayed duration corresponding to the first played video, and the method further includes:
[0021] If a handshake communication interruption is detected between the target domain and other domains, the pre-configured second control program corresponding to the target domain is controlled to count down according to the unplayed time.
[0022] Once the countdown ends, the second control program loads the second decoding resource driver.
[0023] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0024] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0025] In conjunction with the first aspect, in one embodiment, the method further includes:
[0026] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0027] The second control program determines whether the countdown has ended after receiving the driver loading request.
[0028] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0029] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0030] In conjunction with the first aspect, in one embodiment, the method further includes:
[0031] If the handshake communication between the target domain and other domains fails, the pre-configured second control program corresponding to the target domain will count down according to the maximum duration of the video that can be played in other domains.
[0032] Once the countdown ends, the second control program loads the second decoding resource driver.
[0033] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0034] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0035] In conjunction with the first aspect, in one embodiment, the method further includes:
[0036] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0037] The second control program checks the registration status of the decoding resources in the domain controller manager after receiving the driver loading request;
[0038] If the registration status is unregistered, then the step of loading the second decoding resource driver by the second control program is executed;
[0039] If the registration status is "registered", the second control program will determine whether the countdown has ended.
[0040] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0041] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0042] Secondly, embodiments of this application provide a decoding resource domain inter-domain flow control device, comprising:
[0043] The resource isolation module is used to perform hardware domain isolation on the hardware platform through the domain controller manager to generate a target domain that starts up slower than other domains. The scope of the target decoding resources is not restricted.
[0044] The first control module is used to control the pre-configured first decoding resource driver to operate the target decoding resource to decode the first playback video when other domains are in the system startup state, and to unload the first decoding resource driver after the target decoding resource is used up.
[0045] The second control module is used to control the target domain to perform handshake communication with other domains when the target domain is in the system startup state, so that the target domain can obtain target information corresponding to other domains, and control the pre-configured second decoding resource to drive the operation of the target decoding resource to decode the second playback video according to the target information, so as to realize the flow of the target decoding resource. The target information includes the usage status of the decoding resource.
[0046] In conjunction with the second aspect, in one implementation, the first control module is specifically used for:
[0047] Start the pre-configured first control program corresponding to other domains, so as to load the first decoding resource driver through the first control program;
[0048] The first decoding resource driver is controlled to register with the domain controller so that only the first decoding resource driver can operate on the target decoding resource after registration. The second decoding resource driver is allowed to preempt registration.
[0049] The first decoding resource drives the operation of the target decoding resource to decode the first playback video and sends target information to the target domain.
[0050] In conjunction with the second aspect, in one implementation, the second control module is specifically used for:
[0051] Start the pre-configured second control program corresponding to the target domain;
[0052] The second control program establishes communication with the first control program through inter-domain communication technology in order to obtain target information from the first control program;
[0053] When the target information detects that the usage status of the decoding resource is "used up" or "used abnormally", the second control program loads the second decoding resource driver.
[0054] Control the second decoding resource driver to register with the domain controller manager so that only the second decoding resource driver can operate on the target decoding resource after registration;
[0055] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0056] In conjunction with the second aspect, in one implementation, the target information further includes the unplayed duration corresponding to the first played video, and the second control module is further configured to:
[0057] If a handshake communication interruption is detected between the target domain and other domains, the pre-configured second control program corresponding to the target domain is controlled to count down according to the unplayed time.
[0058] Once the countdown ends, the second control program loads the second decoding resource driver.
[0059] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0060] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0061] In conjunction with the second aspect, in one implementation, the second control module is further configured to:
[0062] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0063] The second control program determines whether the countdown has ended after receiving the driver loading request.
[0064] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0065] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0066] In conjunction with the second aspect, in one implementation, the second control module is further configured to:
[0067] If the handshake communication between the target domain and other domains fails, the pre-configured second control program corresponding to the target domain will count down according to the maximum duration of the video that can be played in other domains.
[0068] Once the countdown ends, the second control program loads the second decoding resource driver.
[0069] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0070] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0071] In conjunction with the second aspect, in one implementation, the second control module is further configured to:
[0072] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0073] The second control program checks the registration status of the decoding resources in the domain controller manager after receiving the driver loading request;
[0074] If the registration status is unregistered, then the step of loading the second decoding resource driver by the second control program is executed;
[0075] If the registration status is "registered", the second control program will determine whether the countdown has ended.
[0076] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0077] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0078] Thirdly, embodiments of this application provide a decoding resource domain inter-domain flow control device, the decoding resource domain inter-domain flow control device including a processor, a memory, and a decoding resource domain inter-domain flow control program stored in the memory and executable by the processor, wherein when the decoding resource domain inter-domain flow control program is executed by the processor, it implements the steps of the decoding resource domain inter-domain flow control method as described above.
[0079] Fourthly, embodiments of this application provide a computer-readable storage medium storing a decoding resource domain flow control program, wherein when the decoding resource domain flow control program is executed by a processor, it implements the steps of the decoding resource domain flow control method as described above.
[0080] The beneficial effects of the technical solutions provided in this application include:
[0081] Hardware domain isolation is achieved through a domain controller to create a target domain that starts slower than other domains. The scope of the target decoding resource is not restricted, providing a basis for the flow of decoding resources between domains. When other domains are in a fully booted state, the pre-configured first decoding resource driver corresponding to the other domains operates the target decoding resource to decode the first playback video. After the target decoding resource is used up, the first decoding resource driver is unloaded. When the target domain is in a fully booted state, it initiates handshake communication with other domains to obtain target information, including the usage status of decoding resources, and controls the pre-configured second decoding resource driver corresponding to the target domain to operate the target decoding resource to decode the second playback video, thus realizing the flow of the target decoding resource. As can be seen, this application not only configures a decoding resource driver for each domain, enabling each domain to operate on the decoding resources, but also establishes an inter-domain communication channel so that different domains can know each other's decoding status, providing a basis for decision-making for the flow of decoding resources. This enables the dynamic flow and reuse of decoding resources between different domains, improves the utilization rate of hardware decoding resources, effectively avoids the consumption of additional CPU resources, reduces the CPU load, and supports the decoding of high-resolution video. Attached Figure Description
[0082] Figure 1 This is a schematic diagram of hardware platform domain isolation and VPU resource distribution in existing technologies;
[0083] Figure 2 This is a flowchart illustrating an embodiment of the decoding resource domain flow control method of this application;
[0084] Figure 3 This is a schematic diagram illustrating the hardware platform domain isolation and VPU resource distribution involved in the embodiments of this application;
[0085] Figure 4 This is a schematic diagram of the lifecycle of the control program involved in the embodiments of this application;
[0086] Figure 5 This is a schematic diagram of the hardware structure of the decoding resource domain flow control device involved in the embodiment of this application. Detailed Implementation
[0087] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0088] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.
[0089] APP (Application): Software applications designed to run on smartphones, tablets, and other mobile devices.
[0090] DCM (Domain Control Manager): The domain controller is responsible for managing the platform's IP authentication, access control, and resource access.
[0091] HAL (Hardware Abstraction Layer): The hardware abstraction layer is an intermediate layer between the operating system kernel and hardware devices.
[0092] HW (Hardware): System hardware resources.
[0093] User Space: User space is a virtual memory region allocated in the operating system. Together with kernel space, it forms a memory protection mechanism used to run user programs and restrict their direct access to hardware resources.
[0094] Kernel: Kernel space is a memory area in the operating system specifically reserved for kernel operation. It is isolated from user space, ensuring that errors in user programs do not affect the stability and security of the kernel.
[0095] RPMSG (Remote Processor Messaging) is a lightweight messaging framework in the Linux kernel for communication between heterogeneous multi-core / multi-processor systems.
[0096] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0097] In a first aspect, embodiments of this application provide a method for controlling the flow of decoding resource domains.
[0098] In one embodiment, reference is made to Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the decoding resource domain flow control method of this application. Figure 2 As shown, the method for controlling the flow of decoding resource domains includes:
[0099] Step S10: Perform hardware domain isolation on the hardware platform through the domain controller to generate a target domain that starts up slower than other domains. The scope of the target decoding resources is not restricted.
[0100] As an example, in this embodiment, when the hardware platform powers on, domain isolation is performed through DCM to divide the system into multiple domains, namely the target domain and other domains. The target domain refers to the domain of the VPU (Vehicle Processing Unit) that is accessible by default in the prior art. For example, if the VPU is configured by default to only be accessible within domain B, then domain B is the target domain. The domains outside the target domain are collectively referred to as other domains. When the target domain and other domains start up simultaneously, the startup speed of other domains is faster than that of the target domain. It should be noted that how to achieve domain isolation through DCM is common knowledge in the art, so it will not be elaborated here for the sake of brevity. In addition, the number of domains obtained can be determined according to the actual needs of the hardware platform and is not limited here. For example, if the vehicle chip system is divided into domain A and domain B, then the target domain is domain B and the other domains are domain A; while if the vehicle chip system is divided into domains A, B, and C, then the target domain is domain B and the other domains are domains A and C.
[0101] It is understood that target decoding resources refer to hardware resources in the hardware platform used for video decoding, such as VPU, and the target decoding resources are limited (i.e., the number of decoding resources is less than the number of domains); however, different isolated domains cannot use target decoding resources at the same time, and the decoding resources are needed in multiple domains; based on this, when the DCM of this embodiment pre-configures hardware domain isolation, it does not restrict the scope of the decoding resource VPU, that is, it does not configure the VPU to be accessible only in domain B (i.e., the target domain). In other words, the VPU can also be accessed in other domains, so as to provide an implementation basis for the flow of decoding resources between domains.
[0102] Step S20: When other domains are in the system startup state, control the pre-configured first decoding resource driver corresponding to other domains to operate the target decoding resource to decode the first playback video. After the target decoding resource is used up, the first decoding resource driver is unloaded.
[0103] As an example, in this embodiment, a decoding resource driver will be pre-configured for each isolation domain, for example, a first decoding resource driver will be pre-configured for other domains (e.g., Figure 3 The VPU driver corresponding to domain A in the middle (A), and at the same time, a second decoding resource driver is pre-configured for the target domain (such as...). Figure 3 (The VPU driver corresponding to domain B is shown in the image). Based on this, after other domain systems have started up, the first decoding resource driver can be directly loaded to operate the target decoding resource to decode the first playback video in other domains. After decoding is completed, the first decoding resource driver is uninstalled. It is worth noting that regardless of whether the driver is successfully uninstalled, the first decoding resource driver will perform a deregistration operation with the DCM. It should be noted that how to perform video decoding processing by operating decoding resources through a driver is common knowledge in this field, so for the sake of brevity, it will not be elaborated here.
[0104] Step S30: When the target domain is in the system startup state, control the target domain to perform handshake communication with other domains so that the target domain can obtain target information corresponding to other domains, and control the pre-configured second decoding resource corresponding to the target domain to drive the target decoding resource to decode the second playback video according to the target information, so as to realize the flow of the target decoding resource. The target information includes the decoding resource usage status.
[0105] In this exemplary embodiment, after the target domain system is started, it will control the target domain to establish communication with other domains through inter-domain communication RPMSG. This allows other domains to transmit playback information (i.e., target information, which includes the usage status of the target decoding resource by other domains, such as in use, use ended, or usage error) to the target domain by generating heartbeat packets (e.g., a communication handshake every 500ms). Based on this, the target domain can determine the current status of the target decoding resource being used by other domains according to the target information, and decide whether to load the second decoding resource driver to operate the target decoding resource, so as to decode the second playback video in the target domain through the target decoding resource, thereby realizing the transfer of the target decoding resource from other domains to the target domain.
[0106] As can be seen, this embodiment not only configures a decoding resource driver for each domain, enabling each domain to operate on the decoding resources, but also establishes an inter-domain communication channel so that different domains can know each other's decoding status, providing a basis for decision-making for the flow of decoding resources. This enables the dynamic flow and reuse of decoding resources between different domains, which not only improves the utilization rate of hardware decoding resources, but also eliminates the need for decoding through the CPU, effectively avoiding the consumption of additional CPU resources, reducing the CPU load, and supporting the decoding of high-resolution videos.
[0107] Further, in one embodiment, the control of the pre-configured first decoding resource corresponding to other domains to drive the target decoding resource to decode the first playback video includes:
[0108] Start the pre-configured first control program corresponding to other domains, so as to load the first decoding resource driver through the first control program;
[0109] The first decoding resource driver is controlled to register with the domain controller so that only the first decoding resource driver can operate on the target decoding resource after registration. The second decoding resource driver is allowed to preempt registration.
[0110] The first decoding resource drives the operation of the target decoding resource to decode the first playback video and sends target information to the target domain.
[0111] In this exemplary embodiment, a control program (i.e., a first control program) will be pre-configured for other domains to implement drive control of their respective domains and data interaction with the target domain. Specifically, after the other domain systems have started, the first control program is started, causing it to load the first decoding resource driver. The first decoding resource driver registers with the DCM and allows the second decoding resource driver in the target domain to preemptively register. It is worth noting that the VPU hardware can be accessed on the system platform, and the operation of the application controlling the VPU hardware to perform decoding must be controlled by the VPU driver. That is, there is only one VPU hardware, but there can be multiple drivers. However, only one VPU driver can operate the VPU hardware at the same time, because multiple drivers operating the VPU hardware at the same time will cause hardware abnormalities. Therefore, after the first decoding resource driver completes its registration, only the first decoding resource driver can operate the target decoding resource. Based on this, the first control program can operate the target decoding resource through the first decoding resource driver to decode the first playback video and send target information, including the decoding resource usage status, to the target domain through inter-domain communication RPMSG.
[0112] Further, in one embodiment, the control target domain performs handshake communication with other domains to enable the target domain to obtain target information corresponding to other domains, and controls the pre-configured second decoding resource corresponding to the target domain to drive the operation of the target decoding resource to decode the second playback video according to the target information, including:
[0113] Start the pre-configured second control program corresponding to the target domain;
[0114] The second control program establishes communication with the first control program through inter-domain communication technology in order to obtain target information from the first control program;
[0115] When the target information detects that the usage status of the decoding resource is "used up" or "used abnormally", the second control program loads the second decoding resource driver.
[0116] Control the second decoding resource driver to register with the domain controller manager so that only the second decoding resource driver can operate on the target decoding resource after registration;
[0117] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0118] As an example, in this embodiment, a control program (i.e., a second control program) will be pre-configured for the target domain to achieve drive control of its domain and data interaction with other domains. Specifically, after the target domain system is started, the second control program is launched. At this time, the second control program will establish communication with the first control program through inter-domain communication RPMSG to obtain target information from the first control program. When the target information detects that the usage status of the decoding resources of other domains is "end of use" or "abnormal use", the second control program will load the second decoding resource driver so that the second decoding resource driver can register normally with the DCM (when the decoding resource usage status is "end of use") or preemptively register (i.e., when the decoding resource usage status is "abnormal use"). It can be understood that after the second decoding resource driver completes registration, only the second decoding resource driver can operate the target decoding resource. Based on this, the second playback video can be decoded by operating the target decoding resource through the second decoding resource driver.
[0119] See Figure 3 and Figure 4 As shown, taking domain B as the target domain and its corresponding second control program as control program-B, the second decoding resource driver as VPU driver-B, and other domains as domain A and their corresponding first control programs as control program-A, and the first decoding resource driver as VPU driver-A as an example: 1) When the system in domain A has started, control program-A is started; 2) Control program-A loads VPU driver-A; 3) VPU driver-A registers with DCM and allows VPU driver-B to preemptively register. After registration, only VPU driver-A can operate the VPU hardware; 4) Control program-A controls video decoding; 5) When the system in domain B has started, control program-B is started. Control program-B and control program-A establish communication through inter-domain communication RPMSG, enabling control program-A to send heartbeat packets to control program-B. -B transmits playback information of VPU used in domain A (i.e., in use / used out / abnormal)); 6) Decoding of domain A is complete; 7) Control program -A uninstalls VPU driver -A, and VPU driver -A performs a deregistration operation with DCM, regardless of whether the driver uninstallation was successful; 8) Control program -A informs control program -B via heartbeat that decoding of domain A is complete (i.e., used out or abnormal information); 9) Control program -A exits voluntarily after receiving a reply from control program -B, and the program ends; 10) Control program -B loads VPU driver -B; 11) If decoding is completed normally, it means that VPU is in an unregistered state in DCM, and VPU driver -B can register normally; if decoding ends abnormally, it means that VPU is in a registered state in DCM, and VPU driver -B will preemptively register in DCM.
[0120] Furthermore, in one embodiment, the target information further includes the unplayed duration corresponding to the first played video, and the method further includes:
[0121] If a handshake communication interruption is detected between the target domain and other domains, the pre-configured second control program corresponding to the target domain is controlled to count down according to the unplayed time.
[0122] Once the countdown ends, the second control program loads the second decoding resource driver.
[0123] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0124] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0125] Exemplary and understandable, during the interaction between the first control program and the second control program, the first control program not only sends the decoding resource usage status of other domains to the second control program, but also sends the unplayed duration information of the first playing video in domain A. That is, the target information also includes the unplayed duration corresponding to the first playing video. Based on this, if the handshake communication between the target domain and other domains is interrupted, it means that the decoding status information of other domains (i.e., the target information) cannot be transmitted to the target domain via heartbeat packets. At this time, the target domain will not know the decoding status of other domains and can only perform its own decoding control based on the unplayed duration received before the interruption. Specifically, the second control program will count down according to the unplayed duration received before the interruption, and load the second decoding resource driver when the countdown ends, so that the second decoding resource driver can be properly registered or preemptively registered with the DCM, thereby operating the target decoding resource through the second decoding resource driver to decode the second playing video.
[0126] Furthermore, in one embodiment, the method further includes:
[0127] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0128] The second control program determines whether the countdown has ended after receiving the driver loading request.
[0129] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0130] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0131] As an example, it should be understood that an App in the target domain may initiate a decoding request to the Hardware Abstraction Layer (HAL) at any time; based on this, see [link to relevant documentation]. Figure 3 As shown, during the countdown, if the App in the target domain (e.g., domain B) initiates a decoding request to HAL, HAL will operate the second decoding resource driver (i.e., VPU driver-B). However, since the control program-B has not yet loaded the VPU driver-B, it cannot find the VPU driver-B device node. Therefore, it initiates a driver loading request to the control program-B. At this time, after receiving the driver loading request, the control program-B will determine whether the countdown has ended. If the countdown has ended, it will load the VPU driver-B and exit after loading is completed. Since the VPU may be in two states in the DCM at this time: (1) Unregistered state, i.e., the driver loading in domain A fails or domain A is unloaded normally after using decoding and successfully deregisters with the DCM; (2) Registered state, i.e., domain A is using decoding or decoding ends abnormally and has not deregistered with the DCM. Based on this, the VPU driver-B will register normally or preemptively register with the DCM, so that the VPU can be operated through the VPU driver-B to decode the second playback video.
[0132] If the countdown has not ended, an error is returned to HAL, which in turn causes HAL to return an error to the App, indicating that the request has failed. Once the control program-B detects that the countdown has ended, it will load the VPU driver-B, thereby registering with the DCM in a normal or preemptive manner through the VPU driver-B.
[0133] Furthermore, in one embodiment, the method further includes:
[0134] If the handshake communication between the target domain and other domains fails, the pre-configured second control program corresponding to the target domain will count down according to the maximum duration of the video that can be played in other domains.
[0135] Once the countdown ends, the second control program loads the second decoding resource driver.
[0136] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0137] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0138] For example, in this embodiment, if the handshake communication between the target domain and other domains fails, that is, if the handshake communication has never been successful, it means that the second control program has not received the unplayed duration and decoding resource usage status sent by the first control program. At this time, the second control program will count down according to the maximum duration of the video that can be played in other domains. The maximum duration can be preset according to actual needs and is not limited here. After the countdown ends, the second control program will load the second decoding resource driver. Since the target decoding resource may be in two states in the DCM at this time: (1) unregistered state, that is, the other domain fails to load the driver or the other domain uses the decoder and then unloads it normally and successfully deregisters with the DCM; (2) registered state, that is, the other domain is using the decoder or the decoder ends abnormally and has not deregistered with the DCM. Based on this, the second decoding resource driver will register normally or preemptively register with the DCM, so that the target decoding resource can be operated through the second decoding resource driver to decode the second playback video.
[0139] Furthermore, in one embodiment, the method further includes:
[0140] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0141] The second control program checks the registration status of the decoding resources in the domain controller manager after receiving the driver loading request;
[0142] If the registration status is unregistered, then the step of loading the second decoding resource driver by the second control program is executed;
[0143] If the registration status is "registered", the second control program will determine whether the countdown has ended.
[0144] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0145] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0146] As an example, it should be understood that an App in the target domain may initiate a decoding request to the HAL at any time; based on this, see [link to relevant documentation]. Figure 3As shown, during the countdown, if an App in the target domain (e.g., domain B) initiates a decoding request to HAL, HAL will operate the second decoding resource driver (i.e., VPU driver-B). However, since the control program-B has not yet loaded VPU driver-B, it cannot find the VPU driver-B device node, and therefore initiates a driver loading request to the control program-B. After receiving the driver loading request, the control program-B, since it has not obtained any playback status information from the control program-A, will check the registration status of the VPU in the DCM. If the registration status is not registered, it will directly load VPU driver-B so that VPU driver-B can register normally with the DCM and inform HAL to notify the App to perform a retry operation (i.e., a retry operation) so that the VPU can be operated by VPU driver-B to decode the second playback video.
[0147] If the registration status is "registered," the second control program will determine whether the countdown has ended. If the countdown has ended, it will load VPU driver-B and register in DCM via VPU driver-B in normal or preemptive mode, thus enabling the VPU to be operated through VPU driver-B to decode the second playback video. If the countdown has not ended, it will return an error to HAL, causing HAL to return a resource failure error to the App, indicating that the request has failed. After the control program-B detects that the countdown has ended, it will load VPU driver-B and exit after loading is complete.
[0148] In summary, the decoding resource inter-domain flow control method proposed in this embodiment realizes the reuse of decoding hardware resources between domains in a platform with domain isolation. It solves the problem of hardware resource flow reuse in video decoding applications when the number of decoding instances is limited, thereby effectively improving the utilization rate of VPU hardware resources, reducing the CPU load, and supporting the decoding of high-resolution videos.
[0149] Secondly, embodiments of this application also provide a decoding resource domain inter-domain flow control device.
[0150] In one embodiment, the decoding resource domain inter-domain flow control device includes:
[0151] The resource isolation module is used to perform hardware domain isolation on the hardware platform through the domain controller manager to generate a target domain that starts up slower than other domains. The scope of the target decoding resources is not restricted.
[0152] The first control module is used to control the pre-configured first decoding resource driver to operate the target decoding resource to decode the first playback video when other domains are in the system startup state, and to unload the first decoding resource driver after the target decoding resource is used up.
[0153] The second control module is used to control the target domain to perform handshake communication with other domains when the target domain is in the system startup state, so that the target domain can obtain target information corresponding to other domains, and control the pre-configured second decoding resource to drive the operation of the target decoding resource to decode the second playback video according to the target information, so as to realize the flow of the target decoding resource. The target information includes the usage status of the decoding resource.
[0154] Furthermore, in one embodiment, the first control module is specifically used for:
[0155] Start the pre-configured first control program corresponding to other domains, so as to load the first decoding resource driver through the first control program;
[0156] The first decoding resource driver is controlled to register with the domain controller so that only the first decoding resource driver can operate on the target decoding resource after registration. The second decoding resource driver is allowed to preempt registration.
[0157] The first decoding resource drives the operation of the target decoding resource to decode the first playback video and sends target information to the target domain.
[0158] Furthermore, in one embodiment, the second control module is specifically used for:
[0159] Start the pre-configured second control program corresponding to the target domain;
[0160] The second control program establishes communication with the first control program through inter-domain communication technology in order to obtain target information from the first control program;
[0161] When the target information detects that the usage status of the decoding resource is "used up" or "used abnormally", the second control program loads the second decoding resource driver.
[0162] Control the second decoding resource driver to register with the domain controller manager so that only the second decoding resource driver can operate on the target decoding resource after registration;
[0163] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0164] Furthermore, in one embodiment, the target information further includes the unplayed duration corresponding to the first played video, and the second control module is further configured to:
[0165] If a handshake communication interruption is detected between the target domain and other domains, the pre-configured second control program corresponding to the target domain is controlled to count down according to the unplayed time.
[0166] Once the countdown ends, the second control program loads the second decoding resource driver.
[0167] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0168] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0169] Furthermore, in one embodiment, the second control module is further configured to:
[0170] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0171] The second control program determines whether the countdown has ended after receiving the driver loading request.
[0172] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0173] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0174] Furthermore, in one embodiment, the second control module is further configured to:
[0175] If the handshake communication between the target domain and other domains fails, the pre-configured second control program corresponding to the target domain will count down according to the maximum duration of the video that can be played in other domains.
[0176] Once the countdown ends, the second control program loads the second decoding resource driver.
[0177] Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration;
[0178] The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
[0179] Furthermore, in one embodiment, the second control module is further configured to:
[0180] During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program.
[0181] The second control program checks the registration status of the decoding resources in the domain controller manager after receiving the driver loading request;
[0182] If the registration status is unregistered, then the step of loading the second decoding resource driver by the second control program is executed;
[0183] If the registration status is "registered", the second control program will determine whether the countdown has ended.
[0184] If the process has ended, then execute the step of loading the second decoding resource driver in the second control program.
[0185] If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
[0186] The functions of each module in the above-mentioned decoding resource domain inter-domain flow control device correspond to the steps in the above-mentioned decoding resource domain inter-domain flow control method embodiment, and their functions and implementation processes will not be described in detail here.
[0187] Thirdly, embodiments of this application provide a decoding resource domain inter-domain flow control device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.
[0188] Reference Figure 5 , Figure 5 This is a schematic diagram of the hardware structure of the decoding resource domain inter-domain transfer control device involved in the embodiments of this application. In the embodiments of this application, the decoding resource domain inter-domain transfer control device may include a processor, a memory, a communication interface, and a communication bus.
[0189] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.
[0190] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the decoding resource domain flow control device, as well as interfaces used for interconnecting the decoding resource domain flow control device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.
[0191] Memory can be various types of storage media, such as Random Access Memory (RAM), Read-only Memory (ROM), Non-volatile RAM (NVRAM), Flash memory, Optical storage, Hard disk, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), etc.
[0192] The processor can be a general-purpose processor, which can call the decoding resource domain inter-domain flow control program stored in memory and execute the decoding resource domain inter-domain flow control method provided in the embodiments of this application. For example, the general-purpose processor can be a CPU. The method executed when the decoding resource domain inter-domain flow control program is called can be referred to in the various embodiments of the decoding resource domain inter-domain flow control method of this application, and will not be repeated here.
[0193] Those skilled in the art will understand that Figure 5 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0194] Fourthly, embodiments of this application also provide a computer-readable storage medium.
[0195] The present application stores a decoding resource domain inter-domain flow control program on a readable storage medium, wherein when the decoding resource domain inter-domain flow control program is executed by a processor, it implements the steps of the decoding resource domain inter-domain flow control method as described above.
[0196] The method implemented when the decoding resource domain flow control program is executed can be referred to in various embodiments of the decoding resource domain flow control method of this application, and will not be repeated here.
[0197] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0198] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0199] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0200] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0201] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0202] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0203] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A method for controlling the flow of decoding resource domains, characterized in that, Includes the following steps: Hardware domain isolation is performed on the hardware platform through the domain controller manager to generate a target domain that starts up slower than other domains. The scope of the target decoding resources is not restricted. When other domains are in the system startup state, the control controls the operation of the target decoding resource to decode the first playback video by the pre-configured first decoding resource driver corresponding to the other domains. After the target decoding resource is used up, the first decoding resource driver is unloaded. When the target domain is in the system startup state, it controls the target domain to perform handshake communication with other domains so that the target domain can obtain target information corresponding to other domains. Based on the target information, it controls the pre-configured second decoding resource corresponding to the target domain to drive the operation of the target decoding resource to decode the second playback video, thereby realizing the flow of the target decoding resource. The target information includes the decoding resource usage status and the unplayed duration corresponding to the first playback video. If a handshake communication interruption is detected between the target domain and other domains, the pre-configured second control program corresponding to the target domain is controlled to count down according to the unplayed time. Once the countdown ends, the second control program loads the second decoding resource driver. Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration; The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
2. The decoding resource domain inter-domain flow control method as described in claim 1, characterized in that, The control, along with other domains, corresponds to a pre-configured first decoding resource that drives the target decoding resource to decode the first playback video, including: Start the pre-configured first control program corresponding to other domains, so as to load the first decoding resource driver through the first control program; The first decoding resource driver is controlled to register with the domain controller so that only the first decoding resource driver can operate on the target decoding resource after registration. The second decoding resource driver is allowed to preempt registration. The first decoding resource drives the operation of the target decoding resource to decode the first playback video and sends target information to the target domain.
3. The decoding resource domain inter-domain flow control method as described in claim 2, characterized in that, The control target domain performs handshake communication with other domains to enable the target domain to obtain target information corresponding to other domains, and controls the pre-configured second decoding resource corresponding to the target domain to drive the operation of the target decoding resource to decode the second playback video according to the target information, including: Start the pre-configured second control program corresponding to the target domain; The second control program establishes communication with the first control program through inter-domain communication technology in order to obtain target information from the first control program; When the target information detects that the usage status of the decoding resource is "used up" or "used abnormally", the second control program loads the second decoding resource driver. Control the second decoding resource driver to register with the domain controller manager so that only the second decoding resource driver can operate on the target decoding resource after registration; The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
4. The method for controlling the flow of decoding resource domains as described in claim 1, characterized in that, The method further includes: During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program. The second control program determines whether the countdown has ended after receiving the driver loading request. If the process has ended, then execute the step of loading the second decoding resource driver in the second control program. If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
5. The method for controlling the flow of decoding resource domains as described in claim 1, characterized in that, The method further includes: If the handshake communication between the target domain and other domains fails, the pre-configured second control program corresponding to the target domain will count down according to the maximum duration of the video that can be played in other domains. Once the countdown ends, the second control program loads the second decoding resource driver. Control the second decoding resource driver to register normally or preemptively with the domain controller manager so that only the second decoding resource driver can operate the target decoding resource after registration; The second decoding resource drives the operation of the target decoding resource to decode the second playback video.
6. The method for controlling the flow of decoding resource domains as described in claim 5, characterized in that, The method further includes: During the countdown, if an application in the target domain initiates a decoding request, the hardware abstraction layer is controlled to search for the second decoding resource driver. If the second decoding resource driver is not found, a driver loading request is sent to the second control program. The second control program checks the registration status of the decoding resources in the domain controller manager after receiving the driver loading request; If the registration status is unregistered, then the step of loading the second decoding resource driver by the second control program is executed; If the registration status is "registered", the second control program will determine whether the countdown has ended. If the process has ended, then execute the step of loading the second decoding resource driver in the second control program. If the process does not end, the hardware abstraction layer returns an error message to the application and executes the step of loading the second decoding resource driver by the second control program after the countdown ends.
7. A decoding resource domain inter-domain flow control device, characterized in that, include: The resource isolation module is used to perform hardware domain isolation on the hardware platform through the domain controller manager to generate a target domain that starts up slower than other domains. The scope of the target decoding resources is not restricted. The first control module is used to control the pre-configured first decoding resource driver to operate the target decoding resource to decode the first playback video when other domains are in the system startup state, and to unload the first decoding resource driver after the target decoding resource is used up. The second control module is used to control the target domain to conduct handshake communication with other domains when the target domain is in the system startup state, so that the target domain can obtain target information corresponding to other domains, and control the pre-configured second decoding resource driver to operate the target decoding resource to decode the second playback video according to the target information, so as to realize the flow of the target decoding resource. The target information includes the usage status of the decoding resource and the unplayed duration corresponding to the first playback video. If a handshake communication interruption is detected between the target domain and other domains, the pre-configured second control program corresponding to the target domain is controlled to count down according to the unplayed duration. After the countdown ends, the second control program is controlled to load the second decoding resource driver; the second decoding resource driver is controlled to register normally or preemptively register with the domain controller so that only the second decoding resource driver can operate the target decoding resource after registration; the second decoding resource driver operates the target decoding resource to decode the second playback video.
8. A decoding resource domain inter-domain flow control device, characterized in that, The decoding resource domain flow control device includes a processor, a memory, and a decoding resource domain flow control program stored in the memory and executable by the processor, wherein when the decoding resource domain flow control program is executed by the processor, it implements the steps of the decoding resource domain flow control method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a decoding resource domain flow control program, wherein when the decoding resource domain flow control program is executed by a processor, it implements the steps of the decoding resource domain flow control method as described in any one of claims 1 to 6.