Display device and camera algorithm resource scheduling method

Abstract

The invention discloses a display device and a scheduling method of camera algorithm resources. A camera connected with the display device is supported to be shared and called by one or more applications. The display device can respond to a calling request initiated by a target application, and determine a to-be-called first camera algorithm and a currently running second camera algorithm. A first algorithm function feature of the first camera algorithm is queried and a second algorithm function feature of the second camera algorithm is detected. If the first algorithm function feature and the second algorithm function feature do not have the same feature, concurrently executing a first camera algorithm and a second camera algorithm; and otherwise, operating the first camera algorithm or maintaining the operation of the second camera algorithm to ensure that no conflict occurs between the algorithms. According to the method, the limitation of a camera exclusive mechanism can be broken through, algorithm conflicts are judged through algorithm function characteristics, non-conflict algorithm concurrence and conflict algorithm ordered scheduling are achieved, and therefore the camera resource utilization rate and the multi-application compatibility are improved, and the user interaction experience is optimized.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a method for scheduling display device and camera algorithm resources. Background Technology

[0002] Display devices refer to terminal devices capable of outputting specific display images, such as smart TVs, mobile terminals, smart advertising screens, and projectors. Taking smart TVs as an example, smart TVs are television products based on Internet application technologies, possessing open operating systems and chips, and having open application platforms. They enable two-way human-computer interaction and integrate multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] Display devices can connect to cameras via device interfaces to enable camera-related functions. In display devices, camera deployment and usage employ an application-exclusive mechanism. Camera resource allocation depends on the device identifier. When an application enables a camera or invokes a camera algorithm, that application exclusively uses the corresponding camera and algorithm resources based on the device identifier. Other applications can only initiate a request to access the camera after the current application releases its camera resources.

[0004] However, in the aforementioned exclusive mechanism, the camera and its algorithm resources can only be used by a single application. Even if the camera algorithm resources used by multiple applications do not conflict, it cannot support multiple applications to use the camera algorithm resources at the same time, which reduces the utilization rate of camera algorithm resources and thus affects the user experience of camera functions in display devices. Summary of the Invention

[0005] This application provides a method for scheduling display device and camera algorithm resources to solve the problem of low utilization of camera algorithm resources in display devices.

[0006] In a first aspect, this application provides a display device, comprising: A display for showing the application interface of a target application, which supports accessing a camera and camera algorithm resources; The device interface is configured to connect to a camera that can be shared and accessed by one or more applications. The controller is configured as follows: In response to the call request initiated by the target application, determine the first camera algorithm to be called and the currently running second camera algorithm; Query the first algorithm function features of the first camera algorithm, and detect the second algorithm function features of the second camera algorithm; In response to the fact that the first algorithm function feature and the second algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently. In response to the fact that the first algorithm function feature and the second algorithm function feature have the same algorithm function feature, the first camera algorithm is run, or the second camera algorithm is kept running.

[0007] The above technical solution has the following beneficial effects or advantages: Differentiated scheduling is achieved by comparing algorithm functional characteristics. It supports the concurrent operation of non-conflicting algorithms and can select one to execute when there is a conflict. This breaks through the exclusive limitation of traditional cameras on hardware and software resources, maximizes resource utilization, and can take into account the algorithm calling needs of multiple applications, thereby enhancing the user experience of camera functions.

[0008] In some embodiments of this disclosure, the controller is configured to maintain the operation of the second camera algorithm, specifically as follows: If the target application is running in the background, the call request is blocked, and the second camera algorithm continues to run.

[0009] The above technical solution has the following beneficial effects or advantages: In response to background application requests, the display device can prioritize ensuring the continuity of the currently running algorithm by blocking the request, thus avoiding interruption of camera function caused by disordered background requests preempting resources, and thereby ensuring the user experience of foreground applications or core algorithms.

[0010] In some embodiments of this disclosure, after the controller blocks the call request, it is further configured to: Register the first listener for the second camera algorithm; Based on the first listening query, the first running result of the second camera algorithm is obtained; If the first running result indicates that the system is idle, the second algorithm function feature of the second camera algorithm is re-detected. In response to the fact that the second algorithm function feature detected by the re-detection does not have the same algorithm function feature as the first algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently.

[0011] The above technical solution has the following beneficial effects or advantages: By monitoring the operation of the camera algorithm, the display device can perform a secondary conflict judgment after the resources are idle. This can avoid long-term idleness of resources and ensure that there are no conflicts with camera functions when they are called back, thereby achieving efficient reuse of camera algorithm resources.

[0012] In some embodiments of this disclosure, the controller executes the first camera algorithm, specifically configured as follows: If the target application is running in the foreground, stop running the second camera algorithm; The first camera algorithm is executed in response to the call request.

[0013] The above technical solution has the following beneficial effects or advantages: Based on the principle of prioritizing foreground applications, the display device promptly releases background camera algorithm resources in case of conflicts, ensuring real-time response of foreground applications and operation of camera functions, thus meeting users' priority usage needs in multi-application interaction scenarios.

[0014] In some embodiments of this disclosure, the controller is further configured to: before executing the determination of the first camera algorithm to be invoked and the currently running second camera algorithm: Register the second listener for the first camera algorithm; Based on the second monitoring query, the second running result of the first camera algorithm is obtained; If the second running result indicates that the system is idle, the first camera algorithm is executed in response to the call request. If the second running result indicates that the computer is not idle, the step of determining the first camera algorithm to be invoked and the currently running second camera algorithm is executed.

[0015] The above technical solution has the following beneficial effects or advantages: The display device monitors and verifies the idle status of the first camera algorithm. When the algorithm to be called is idle, it directly calls the corresponding algorithm resource, which can improve response efficiency. When the algorithm is not idle, it starts the conflict determination logic to ensure the accuracy of conflict determination.

[0016] In some embodiments of this disclosure, the controller is further configured to: before executing the determination of the first camera algorithm to be invoked and the currently running second camera algorithm: Based on the target camera indicated by the call request, query the algorithm capabilities of the target camera; In response to the algorithm capability characterizing an instance where the target camera is deployed with a camera algorithm, the step of determining the first camera algorithm to be invoked and the currently running second camera algorithm is performed; In response to an instance where the algorithm capability characterizes that the target camera has not deployed a camera algorithm, the hardware resources of the target camera are invoked, and the first camera algorithm is run based on the hardware resources.

[0017] The above technical solution has the following beneficial effects or advantages: The display device can be adapted to cameras with different algorithm deployment types. Conflict scheduling is initiated for cameras with deployed camera algorithms, while cameras without deployed camera algorithms directly call hardware resources, thereby achieving compatibility and adaptation for multiple types of cameras and increasing the applicability of the solution.

[0018] In some embodiments of this disclosure, the controller is specifically configured to perform an algorithmic query of the target camera. Call the extension unit interface to generate a query command for the algorithm's capabilities; The algorithm returns byte data in response to the query command; The algorithm state mask is parsed from the algorithm byte data to determine the algorithm's capabilities; the algorithm state mask includes instance information of the algorithm instance.

[0019] The above technical solution has the following beneficial effects or advantages: The display device can accurately obtain the algorithm deployment status of the camera through standardized expansion unit interfaces and status mask parsing, providing precise data support for resource scheduling.

[0020] In some embodiments of this disclosure, the algorithm capability characterizes instances where the target camera has not deployed a camera algorithm, and the controller is further configured to: Before executing the first camera algorithm based on the hardware resources. If the hardware resources of the target camera are already occupied, a preemption notification is generated for the target application. In response to the preemption notification, the hardware resources are released.

[0021] The above technical solution has the following beneficial effects or advantages: For traditional cameras without algorithm deployment, the display device releases camera hardware resources in an orderly manner through preemption notifications, enabling the target application to schedule hardware and algorithm resources normally.

[0022] In some embodiments of this disclosure, the controller executes a second algorithmic function feature for detecting the second camera algorithm, specifically configured as follows: A communication link with the camera is established based on the device interface; The status data of the second camera algorithm is obtained according to the communication link; Parse the feature identifiers corresponding to the algorithm functions of the second camera algorithm from the state data; The functional features of the second algorithm are determined based on the feature identifier.

[0023] The above technical solution has the following beneficial effects or advantages: The display device extracts the algorithmic functional features of the running camera algorithm through communication links and feature identifier parsing, which can provide accurate data support for algorithm conflict judgment.

[0024] Secondly, this application also provides a method for scheduling camera algorithm resources, including: In response to a call request initiated by the target application, determine the first camera algorithm to be invoked and the currently running second camera algorithm; Query the first algorithm function features of the first camera algorithm, and detect the second algorithm function features of the second camera algorithm; In response to the fact that the first algorithm function feature and the second algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently. In response to the existence of the same algorithm function features between the first algorithm function features and the second algorithm function features, the first camera algorithm is run, or the second camera algorithm is kept running.

[0025] As can be seen from the above technical solutions, the scheduling method for display devices and camera algorithm resources provided in this application embodiment allows the camera connected to the display device to be shared and invoked by one or more applications. The display device can respond to an invocation request initiated by a target application, determining the first camera algorithm to be invoked and the currently running second camera algorithm. It then queries the first algorithm functional characteristics of the first camera algorithm and detects the second algorithm functional characteristics of the second camera algorithm. If the first and second algorithm functional characteristics do not share the same algorithm functional characteristics, the first and second camera algorithms are executed concurrently; otherwise, the first camera algorithm is run or the second camera algorithm is kept running to ensure that there is no conflict between the algorithms. This application can overcome the limitations of the camera exclusivity mechanism, determine algorithm conflicts through algorithm functional characteristics, achieve concurrent execution of non-conflicting algorithms and orderly scheduling of conflicting algorithms, thereby improving camera resource utilization and multi-application compatibility, and optimizing the user interaction experience. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic diagram illustrating display device operation scenarios provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 Timing diagrams for camera exclusivity mechanisms provided in some embodiments of this application; Figure 5 A flowchart illustrating a method for scheduling camera algorithm resources provided in some embodiments of this application; Figure 6 This application provides a deployment architecture diagram for camera algorithm interaction control in some embodiments. Figure 7 This application provides an architecture diagram for data processing based on the StreamPipe module, which is provided in some embodiments. Figure 8 A schematic diagram illustrating the multi-application camera algorithm conflict response process provided in some embodiments of this application; Figure 9 Timing diagrams of multi-application camera algorithm conflict response provided in some embodiments of this application; Figure 10 Timing diagrams for camera algorithm monitoring provided in some embodiments of this application; Figure 11 A timing diagram of a camera algorithm resource scheduling method provided in some embodiments of this application; Figure 12 This is a schematic diagram illustrating the processing flow of various types of cameras provided in some embodiments of this application. Detailed Implementation

[0028] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0029] In this embodiment, display device 200 generally refers to a device with screen display and data processing capabilities. For example, display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0030] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, users can operate the display device 200 via touch operation, mobile terminal 300, and control device 100. For example, control device 100 can be a remote control, stylus, gamepad, etc.

[0031] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0032] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0033] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0034] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0035] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0036] In some embodiments, detector 230 is used to collect signals from the external environment or signals interacting with the outside world. For example, detector 230 may include millimeter-wave radar, which can be used to detect whether a user is present within a preset range. Detector 230 may also include a voice acquisition unit to collect voice commands input by the user.

[0037] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0038] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0039] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0040] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0041] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0042] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0043] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0044] In some embodiments, the user input interface 280 can be used to receive instructions input by a user. The user input interface 280 may include at least one of a microphone, touchpad, sensor, remote control, etc. The display device 200 can then receive user-input instructions based on the user input interface 280 to perform interactive functions with the user.

[0045] In some embodiments, to enable user interaction, the display device 200 may run an operating system. An operating system is a computer program that manages and controls the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running applications. The operating system also allows users to interact with the display device 200.

[0046] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0047] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0048] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0049] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0050] like Figure 3As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0051] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0052] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0053] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0054] In some embodiments, the display device 200 connects to a camera via a device interface 240 to implement the camera's related functions. For example, after the display device 200 connects to a camera via a USB interface, the user can use camera-related applications to call the corresponding camera algorithm and combine it with the camera's hardware to implement camera-related functions.

[0055] It should be noted that, in the embodiments of this application, camera-related applications refer to applications or services that can call camera-related functions, and are not applications specifically developed for cameras. For example, applications or services that include sub-functions such as face detection and gesture recognition can all be referred to as camera-related applications.

[0056] In some embodiments, camera-related applications may be running in the foreground or background.

[0057] For example, camera-related applications running in the foreground can be marked as foreground applications (FOREGROUND_APP) or have foreground service (FOREGROUND_SERVICE) permissions by the operating system of the display device 200, and are in a high-priority scheduling queue, enjoying the highest priority in algorithm scheduling; camera-related applications running in the background have no direct user interaction and are in a low-priority scheduling state.

[0058] In some embodiments, the camera algorithm can be deployed locally, with the algorithm deployed in an independent application. Each application can load the algorithm, control its operation, and process data based on this independent application.

[0059] In some embodiments, the camera algorithm can be integrated into the system platform of the display device 200. The camera-related applications only call the platform interface to control the algorithm on and off and transmit data, and receive the results back.

[0060] In some embodiments, the camera algorithm can be deployed on the camera, and the display device 200 can control the algorithm's on / off state and acquire and detect data through a custom protocol.

[0061] In related technologies, the display device 200 employs an exclusive mechanism for accessing camera resources (camera hardware resources and the corresponding camera algorithms). The display device 200 assigns a unique device ID to each camera. When a camera-related application A accesses camera resources, application A exclusively occupies those resources. During this time, any requests from other applications to access these resources are intercepted. In other words, application A exclusively occupies the camera resources, regardless of whether the camera algorithm requested by other applications conflicts with the currently running camera algorithm. Applications must wait for application A to actively release the camera resources before they can access them again. Therefore, if a user initiates a camera access request to the display device 200 through a new application, the display device 200 will be unable to respond, resulting in a degraded user experience.

[0062] In some embodiments, when application B initiates a call request to display device 200, if application A already occupies camera resources on display device 200, display device 200 loads camera resources through application B and forces the process of application A to release camera resources, thereby realizing the exclusive access mechanism of the newly requesting application (application B) to the camera.

[0063] In some embodiments, camera-related applications load camera resources through the camera platform interface to launch the camera and preview the camera-related interface.

[0064] For example, such as Figure 3 As shown, the application layer of the display device 200 is configured with camera-related application 1 and camera-related application 2. For example... Figure 4 As shown, display device 200 launches application 1. Application 1 calls upon (loads) the camera, thus launching and previewing the camera through the camera platform interface. Based on the above operation events, the camera platform interface returns the launch preview result to application 1, causing application 1 to continue loading the camera algorithm corresponding to the camera. At this time, application 2 needs to use camera resources, so application 2 loads the camera through the camera platform interface, realizing the preemption of the camera. The camera platform interface then returns the preemption result to application 1, causing application 1 to passively disconnect from the camera.

[0065] Clearly, the exclusive mechanism of the aforementioned cameras cannot enable the concurrent execution of different camera algorithms. That is, when a single application is using a certain camera algorithm, other applications cannot use any camera algorithm in parallel, which reduces the utilization of camera resources and prevents multiple applications from working together to implement camera-related functions, thereby affecting the user experience of the camera functions in the display device 200.

[0066] To address the aforementioned issues, this application provides a display device 200 that determines whether there are conflicts between camera algorithms based on the functional characteristics of the algorithms in different camera algorithms; and executes multiple algorithms concurrently when there are no conflicts, thereby improving the utilization rate of the camera algorithms.

[0067] In some embodiments, the display device 200 includes a display 260, a device interface 240, and a controller 250. The display 260 can be used to display the application interface of a target application, which can be a camera-related application as described in the above embodiments, supporting the use of camera and camera algorithm resources. The device interface 240 is configured to connect to the camera and can be a built-in interface or an external interface, such as a USB interface.

[0068] Accordingly, in some embodiments, when the device interface 240 is a built-in interface, the camera can be described as a built-in camera of the display device 200; when the device interface 240 is an external interface, the camera can be described as an external camera of the display device 200.

[0069] In some embodiments, the camera type includes general-purpose cameras and AI cameras. General-purpose cameras are plug-and-play in the display device 200, transmitting real-time images or pictures through a device interface 240 (such as USB), and do not contain AI algorithms; AI cameras support embedded algorithm modules, enabling algorithm loading or shutdown control through a dedicated interface (without physically turning on the camera), and the camera algorithm is deployed at the camera end (remote end).

[0070] like Figure 5 As shown, in some embodiments, the controller 250 of the display device 200 is configured to perform the following steps: S501. In response to the call request initiated by the target application, determine the first camera algorithm to be called and the currently running second camera algorithm.

[0071] In some embodiments, the target application is configured at the application layer (e.g., Figure 3 As shown, the display device 200 can initiate a call request through the target application based on the application layer.

[0072] In some embodiments, the invocation request can also be described as a request to use the camera or a request to invoke camera resources. It is a request triggered by the target application when using camera-related functions. The invocation request instructs the target application to run a first camera algorithm, thereby achieving the corresponding function. That is, the first camera algorithm is the camera algorithm to be used but not currently running.

[0073] For example, when a user launches a fitness yoga app (target application) on display device 200, selects an upper limb stretching course, and clicks "Start Training," the fitness yoga app sends a request to display device 200 to call the limb detection algorithm and gesture detection algorithm (first camera algorithm), thereby enabling limb movement calibration and gesture control to pause or resume the course.

[0074] In some embodiments, the number of second camera algorithms can be single or multiple. When multiple camera algorithms are currently running on the display device 200, the number of second camera algorithms is multiple. That is, the second camera algorithm may include one or more currently running camera algorithms.

[0075] S502, query the first algorithm function features of the first camera algorithm, and detect the second algorithm function features of the second camera algorithm.

[0076] Since there can be many types of camera algorithms, and the algorithmic functional features involved in different types of camera algorithms are also different, each camera algorithm can contain one or more algorithmic functional features.

[0077] For example, camera algorithms may include distance detection algorithms, limb detection algorithms, face detection algorithms, gesture detection algorithms, etc. The algorithm functional features involved include target distance measurement, near-distance threshold alarm, long-distance recognition, distance dynamic tracking, multi-target distance differentiation, distance interval classification, human presence detection, limb joint localization, limb action recognition, human posture determination, multi-person limb differentiation, limb activity range detection, face presence detection, face number statistics, face location localization, face orientation recognition, child face recognition, adult face recognition, face activity monitoring, OK gesture, mute gesture, swipe up gesture, swipe down gesture, long press gesture, etc.

[0078] It should be noted that the algorithmic features involved in the above-mentioned camera algorithm are only illustrative examples and are not intended to limit it. The camera algorithm may include more or fewer algorithmic features, and this application does not limit this.

[0079] In some embodiments, each algorithm function feature corresponds to a unique feature identifier. For example, the feature identifier corresponding to the OK gesture is {“OK_gesture”}. If Algorithm 1 uses the OK gesture algorithm function feature, then the set of function features corresponding to Algorithm 1 includes {“OK_gesture”}.

[0080] In some embodiments, when querying the first algorithmic functional features of the first camera algorithm, the display device 200 can read the feature identifier corresponding to each algorithmic functional feature in the first camera algorithm to form a first set. The first algorithmic functional features of the first camera algorithm can be determined through the first set.

[0081] Similarly to the functional features corresponding to the first camera algorithm described above, in some embodiments, when detecting the second algorithm functional features of the second camera algorithm, the display device 200 can detect the feature identifier corresponding to each algorithm functional feature in the second camera algorithm, forming a second set. The second algorithm functional features of the second camera algorithm can be determined through this second set.

[0082] In some embodiments, when there are multiple second camera algorithms, the display device 200 needs to detect the feature identifier of the algorithm function feature corresponding to each second camera algorithm to obtain a second set corresponding to each second camera algorithm.

[0083] like Figure 6 As shown, in some embodiments, the application layer of the display device 200 includes a UI layer, which can be used to implement algorithm management, algorithm mutual exclusion, and local detection algorithms. The framework layer of the display device 200 includes a StreamPipe module, a CameraControl module, and a platform camera standard interface. StreamPipe provides an algorithm query interface to the UI and enables data communication with remote AI-enabled cameras via CameraControl, including command control and data transmission. CameraControl is used to customize data communication methods to achieve data communication between the display device and the camera, including algorithm capability querying and control. The platform camera standard interface describes the basic functions (such as startup and preview) provided by the operating system platform for controlling and calling the camera. Type 1 corresponds to the remote detection algorithm scenario for general cameras, and Type 2 corresponds to the scenario without a detection algorithm for AI cameras.

[0084] In some embodiments, the display device 200 uses the StreamPipe module to query a first set corresponding to the first algorithm functional features and a second set corresponding to the second algorithm functional features. The target application initiates a request to load the StreamPipe module. Figure 7 As shown, the StreamPipe module is used to manage all camera algorithms and their corresponding capability interfaces, implement the camera algorithm registration process, and register and mark the camera's algorithm capabilities to the management queue.

[0085] In some embodiments, when the StreamPipe module of the display device 200 implements the camera algorithm registration process, it queries the camera algorithm capability topology through the UVC Extension Unit Interface (XU interface) of the CameraControl service. The algorithm capability topology is algorithm byte data containing data information such as algorithm instance type and number of algorithm instances.

[0086] For example, the communication protocol of the CameraControl service is as follows: Field: UVC_EXTENSION_SELECTOR_ID; Type: Array; Description of the data in the array is as follows: Get Status: ACTION, 01 indicates getting algorithm status, DATA: D0-D3 (4 bytes): Switch status bitmask (taking the first byte as an example), o0x00: All functions are off, o0x01: Face detection is on, o0x02: Gesture detection is on, o0x03: Face detection and gesture detection are on simultaneously (0x01 | 0x02), o0x04: Limb detection is on, o0x08: Distance detection is on, o0x0F: All 4 functions are on simultaneously (0x01 | 0x02 | 0x04 | 0x08).

[0087] Setting status: ACTION, 02 indicates setting algorithm status, DATA: 64 bytes of data, D0-D3 (4 bytes): switch status bitmask (taking the first byte as an example), o0x00: all functions off, o0x01: face detection on, o0x02: gesture detection on, o0x03: face detection and gesture detection on simultaneously (0x01 | 0x02), o0x04: limb detection on, o0x08: distance detection on, o0x0F: all 4 functions on simultaneously (0x01 | 0x02 | 0x04 | 0x08).

[0088] Setting status: ACTION, 03 indicates reading data, DATA: 64 bytes of data, D0 (1 byte): 0 packet index (high 4 bits), 0 packet quantity (low 4 bits), D1-D62 (26 bytes): algorithm encrypted data, D63 (1 byte): CRC checksum.

[0089] Setting status: ACTION, 04 indicates obtaining the algorithm list, DATA: 64 bytes of data, D0-D3 (4 bytes): algorithm status bitmask (taking the first byte as an example), o0x01: face detection, o0x02: gesture detection, o0x04: limb detection, o0x08: distance detection.

[0090] When querying the algorithm capabilities corresponding to the camera, the display device 200 sets action=04 in the XU command to obtain the list of algorithms deployed on the camera (including algorithm instances) and synchronously calls the local algorithm SDK to obtain the local camera algorithm list.

[0091] For example, the list of camera algorithms is described as Algorithm_info {Int sourceType: identifies the source of the algorithm, Int Type: identifies the type of the algorithm, Feature_list: describes the supported types. Wherein, sourceType = 1 indicates that the algorithm is deployed on the main device, and sourceType = 4 indicates that the algorithm is deployed on an external camera; Type represents the list of algorithm types: 0x01: face detection, 0x02: gesture detection, 0x04: limb detection, 0x08: distance detection; Feature_list defines the supported scenarios of the algorithm; 0x02 (gesture detection): specifies the supported gesture types (e.g., OK gesture, Silence gesture); 0x04 (limb detection): distinguishes the detection type (limb data or posture detection).

[0092] Accordingly, in some embodiments, when the display device 200 detects the second algorithm functional features of the second camera algorithm, it establishes a communication link with the camera based on the device interface 240, and obtains the status data of the second camera algorithm (such as the 64 bytes of data mentioned above) according to the communication link. Then, it parses the feature identifier corresponding to the algorithm functional features of the second camera algorithm from the status data, and determines the second algorithm functional features based on the feature identifier. The communication link can be implemented based on CameraControl.

[0093] S503. In response to the fact that the first algorithm function feature and the second algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently.

[0094] After determining the first algorithm function characteristics and the second algorithm function characteristics, the display device 200 compares the first algorithm function characteristics and the second algorithm function characteristics. If there are no identical algorithm function characteristics, it means that the first camera algorithm to be used in the current call request does not conflict with other running second camera algorithms. In this case, the first camera algorithm and the second camera algorithm are executed concurrently, thereby realizing the parallel processing of multiple algorithms.

[0095] In some embodiments, when comparing the functional features of the first algorithm and the functional features of the second algorithm, the display device 200 calculates the intersection of the first set and the second set. If the intersection is an empty set, it indicates that the first set and the second set do not have the same algorithm functional features. At this time, the first camera algorithm and the second camera algorithm can run concurrently. If the intersection of the feature identifier set (A_set) of the first camera algorithm requested by the call request and the feature identifier sets (B1_set, B2_set, … Bn_set) of all current second camera algorithms is an empty set, it is determined that there is no conflict, and the first camera algorithm instance and the second camera algorithm instance are allowed to start concurrently.

[0096] For example, the first set of functional features corresponding to the first camera algorithm includes the feature identifier {“OK_gesture”}, while the second set of functional features corresponding to the algorithms of all currently active application instances (the second camera algorithm) are {“mute_gesture”}, {“volume up_gesture”}, etc., none of which include {“OK_gesture”}. Therefore, it is determined that the intersection of the first set and all second sets is empty. At this time, the first camera algorithm can run immediately and execute concurrently with the existing second camera algorithm instances.

[0097] It should be noted that the running of the first camera algorithm described in the embodiments of this application can also be described as starting the first camera algorithm, an instance of starting the first camera algorithm, executing the first camera algorithm, etc.

[0098] S504. In response to the fact that the first algorithm function feature and the second algorithm function feature have the same algorithm function feature, run the first camera algorithm or continue running the second camera algorithm.

[0099] If the display device 200 compares the functional features of the first algorithm and the functional features of the second algorithm, and there are identical algorithm functional features, it means that the first camera algorithm to be used in the current call request conflicts with other running second camera algorithms. In this case, the first camera algorithm and the second camera algorithm cannot be executed concurrently. Only one of the first camera algorithm and the second camera algorithm can be run to ensure that the camera function can be used normally and to avoid running errors.

[0100] It should be noted that the embodiments of this disclosure have at least one identical algorithmic function feature, and the display device 200 executes step S504.

[0101] In some embodiments, the display device 200 calculates the intersection of the first set and the second set. If the intersection is not empty, it is determined that the first camera algorithm and the second camera algorithm conflict, and only one can be processed. For example, a conflict is determined to exist when the intersection (A_set ∩ B_set) of the feature identifier set (A_set) of the requested first camera algorithm and the feature identifier set (B_set) of any current second camera algorithm is not empty.

[0102] For example, if the first set of functional features corresponding to the first camera algorithm contains the feature identifier {"OK_gesture"}, and the algorithm of another active application instance (the second camera algorithm) also contains {"OK_gesture"}, then the intersection is {"OK_gesture"} (not empty), triggering a conflict determination.

[0103] In this embodiment, the display device 200 can respond to camera algorithm call requests from target applications, query the first set corresponding to the first camera algorithm through the StreamPipe module, and simultaneously detect the second set corresponding to the current second camera algorithm. It obtains the camera-side and local algorithm lists and statuses via the XU interface of the CameraControl service, and determines conflicts by comparing the intersection of the first and second sets. If there is no intersection, the first and second camera algorithms run concurrently; if there is an intersection, one is selected for execution according to rules. This overcomes the limitations of the camera exclusivity mechanism, enabling concurrent execution of non-conflicting algorithms, improving camera algorithm resource utilization, adapting to cross-device scenarios for both general and AI cameras, and ultimately enhancing the user experience of camera functionality.

[0104] For example, in a scenario with non-exclusive camera algorithms (S503): When a user uses a smart TV equipped with an AI camera, a child eye protection monitoring application is already running in the background, operating a distance detection algorithm (second camera algorithm). The corresponding feature set of the second algorithm is {"target distance measurement", "near distance threshold alarm"}. At the same time, the user opens a fitness yoga app (the target application) in the foreground, initiating a request to call the limb detection algorithm (first camera algorithm). The corresponding feature set of the first algorithm is {"limb joint localization", "limb movement recognition"}. The TV queries and compares the two feature sets through the StreamPipe module, finding no identical algorithm features (empty intersection), determining no conflict, and then concurrently running the distance detection algorithm and the limb detection algorithm. That is, the background continuously monitors the distance between the child and the TV and issues an alarm when the distance exceeds the threshold, while the foreground identifies the user's limb movements in real time and calibrates the yoga posture, thereby achieving high-efficiency reuse of multi-application algorithm resources.

[0105] For mutually exclusive camera algorithm scenarios (S504): A user is using a smart TV equipped with an AI camera, currently running a face detection algorithm (second camera algorithm) via a video call app. The corresponding feature set of the second algorithm includes {"face location" and "adult face recognition"}. During this time, the user launches a home security monitoring application (target application), initiating a call request to the face detection algorithm (first camera algorithm). The corresponding feature set of the first algorithm also includes {"face location"}. The TV detects that the intersection of the two feature sets is {"face location"} (not empty), determining a conflict, and can only execute one of the face detection algorithms.

[0106] Regarding step S504, in some embodiments, when the first algorithm functional feature and the second algorithm functional feature have the same algorithm functional feature, the display device 200 blocks the call request and maintains the operation of the second camera algorithm when the target application is running in the background. That is, when the target application is running in the background, if the camera algorithm (first camera algorithm) to be used by the call request initiated by the target application conflicts with the currently running camera algorithm (second camera algorithm), the call request is blocked according to the principle of higher priority of the foreground application, and the first camera algorithm is not run in response to the call request, while the operation of the second camera algorithm corresponding to the foreground application is maintained.

[0107] For example, if the display device 200 finds that the first set contains the feature identifier {“OK_gesture”} and the second set also contains the feature identifier {“OK_gesture”}, it will block the call request, maintain the allocation of resources for the remaining camera algorithms (second camera algorithm), and wait for the release of the resources corresponding to the second camera algorithm.

[0108] like Figure 8 As shown, in some embodiments, after the display device 200 blocks the call request, it also listens to the second camera algorithm, and when the second camera algorithm is in an idle state, it re-compares the corresponding functional features of the first camera algorithm and the second camera algorithm, as follows: S801, Register the first listening of the second camera algorithm.

[0109] In order to monitor the operation of the second camera algorithm, the display device 200 also registers a listener for the second camera algorithm to obtain the usage status of the second camera algorithm in real time.

[0110] In some embodiments, the target application of the display device 200 application layer registers a first listener for the usage status of the second camera algorithm with the StreamPipe module.

[0111] S802, the first running result of the algorithm for querying the second camera based on the first listening query.

[0112] After the display device 200 registers the first listener for the second camera algorithm, it queries the first running result of the second camera algorithm based on the first listener. The first running result can be used to indicate whether the second camera algorithm is idle. If the first running result indicates that it is idle, it means that the second camera algorithm has completed running; if the first running result indicates that it is not idle, it means that the second camera algorithm is still running.

[0113] S803. If the first running result indicates that the system is idle, re-detect the second algorithm function features of the second camera algorithm.

[0114] When the display device 200 finds that the first running result indicates that it is idle, the conflict between the previously determined first camera algorithm and this second camera algorithm has been resolved. At this time, the display device 200 should re-detect the algorithm function characteristics of the second camera algorithm, that is, re-execute the above conflict judgment logic (same as S503-S504).

[0115] S804. In response to the fact that the second algorithm function feature and the first algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently.

[0116] When the second set being re-tested does not have the same algorithmic functional characteristics as the first set, the display device 200 can unblock the call request and concurrently run the first camera algorithm and the second camera algorithm.

[0117] In some embodiments, after the second camera algorithm of the display device 200 has completed its operation, it can send back a completion flag to mark the result of the second camera algorithm as an idle state.

[0118] In some embodiments, the idle or non-idle state of the execution result can be determined by the value of a specified field. The display device 200 can read the value of the specified field to determine the state represented by the execution result.

[0119] In some embodiments, the first algorithm and the second algorithm share the same algorithmic functional features in step S504. When the target application is running in the foreground, the display device 200 stops running the second camera algorithm and responds to the call request to run the first camera algorithm. In other words, when the target application is running in the foreground and initiates a call request for the first camera algorithm, the display device 200 stops running the second camera algorithm and responds to the call request for the first camera algorithm to run the first camera algorithm, ensuring timely response from high-priority foreground applications.

[0120] For example, such as Figure 9 As shown, when the camera service process is currently holding and executing a certain camera algorithm instance (second camera algorithm 1), it registers a conflict listener for algorithm 1. If the foreground application process (target application) requests to start a new first camera algorithm instance (requesting feature identifier set Features) through the system interface, it sends a call request. StreamPipe, as the algorithm instance status monitoring interface, captures and sends back the call request and its associated Requested_Features to the camera service process in real time. As the current holder of algorithm resources, the camera service process, upon receiving the call request, immediately calculates the intersection of the first set of functional features corresponding to the first camera algorithm and the second set of functional features corresponding to algorithm 1 (identifier set Current_Features), where the intersection Conflict_Set = Requested_Features ∩ Current_Features. If the intersection is not empty, the camera service process exits algorithm detection (stops running algorithm 1) and enters the listening process for whether the algorithm is idle, causing the target application process to run the first camera algorithm; if the intersection is empty, the camera service process maintains the current resource allocation state (maintains the second camera algorithm).

[0121] It should be noted that stopping the execution of the camera algorithm (such as the first camera algorithm or the second camera algorithm) as described in the embodiments of this application can also mean turning off the execution of this camera algorithm, exiting the execution of this camera algorithm, releasing the system resources that call this camera algorithm, etc.

[0122] To expedite the response time, in some embodiments, after receiving a call request initiated by the target application, the display device 200 also registers a second listener for the first camera algorithm and queries the second running result of the first camera algorithm based on the second listener. That is, it determines whether the instance of the first camera algorithm is idle by listening to the first camera algorithm. If the second running result indicates that the first camera algorithm is idle (the instance of the first camera algorithm is idle), the first camera algorithm is run in response to the call request; if the second running result indicates that the first camera algorithm to be called and the currently running second camera algorithm are determined (S501).

[0123] In other words, the display device 200 initiates a call request through the target application, first checking whether the instance of the first camera algorithm in the current display device 200 is idle. If it is idle, the first camera algorithm instance can be used directly to execute the first camera algorithm without conflict checking. For example, when the target application starts the first camera algorithm, the display device 200 performs a pre-verification check on the remotely deployed algorithm (deployment location: camera end). When the first camera algorithm instance is idle (not occupied by any application process), the camera service process can initiate a request to use the camera algorithm. At the same time, a data usage listener for the camera algorithm is added, waiting for the return of detection data. If the algorithm is in use, the conflict checking in steps S501-S504 is performed.

[0124] For example, a user is using a smart TV equipped with an AI camera and is currently running a face detection algorithm (second camera algorithm) through a video call app. The corresponding feature set of the second algorithm includes {"face location" and "adult face recognition"}. During this time, the user launches a home security monitoring application (target application) and initiates a call request to the face detection algorithm (first camera algorithm). The corresponding feature set of the first algorithm also includes {"face location"}. The TV detects that the intersection of the two feature sets is {"face location"} (not empty), and determines that there is a conflict. Since the video call app is a foreground application, it maintains the operation of its face detection algorithm according to the priority rules, blocks the call request of the security monitoring application, and registers a listener. After the video call ends (the second algorithm releases resources), the listener mechanism triggers a wake-up, and the face detection algorithm of the security monitoring application can then start running.

[0125] In some embodiments, the applications associated with the first camera algorithm initiated by the target application and the second camera algorithm are both running in the foreground. The display device 200 can select the camera algorithm corresponding to the application with the higher priority according to the preset application priority.

[0126] like Figure 10 As shown, in some embodiments, the camera service loads the algorithm deployed on the camera, registers the algorithm's usage status with StreamPipe, and queries whether it is in use. StreamPipe sends back the algorithm's running results. After parsing, if the algorithm is being used, the camera service waits for its use to complete based on the registered listener. After receiving the feedback flag indicating that the algorithm has been used, the camera detection algorithm is started, and the detection data is updated in real time through the algorithm data feedback interface, so as to achieve accurate invocation when the algorithm is idle.

[0127] In some embodiments, the display device 200 is further configured with a setting switch, which is used to turn the algorithm detection logic on or off. The algorithm detection logic includes the detection, judgment and analysis of the camera algorithm in the above embodiments, which will not be described in detail here.

[0128] For example, such as Figure 11 As shown, the camera service first starts and listens for the setting switch, simultaneously turning on the camera algorithm detection switch. The service parses the switch status and marks it as hand_switch. When hand_switch=1, the service queries the camera algorithm capabilities, sends a request to the remote camera service through CameraControl, obtains and analyzes the algorithm list, and determines whether the remote algorithm is supported. If supported, conflict detection is performed (steps S501-S504); if not supported, a conflict detection method is used. Figure 4 The hardware resource usage is shown; when hand_switch=0, camera detection is directly turned off.

[0129] To adapt to different deployment scenarios of general-purpose cameras and AI cameras, such as Figure 12 As shown, in some embodiments, after receiving a call request initiated by the target application, the display device 200 further performs the following steps: S1201. Based on the target camera indicated in the call request, query the algorithm capabilities of the target camera.

[0130] The display device 200 queries the algorithm capabilities of the target camera to determine the type of the target camera, such as an AI camera or a general-purpose camera.

[0131] In some embodiments, the display device 200 generates a query instruction (such as an XU command) for algorithm capabilities via an extension unit interface (such as an XU interface). In response to the query instruction returning algorithm byte data, an algorithm state mask is parsed from the algorithm byte data to determine the algorithm capabilities. The algorithm state mask includes instance information of the algorithm instance.

[0132] For example, instance information may include `Algorithm_info {Int sourceType: identifies the algorithm source, Int Type: identifies the algorithm type, Feature_list: describes the supported types. Here, `sourceType = 1` indicates the algorithm is deployed on the main machine, and `sourceType = 4` indicates the algorithm is deployed on an external camera; `Type` represents the list of algorithm types: 0x01: face detection, 0x02: gesture detection, 0x04: limb detection, 0x08: distance detection; `Feature_list` defines the supported scenarios for the algorithm; 0x02 (gesture detection): specifies the supported gesture types (e.g., OK gesture, Silence gesture); 0x04 (limb detection): distinguishes the detection type (limb data or posture detection), etc.}`

[0133] S1202, In response to an instance where the target camera is equipped with a camera algorithm, the step of determining the first camera algorithm to be invoked and the currently running second camera algorithm is performed.

[0134] After the display device 200 queries the algorithm capability, if the algorithm capability characterizes an instance of camera algorithm deployed on the camera end, indicating that the camera is an AI camera that supports camera algorithm, then the steps of determining the first camera algorithm to be called and the currently running second camera algorithm are executed (S501-S504).

[0135] S1203. In response to an instance where the algorithm capability characterization target camera has not deployed a camera algorithm, call the camera's hardware resources and run the first camera algorithm based on the hardware resources.

[0136] If the algorithm capability characterization instance does not have a camera algorithm deployed on the camera, it indicates that the camera is a general-purpose camera and does not support camera algorithms. In this case, an exclusive mechanism is required. The display device 200 loads the hardware resources of the target camera and directly runs the first camera algorithm, without needing to perform the aforementioned conflict judgment, thereby improving the response speed of the display device 200. For example, when the algorithm capability characterization instance does not have a camera algorithm deployed on the target camera, the display device 200 can, according to... Figure 4 The first camera algorithm is executed in the manner shown.

[0137] For general-purpose cameras, in some embodiments, when the hardware resources of the target camera are already occupied, the display device 200 can generate a preemption notification for the target application and, in response to the preemption notification, release the hardware resources so that the target application can occupy the hardware resources to run the first camera algorithm without conflict.

[0138] For example, when display device 200 connects to a general-purpose USB camera (without local algorithm deployment, only providing data acquisition functionality), the default priority list is "video call application > fitness application > background security monitoring application". When the security monitoring application is using camera hardware resources to run a local human presence detection algorithm, if the user launches the video call application and initiates a camera access request, display device 200 will respond to the request, disconnecting the background security monitoring application from the camera and loading the camera through the video call application.

[0139] The display device detected that no camera algorithm instance was deployed on the camera and that there was hardware resource contention. After reading the priority list, it determined that the video call application had a higher priority. It then terminated the algorithm operation of the background security monitoring application, released the camera hardware resources, and allocated resources to the video call application to start the face detection algorithm, thereby ensuring the normal use of the core interactive scenarios in the foreground.

[0140] In some embodiments, if the algorithm capability characterizes an instance where the target camera has not deployed a camera algorithm, the display device 200 can also read a preset priority list, which includes the priority of each application. When there is preemption of camera hardware resources, the display device 200 can determine the camera algorithm with the higher priority based on the priority list.

[0141] For example, display device 200 connects to a general-purpose USB camera (without local algorithm deployment, only providing data acquisition functionality), with a default priority list of "video call application > fitness application > background security monitoring application". When a user launches the video call application, it loads the camera's hardware resources and runs a face detection algorithm based on those resources. At this time, if the background security monitoring application initiates a request to execute a local human presence detection algorithm, display device 200 maintains the video call application's exclusive access to the camera resources and does not respond to the request from the background security monitoring application.

[0142] It should be noted that the above method is only an illustrative example and is not intended to limit the specific implementation of this application.

[0143] Based on the aforementioned display device 200, this application also provides a method for scheduling camera algorithm resources, such as... Figure 5 As shown, the scheduling method includes the following steps: S501. In response to the call request initiated by the target application, determine the first camera algorithm to be called and the currently running second camera algorithm; S502. Query the first algorithm function features of the first camera algorithm, and detect the second algorithm function features of the second camera algorithm. S503. In response to the fact that the first algorithm function feature and the second algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently. S504. In response to the fact that the first algorithm function feature and the second algorithm function feature have the same algorithm function feature, run the first camera algorithm or continue running the second camera algorithm.

[0144] Based on the above embodiments, the scheduling method described in this application can, in response to a call request initiated by a target application, determine the first camera algorithm to be called and the currently running second camera algorithm. It then queries the first algorithm functional characteristics of the first camera algorithm and detects the second algorithm functional characteristics of the second camera algorithm. If the first and second algorithm functional characteristics do not share the same algorithm functional characteristics, the first and second camera algorithms are executed concurrently; otherwise, the first camera algorithm is run or the second camera algorithm is kept running to ensure that no conflict occurs between the algorithms. This application can overcome the limitations of the camera exclusivity mechanism, determine algorithm conflicts through algorithm functional characteristics, achieve concurrent execution of non-conflicting algorithms and orderly scheduling of conflicting algorithms, thereby improving the utilization rate of camera algorithm resources and multi-application compatibility, and ultimately optimizing the user interaction experience.

[0145] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: A display for showing the application interface of a target application, which supports accessing a camera and camera algorithm resources; The device interface is configured to connect to a camera that can be shared and accessed by one or more applications. The controller is configured as follows: In response to the call request initiated by the target application, determine the first camera algorithm to be called and the currently running second camera algorithm; Query the first algorithm function features of the first camera algorithm, and detect the second algorithm function features of the second camera algorithm; In response to the fact that the first algorithm function feature and the second algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently. In response to the existence of the same algorithm function features between the first algorithm function features and the second algorithm function features, the first camera algorithm is run, or the second camera algorithm is kept running.

2. The display device according to claim 1, characterized in that, The controller executes the algorithm to maintain the operation of the second camera, specifically configured as follows: If the target application is running in the background, the call request is blocked, and the second camera algorithm continues to run.

3. The display device according to claim 2, characterized in that, After the controller blocks the call request, it is further configured to: Register the first listener for the second camera algorithm; Based on the first listening query, the first running result of the second camera algorithm is obtained; If the first running result indicates that the system is idle, the second algorithm function feature of the second camera algorithm is re-detected. In response to the fact that the second algorithm function feature detected by the re-detection does not have the same algorithm function feature as the first algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently.

4. The display device according to claim 1, characterized in that, The controller executes the algorithm of the first camera, specifically configured as follows: If the target application is running in the foreground, stop running the second camera algorithm; The first camera algorithm is executed in response to the call request.

5. The display device according to claim 1, characterized in that, Before the controller executes the process of determining the first camera algorithm to be invoked and the currently running second camera algorithm, it is also configured to: Register the second listener for the first camera algorithm; Based on the second monitoring query, the second running result of the first camera algorithm is obtained; If the second running result indicates that the system is idle, the first camera algorithm is executed in response to the call request. If the second running result indicates that the computer is not idle, the step of determining the first camera algorithm to be invoked and the currently running second camera algorithm is executed.

6. The display device according to claim 1 or 5, characterized in that, Before the controller executes the process of determining the first camera algorithm to be invoked and the currently running second camera algorithm, it is also configured to: Based on the target camera indicated by the call request, query the algorithm capabilities of the target camera; In response to the algorithm capability characterizing an instance where the target camera is deployed with a camera algorithm, the step of determining the first camera algorithm to be invoked and the currently running second camera algorithm is performed; In response to an instance where the algorithm capability characterizes that the target camera has not deployed a camera algorithm, the hardware resources of the target camera are invoked, and the first camera algorithm is run based on the hardware resources.

7. The display device according to claim 6, characterized in that, The controller is specifically configured to perform an algorithm to query the target camera. Call the extension unit interface to generate a query command for the algorithm's capabilities; The algorithm returns byte data in response to the query command; The algorithm state mask is parsed from the algorithm byte data to determine the algorithm's capabilities; the algorithm state mask includes instance information of the algorithm instance.

8. The display device according to claim 6, characterized in that, The algorithm capability characterizes instances where the target camera has not deployed a camera algorithm. Before the controller executes the first camera algorithm based on the hardware resources, it is further configured to: If the hardware resources of the target camera are already occupied, a preemption notification is generated for the target application. In response to the preemption notification, the hardware resources are released.

9. The display device according to claim 1, characterized in that, The controller executes the second algorithm function feature of the second camera algorithm, specifically configured as follows: A communication link with the camera is established based on the device interface; The status data of the second camera algorithm is obtained according to the communication link; Parse the feature identifiers corresponding to the algorithm functions of the second camera algorithm from the state data; The functional features of the second algorithm are determined based on the feature identifier.

10. A method for scheduling camera algorithm resources, characterized in that, include: In response to a call request initiated by the target application, determine the first camera algorithm to be invoked and the currently running second camera algorithm; Query the first algorithm function features of the first camera algorithm, and detect the second algorithm function features of the second camera algorithm; In response to the fact that the first algorithm function feature and the second algorithm function feature do not have the same algorithm function feature, the first camera algorithm and the second camera algorithm are run concurrently. In response to the existence of the same algorithm function features between the first algorithm function features and the second algorithm function features, the first camera algorithm is run, or the second camera algorithm is kept running.

Images