Display device and data synchronization method

Abstract

The invention provides a display device and a data synchronization method, which are applied to the technical field of display devices, and the data synchronization method can respond to a first data request instruction of a first application, and send a first image obtained according to an original image collected by an image collector to the first application. And under the condition that the first application occupies the calling permission of the image collector, a first data request instruction of the second application is received, and the second image is sent to the second application. Therefore, the second application can execute corresponding processing according to the second image. The second image is obtained by copying or mapping the first image, so that the content of the second image is the same as that of the first image. The problem that when the calling permission of the image collector is occupied by the first application, the function achieved by the second application based on the image collected by the image collector is invalid is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a data synchronization method. Background Art

[0002] Display devices are intelligent devices that can present user interfaces and support user interaction. Taking smart TV as an example, smart TV is based on Internet application technology, has an open operating system and chip, has an open application platform, can realize two-way human-computer interaction, and integrates multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] In order to provide users with a better experience of using smart TVs, current smart TVs all support camera functions. The camera can be used to take photos, make video calls, and provide home care. The AI ​​recognition algorithm can also be used to recognize images captured by the camera, thereby realizing various operations such as face unlocking, face authentication, and gesture control.

[0004] However, currently, the calling permission of the camera on smart TV can only be occupied by one application at a time. If the calling permission of the camera is already occupied when the current application requests to obtain the calling permission of the camera, the function implemented by the current application based on the image collected by the camera will be invalid. For example, during a video chat, the gesture recognition function fails and the user cannot adjust the volume by gestures, but can only adjust the volume by remote control. This reduces the interactive experience of the intelligent system. Summary of the invention

[0005] The present application provides a display device and a data synchronization method to solve the problem that when an image collector is occupied, functions implemented by other applications based on images collected by the image collector become invalid.

[0006] In a first aspect, some embodiments of the present application provide a display device, comprising: a display; an image collector, configured to collect a first image; and a controller, connected to the display and the image collector, and configured to perform the following program steps:

[0007] In response to a first data request instruction of the first application, a first image is sent to the first application; the first image is obtained according to an original image acquired by the image acquirer;

[0008] In the case where the first application occupies the calling authority of the image collector, if the first data request instruction of the second application is received, the second image is sent to the second application; the second image is obtained by copying or mapping the first image.

[0009] After the controller receives the first data request instruction of the first application, the first image collected by the image collector is sent to the first application, so that the first application can perform corresponding processing according to the first image. In the process of the controller sending the first image to the first application, the first application occupies the calling authority of the image collector. In the case where the first application occupies the calling authority of the image collector, if the controller receives the first data request instruction of the second application, the second image is sent to the second application, so that the second application can perform corresponding processing according to the second image. Since the second image is obtained by copying or mapping the first image, the content of the second image is the same as that of the first image. The problem of the failure of the function implemented by the second application based on the image collected by the image collector when the calling authority of the image collector is occupied by the first application is solved.

[0010] In some embodiments, executing the data request instruction in response to the first application to send the first image acquired by the image collector to the first application is specifically configured as follows:

[0011] Call the camera service to start the image collector, control the image collector to record the physical environment; use the first camera data provider process to obtain the original image recorded by the image collector; use the first camera data provider process to format the original image to obtain a first image; send the first image to the first application through the camera interface.

[0012] Since the data format of the original image is not convenient for subsequent processing, format processing of the original image can obtain a first image that is convenient for subsequent processing.

[0013] In some embodiments, when the first application occupies the calling permission of the image collector, if the controller receives a data request instruction from the second application, the controller sends the second image to the second application, which is specifically configured as follows:

[0014] Call the camera service to start the virtual image collector; copy or map the first image to obtain a second image; use the second camera data provider process to obtain the second image; and send the second image to the second application through the camera interface.

[0015] Since the virtual image collector needs a data source, by using the second image as the data source of the virtual image collector, the second application can realize the corresponding function based on the second image. This solves the problem that when the calling permission of the image collector is occupied by the first application, the function realized by the second application based on the image collected by the image collector becomes invalid.

[0016] In some embodiments, the controller is further configured to:

[0017] In response to the start of the first service process, it is detected whether the calling permission of the image collector is occupied; if the calling permission of the image collector is not occupied, the calling permission of the image collector is allocated to the first service process so that the first service process obtains the first image; if the calling permission of the image collector is occupied, the camera service is called to register a virtual image collector, and the calling permission of the virtual image collector is allocated to the first service process so that the first service process obtains the second image.

[0018] Since the second image is obtained by copying or mapping the first image, after the first service process is started, if the calling permission of the image collector is not occupied, it means that the image collector is not turned on, and there is no first image to copy and map. The calling permission of the image collector is assigned to the first service process. If the calling permission of the image collector is occupied, it means that the image collector has been turned on, and the first image can be copied or mapped to obtain the second image. The calling permission of the virtual image collector is assigned to the first service process. This allows the first service process to obtain images stably.

[0019] In some embodiments, the first service process includes at least one image algorithm model; after the first service process is started, the controller is further configured to: control the first service process to initialize the image algorithm model.

[0020] Initializing the image algorithm model can improve the processing speed, stability, and accuracy of the model.

[0021] In some embodiments, the controller is further configured to: receive a second data request instruction from the first application; control the first service process to call the target image algorithm model to process the first image according to the second data request instruction from the first application; the target image algorithm model is one of the image algorithm models included in the first service process; and send the processing result to the first application.

[0022] Integrating multiple image algorithm models in the first service process can provide multiple image processing capabilities for the first application, facilitate unified management of image algorithm models, and can feedback processing results based on the actual needs of the first application, thereby improving the overall processing efficiency of the system.

[0023] In some embodiments, the controller is further configured to: receive a second data request instruction from a second application; control the first service process to call a target image algorithm model to process the second image according to the second data request instruction from the second application; and send the processing result to the second application.

[0024] Integrating multiple image algorithm models in the first service process can provide multiple image processing capabilities for the second application, facilitate unified management of image algorithm models, and be able to feedback processing results based on the actual needs of the second application, thereby improving the overall processing efficiency of the system.

[0025] In some embodiments, the controller is further configured to: register the virtual image collector in a camera list of the hardware abstraction layer according to the data structure of the image collector.

[0026] The virtual image collector is registered using the same data structure as the image collector, so that the virtual image collector of the display device can more accurately simulate the function of the image collector.

[0027] In some embodiments, after the controller executes an operation in response to the first data request instruction of the first application, the controller is further configured to: if it is detected that the calling permission of the image collector is occupied by the first service process, release the calling permission of the image collector occupied by the first service process, and assign the calling permission of the image collector to the first application, so that the first application obtains the first image; call the camera service to start the virtual image collector, and assign the calling permission of the virtual image collector to the first service process, so that the first service process obtains the second image.

[0028] The display device 200 can preferentially ensure that the first application occupies the calling authority of the image collector, thereby avoiding conflicts between the first application and the first service process when occupying the calling authority of the image collector.

[0029] In a second aspect, an embodiment of the present application further provides a data synchronization method, the method comprising the following procedural steps:

[0030] In response to a data request instruction of a first application, a first image is sent to the first application; the first image is obtained based on an original image acquired by an image collector; when the first application occupies the calling authority of the image collector, if a data request instruction of a second application is received, the second image is sent to the second application; the second image is obtained by copying or mapping the first image.

[0031] It can be seen from the above technical solutions that the display device 200 and data synchronization method provided in some embodiments of the application can respond to the first data request instruction of the first application and send the first image obtained based on the original image collected by the image collector to the first application. And when the first application occupies the calling authority of the image collector, it can receive the first data request instruction of the second application and send the second image to the second application. So that the second application can perform corresponding processing according to the second image. Since the second image is obtained by copying or mapping the first image, the content of the second image is the same as that of the first image. The problem of the failure of the function implemented by the second application based on the image collected by the image collector when the calling authority of the image collector is occupied by the first application is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0034] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0035] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0036] Figure 4 A schematic diagram of a processing flow of a controller of a display device in response to a data request instruction of an application provided in some embodiments of the present application;

[0037] Figure 5 One of the user interface schematic diagrams of a display device provided in some embodiments of the present application;

[0038] Figure 6 A second schematic diagram of a user interface of a display device provided in some embodiments of the present application;

[0039] Figure 7 A schematic diagram of a system setting interface of a display device provided in some embodiments of the present application;

[0040] Figure 8 One of the example diagrams of controlling the start of a first service process through a voice command of a display device provided in some embodiments of the present application;

[0041] Fig. 9 One of the example diagrams of controlling the start of a first service process through a voice command of a display device provided in some embodiments of the present application;

[0042] Fig.10 One of the example diagrams of a display device controlling the start of a first service process through a gesture command provided in some embodiments of the present application;

[0043] Fig.11 One of the example diagrams of a display device controlling the start of a first service process through a gesture command provided in some embodiments of the present application;

[0044] Fig.12 A schematic diagram of a processing flow of a controller of a display device in response to a first data request instruction of a first application provided in some embodiments of the present application;

[0045] Fig.13 A schematic diagram of software configuration including a first service process of a display device provided in some embodiments of the present application;

[0046] Fig.14 A schematic diagram of a flow chart of a first service process of a display device provided in some embodiments of the present application;

[0047] Fig.15 One of the schematic diagrams of application scenarios of the display device provided in some embodiments of the present application;

[0048] Fig.16 A second schematic diagram of an application scenario of a display device provided in some embodiments of the present application;

[0049] Fig.17 A schematic diagram of a data synchronization method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0051] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0052] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0053] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0054] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0055] In the embodiments of the present application, the display device generally refers to a device with image display and data processing capabilities. For example, the display device includes but is not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0056] Figure 1 Schematic diagram of an operation scenario between a display device 200 and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0057] The mobile terminal 300 can be used as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0058] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0059] like Figure 1As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0060] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0061] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0062] In some embodiments, the display device 200 may include at least one of a tuner 210 , a communication device 220 , a device interface 240 , a controller 250 , a display 260 , an audio output device 270 , a user input interface 280 , and a memory 290 .

[0063] The device interface 240 is connected to an image collector, which is used to collect external environment or external interaction signals. The image collector includes a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures.

[0064] In some embodiments, the camera may include a lens assembly, in which a photosensitive element and a lens are provided. The lens may refract light through a plurality of lenses so that the light is irradiated on the photosensitive element. The photosensitive element may be made of a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) material. The light signal captured by the photosensitive element is converted into an electrical signal, and then the electrical signal is converted into an image and output. In some embodiments, the camera may also output images frame by frame at a preset sampling frequency to form an image stream (also referred to as a video).

[0065] In some embodiments, the camera can be independent of the display device 200 and connected to the display device 200 through a USB interface, an HDMI interface, etc. In addition, the camera can also be integrated inside the display device 200 and connected through a USB interface or a mobile industry serial interface.

[0066] The device interface 240 is also connected to a sound collector, such as a microphone, for receiving external sounds.

[0067] In some embodiments, a microphone is a transducer that converts sound signals into electrical signals. Its main function is to collect the sound emitted by the user or the sound emitted by the surrounding environment and convert it into an audio signal that can be processed by the electronic device.

[0068] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0069] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0070] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0071] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and an embedded neural network processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device 200 and responds to the user's operation through various software control programs stored in the memory 290. The controller 250 controls the overall operation of the display device 200.

[0072] In some embodiments, the central processing unit is used to execute operating system instructions and application instructions stored in the memory 290, as well as various interactive instructions received from external inputs, so as to ultimately display and play various audio and video contents.

[0073] In some embodiments, the video processor is used to perform at least one of the following video processing operations on the received external video signal according to a standard codec protocol, such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc., to obtain an image data stream that can be displayed or played on the display device 200.

[0074] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, and a display formatting module. The demultiplexing module is used to decompose the composite audio and video signal input externally into multiple independent audio and video signals. The video decoding module is used to process the independent audio and video signals obtained after demultiplexing (for example, decoding and scaling processing, etc.). The image synthesis module is used to perform superposition and mixing processing on the independent audio and video signals after decoding and scaling processing according to the GUI signal input by the user or generated by itself, so as to generate an image data stream for display; the frame rate conversion module is used to convert the playback frame rate of the image data stream; the display formatting processing module is used to change the display format of the image data stream, for example, the RGB display format.

[0075] In some embodiments, the audio processor is used to decompress and decode the received external audio signal according to a standard codec protocol, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain sound data that can be played in a speaker.

[0076] In some embodiments, the embedded neural network processor can accelerate the processing speed of voice recognition and image recognition, so that the user's voice commands or gesture commands can be recognized more quickly, and the TV can be operated (for example, switching channels, searching for programs, adjusting the volume, etc.). It can also better realize image recognition and processing functions, such as identifying and analyzing characters and scenes in video content, and providing users with relevant information or recommendations. It can also optimize the picture and improve the picture quality, such as improving low-resolution video content to a higher definition.

[0077] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0078] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface 280 receives the user input command through the graphical user interface (GUI).

[0079] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0080] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0081] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may control the display device 200 to provide a user interface, for example, the operating system may directly control the display device 200 to provide a user interface, or may provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0082] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device 200 .

[0083] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the hardware abstraction layer (HardwareAbstr action Layer) and the kernel layer.

[0084] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0085] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0086] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes a view system (View System), managers (Managers), a content provider (Content Provider), a camera service (Camera Service), etc.

[0087] Among them, the view system can design and implement the interface and interaction of the application. The view system includes lists, grids, text boxes, buttons, etc.

[0088] In some embodiments, the manager includes at least one of the following modules: an activity manager (ActivityManager) for interacting with all activities running in the system; a location manager (Location Manager) for providing system services or applications with access to system location services; a package manager (Package Manager) for retrieving various information related to application packages currently installed on the device; a notification manager (NotificationManager) for controlling the display and clearing of notification messages; and a window manager (Window Manager) for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0089] In some embodiments, the camera service is an important component responsible for managing and controlling camera functions, and is mainly used for device management, parameter configuration, data transmission, etc. The camera service provides a camera interface (e.g., Camera API or Camera2API) for the application layer. The application can communicate with the camera service through the camera interface to control the camera startup, shutdown, and setting of camera configuration parameters (e.g., exposure parameters, focus parameters, resolution, etc.). In addition, the camera service can also communicate with the camera hardware through the hardware abstraction layer. Control the camera configuration parameters, obtain images captured by the camera, etc.

[0090] The hardware abstraction layer is a layer between hardware and software. The operating system interacts with the hardware through the hardware abstraction layer. The hardware abstraction layer provides a set of standard interfaces, allowing the operating system to shield the differences in hardware and easily communicate with different hardware components. For example, camera hardware produced by different manufacturers may have different underlying control methods and data formats. Through the hardware abstraction layer, the operating system can use a unified interface to call the camera to take pictures or record videos.

[0091] The hardware abstraction layer includes a camera data provider process. In some embodiments, the camera data provider process is usually closely related to the camera service. The camera data provider is mainly used to convert the format of the original image collected by the camera. For example, the data format of the original image collected by a general camera is the Bayer format, and the camera data provider process converts the Bayer format of the original image into a format that the operating system can recognize, such as (BMP, JPEG, PNG format, etc.).

[0092] The kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as multitasking and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0093] In some embodiments, the camera driver is used to connect the camera hardware and the operating system software, enabling the operating system to identify the camera hardware device and implement basic operations such as turning the camera on, off, taking pictures, and recording videos. For example, when the application is opened, the application will send a startup instruction to the camera driver through the camera service, and the camera driver will convert the startup instruction into an electrical signal or operation command that the camera can understand, thereby controlling the camera to turn on. In addition, the camera driver can also accurately and completely send the original image captured by the camera to the operating system memory, so that the operating system software (for example, the camera data provider process) can obtain accurate and complete original images for processing.

[0094] like Figure 4 As shown, the processing flow of the controller 250 in response to the data request instruction of the application is introduced:

[0095] In response to the first data request instruction of the application, the controller 250 calls an open camera method in the camera service of the framework layer, and controls the camera service to call a camera data provider process of the hardware abstraction layer.

[0096] In some embodiments, the hardware abstraction layer may be called through HIDL, where HIDL (Hardware Interface Definition Language) refers to a hardware interface definition language used for inter-process communication.

[0097] The controller 250 establishes a communication connection with the camera driver using the camera data provider process.

[0098] In some embodiments, the camera data provider process can establish a communication connection with the camera through a Universal Serial Bus Video Class (UVC protocol) or a Mobile Industry Processor Interface (MIPI protocol).

[0099] The controller 250 sends a start instruction to the camera driver.

[0100] In some embodiments, the start instruction is a software-level instruction and cannot directly control the camera to start. The camera driver converts the start instruction into a start signal to control the camera to start.

[0101] After the camera is turned on, it starts recording the surrounding environment and collecting original images.

[0102] The camera sends the raw image to the camera driver.

[0103] The camera driver sends the raw image to the camera data provider process.

[0104] In some embodiments, after acquiring the raw image, the camera driver sends the raw image to the camera data provider process via the UVC protocol or the MIPI protocol.

[0105] The camera data provider process formats the raw image to obtain the image.

[0106] In some embodiments, the camera data provider process converts the raw image in Bayer format to an image in BMP or JPEG or PNG format.

[0107] The controller 250 calls the camera interface provided by the camera service and sends the image to the application.

[0108] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device 200 and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0109] The camera calling mechanism in the current operating system is that the camera calling permission can only be occupied by one application at the same time. When the camera calling permission is occupied by a certain application, other applications cannot obtain the camera calling permission. For example, in some application scenarios, the user cannot use gestures to control the display device 200 during video chatting. This is because the application that implements the video chat function occupies the camera calling permission during the video chatting process, so that the application that implements the gesture control function cannot obtain the image collected by the camera, which leads to the failure of the gesture control function.

[0110] The present application provides a display device 200 that can meet the needs of multiple applications to simultaneously obtain images captured by a camera, thereby solving the problem of failure of functions implemented by other applications based on images captured by the camera when the camera's calling permission is occupied by a certain application.

[0111] In some embodiments, the display device 200 includes a display 260, a device interface 240, and a controller 250. The device interface 240 is connected to the camera. The controller 250 sends the first image captured by the camera to the first application in response to the first data request instruction of the first application. In the case where the first application occupies the calling authority of the camera, if the controller 250 receives the first data request instruction of the second application, it sends the second image to the second application. Since the second image is obtained by copying or mapping the first image, the second application can implement the corresponding function based on the second image, thereby solving the problem that the function implemented by the second application based on the image captured by the camera is invalid when the calling authority of the camera is occupied by the first application.

[0112] The display device 200 may be installed with applications, including some applications that implement certain functions based on images captured by the camera. For example, some functions implemented by these applications based on images captured by the camera include: video chat, gesture control, camera shooting, etc. In some embodiments, the first application and the second application are applications that implement certain functions based on images captured by the camera.

[0113] The user can view the applications installed in the display device 200. For example, the user can send a query instruction to the display device 200, and the query instruction is used to query the applications installed in the display device 200. In response to the query instruction, the controller 250 can control the display to display the application list. For example, Figure 5 FIG. 2 shows one of the user interface schematic diagrams of the display. The user can select “My Applications” in the user interface to view the applications installed in the display device 200. Figure 6In the second user interface schematic diagram of the display shown, the display displays an application list, and the application list includes a first application and a second application, etc.

[0114] In addition, the user can also select the application that he wants to run in the application list. In some embodiments, according to different interaction modes between the display device 200 and the user, the user can input control instructions for running the application in different interaction modes.

[0115] In some embodiments, the user can enter the application list interface by interactively pressing the buttons on the control device 100 (such as a remote control), and adjust the cursor position in the application list interface by pressing the "up", "down", "left" and "right" keys of the control device. After controlling the cursor position to be on the target application icon, the target application is started by pressing the "OK" key.

[0116] In some embodiments, the user may also input control instructions through other interactive methods.

[0117] In some embodiments, for a display device 200 supporting a touch function, a user may directly click on a target application icon displayed on the display to start the target application.

[0118] In some embodiments, for a display device 200 supporting a voice control function, a user can start a target application by inputting a voice command. For example, a user can start a first application by inputting a voice command of "open the first application".

[0119] In some embodiments, for a display device 200 that supports gesture control, a user can start a target application by inputting a gesture command. For example, a user can adjust the cursor position in the application list interface by "sliding a finger in the air", and after controlling the cursor position to be located at the target application icon, the target application can be started by "making a fist".

[0120] In some implementations, after the application is started, the application sends a first data request instruction to the controller 250 , and the controller 250 sends the image captured by the camera to the application in response to the first data request instruction of the application.

[0121] In some embodiments, after being started, some applications directly send a first data request instruction to the controller 250 to request to obtain images captured by the camera, for example, some gesture control applications or camera applications.

[0122] After being started, some applications need to wait for the user to turn on the camera function within the application before sending a first data request instruction to the controller 250, such as some video chat applications.

[0123] In some embodiments, the controller 250 sends the first image captured by the camera to the first application in response to the first data request instruction of the first application, and specifically performs the following steps: calling the camera service to turn on the camera, controlling the camera to record the physical environment to obtain the original image, using the first camera data provider process to format the original image to obtain the first image, and sending the first image to the first application through the camera interface.

[0124] In some embodiments, reference may be made to the above Figure 4 In an embodiment, the controller 250 responds to the first data request instruction of the first application, calls the open camera method in the camera service of the framework layer, and controls the camera service to call the camera data provider process of the hardware abstraction layer. Then, the controller 250 uses the camera data provider process to establish a communication connection with the camera driver, and then, the controller 250 sends an opening instruction to the camera driver. After the camera is turned on, it starts to record the surrounding environment and collects the original image. The camera driver sends the original image to the camera data provider process. Then, the camera data provider process formats the original image to obtain an image. Finally, the controller 250 calls the camera interface provided by the camera service and sends the image to the application.

[0125] In some embodiments, when the first application occupies the calling permission of the camera, if the controller receives the first data request instruction of the second application, the second image is sent to the second application, and specifically performs the following steps: calling the camera service to turn on the virtual camera, copying or mapping the first image to obtain the second image, sending the second image to the second camera data provider process, and sending the second image to the second application through the camera interface.

[0126] Among them, the virtual camera is a camera implemented at the software level, without corresponding hardware implementation. The virtual camera can simulate the functions of the camera. The virtual camera includes a data source for simulating the "capture" function of the camera. The data source may include, for example: a local video file of the device, a screen capture, network stream data, etc. In some embodiments of the present application, the data source of the virtual camera may include a second image.

[0127] In some embodiments, before the controller 250 calls the camera service to start the virtual camera, it registers the virtual camera in the camera list of the hardware abstraction layer according to the camera data structure.

[0128] In some embodiments, the data structure of the virtual camera is the same as the data structure of the camera. A copy of the data structure of the camera can be used as the data structure of the virtual camera, and then it is registered in the camera list of the hardware abstraction layer. The registered virtual camera can be called by the application.

[0129] These data structures include, for example: configuration parameters (e.g., resolution, frame rate, exposure time), calibration parameters (focal length, principal point coordinates, rotation and translation relationship between the camera and the coordinate system), data format, etc.

[0130] In some embodiments, the ID of the camera in the camera list is 1. In order to distinguish the camera from the virtual camera, the ID of the virtual camera in the camera list is -1.

[0131] In some embodiments, the second image is a copy or mapping of the first image.

[0132] If the second image is obtained by copying the first image, after the first image is deleted from the memory, the second image does not disappear along with the deletion of the first image. The second image obtained by copying will not be affected by the deletion of the first image, thus increasing the stability of the data.

[0133] If the second image is obtained by mapping the first image, then after the first image is deleted from the memory, the second image disappears along with the first image. The processing speed of obtaining the second image by mapping is faster, which can provide more processing time for subsequent processes and improve the overall processing efficiency.

[0134] In this way, when the first application occupies the calling permission of the camera, the controller 250 opens the virtual camera by calling the camera service, so that the second application can obtain the calling permission of the virtual camera. Since the data source of the virtual camera is the second image obtained by copying or mapping the first image, the second application can obtain the second image by occupying the calling permission of the virtual camera and implement the corresponding function based on the second image.

[0135] In some application scenarios, the image parameters required by the first application and the second application may be different. For example, the first application is a video chat application that requires high-definition images for display, so the image is required to have a higher resolution. The second application is a gesture control application. A higher-resolution image will increase the computing resources of the model and reduce the efficiency of the model output recognition results. Therefore, the image resolution is required to be relatively lower.

[0136] In some embodiments, if the first application requires the image to be acquired with a resolution of 1080P and the second application requires the image to be acquired with a resolution of 720P, then after obtaining the second image with a resolution of 1080P, the second camera data provider process resamples the second image so that the resolution of the second image is reduced to 720P to obtain a third image that meets the requirements of the second application.

[0137] In some embodiments, the display device 200 may also set a first service process. After the first service process is started, it can simultaneously meet the needs of multiple applications for acquiring images.

[0138] In some embodiments, the first service process can be connected to multiple applications of the application layer, and to the first camera data provider process and the second camera data provider process of the hardware abstraction layer. According to the data request instruction of the application, the first image is obtained from the first camera data provider process, or the second image is obtained from the second camera data provider process, and the image is distributed to the corresponding application.

[0139] In some embodiments, controlling the opening / closing of the first service process may be integrated into a system setting application of the display device 200, and a user may control the opening or closing of the first service process through a control for controlling the opening / closing of the first service process in the system setting application.

[0140] like Figure 7 As shown, Figure 7 A schematic diagram of the system setting interface is shown, and the user can control the cursor to move to the "intelligent recognition service" option through the remote control, and click the OK key to control the first service process to be turned on or off.

[0141] The user may also control the opening or closing of the first service process by inputting voice commands or gesture commands.

[0142] like Figure 8 As shown, Figure 8 One of the example diagrams showing the control of opening the first service process through voice commands is shown. The user can open the "intelligent recognition service" pop-up window by inputting the voice command "open intelligent recognition service". Fig. 9 As shown, Fig. 9 The second example diagram shows controlling the start of the first service process through voice commands, where the user voice inputs "confirm start" to control the start of the first service process.

[0143] like Fig.10 As shown, Fig.10 One of the example diagrams showing the control of opening the first service process through gesture instructions is shown. The user can make an "open palm" gesture to open the "intelligent recognition service" pop-up window. Fig.11 As shown, Fig.11 The second example diagram shows controlling the start of the first service process through a gesture instruction, where the user performs a "tap with the index finger" action to control the start of the first service process.

[0144] In some embodiments, after the first service process is started, it will preferentially occupy the calling permission of the camera. If the calling permission of the camera has been occupied by the application, the first service process will occupy the calling permission of the virtual camera (virtual image collector).

[0145] In some embodiments, the controller 250 is also used to execute: in response to the start of the first service, detecting whether the calling permission of the camera is occupied; if the calling permission of the camera is not occupied, assigning the calling permission of the camera to the first service process so that the first service process obtains the first image; if the calling permission of the camera is occupied, calling the camera service to register a virtual camera, and assigning the calling permission of the virtual camera to the first service process so that the first service process obtains the second image.

[0146] In some embodiments, the camera service may be called to obtain the camera's attribute information to determine whether the camera's calling permission is occupied. For example, the camera's attribute information may be obtained by calling the get Camera Characteristics method, and the camera's attribute information may be used to determine whether the camera is occupied.

[0147] In some embodiments, it is possible to determine whether the camera's calling permission is occupied by catching exceptions. For example, if a runtime exception (Runtime Exception) is returned when calling the camera service's open camera method, it means that the camera's calling permission is occupied.

[0148] After the first application is started and sends the first data request instruction, if the camera calling authority is occupied by the first service process, the camera calling authority occupied by the first service process can be released and the camera calling authority can be allocated to the first application.

[0149] In some embodiments, the controller 250 is also used to execute: if it is detected that the calling permission of the camera is occupied by the first service process, release the calling permission of the camera occupied by the first service process, and assign the calling permission of the camera to the first application so that the first application obtains the first image; call the camera service to turn on the virtual camera, and assign the calling permission of the virtual camera to the first service process so that the first service process obtains the second image.

[0150] like Fig.12 As shown, the processing flow of the controller 250 in response to the first data request instruction of the first application is exemplarily introduced:

[0151] S1201. Receive a first data request instruction from a first application.

[0152] In some embodiments, the first application may send a first data request instruction after being started. For example, some camera applications send a first data request instruction after being started so as to display a preview image in a timely manner.

[0153] In another example, the first application may send a first data request instruction in response to a user operation after being started. For example, some video chat applications may send a first data request instruction to display their own screen after the user selects a video chat object and triggers a control to start a video chat.

[0154] S1202. Determine whether the camera's calling permission is occupied. If so, execute S1210; if not, execute S1203.

[0155] In some embodiments, the above-mentioned embodiments can be used to obtain the attribute information of the camera by calling the camera service to determine whether the camera's calling permission is occupied, or exception capture can be used to determine whether the camera's calling permission is occupied.

[0156] S1203, control to open the camera.

[0157] In some embodiments, the above Figure 4 The method of opening the camera is introduced in the embodiment.

[0158] S1204: Send the first image to the first application.

[0159] In some embodiments, the above Figure 4 The embodiment introduces a method for sending a first image to a first application.

[0160] S1210, determine whether the first service process is running, if so, execute S1220, if not, execute S1211.

[0161] In some embodiments, if the first service process is not running, the camera call permission may be occupied by other applications. In this case, S1211 is executed to open the virtual camera so that the first application can obtain the second image. If the first service process is already running, the camera call permission may be occupied by the first service process. In this case, S1220 is executed to release the camera call permission occupied by the first service process and allocate the camera call permission to the first application.

[0162] S1211. Control opening of a virtual camera.

[0163] In some embodiments, the method for opening a virtual camera described in the above embodiments may be used.

[0164] S1212: Acquire a second image using a copy mode, and send the second image to the first application.

[0165] S1213: Acquire a second image using a mapping mode, and send the second image to the first application.

[0166] S1220: Release the calling permission of the first service process to occupy the camera.

[0167] In some embodiments, the permission management library of the operating system (for example, Rx Permissions) can be used to manage the calling permissions of the camera. The permission management library can manage the calling permissions of the hardware through responsive programming. For example, after the above step S1210 determines that the first service process is already running, the calling permissions of the camera occupied by the first service process are released.

[0168] S1221. Allocate camera calling authority to the first application.

[0169] In some embodiments, after executing the above step S1220 to release the first service process from occupying the calling permission of the camera, the calling permission of the camera can also be allocated to the first application through the permission management library.

[0170] S1222. Control opening of a virtual camera.

[0171] In some embodiments, the method for opening a virtual camera described in the above embodiments may be used.

[0172] S1223: Acquire the second image using the copy mode, and send the second image to the first service process.

[0173] S1224. Acquire the second image using the mapping mode, and send the second image to the first service process.

[0174] According to the above Fig.12 The processing flow described in the example can, after receiving the first data request of the first application, give priority to ensuring that the first application occupies the camera calling permission when no other application occupies the camera calling permission, thereby avoiding conflicts between the first application and the first service process when occupying the camera calling permission.

[0175] At present, the real demand of most applications is not to obtain images captured by cameras, but to obtain the processing results after customized processing of images. Customized processing of images includes, for example, identifying image content, image enhancement, image synthesis, etc. After the application obtains these processing results, it can directly implement the corresponding functions based on the processing results.

[0176] For example, the real requirement of some gesture control applications is to obtain gesture actions in images. The gesture control application can match the gesture actions with the saved gesture instructions. If the match with a saved gesture instruction is successful, the gesture instruction is sent, and the controller 250 controls the display device 200 to perform the corresponding action according to the gesture instruction.

[0177] For another example, the real requirement of some camera applications is to synthesize objects such as faces or bodies in images with special effects to obtain special effect images. These special effect images can be sent to a display for display.

[0178] If each application needs to perform customized processing on the image after acquiring the image captured by the camera, it will consume a lot of computing resources. In addition, different applications may have the same requirements (for example, application A and application B both need to obtain gestures in the image), and repeated processing will cause a waste of computing resources.

[0179] A display device 200 provided in an embodiment of the present application includes a first service process. The first service process can provide at least one image processing capability to an application through a service interface, and the application can call the image processing capability provided by the first service process through the service interface. After obtaining the needs of the application, the first service process processes the image to obtain a processing result, and calls the processing result back to the application. In this way, since the first service process integrates a variety of image processing capabilities, customized processing results can be output according to the actual needs of the application. In addition, if multiple applications simultaneously call an image processing capability provided by the first service process, the first service process only needs to perform one operation and then call back the processing results to these applications respectively, which can save a lot of computing resources.

[0180] like Fig.13 As shown in FIG. 1 , a software configuration diagram including a first service process is introduced. It can be known that, Figure 3 Components similar to the software configuration shown are not repeated here.

[0181] exist Fig.13 In the application layer, a service interface is provided, and the service interface provides the image processing capability possessed by the first service process for the application.

[0182] The first service process provides multiple image algorithm models at the application framework layer, and each image algorithm model is responsible for providing an image processing capability, such as content recognition, image enhancement, and image synthesis.

[0183] The first service process is connected to the first camera data provider process and the second camera data provider process, and is used to obtain a first image provided by the first camera data provider process or a second image provided by the second camera data provider process.

[0184] The controller 250 may copy or map the first image to obtain the second image.

[0185] The second camera data provider shares the camera interface provided by the camera service with the first camera data provider. The controller 250 may send the processing result output by the first service process to the application through the camera structure.

[0186] In some embodiments, the controller 250 turns on the camera in response to a first data request instruction of the first application, and controls the first camera data provider process to obtain the original image captured by the camera, formats the original image to obtain the first image, and then sends the first image to the first application through the camera interface provided by the camera service.

[0187] In some embodiments, the controller 250 turns on the virtual camera in response to a second data request instruction of the second application, obtains a second image through a second camera data provider process, sends the second image to the first service process for customized processing, and then sends the processing result output by the first service process to the second application through the camera interface provided by the camera service.

[0188] In some embodiments, the image algorithm model of the first service process can be expanded by means of a plug-in, so that the first service process can provide more image processing capabilities.

[0189] In some embodiments, after the first service process is started, the controller 250 is further configured to execute:

[0190] Control the first service process to initialize the image algorithm model.

[0191] In some embodiments, the values ​​of various parameters in the image algorithm model (eg, weight values, bias values, etc.) are set to initial values.

[0192] In some embodiments, the above-mentioned embodiments can be used to obtain the attribute information of the camera by calling the camera service to determine whether the camera's calling permission is occupied, or exception capture can be used to determine whether the camera's calling permission is occupied.

[0193] Since the data source of the virtual camera needs to rely on the camera, when the calling permission of the camera is not occupied, the camera is opened first and the first image is sent to the first service process.

[0194] In some embodiments, the controller 250 is also used to perform:

[0195] Receive a second data request instruction from the first application, and control the first service process to call a target image algorithm model to process the first image according to the second data request instruction of the first application; the target image algorithm model is one of the image algorithm models included in the first service process, and the processing result is sent to the first application.

[0196] In some embodiments, the second data request instruction includes a handle that can call an image processing model. These handles are provided to the application by the first service process through a service interface. The application can obtain the image processing capabilities of the first service process based on these handles, and then encapsulate the corresponding handles in the second data request instruction according to the actual needs of the application to use the corresponding image processing capabilities.

[0197] In some application scenarios, taking the first application as a gesture control application as an example, the first application needs to obtain the user's gesture movements, and the second data request instruction includes a handle for calling image content recognition. After obtaining the image captured by the camera, the controller 250 calls the image algorithm model that can recognize the image content for processing according to the second data request instruction, and obtains the processing result of the image algorithm model, and the processing result is, for example, "clenching a fist". The controller 250 sends the processing result to the first application, and the first application queries the gesture instruction corresponding to "clenching a fist" according to the processing result. If the first application queries the gesture instruction corresponding to "clenching a fist", the gesture instruction is sent, and the controller 250 executes the gesture instruction.

[0198] like Fig.14 As shown, a schematic diagram of the process flow of the first service process is exemplarily introduced:

[0199] The controller 250 starts the first service process in response to the user's operation of opening the first service. The controller 250 controls the first service process to initialize the image algorithm model and open the camera to obtain the first image. In response to the second data request instruction of the first application, the controller 250 calls the target image algorithm model to process the first image to obtain a processing result, and calls the processing result back to the first application.

[0200] In some embodiments, after controlling the first service process to initialize the image algorithm model, the camera can be turned on to obtain the first image after responding to the second data request instruction of the first application, thereby avoiding waste of resources.

[0201] In some embodiments, the controller 250 is also used to perform:

[0202] Receive a second data request instruction from the second application, control the first service process to call the target image algorithm model to process the second image according to the second data request instruction from the second application, and send the processing result to the second application.

[0203] In some embodiments, the processing process of the controller 250 receiving the second data request instruction of the first application is similar to that in the above example, and will not be repeated here.

[0204] In some embodiments, taking the first application as a video chat application and the second application as a gesture control application as an example, the application scenario of the display device 200 provided in the embodiments of the present application is exemplarily introduced.

[0205] like Fig.15 In one of the application scenario diagrams of the display device 200 shown in the figure, a user uses a video chat application to conduct a video conference, and the display device 200 displays the user interface of the video chat application, which includes multiple video windows, wherein video window 1 displays the image captured by the camera. During the video conference, the user wants to adjust the volume of the display device 200. Fig.16 In one of the application scenario diagrams of the display device 200 shown, the user makes a "slide finger upward" action, and the display device 200 displays a volume adjustment control and controls the volume to increase.

[0206] In addition, the present application also provides a data synchronization method, such as Fig.17 As shown, the method includes:

[0207] S1701. In response to a first data request instruction of a first application, send a first image to the first application; the first image is obtained based on an original image captured by a camera.

[0208] S1702: When the first application occupies the calling authority of the camera, if a first data request instruction of the second application is received, the second image is sent to the second application; the second image is obtained by copying or mapping the first image.

[0209] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a data synchronization method. The data synchronization method can respond to the first data request instruction of the first application and send the first image obtained based on the original image captured by the camera to the first application. And when the first application occupies the calling authority of the camera, it can receive the first data request instruction of the second application and send the second image to the second application. So that the second application can perform corresponding processing according to the second image. Since the second image is obtained by copying or mapping the first image, the content of the second image is the same as that of the first image. The problem of the failure of the function implemented by the second application based on the image captured by the camera when the calling authority of the camera is occupied by the first application is solved.

[0210] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0211] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0212] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0213] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: monitor; Device interface; The device interface is configured to connect to the image collector; A controller is connected to the display and the image collector and is configured as follows: In response to a first data request instruction of a first application, sending a first image to the first application; the first image is obtained according to an original image acquired by the image acquirer; In the case where the first application occupies the calling authority of the image collector, if a first data request instruction of a second application is received, the second image is sent to the second application; the second image is obtained by copying or mapping the first image.

2. The display device according to claim 1, characterized in that The controller executes a data request instruction in response to the first application, and sends the first image acquired by the image collector to the first application, and is specifically configured as follows: Call the camera service to start the image collector; Controlling the image collector to record the physical environment; Using a first camera data provider process to obtain the original image recorded by the image collector; Using a first camera data provider process to perform format processing on the original image to obtain a first image; The first image is sent to the first application through a camera interface.

3. The display device according to claim 2, characterized in that The controller executes, when the first application occupies the calling authority of the image collector, sending the second image to the second application if a data request instruction of the second application is received, and is specifically configured as follows: Call the camera service to start the virtual image collector; Copying or mapping the first image to obtain the second image; acquiring the second image using a second camera data provider process; The second image is sent to the second application through the camera interface.

4. The display device according to any one of claims 1 to 3, characterized in that: The controller is also configured to: In response to the first service process being started, detecting whether the calling permission of the image collector is occupied; If the calling permission of the image collector is not occupied, assigning the calling permission of the image collector to the first service process, so that the first service process obtains the first image; If the calling permission of the image collector is occupied, the camera service is called to register the virtual image collector, and the calling permission of the virtual image collector is allocated to the first service process, so that the first service process obtains the second image.

5. The display device according to claim 4, characterized in that The first service process includes at least one image algorithm model; After the first service process is started, the controller is further configured to: Control the first service process to initialize the image algorithm model.

6. The display device according to claim 5, characterized in that The controller is also configured to: receiving a second data request instruction of the first application; According to the second data request instruction of the first application, the first service process is controlled to call a target image algorithm model to process the first image; the target image algorithm model is one of the image algorithm models included in the first service process; The processing result is sent to the first application.

7. The display device according to claim 6, characterized in that The controller is also configured to: receiving a second data request instruction of the second application; According to the second data request instruction of the second application, controlling the first service process to call the target image algorithm model to process the second image; The processing result is sent to the second application.

8. The display device according to claim 3, characterized in that Before calling the camera service to start the virtual image collector, the controller is further configured as follows: The virtual image collector is registered in the camera list of the hardware abstraction layer according to the data structure of the image collector.

9. The display device according to claim 4, characterized in that: After the controller responds to the first data request instruction of the first application, it is further configured to: If it is detected that the calling permission of the image collector is occupied by the first service process, the calling permission of the image collector occupied by the first service process is released, and the calling permission of the image collector is allocated to the first application, so that the first application obtains the first image; The camera service is called to start a virtual image collector, and the calling authority of the virtual image collector is allocated to the first service process, so that the first service process obtains the second image.

10. A data synchronization method, characterized in that: The method comprises: In response to a data request instruction of a first application, a first image is sent to the first application; the first image is obtained according to an original image acquired by an image collector; In the case where the first application occupies the calling authority of the image collector, if a data request instruction of a second application is received, the second image is sent to the second application; the second image is obtained by copying or mapping the first image.

Images