Display device and display method

Abstract

The invention relates to the technical field of display equipment, and provides display equipment and a display method.The display equipment comprises a displayer, an image collecting device and a controller, and the controller is coupled with the displayer and the image collecting device and is configured to respond to a first instruction received by a first application, generate a camera starting instruction according to an input channel of the first instruction and send the camera starting instruction to the first application; according to the method, a camera starting instruction is set, so that the camera starting instruction contains information representing a first instruction demand, and therefore, after a corresponding image acquisition device is started according to the camera starting instruction and acquisition parameters of the image acquisition device are set, the started image acquisition device can acquire a target image or a target video meeting the first instruction demand, and in other words, the target image or the target video can be acquired. After the feature information is extracted from the target image or the target video, the verification passing probability of the feature information is improved, and the execution of the first operation is realized, so that the user operation difficulty is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a display method. Background Art

[0002] With the development of smart TVs, the media functions of smart TVs have become more diverse. Traditional forms of television are one-way media push-type communications, while smart TVs with interactive experience functions can realize multiple media forms. For example, common media forms of human-computer interaction include somatosensory, voice and video capture.

[0003] Among them, video capture generally uses cameras to obtain people's facial features, expression information, movement posture information, etc., and through algorithms such as facial recognition, posture recognition, and related business logic processing, it can meet the needs of scenarios such as account login, payment, video conferencing, gesture control, and interactive games.

[0004] However, the requirements for images or videos are different in different scenarios. In actual applications, directly captured images or videos are difficult to meet the image or video requirements of all scenarios. Users are required to adjust the image or video capture method or the captured images or videos according to the scenario requirements. Summary of the invention

[0005] The present application provides a display device and a display method, so as to provide a convenient image or video acquisition method when a user needs to acquire images or videos, so as to improve the user experience.

[0006] The present application provides a display device, the display device comprising:

[0007] a display configured to display a user interface of the first application;

[0008] An image acquisition device, coupled to the display and configured to acquire a target image or a target video;

[0009] A controller is coupled to the display and the image acquisition device and is configured to:

[0010] In response to the first application receiving the first instruction, according to the input channel of the first instruction, controlling the first application to generate a camera start instruction, the first application being an application capable of invoking the image acquisition device;

[0011] In response to the camera start instruction, start the corresponding image acquisition device and set the acquisition parameters of the image acquisition device;

[0012] Drive the image acquisition device to acquire a target image or a target video;

[0013] driving the display to display a target image or a target video in the first application, and identifying feature information of the target image or the target video;

[0014] In response to the characteristic information, a first operation is performed, where the first operation is an operation corresponding to the first instruction.

[0015] In this way, a camera start-up instruction is generated according to the input channel of the first instruction, so that the camera start-up instruction contains information representing the requirements of the first instruction. After the corresponding image acquisition device is started according to the camera start-up instruction and the acquisition parameters of the image acquisition device are set, it is ensured that the started image acquisition device can acquire the target image or target video that meets the requirements of the first instruction. In other words, after the feature information is extracted from the target image or target video, the probability of the feature information passing the verification is increased, and the execution of the first operation is realized, thereby reducing the difficulty of user operation and improving the user experience.

[0016] In a possible implementation, in the step of controlling the first application to generate a camera start instruction according to the input channel of the first instruction, the controller is specifically configured as follows:

[0017] In response to the first instruction, obtaining a device list of the image acquisition device, the device list including camera identity information and camera parameter information;

[0018] According to the input channel of the first instruction, matching the camera parameters in the device list to determine the identity information and acquisition parameters of the selected camera;

[0019] Generate camera startup instructions based on camera identity information and acquisition parameters.

[0020] In this way, when the camera is started using the camera start instruction, the camera can meet the functional requirements of the first instruction as much as possible, thereby simplifying user operations and improving user experience.

[0021] In a possible implementation, in the step of starting the corresponding image acquisition device in response to the camera start instruction and setting acquisition parameters of the image acquisition device, the controller is specifically configured as follows:

[0022] Create a collection session according to the camera startup instruction.

[0023] Get the camera identity information, and add the camera corresponding to the camera identity information to the collection session;

[0024] The display is driven to display a parameter setting interface on the user interface of the first application, and in response to a user's determination operation on the acquisition parameters, the acquisition parameters of the image acquisition device are configured.

[0025] In this way, the configured acquisition parameters can be displayed to the user for viewing, so that the user can know the acquisition parameters currently set by the image acquisition device.

[0026] In a possible implementation, after driving the display to display the parameter setting interface on the user interface of the first application, the controller is further configured to:

[0027] In response to the parameter adjustment value input by the user, the parameter information on the parameter setting interface is adjusted.

[0028] In this way, when the user is relatively familiar with the image or video acquisition function of the display device, a channel is provided for the user to independently set the acquisition parameters, providing the user with more choices, thereby improving the user experience.

[0029] In a possible implementation, after driving the display to display the parameter setting interface on the user interface of the first application, the controller is further configured to:

[0030] Acquire a first image determined by a user, and extract image information of the first image;

[0031] According to the image information of the first image, the parameter information on the parameter setting interface is adjusted.

[0032] In this way, when the user is not clear about the appropriate values ​​of the acquisition parameters, and the acquisition parameters set by the controller according to the camera startup instruction are inappropriate, the acquisition parameters of the image acquisition device can be set by extracting appropriate parameter information through existing images that meet the requirements, thereby improving the user experience.

[0033] In a possible implementation, after driving the display to display the parameter setting interface on the user interface of the first application, the controller is further configured to:

[0034] In response to a switching instruction input by a user, a camera corresponding to the switching instruction is started.

[0035] In this way, users are provided with a channel to switch cameras independently, thereby improving user experience.

[0036] In a possible implementation, after driving the image acquisition device to acquire the target image or target video, the controller is further configured to: add the acquired target image or target video to the acquisition session, release the acquisition session resources, and release the camera resources. Before releasing the acquisition session resources and releasing the camera resources, the controller is further configured to: output the acquired target image or target video to the local database.

[0037] In this way, the target image or target video can be saved for the user, and the parameter information saved in the target image or target video can be used as a data source for setting acquisition parameters in the subsequent acquisition parameter setting process. For example, the target image is used as the first image in the subsequent acquisition parameter setting process.

[0038] In a possible implementation, after driving the display to display the parameter setting interface on the user interface of the first application, the controller is further configured to:

[0039] In response to a user query operation on the collection parameters, driving the display to display the historical parameters on the user interface of the first application;

[0040] In response to a user's determination operation on the historical parameters, acquisition parameters of the image acquisition device are configured according to the historical parameters.

[0041] In this way, by displaying historical parameters, users are provided with a reference when setting acquisition parameters, thereby improving the user experience.

[0042] The present application also provides a display method, which is applied to a display device, the display device comprising: a display and an image acquisition device coupled to the display; the display method comprises:

[0043] In response to the first application receiving the first instruction, according to the input channel of the first instruction, controlling the first application to generate a camera start instruction, the first application being an application capable of invoking the image acquisition device;

[0044] In response to the camera start instruction, start the corresponding image acquisition device and set the acquisition parameters of the image acquisition device;

[0045] Drive the image acquisition device to acquire a target image or a target video;

[0046] driving the display to display a target image or a target video in the first application, and identifying feature information of the target image or the target video;

[0047] In response to the characteristic information, a first operation is performed, where the first operation is an operation corresponding to the first instruction.

[0048] The display device and display method provided in the embodiments of the present application generate a camera startup instruction according to the input channel of the first instruction, so that the camera startup instruction contains information representing the requirements of the first instruction. In this way, after starting the corresponding image acquisition device according to the camera startup instruction and setting the acquisition parameters of the image acquisition device, it is ensured that the started image acquisition device can acquire a target image or target video that meets the requirements of the first instruction. In other words, after extracting feature information from the target image or target video, the probability of the feature information passing the verification is increased, and the execution of the first operation is realized, thereby reducing the difficulty of user operation and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0051] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0052] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0053] Figure 4 Schematic diagram of a process that a controller is configured to execute in some embodiments of the present application Figure 1 ;

[0054] Figure 5 Schematic diagram of a process that a controller is configured to execute in some embodiments of the present application Figure 2 ;

[0055] Figure 6 Schematic diagram of a process that a controller is configured to execute in some embodiments of the present application Figure 3 ;

[0056] Figure 7 User interface diagram provided for some embodiments of the present application Figure 1 ;

[0057] Figure 8 User interface diagram provided for some embodiments of the present application Figure 2 ;

[0058] Fig. 9Schematic diagram of the system architecture of the display device provided in some embodiments of the present application Figure 1 ;

[0059] Fig.10 Schematic diagram of the system architecture of the display device provided in some embodiments of the present application Figure 2 ;

[0060] Fig.11 A timing control schematic diagram provided for some embodiments of the present application. DETAILED DESCRIPTION

[0061] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0062] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0063] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0064] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0065] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0066] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0067] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0068] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0069] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0070] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0071] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0072] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0073] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0074] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0075] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0076] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0077] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0078] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0079] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0080] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0081] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0082] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0083] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface. For example, the operating system can directly control the display device to provide a user interface, or provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0084] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0085] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0086] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0087] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0088] like Figure 3 As shown, Figure 3 A software configuration diagram of a display device provided for some embodiments of the present application. In some embodiments, the system of the display device 200 can be divided into three layers, namely, an application layer, a middleware layer, and a hardware layer from top to bottom.

[0089] The application layer mainly includes applications on the TV and the application framework. Among them, applications are mainly applications developed based on the browser, such as HTML5 APPs and native applications.

[0090] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0091] Native apps can support online or offline, message push or local resource access.

[0092] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0093] The hardware layer mainly includes HAL interface, hardware and driver. Among them, HAL interface is the unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver (Universal Serial Bus: USB), HDMI driver (High Definition Multimedia Interface: HDMI), sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0094] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0095] With the development of smart TVs, the media functions of smart TVs are becoming more diverse. For example, smart TVs provide image capture functions. Applications installed on smart TVs can use image capture functions to implement specific operations. For example, through algorithms such as facial recognition and gesture recognition, as well as related business logic processing, they can meet the needs of scenarios such as account login, payment, video conferencing, gesture control, and interactive games. However, in different scenarios, the requirements for images or videos are different. In actual applications, directly captured images or videos are difficult to meet the image or video requirements of all scenarios. Users are required to adjust the image or video capture method or the captured images or videos according to the scenario requirements.

[0096] In response to the above technical problems, an embodiment of the present application provides a display device, the display device includes a display, an image acquisition device and a controller, the display is configured to display a user interface of a first application. The first application is an application that can call the image acquisition device. The image acquisition device is coupled to the display, and the image acquisition device is configured to acquire a target image or a target video. In actual application, the display device can usually be connected to multiple cameras, and the positions and functions of the multiple cameras are different to cope with different shooting requirements. The multiple cameras can be controlled by multiple image acquisition devices, and the multiple cameras can also be driven by a single image acquisition device. The controller is coupled to the display and the image acquisition device, and if Figure 4 As shown, the controller is configured to execute S410 to S450.

[0097] S410, in response to the first application receiving the first instruction, controlling the first application to generate a camera start instruction according to an input channel of the first instruction.

[0098] The input channel refers to the data source of the first instruction. For example, the first application is an application that can implement the permission unlocking function. When the user triggers the face unlocking control in the first application, the controller drives the first application to generate a camera startup instruction. The camera startup instruction contains information indicating that the first application needs to implement the face unlocking function. For another example, when the user triggers the scan code unlocking control in the first application, the controller drives the first application to generate a camera startup instruction. The camera startup instruction contains information indicating that the first application needs to implement the scan code unlocking function. For another example, the first application is an application that can implement the motion recognition function. When the user triggers the motion recognition control in the first application, the controller drives the first application to generate a camera startup instruction. The camera startup instruction contains information indicating that the first application needs to implement the motion recognition function. In the above examples, the first instruction is: a signal generated by the user triggering the face unlocking control, a signal generated by the user triggering the scan code unlocking control, or a signal generated by the user triggering the motion recognition control.

[0099] The present application controls the first application to generate a camera startup instruction according to the input channel of the first instruction, so that the camera startup instruction includes information representing the requirements of the first instruction.

[0100] S420, in response to the camera start instruction, start the corresponding image acquisition device and set acquisition parameters of the image acquisition device.

[0101] When the camera startup instruction includes information representing the requirements of the first instruction, the controller can start the corresponding image acquisition device according to the information representing the requirements of the first instruction in response to the camera startup instruction. When there is only one image acquisition device and one image acquisition device drives multiple cameras, the corresponding image acquisition device is started to indicate the identity of the camera. By starting the corresponding image acquisition device, it can be ensured that the started image acquisition device can meet the requirements of the first instruction, avoiding the situation where the user needs to manually select or switch the camera to meet the requirements of the first instruction after starting the image acquisition device function. At the same time, the acquisition parameters of the image acquisition device can be preliminarily set according to the information representing the requirements of the first instruction, wherein setting the acquisition parameters of the image acquisition device includes setting the working parameters of the camera, avoiding the situation where the user is not clear about the parameter setting rules of the image acquisition device, and providing the user with convenient operation, thereby improving the user experience.

[0102] S430, driving the image acquisition device to acquire a target image or a target video.

[0103] In the process of driving the image acquisition device to acquire the target image or target video, the image acquisition device controls the corresponding camera to acquire data, and can display a preview interface of the acquired content in the first application. In this way, when the user needs to adjust the shooting angle, it is convenient for the user to adjust the camera posture corresponding to the image acquisition device, or adjust the position of the target object.

[0104] S440: Drive the display to display the target image or target video in the first application, and identify feature information of the target image or target video.

[0105] The feature information corresponds to the requirement of the first instruction. For example, if the requirement of the first instruction is to realize the face unlocking function, the feature information is the face feature information. For another example, if the requirement of the first instruction is to realize the code scanning unlocking function, the feature information is the QR code or barcode feature information. For another example, if the requirement of the first instruction is the motion recognition function, the feature information is the motion feature information.

[0106] Specifically, the required feature information can be extracted based on geometric features. For example, when extracting facial feature information, the distance and ratio between facial points on the image are used as features. This geometric feature extraction method has a fast recognition speed, requires less memory on the device, and is less sensitive to light.

[0107] S450, executing a first operation in response to the characteristic information, where the first operation is an operation corresponding to the first instruction.

[0108] The first operation refers to an operation that implements the function corresponding to the first instruction. It should be noted that before executing the first operation, it is necessary to ensure that the feature information is verified. For example, the first instruction is generated when the user triggers the face unlock control. After the face feature information is verified, the corresponding permission unlock function is implemented.

[0109] The technical solution provided in the embodiment of the present application generates a camera startup instruction according to the input channel of the first instruction, so that the camera startup instruction contains information representing the requirements of the first instruction. In this way, after starting the corresponding image acquisition device according to the camera startup instruction and setting the acquisition parameters of the image acquisition device, it is ensured that the started image acquisition device can acquire the target image or target video that meets the requirements of the first instruction. In other words, after the feature information is extracted from the target image or target video, the probability of the feature information passing the verification is increased, and the execution of the first operation is realized, thereby reducing the difficulty of user operation and improving the user experience.

[0110] In some embodiments, in the step of controlling the first application to generate a camera start instruction according to the input channel of the first instruction, as Figure 5 As shown, the controller is specifically configured to execute S510 to S530.

[0111] S510, in response to the first instruction, obtaining a device list of the image acquisition device, wherein the device list includes camera identity information and camera parameter information.

[0112] The camera parameter information includes the camera's device information and device capabilities; for example, the device information includes: the resolution range supported by the camera, the frame rate range, the captured image data format, the flash mode (automatic, forced, and disabled), the focus mode, and the exposure mode. The device capabilities include night vision, motion detection, and face recognition functions of advanced smart cameras.

[0113] S520: Match camera parameters in the device list according to the input channel of the first instruction to determine the identity information and acquisition parameters of the selected camera.

[0114] The input channel of the first instruction can represent the functional requirements of the first instruction. In this way, according to the functional requirements represented by the input channel, the corresponding camera parameters are matched in the device list to determine the camera that meets the requirements, and recommend parameter settings, that is, determine the identity information and acquisition parameters of the selected camera.

[0115] S530: Generate a camera start instruction according to the camera identity information and acquisition parameters.

[0116] In this way, when the image acquisition device is started using the camera start instruction, the image acquisition device can be enabled to meet the functional requirements of the first instruction as much as possible, thereby simplifying user operations and improving user experience.

[0117] In some embodiments, in the step of starting the corresponding image acquisition device in response to the camera start instruction and setting the acquisition parameters of the image acquisition device, as shown in FIG. Figure 6 As shown, the controller is specifically configured to execute S610 to S630.

[0118] S610: Create a collection session according to the camera start instruction.

[0119] The acquisition session refers to a session created when using the camera API (Application Programming Interface) to process the camera data stream, which is used to implement the photo, video and preview functions.

[0120] S620: Obtain camera identity information, and add the camera corresponding to the camera identity information to the acquisition session.

[0121] S630: driving the display to display a parameter setting interface on the user interface of the first application, and configuring acquisition parameters of the image acquisition device in response to a user's determination operation on acquisition parameters.

[0122] Among them, after driving the display to display the parameter setting interface on the user interface of the first application, the parameter setting interface displays the default image acquisition device acquisition parameters, which are the parameter values ​​set by the controller in response to the camera startup instruction in step S420.

[0123] In this way, the configured acquisition parameters can be displayed to the user for viewing, so that the user can know the acquisition parameters currently set by the image acquisition device, thereby improving the user experience.

[0124] In some cases, the user can also modify the value of the acquisition parameter in the parameter setting interface. For example, after driving the display to display the parameter setting interface on the user interface of the first application, the controller obtains the parameter adjustment value input by the user, and adjusts the parameter information on the parameter setting interface based on the parameter adjustment value input by the user, and modifies the acquisition parameters of the image acquisition device in response to the adjusted parameter information. In this way, when the user is familiar with the image or video acquisition function of the display device, a channel is provided for the user to independently set the acquisition parameters, providing the user with more choices, thereby improving the user experience.

[0125] In some cases, when the user is not clear about the parameters that need to be adjusted, the acquisition parameters of the current acquisition process can also be set by providing an existing image that meets the requirements. For example, a first image determined by the user is obtained, and the image information of the first image is extracted; according to the image information of the first image, the parameter information on the parameter setting interface is adjusted. The first image is an existing image in the display device. In this way, when the user is not clear about the appropriate value of the acquisition parameter, and the acquisition parameter set by the controller according to the camera startup instruction is not appropriate, the acquisition parameters of the image acquisition device can be set by using an existing image that meets the requirements and then extracting the appropriate parameter information, thereby improving the user experience.

[0126] In some cases, after the display is driven to display the parameter setting interface on the user interface of the first application, the controller is further configured to start the camera corresponding to the switching instruction in response to the switching instruction input by the user. In this way, a channel for autonomous camera switching is provided to the user, thereby improving the user experience.

[0127] In some embodiments, after driving the image acquisition device to acquire the target image or target video, the controller is further configured to: add the acquired target image or target video to the acquisition session, release the acquisition session resources, and release the camera resources. Before releasing the camera resources, the acquired target image or target video can also be output to the local database according to user needs. For example, the user sets a rule in advance to save the target image or target video to the local database. For another example, Figure 7As shown, when the display is driven to display the target image or target video in the first application, a save control is set in the first application. When the user triggers the save control, the controller outputs the acquired target image or target video to the local database. In this way, the target image or target video can be saved for the user, and the parameter information saved in the target image or target video can be used as a data source for setting acquisition parameters in the subsequent acquisition parameter setting process, for example, the target image is used as the first image in the subsequent acquisition parameter setting process.

[0128] In some embodiments, after driving the display to display the parameter setting interface on the user interface of the first application, the controller is further configured to: in response to the user's determination operation on the acquisition parameters, configure the acquisition parameters of the image acquisition device. Figure 8 As shown, a confirmation control is displayed on the user interface of the first application, and the user can trigger the confirmation control after confirming the acquisition parameters. In this way, the user can input customized acquisition parameters.

[0129] In some embodiments, after driving the display to display the parameter setting interface on the user interface of the first application, the controller is further configured to: in response to the user's query operation on the acquisition parameters, drive the display to display the historical parameters on the user interface of the first application; in response to the user's determination operation on the historical parameters, configure the acquisition parameters of the image acquisition device according to the historical parameters. For example, on the parameter setting interface, a query control is set on the parameter setting bar. After the user triggers the query control, the most recently set historical parameters are displayed, and after the user selects the historical parameters, the corresponding parameters on the parameter setting interface are set to the values ​​of the historical parameters. In this way, by displaying the historical parameters, when the user sets the acquisition parameters, a reference is provided to the user, thereby improving the user experience.

[0130] In order to more clearly illustrate the technical solution of the present application, the technical solution of the present application is introduced from the perspective of system architecture. Fig. 9As shown in the figure, in the system architecture of the display device, the CameraService framework layer is set between the Camera App (application layer) and the Kernel layer (Hardware Abstraction Layer, HAL, hardware abstraction layer), and the CameraService framework layer can interact with the application layer and the Kernel layer. The CameraService framework layer is responsible for the camera session management, device input and data output. Camera biz provides a docking biz interface to facilitate the application development of the Camera App. The Camera Adapt Layer is an encapsulation interface for the underlying data interaction, and establishes a connection with the Kernel through the underlying V4L2 or Camera driver interface.

[0131] like Fig.10 As shown in the figure, the CameraService framework layer abstracts the camera functions into Camera management (CameraManager), session management (CameraSession), device input (CameraInput) and data output (CameraOutput).

[0132] Among them, CameraManager: is responsible for Camera device management, including camera ID, device information, device capabilities and other information, supports multi-camera management, and completes the creation of CameraSession, CameraInput and CameraOutput instances.

[0133] CameraSession: Responsible for maintaining the camera lifecycle, configuring the session input (device input) and output (data output), and controlling the start and end of the session.

[0134] CameraInput: Responsible for setting the input parameters of the input device and issuing hardware control parameters, for example, setting the flash mode (on or off), focus mode, and exposure mode.

[0135] CameraOutput: Responsible for the subdivision into preview output, photo output and video output according to the actual business functions. Supports the setting of output parameters, such as picture format, resolution setting, video frame rate, brightness and contrast setting.

[0136] CameraSession, CameraInput and CameraOutput are all created by the camera management class CameraManager. The collected data is stored in shared memory, and the maximum supported data cache frame number can be configured, which is implemented in the queue mode.

[0137] For example, taking the acquisition of target image as an example, Fig.11 As shown in the figure, the camera service architecture layer photo-taking process includes:

[0138] S1101, a first application generates a camera start instruction.

[0139] The camera startup command is triggered by the user through the first application. For example, the first application is an application that can implement the permission unlocking function. When the user triggers the face unlocking control in the first application, the camera startup command is generated according to the input channel of the first command.

[0140] S1102 to S1103, obtaining a device list of the image acquisition apparatus.

[0141] This function is implemented in the Camera management module to obtain available camera information, camera device information and device capabilities. The device information includes the resolution range, frame rate range, captured image data format, flash mode (auto, forced, disabled), focus mode and exposure mode supported by the camera; the device capabilities include night vision, motion detection and face recognition functions of advanced smart cameras.

[0142] S1104 to S1106: creating a collection session.

[0143] This function is implemented in the Camera session module, which represents a session process of the camera device and maintains the camera status and life cycle management. A camera's interactive process such as taking pictures and recording videos is a Camera session process.

[0144] The Camera session is created in the Camera management module. The camera input and output parameters are configured in the session module, and the start and end of the session are controlled.

[0145] S1107 to S1110, adding acquisition parameters of the image acquisition device to the acquisition session.

[0146] The Camera input module is created in the Camera management module and is mainly responsible for the configuration of the camera input. After the camera input is added to the acquisition session, the corresponding image or video data will be generated according to the specified configuration.

[0147] Set the acquisition parameters of the image acquisition device, such as: flash mode (automatic, forced and disabled), focus mode (automatic, manual, single autofocus, continuous autofocus and autofocus lock), exposure mode (automatic, manual, aperture priority and shutter priority), brightness, gain, contrast, GAMMA correction, contrast, data format (RAW / JPEG / MPEG / YUV / H.264 / H.265), frame rate and resolution (VGA: 640x480 pixels, HD: 1280x720 pixels, Full HD: 1920x1080 pixels, 4K: 3840x2160 pixels).

[0148] S1111 to S1114, adding the camera photo output to the collection session.

[0149] The Camera output module is created in the Camera management module and is mainly responsible for the output of the camera. It supports data encoding and decoding functions. According to the actual business functions, it is further divided into preview output, photo output and video output. Among them, preview output refers to the preview display of camera data, which supports the display and playback functions of different data formats. Photo output refers to the local image storage function of camera data, which supports common file formats such as jpg, png, bmp and other formats. Video output refers to the local video storage function of camera data, which supports common file formats such as mp4, avi, wmv and other formats.

[0150] S1115-S1118, take images and call the underlying data interface to collect images.

[0151] The camera output module calls the camera driver interface to obtain the real-time camera data stream. It also previews and saves the data to the required local file according to different camera output types.

[0152] S1119, releasing the acquisition session resources.

[0153] The camera management module completes the end of the session life cycle and releases the camera session resources.

[0154] S1120-S1121, release the camera input and output, and turn off the camera.

[0155] The camera management module releases the camera input and output resources and turns off the camera.

[0156] The embodiment of the present application further provides a display method, which is applied to a display device, the display device comprising: a display and an image acquisition device coupled to the display; the display method comprises:

[0157] In response to the first application receiving the first instruction, the first application is controlled to generate a camera start instruction according to an input channel of the first instruction, and the first application is an application that can call the image acquisition device.

[0158] In response to the camera start-up instruction, the corresponding image acquisition device is started, and acquisition parameters of the image acquisition device are set.

[0159] The image acquisition device is driven to acquire a target image or a target video.

[0160] The display is driven to display a target image or a target video in the first application, and feature information of the target image or the target video is identified.

[0161] In response to the characteristic information, a first operation is performed, where the first operation is an operation corresponding to the first instruction.

[0162] The display device and display method provided in the embodiments of the present application generate a camera startup instruction according to the input channel of the first instruction, so that the camera startup instruction contains information representing the requirements of the first instruction. In this way, after starting the corresponding image acquisition device according to the camera startup instruction and setting the acquisition parameters of the image acquisition device, it is ensured that the started image acquisition device can acquire a target image or target video that meets the requirements of the first instruction. In other words, after extracting feature information from the target image or target video, the probability of the feature information passing the verification is increased, and the execution of the first operation is realized, thereby reducing the difficulty of user operation and improving the user experience.

Claims

1. A display device, characterized in that: The display device comprises: a display configured to display a user interface of the first application; An image acquisition device, coupled to the display and configured to acquire a target image or a target video; A controller is coupled to the display and the image acquisition device and is configured to: In response to the first application receiving a first instruction, controlling the first application to generate a camera start instruction according to an input channel of the first instruction, wherein the first application is an application that can call the image acquisition device; In response to the camera start instruction, start the corresponding image acquisition device and set acquisition parameters of the image acquisition device; Driving the image acquisition device to acquire a target image or a target video; driving the display to display a target image or a target video in the first application, and identifying feature information of the target image or the target video; In response to the characteristic information, a first operation is performed, where the first operation is an operation corresponding to the first instruction.

2. The display device according to claim 1, characterized in that In the step of controlling the first application to generate a camera start instruction according to the input channel of the first instruction, the controller is specifically configured to: In response to the first instruction, obtaining a device list of the image acquisition device, the device list including camera identity information and camera parameter information corresponding to the image acquisition device; According to the input channel of the first instruction, matching the camera parameters in the device list to determine the identity information and acquisition parameters of the selected camera; The camera start instruction is generated according to the camera identity information and acquisition parameters.

3. The display device according to claim 1 or 2, characterized in that: In the step of starting the corresponding image acquisition device in response to the camera start instruction and setting acquisition parameters of the image acquisition device, the controller is specifically configured as follows: Creating a collection session according to the camera startup instruction; Acquire camera identity information, and add the camera corresponding to the camera identity information to the acquisition session; The display is driven to display a parameter setting interface on the user interface of the first application, and in response to a user's determination operation on the acquisition parameters, the acquisition parameters of the image acquisition device are configured.

4. The display device according to claim 3, characterized in that After driving the display to display a parameter setting interface on the user interface of the first application, the controller is further configured to: In response to the parameter adjustment value input by the user, the parameter information on the parameter setting interface is adjusted.

5. The display device according to claim 3, characterized in that After driving the display to display a parameter setting interface on the user interface of the first application, the controller is further configured to: Acquire a first image determined by a user, and extract image information of the first image; According to the image information of the first image, the parameter information on the parameter setting interface is adjusted.

6. The display device according to claim 3, characterized in that After driving the display to display a parameter setting interface on the user interface of the first application, the controller is further configured to: In response to a switching instruction input by a user, a camera corresponding to the switching instruction is started.

7. The display device according to claim 3, characterized in that After driving the image acquisition device to acquire the target image or target video, the controller is further configured to: The captured target image or the target video is added to a capture session, the capture session resources are released, and the camera resources are released.

8. The display device according to claim 7, characterized in that Before releasing the acquisition session resources and releasing the camera resources, the controller is further configured to: The captured target image or the target video is output to a local database.

9. The display device according to claim 1, characterized in that: After driving the display to display a parameter setting interface on the user interface of the first application, the controller is further configured to: In response to a user query operation on the collection parameters, driving the display to display the historical parameters on the user interface of the first application; In response to a user's determination operation on the historical parameters, acquisition parameters of the image acquisition device are configured according to the historical parameters.

10. A display method, characterized in that: The display method is applied to a display device, the display device comprising: a display and an image acquisition device coupled to the display; the display method comprises: In response to a first application receiving a first instruction, controlling the first application to generate a camera start instruction according to an input channel of the first instruction, wherein the first application is an application that can call the image acquisition device; In response to the camera start instruction, start the corresponding image acquisition device and set acquisition parameters of the image acquisition device; Driving the image acquisition device to acquire a target image or a target video; driving the display to display a target image or a target video in the first application, and identifying feature information of the target image or the target video; In response to the characteristic information, a first operation is performed, where the first operation is an operation corresponding to the first instruction.

Images