A screen control method, device, storage medium and terminal equipment
By generating preset icons on the display terminal and recognizing user operation intentions, the problems of high hardware costs and complex manufacturing of large-size screens are solved, achieving interactive effects without physical buttons, reducing costs and simplifying the process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN TCL NEW-TECH CO LTD
- Filing Date
- 2021-07-02
- Publication Date
- 2026-06-16
AI Technical Summary
Existing technologies require increased hardware costs and complex manufacturing processes to achieve touch screen control on televisions, especially for large-screen displays where the costs are high and the design is affected.
By recognizing the user's operational intent, a preset identifier is generated on the current screen of the display terminal, the preset identifier in the target image is obtained and extracted, and the corresponding operation command is executed to achieve interaction without physical buttons.
It reduces the manufacturing cost of large-size displays, simplifies the processing technology, and has a more concise appearance, while providing a touchscreen-like operating experience.
Smart Images
Figure CN115576472B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of electronic communication technology, and more particularly to the field of screen control technology, and especially to a screen control method, device, storage medium and terminal equipment. Background Technology
[0002] Currently, most ways to interact with a TV on the market are either through remote control or through touchscreen control. Touchscreen control is the most widely used method, and it significantly improves the user experience. To enable touchscreen control on TVs, most devices rely on touchscreens, and some also add additional detection devices around the screen to achieve the same effect.
[0003] However, all of these solutions require significant increases in hardware costs and have very high manufacturing requirements. Even if the cost is acceptable, they may not be widely adopted due to manufacturing issues, especially when applied to mainstream 65 / 75-inch or larger screens, where the corresponding manufacturing costs are very high and the appearance design is also severely affected. Summary of the Invention
[0004] This application provides a screen control method, apparatus, storage medium, and terminal device, which enables ordinary displays without touch screens to interact with the display without the need for physical buttons, while reducing the manufacturing cost of the display and simplifying the processing technology.
[0005] One embodiment of this application provides a screen control method, including:
[0006] Identify the user's operational intent based on the current display content on the display terminal;
[0007] According to the operation intention and the preset identifier generation rules, preset identifiers are generated on each sub-area of the current screen of the display terminal.
[0008] Obtain a target image containing the sub-region selected by the user;
[0009] Extract the preset identifier corresponding to the sub-region in the target image, and execute the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0010] In the screen control method described in this application embodiment, the identifier generation rule includes:
[0011] Based on the intent category of the operation intent, the location distribution strategy corresponding to the intent category is obtained from a pre-created mapping table between intent categories and location distribution strategies.
[0012] In the screen control method described in this application embodiment, after obtaining the location distribution strategy corresponding to the intent category from a pre-created mapping table between intent categories and location distribution strategies, the identifier generation rule further includes:
[0013] Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions;
[0014] Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category.
[0015] Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
[0016] In the screen control method described in this application embodiment, the identifier generation rule further includes:
[0017] The image clarity of the preset marks on different sub-regions is calculated according to the preset display optimization rules.
[0018] Based on the image clarity, the clarity of preset labels corresponding to different sub-regions is adjusted.
[0019] In the screen control method described in this application embodiment, obtaining the target image containing the sub-region selected by the user includes:
[0020] Acquire a target image captured and uploaded by a shooting terminal. The target image includes an obscured sub-region on the display terminal and a preset identifier corresponding to the obscured sub-region.
[0021] In the screen control method described in this application embodiment, the shooting device is provided with a control. The control is configured such that when the control is triggered, the shooting terminal acquires the original image of the current screen of the display terminal, marks the original image as the target image, and filters out the original images without the marking.
[0022] Accordingly, another aspect of this application embodiment also provides a screen control device, including:
[0023] The recognition module is used to identify the user's operational intent based on the current display content of the display terminal;
[0024] The generation module is used to generate preset identifiers on each sub-area pre-divided in the current screen of the display terminal according to the operation intention and the preset identifier generation rules.
[0025] The acquisition module is used to acquire a target image containing the sub-region selected by the user;
[0026] The execution module is used to extract the preset identifier corresponding to the sub-region in the target image, and execute the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0027] Accordingly, in another aspect, this application also provides a storage medium storing a plurality of instructions adapted for loading by a processor to execute the screen control method described above.
[0028] Accordingly, another aspect of this application embodiment also provides a terminal device, including a processor and a memory, wherein the memory stores a plurality of instructions, and the processor loads the instructions to execute the screen control method described above.
[0029] This application provides a screen control method, apparatus, storage medium, and terminal device. The method identifies the user's operational intent based on the current display content of the display terminal; generates preset identifiers on each pre-divided sub-region of the current screen of the display terminal according to the operational intent and pre-set identifier generation rules; acquires a target image containing the sub-region selected by the user; extracts the preset identifier corresponding to the sub-region in the target image; and executes an operation command corresponding to the preset identifier on the display terminal according to the preset identifier. This application enables ordinary displays without touchscreens to interact with the display without physical buttons. Especially when applied to large-size displays, it can reduce manufacturing costs and simplify processing, resulting in a simpler and more user-friendly appearance. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 This is a flowchart illustrating the screen control method provided in an embodiment of this application.
[0032] Figure 2 This is a schematic diagram illustrating a specific embodiment of the screen control method provided in this application.
[0033] Figure 3 This is a schematic diagram illustrating the composition strategy of preset identifiers in the screen control method provided in the embodiments of this application.
[0034] Figure 4 This is a schematic diagram of the structure of the screen control device provided in the embodiments of this application.
[0035] Figure 5 This is another schematic diagram of the screen control device provided in the embodiments of this application.
[0036] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation
[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.
[0038] This application provides a screen control method that can be applied to terminal devices. The terminal device can be a smartphone, tablet, or other similar device. Specifically, the screen control method provided in this application allows ordinary terminal devices without touchscreens to interact with the terminal device without physical controls. In simpler terms, it enables functions such as text input or app control without relying on a remote control or other physical buttons. This is particularly beneficial when applied to mainstream 65 / 75-inch or larger screens, reducing display manufacturing costs and simplifying processing, resulting in a cleaner and more streamlined appearance.
[0039] Please see Figure 1 , Figure 1 This is a flowchart illustrating a screen control method provided in an embodiment of this application. The screen control method, applied in a terminal device, may include the following steps:
[0040] Step 101: Identify the user's operational intent based on the current display content on the display terminal.
[0041] In this embodiment, when a user needs to interact with a terminal device implemented using the screen control method provided in this application, the terminal device first identifies the user's operational intent based on the current display content of the display terminal (e.g., an LCD screen). Specifically, the currently running sub-thread of the terminal device can be identified through the underlying thread, and combined with the current display screen of the display terminal, the application currently being operated by the user can be determined. For example, if the currently running sub-thread of the terminal device is the "music player search box," and the current display screen is the "virtual keyboard," then it can be determined that the user's operational intent is to input text in the music player search box using the virtual keyboard.
[0042] In other embodiments, the operational intent may also be to draw in a drawing app, etc.
[0043] Step 102: Based on the operation intention and the pre-set identifier generation rules, generate preset identifiers on each of the pre-divided sub-areas of the current screen of the display terminal.
[0044] In this embodiment, after the user's operation intention is recognized by the terminal device, a preset identifier is generated on each sub-area pre-divided on the current screen of the display terminal according to the preset identifier generation rules.
[0045] The identifier generation rules include:
[0046] Based on the intent category of the operation intent, the location distribution strategy corresponding to the intent category is obtained from a pre-created mapping table between intent categories and location distribution strategies. The location distribution strategy includes a preset identifier category corresponding to each sub-region and the sub-region division rules.
[0047] It's important to note that the mapping table between the intent categories of operational intentions and the location distribution strategies can be understood as pre-establishing a one-to-one correspondence between various operational intent categories and specified location distribution strategies. Once the intent category of a user's operational intention is obtained, the corresponding preset identifier location distribution strategy can be retrieved by looking up the table. For example, assuming the intent category corresponding to the operational intention is text input, the location distribution strategy corresponding to this intent category, i.e., the sub-region division rule, divides each key on the virtual keyboard. The preset identifier category corresponding to each sub-region is text input. Continuing with the text input example, the preset identifier for this intent category can include the position coordinates of the sub-region where the preset identifier is located and the key information (e.g., numbers or letters) corresponding to that sub-region. (Parameters are not specified.) Figure 2 As shown, Figure 2 The image on the left shows a standard interface without preset icons. Figure 2 The image on the right shows the interface with preset icons.
[0048] In another embodiment, the identifier generation rule further includes:
[0049] Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions;
[0050] Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category.
[0051] Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
[0052] In this embodiment, the construction strategy is used to construct preset identifiers on different sub-regions. The construction strategy includes preset identifier categories and other information. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of a pre-defined identifier composition strategy, in which the pre-defined identifier category information is recorded in the identifier header and identifier footer.
[0053] The calculation process for the strategy code is as follows:
[0054] F1=a1*APL / 255+a2*P / 255+a3*(((APL+P) / 2) / 255)^2.2;
[0055] Among them, F1 is the code for the strategy;
[0056] APL is the average signal level. Taking an 8-bit signal as an example, the signal distribution ranges from 0 to 255. Here, we first obtain the number of signals SN(i) corresponding to each order signal i, and then...
[0057] APL=(SN(0)*0+SN(1)*1+…+SN(255)*255) / (SN(0)+SN(1)+…+SN(255));
[0058] P represents the image purity. Taking an 8-bit signal as an example, the initial value of P is 0. If the corresponding SN(i) > 5% * (SN(0) + SN(1) + ... + SN(255)), then P + 1. i ranges from 0 to 255. a1 = 0.3, a2 = 0.2, a3 = 0.5.
[0059] In some embodiments, the identifier generation rule further includes:
[0060] The image clarity of the preset marks on different sub-regions is calculated according to the preset display optimization rules.
[0061] Based on the image clarity, the clarity of preset labels corresponding to different sub-regions is adjusted.
[0062] In this embodiment, due to differences in pixel levels and display effects across different display terminals, the brightness of background images displayed by different apps also varies. Failure to adjust the display effect of the preset logo according to these variations will negatively impact the user's viewing experience or the imaging effect of the camera. To ensure the preset logo doesn't affect the viewing experience, it needs to be as faint as possible, or even invisible to the naked eye, while simultaneously ensuring that the information contained in the preset logo is captured in the target image by the camera. Therefore, it is necessary to adjust the image clarity of the preset logo in different sub-regions using pre-defined display optimization rules to ensure that the preset logo doesn't affect the viewing experience while simultaneously capturing the information contained in the preset logo in the target image captured by the camera.
[0063] The display optimization rules include:
[0064] Calculate the APL (average picture level) value corresponding to the sub-region for which the preset identifier is to be generated;
[0065] Based on the calculated APL value, the image clarity of the preset marker in the corresponding sub-region is calculated using a predetermined clarity algorithm.
[0066] Sharpness algorithm:
[0067] F2(APL)=a+b*(APL / 255)^2.2;
[0068] Where F2 represents the image sharpness, a = 0.3, b = 0.7.
[0069] Step 103: Obtain the target image containing the sub-region selected by the user.
[0070] In this embodiment, the terminal device acquires the target image captured and uploaded by the shooting terminal.
[0071] It should be clarified that the shooting terminal refers to a device with both shooting and data transmission capabilities, and is not limited here. The target image includes the obscured sub-region on the display terminal and a preset identifier corresponding to the obscured sub-region. In a specific application scenario, when a user performs an operation on the current display terminal of the terminal device, such as text input, the shooting terminal captures the target image containing the user's text input operation. Assuming that a sub-region in the target image is obscured by the user's finger, the preset identifier corresponding to the selected sub-region is obtained, and corresponding operations are performed based on the information contained in the selected preset identifier.
[0072] Furthermore, in order to enable the shooting terminal to effectively acquire the preset markers on the sub-area, multiple identical preset markers can be set in each sub-area to ensure that the shooting terminal can capture the preset markers.
[0073] Furthermore, a control can be set on the shooting device, which is configured so that when the control is triggered, the shooting terminal acquires the original image of the current screen of the display terminal, marks the original image as the target image, and filters out the original images without the mark.
[0074] Since a user's presence in a sub-area may not necessarily indicate a confirmed selection, to prevent the camera from misidentifying the image captured in this situation as the target image, a handheld camera can be used in this embodiment. The user holds the camera while operating the display terminal, and when selecting a sub-area, covers it with their finger and presses a control on the camera to confirm. The camera then captures the original image of the current screen on the display terminal, marks it as the target image, and filters out unmarked original images. Only marked original images are uploaded to the terminal device for the next step of pre-defined identifier extraction.
[0075] Step 104: Extract the preset identifier corresponding to the sub-region in the target image, and execute the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0076] In this embodiment, a preset identifier corresponding to a sub-region in the target image is extracted. Based on the preset identifier, an operation instruction corresponding to the preset identifier is executed on the display terminal. When the category corresponding to the preset identifier is text input, the corresponding operation instruction inputs the number or text on the sub-region corresponding to the preset identifier into the search box, thus achieving a touch screen-like operation effect and enabling real-time interaction with the terminal device.
[0077] Specifically, the Halcon algorithm can be used to extract preset identifiers corresponding to sub-regions in the target image. The Halcon algorithm includes the following steps:
[0078] 1. Select the feature region of the target image and crop the feature region to obtain the first image;
[0079] 2. Perform grayscale conversion on the first image to obtain the second image, completing the conversion between RGB and gray.
[0080] 3. Noise removal is performed on the feature regions of the second image, and blurring or filtering algorithms are used to process the image noise and impurities to obtain the third image;
[0081] 4. Use thresholding to extract feature edges of feature regions in the third image;
[0082] 5. Using connected regions and area information, select feature regions to obtain preset labels in the target image.
[0083] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0084] In practice, this application is not limited by the execution order of the described steps. Without causing conflicts, some steps may be performed in other orders or simultaneously.
[0085] As can be seen from the above, the screen control method provided in this application identifies the user's operation intention based on the current display content of the display terminal; generates preset identifiers on each sub-region of the current screen of the display terminal according to the operation intention and a pre-set identifier generation rule; obtains a target image containing the sub-region selected by the user; extracts the preset identifier corresponding to the sub-region in the target image; and executes the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier. This application embodiment enables ordinary displays without touchscreens to interact with the display without the need for physical buttons. Especially when applied to mainstream 65 / 75-inch or larger screens, it can reduce the manufacturing cost of the display and simplify the processing technology, resulting in a more concise appearance.
[0086] This application also provides a screen control device, which can be integrated into a terminal device. The terminal device can be a smartphone, tablet computer, or other similar device.
[0087] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of the screen control device provided in an embodiment of this application. The screen control device 30 may include:
[0088] The recognition module 31 is used to recognize the user's operation intention based on the current display content of the display terminal;
[0089] The generation module 32 is used to generate preset identifiers on each of the pre-divided sub-areas of the current screen of the display terminal according to the operation intention and the preset identifier generation rules.
[0090] The acquisition module 33 is used to acquire a target image containing the sub-region selected by the user;
[0091] The execution module 34 is used to extract the preset identifier corresponding to the sub-region in the target image, and execute the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0092] In some embodiments, the identifier generation rule further includes:
[0093] Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions;
[0094] Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category.
[0095] Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
[0096] In some embodiments, the acquisition module 33 is used to acquire a target image captured and uploaded by the shooting terminal, the target image including an obscured sub-region on the display terminal and a preset identifier corresponding to the obscured sub-region.
[0097] In some embodiments, the shooting device is provided with a control, which is configured such that when the control is triggered, the shooting terminal acquires the original image of the current screen of the display terminal, marks the original image as the target image, and filters out the original images without the marking.
[0098] In some embodiments, each sub-region contains one or more identical preset identifiers.
[0099] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities.
[0100] As can be seen from the above, the screen control device 30 provided in this application embodiment identifies the user's operation intention based on the current display content of the display terminal through the identification module 31; the generation module 32 generates preset identifiers on each sub-region of the current screen of the display terminal according to the operation intention and the preset identifier generation rules; the acquisition module 33 acquires a target image containing the sub-region selected by the user; and the execution module 34 extracts the preset identifier corresponding to the sub-region in the target image and executes the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0101] Please see Figure 5 , Figure 5This is another schematic diagram of the screen control device provided in an embodiment of this application. The vibration screen control device 30 includes a memory 120, one or more processors 180, and one or more application programs, wherein the one or more application programs are stored in the memory 120 and configured to be executed by the processor 180; the processor 180 may include a first acquisition module 31, a second acquisition module 32, a selection module 33, and a control module 34. For example, the structure and connection relationship of the above components can be as follows:
[0102] Memory 120 can be used to store applications and data. The applications stored in memory 120 contain executable code. Applications can be composed of various functional modules. Processor 180 executes various functional applications and data processing by running the applications stored in memory 120. Furthermore, memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 120 may also include a memory controller to provide processor 180 with access to memory 120.
[0103] The processor 180 is the control center of the device, connecting various parts of the terminal through various interfaces and lines. It performs various functions and processes data by running or executing applications stored in the memory 120 and calling data stored in the memory 120, thereby providing overall monitoring of the device. Optionally, the processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications.
[0104] Specifically, in this embodiment, the processor 180 loads the executable code corresponding to the processes of one or more applications into the memory 120 according to the following instructions, and the processor 180 runs the applications stored in the memory 120 to achieve various functions:
[0105] The recognition module 31 is used to recognize the user's operation intention based on the current display content of the display terminal;
[0106] The generation module 32 is used to generate preset identifiers on each of the pre-divided sub-areas of the current screen of the display terminal according to the operation intention and the preset identifier generation rules.
[0107] The acquisition module 33 is used to acquire a target image containing the sub-region selected by the user;
[0108] The execution module 34 is used to extract the preset identifier corresponding to the sub-region in the target image, and execute the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0109] In some embodiments, the identifier generation rule further includes:
[0110] Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions;
[0111] Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category.
[0112] Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
[0113] In some embodiments, the acquisition module 33 is used to acquire a target image captured and uploaded by the shooting terminal, the target image including an obscured sub-region on the display terminal and a preset identifier corresponding to the obscured sub-region.
[0114] In some embodiments, the shooting device is provided with a control, which is configured such that when the control is triggered, the shooting terminal acquires the original image of the current screen of the display terminal, marks the original image as the target image, and filters out the original images without the marking.
[0115] In some embodiments, each sub-region contains one or more identical preset identifiers.
[0116] This application also provides a terminal device. The terminal device may be a smartphone, tablet computer, or other similar device.
[0117] Please see Figure 6 , Figure 6 A schematic diagram of the structure of a terminal device provided in an embodiment of this application is shown. This terminal device can be used to implement the vibration adjustment method provided in the above embodiments. The terminal device 1200 can be a smartphone or a tablet computer.
[0118] like Figure 6As shown, the terminal device 1200 may include an RF (Radio Frequency) circuit 110, a memory 120 including one or more (only one is shown in the figure) computer-readable storage media, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a transmission module 170, a processor 180 including one or more (only one is shown in the figure) processing cores, and a power supply 190, etc. Those skilled in the art will understand that... Figure 6 The structure of the terminal device 1200 shown does not constitute a limitation on the terminal device 1200, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0119] RF circuit 110 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby enabling communication with communication networks or other devices. RF circuit 110 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 110 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks.
[0120] The memory 120 can be used to store software programs and modules, such as the program instructions / modules corresponding to the vibration adjustment method in the above embodiment. The processor 180 executes various functional applications and data processing by running the software programs and modules stored in the memory 120. It can automatically select the vibration reminder mode for screen control according to the current scenario of the terminal device, ensuring that scenarios such as meetings are not disturbed, while also ensuring that users can sense incoming calls, thus improving the intelligence of the terminal device. The memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 120 may further include memory remotely located relative to the processor 180, and these remote memories can be connected to the electronic device 1200 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0121] The input unit 130 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 130 may include a touch-sensitive surface 131 and other input devices 132. The touch-sensitive surface 131, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 131), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 131 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 180, and can receive and execute commands from the processor 180. In addition, the touch-sensitive surface 131 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0122] Display unit 140 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of terminal device 1200. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 140 may include display panel 141, optionally configured as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, touch-sensitive surface 131 may cover display panel 141. When touch-sensitive surface 131 detects a touch operation on or near it, it transmits the information to processor 180 to determine the type of touch event. Subsequently, processor 180 provides corresponding visual output on display panel 141 according to the type of touch event. Although in Figure 4 In this embodiment, the touch-sensitive surface 131 and the display panel 141 are implemented as two separate components to realize input and output functions. However, in some embodiments, the touch-sensitive surface 131 and the display panel 141 can be integrated to realize input and output functions.
[0123] The terminal device 1200 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 141 according to the ambient light level, and the proximity sensor can turn off the display panel 141 and / or backlight when the terminal device 1200 is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that the terminal device 1200 may also be configured with, such as a gyroscope, barometer, hygrometer, thermometer, and infrared sensor, will not be described in detail here.
[0124] Audio circuitry 160, speaker 161, and microphone 162 provide an audio interface between the user and terminal device 1200. Audio circuitry 160 converts received audio data into electrical signals, which are then transmitted to speaker 161, where they are converted into sound signals for output. Conversely, microphone 162 converts collected sound signals into electrical signals, which are received by audio circuitry 160, converted back into audio data, and then processed by processor 180 before being transmitted via RF circuitry 110 to, for example, another terminal, or output to memory 120 for further processing. Audio circuitry 160 may also include an earphone jack to facilitate communication between peripheral headphones and terminal device 1200.
[0125] Terminal device 1200, through transmission module 170 (e.g., Wi-Fi module), can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 4 The transmission module 170 is shown, but it is understood that it is not a necessary component of the terminal device 1200 and can be omitted as needed without changing the nature of the invention.
[0126] The processor 180 is the control center of the terminal device 1200. It connects to various parts of the mobile phone via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 120, and by calling data stored in the memory 120, it performs various functions of the terminal device 1200 and processes data, thereby providing overall monitoring of the mobile phone. Optionally, the processor 180 may include one or more processing cores; in some embodiments, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 180.
[0127] The terminal device 1200 also includes a power supply 190 (such as a battery) to power various components. In some embodiments, the power supply can be logically connected to the processor 180 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 190 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0128] Although not shown, the terminal device 1200 may also include a camera (such as a front-facing camera and a rear-facing camera), a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit 140 of the terminal device 1200 is a touch screen display, and the terminal device 1200 also includes a memory 120 and one or more programs, one or more of which are stored in the memory 120 and configured to be executed by one or more processors 180. One or more programs contain instructions for performing the following operations:
[0129] Identify the user's operational intent based on the current display content on the display terminal;
[0130] According to the operation intention and the preset identifier generation rules, preset identifiers are generated on each sub-area of the current screen of the display terminal.
[0131] Obtain a target image containing the sub-region selected by the user;
[0132] Extract the preset identifier corresponding to the sub-region in the target image, and execute the operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier.
[0133] In some embodiments, the identifier generation rule further includes:
[0134] Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions;
[0135] Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category.
[0136] Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
[0137] In some embodiments, the shooting device is provided with a control, which is configured such that when the control is triggered, the shooting terminal acquires the original image of the current screen of the display terminal, marks the original image as the target image, and filters out the original images without the marking.
[0138] In some embodiments, each sub-region contains one or more identical preset identifiers.
[0139] This application also provides a terminal device. The terminal device may be a smartphone, tablet computer, or other similar device.
[0140] As can be seen from the above, this application embodiment provides a terminal device 1200, which performs the following steps: identifying the user's operation intention based on the current display content of the display terminal; generating preset identifiers on each sub-region of the current screen of the display terminal according to the operation intention and a preset identifier generation rule; acquiring a target image containing the sub-region selected by the user; extracting the preset identifier corresponding to the sub-region in the target image; and executing an operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier. This application embodiment enables ordinary displays without touch screens to interact with the display without the need for physical buttons. Especially when applied to mainstream 65 / 75-inch or larger screens, it can reduce the manufacturing cost of the display and simplify the processing technology, resulting in a simpler appearance.
[0141] This application also provides a storage medium storing a computer program. When the computer program is run on a computer, the computer executes the screen control method described in any of the above embodiments.
[0142] It should be noted that, for the screen control method described in this application, those skilled in the art will understand that all or part of the processes of the screen control method described in the embodiments of this application can be implemented by a computer program controlling the related hardware. The computer program can be stored in a computer-readable storage medium, such as in the memory of a terminal device, and executed by at least one processor within the terminal device. During execution, it can include the processes of the embodiments of the vibration adjustment method described. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), etc.
[0143] For the vibration adjustment device described in this application embodiment, its functional modules can be integrated into a single processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0144] The screen control method, apparatus, storage medium, and terminal device provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application; at the same time, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A screen control method, characterized in that, include: Identify the user's operational intent based on the current display content on the display terminal; According to the operation intention and the preset identifier generation rules, preset identifiers are generated on each sub-area of the current screen of the display terminal. Obtain a target image containing the sub-region selected by the user; Extract a preset identifier corresponding to the sub-region in the target image, and execute an operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier; The identifier generation rules include: Based on the intent category of the operation intent, the location distribution strategy corresponding to the intent category is obtained from a pre-created mapping table between intent categories and location distribution strategies; After obtaining the location distribution strategy corresponding to the intent category from the pre-created mapping table between intent categories and location distribution strategies, the identifier generation rule further includes: Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions; Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category. Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
2. The screen control method as described in claim 1, characterized in that, The identifier generation rules also include: The image clarity of the preset marks on different sub-regions is calculated according to the preset display optimization rules. Based on the image clarity, the clarity of preset labels corresponding to different sub-regions is adjusted.
3. The screen control method as described in claim 1, characterized in that, The step of obtaining the target image containing the sub-region selected by the user includes: Acquire a target image captured and uploaded by a shooting terminal. The target image includes an obscured sub-region on the display terminal and a preset identifier corresponding to the obscured sub-region.
4. The screen control method as described in claim 3, characterized in that, The shooting terminal is equipped with a control, which is configured to, when the control is triggered, acquire the original image of the current screen of the display terminal, mark the original image as the target image, and filter out the original images without the mark.
5. The screen control method as described in claim 1, characterized in that, Each of the sub-regions contains one or more identical preset identifiers.
6. A screen control device, characterized in that, include: The recognition module is used to identify the user's operational intent based on the current display content of the display terminal; The generation module is used to generate preset identifiers on each sub-area pre-divided in the current screen of the display terminal according to the operation intention and the preset identifier generation rules. The acquisition module is used to acquire a target image containing the sub-region selected by the user; The execution module is used to extract a preset identifier corresponding to the sub-region in the target image, and execute an operation instruction corresponding to the preset identifier on the display terminal according to the preset identifier; The identifier generation rules include: Based on the intent category of the operation intent, the location distribution strategy corresponding to the intent category is obtained from a pre-created mapping table between intent categories and location distribution strategies; The identifier generation rules also include: Calculate the composition strategy code corresponding to different sub-regions based on the image purity and APL value of different sub-regions; Based on the composition strategy code, the composition strategy corresponding to the composition strategy code is obtained from a pre-created mapping table between composition strategy codes and composition strategies. The composition strategy includes a preset identifier category. Based on the aforementioned composition strategy, preset identifiers are constructed for different sub-regions.
7. A storage medium, characterized in that, The storage medium stores a plurality of instructions, which are adapted for loading by a processor to execute the screen control method according to any one of claims 1 to 5.
8. A terminal device, characterized in that, The device includes a processor and a memory, the memory storing a plurality of instructions, the processor loading the instructions to execute the screen control method according to any one of claims 1 to 5.