Control method and device

Abstract

The embodiment of the invention provides a control method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the field of human-computer interaction. The control method is used in a client in which a target application program is installed. The method comprises the following steps: acquiring a plurality of head images of a target object in a target time window under the condition of running a target application program; determining a target action of the target object according to the plurality of head images; generating a target control instruction according to the target action; and in response to the target control instruction, controlling the target application program to execute a corresponding target operation. According to the technical scheme provided by the embodiment of the invention, accurate control on the target application program can be realized under the condition that equipment is not required to be contacted.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of human-computer interaction technology, and in particular, to a control method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the widespread adoption of mobile devices and the continuous expansion of application functionality, users are increasingly demanding a more interactive experience when using applications in various scenarios. Therefore, existing technologies offer contactless control methods to enhance the user experience. However, these existing contactless control methods suffer from low recognition accuracy and are unable to obtain the desired control commands to control applications.

[0003] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0004] The embodiments of the present application provide a control method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product to solve or alleviate one or more of the technical problems raised above.

[0005] One aspect of an embodiment of the present application provides a control method, the method comprising: While running the target application, acquiring multiple head images of a target object within a target time window; determining a target action of the target object based on the multiple head images; Generate a target control instruction according to the target action; and In response to the target control instruction, the target application is controlled to perform a corresponding target operation.

[0006] Optionally, determining the target action of the target object according to the multiple head images includes: acquiring an angle sequence based on the multiple head images, wherein the head angle sequence includes multiple head pitch angles sorted in chronological order; determining an angle reference value according to the multiple head pitch angles; determining an angle reference range according to the angle reference value; A target action of the target object is determined according to the multiple head pitch angles and the angle reference range.

[0007] Optionally, determining the target action of the target object according to the multiple head pitch angles and the angle reference range includes: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be an upward nodding action: (1) a first head pitch angle in the angle sequence is higher than a first preset threshold value of the upper limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

[0008] Optionally, determining the target action of the target object according to the multiple head pitch angles and the angle reference range includes: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be a downward nodding action: (1) The first head pitch angle in the angle sequence is lower than a second preset threshold value of the lower limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

[0009] Optionally, determining an angle reference value according to the multiple head pitch angles includes: acquiring a maximum head pitch angle and a minimum head pitch angle from the plurality of head pitch angles; An average value is taken for the maximum head pitch angle and the minimum head pitch angle, and the angle reference value is dynamically updated.

[0010] Optionally, determining the target action of the target object according to the multiple head images includes: Acquire multiple eye position parameters according to the multiple head images; wherein the eye position parameters include longitudinal distances and lateral distances of key points of the eyes; When the ratio of the longitudinal distance to the lateral distance of the eye position parameter is less than a third preset threshold, determining that the head image corresponding to the eye position parameter is in an eye-closed state; wherein one eye position parameter corresponds to one head image; When a preset number of consecutive head images are all in the eyes-closed state, the target action is determined to be blinking.

[0011] Optionally, generating a target control instruction according to the target action includes: generating a first control instruction when a single blink occurs within a preset time range; When a preset number of multiple blink events occur within the preset time range, a second control instruction is generated.

[0012] Optionally, it is characterized in that: The target applications include: comic reading applications; The target operations include: turning the page up and turning the page down.

[0013] Another aspect of an embodiment of the present application provides a control device, the device comprising: an acquisition module, configured to acquire a plurality of head images of a target object within a target time window while running the target application; a determination module, configured to determine a target action of the target object based on the multiple head images; a generating module, configured to generate a target control instruction according to the target action; and The control module is configured to control the target application to execute a corresponding target operation in response to the target control instruction.

[0014] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0015] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0016] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0017] The embodiments of the present application employing the above-mentioned technical solution offer the following advantages: by acquiring multiple head images of a target subject within a target time window and analyzing and determining head movements based on the temporal changes and motion trends between these multiple head images, the accuracy of target movement recognition can be effectively improved. Furthermore, the identified target movement can be used to generate desired target control instructions, enabling accurate control of the target application without requiring contact with the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0019] Figure 1 The following schematically shows an operating environment diagram of the control method according to the first embodiment of the present application; Figure 2 The flowchart of the control method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows the Figure 2 Sub-flowchart of step S202; Figure 4 Schematically shows the Figure 3 Sub-flowchart of step S302; Figure 5 Schematically shows the Figure 2 Sub-flowchart of step S202; Figure 6 Schematically shows the Figure 2 Sub-flowchart of step S204; Figure 7 Schematically illustrates an exemplary application of the control method according to the first embodiment of the present application; Figure 8 Schematically illustrates an exemplary application of the control method according to the first embodiment of the present application; Figure 9 A block diagram schematically shows a control device according to the second embodiment of the present application; and Figure 10 The following schematically shows a hardware architecture diagram of a computer device according to the third embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] It should be noted that the descriptions of "first", "second", etc. in the embodiments of the present application are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] It should be noted that in all aspects of this application involving data collection, storage, use, transmission, and processing, all data must strictly adhere to the laws, regulations, industry standards, and regulatory requirements of the data's source and use locations, as well as the relevant countries and regions, to ensure the legality and compliance of data activities. During the collection phase, the purpose, method, and scope of collection will be clearly stated to the data subject in a conspicuous manner. Collection will be carried out only after obtaining the data subject's legal authorization, ensuring that the collection process adheres to the "minimum necessary" principle and does not exceed the scope of data collection. During storage, the storage period will be limited, and after the storage purpose has been achieved, data will be promptly deleted, anonymized, or encrypted. During the use phase, strict data security protection mechanisms will be implemented, using field-level desensitization technology to process raw data according to pre-set desensitization rules. A variety of desensitization strategies, such as data generalization, data anonymization, and data encryption, will be employed for different data types to effectively mitigate the risk of sensitive information leakage and ensure that the data ultimately used is securely desensitized data, fully protecting the rights and interests of data subjects and data security. During the transmission and processing phases, the confidentiality and security of data will be ensured during transmission and processing.

[0023] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed. They are only used to facilitate the description of this application and to distinguish each step. Therefore, they cannot be understood as limitations on this application.

[0024] The present application provides a control technology solution. In this technology solution, (1) the pitch angle of the user's head in the normal position is dynamically calculated and the angle reference value is updated to resolve detection errors caused by different pitch angles of the user's normal head. (2) Specific recognition angles for nodding, nodding, and returning to the normal position are set to effectively ensure the timeliness and accuracy of the response to nodding behavior, while minimizing recognition failures that may be caused by device shaking. (3) By setting a specific blink recognition time and reset interval, the accurate recognition of single blinks and consecutive blinks is effectively guaranteed, reducing the user's somatosensory delay. See below for details.

[0025] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0026] like Figure 1As shown, client 10000 includes various types of electronic devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service kiosks, service robots, gaming systems, thin clients, various messaging devices, sensors, or other electronic devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as Google ChromeOS); or various mobile operating systems, such as Microsoft Windows, Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices can include cellular phones, smartphones, tablets, personal digital assistants, etc. Wearable devices can include head-mounted displays (such as smart glasses), etc. Gaming systems can include various handheld gaming devices and internet-enabled gaming devices. Client devices are capable of executing a variety of different applications, such as various internet-related applications, communication applications (such as email applications), and short message service (SMS) applications, and can use various communication protocols.

[0027] Client 10000 may include an input / output interface. The input interface may include a touchpad, a touch screen, a mouse, a keyboard, or other sensing elements. The input interface may be configured to receive user commands, which may cause client 10000 to perform various operations, such as controlling an application. The output interface is used to output information to the user, such as display information.

[0028] Based on the operating environment described above, the client 10000 may also be installed with one or more application programs, such as a comic reading program and an e-book reading program.

[0029] The following describes the technical solutions of the present application through multiple embodiments, taking the client 10000 as the execution subject. It should be noted that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments described here.

[0030] Example 1 Figure 2 The flowchart of the control method according to the first embodiment of the present application is schematically shown.

[0031] This control method is used in the client where the target application is installed, such as Figure 2 As shown, the control method may include steps S200 to S206, wherein: Step S200 : acquiring a plurality of head images of a target object within a target time window while running the target application.

[0032] Step S202: determining a target action of the target object according to the multiple head images.

[0033] Step S204: generating a target control instruction according to the target action.

[0034] Step S206 , in response to the target control instruction, controlling the target application to execute a corresponding target operation.

[0035] The control method provided in this embodiment effectively improves the accuracy of target motion recognition by acquiring multiple head images of a target subject within a target time window and analyzing and determining head motion based on the temporal changes and motion trends between these multiple head images. Furthermore, the identified target motion can be used to generate desired control instructions, achieving accurate control of the target application without requiring any contact with the device.

[0036] The following combination Figure 2 , each step in steps S200~S206 and other optional steps are described in detail.

[0037] Step S200 , while running the target application, acquiring multiple head images of the target object within a target time window.

[0038] The target application can be a comic book reading program, a video player program, a map navigation program, a slide show player program, etc., and is not limited here. A target subject can be photographed within a target time window using an image acquisition device such as a camera to generate multiple head images. Head images can be acquired in real time within the target time window, or at fixed time intervals. A time window refers to a time range within a predetermined or dynamically determined time period during which data is collected. For example, if a user uses a comic book reading program to read comics, multiple head images of the user can be obtained with the user's authorization to control the comic book reading.

[0039] Step S202 , determining a target action of the target object based on the multiple head images.

[0040] Multiple head images can be sorted chronologically to facilitate analysis and determination of the target object's target action through temporal changes and motion trends among the multiple head images. The target action can be a preset action with specific meaning. Multiple head images can be analyzed using a framework such as a multimedia machine learning framework to obtain key point data of the head, and then the motion trend of the target object can be analyzed based on the key point data of the head. The multimedia machine learning framework has low-latency image and video processing capabilities, can support target detection, tracking, and recognition tasks in dynamic scenes, and can be used in real-time computer vision (CV) and media processing fields.

[0041] The following will provide a further exemplary introduction to determining a target action based on multiple head images through more embodiments.

[0042] In an optional embodiment, if Figure 3 As shown, step S202 may include: Step S300: Acquire an angle sequence based on the multiple head images, where the head angle sequence includes multiple head pitch angles sorted in chronological order.

[0043] Step S302: determining an angle reference value according to the multiple head pitch angles.

[0044] Step S304: determining an angle reference range according to the angle reference value.

[0045] Step S306 : determining the target action of the target object according to the multiple head pitch angles and the angle reference range.

[0046] The corresponding head posture matrix can be obtained from the head image through the multimedia machine learning framework, and then the corresponding head pitch angle can be calculated based on the head posture matrix. The head posture matrix is ​​used to represent the direction (orientation) and position (coordinates) of the human head in three-dimensional space. The head pitch angle refers to the angle of up and down rotation with the line connecting the two ears as the axis, which is used to measure the degree of up and down rotation of the head. The angle reference value can be used to represent the specific reference value of the head in a static or default state. A tolerance range or error interval can be set based on the angle reference value to determine the angle reference range. The angle reference range can be used as a reference standard for judging whether the target object's head is in a normal position. In this embodiment, the angle reference value is dynamically calculated through multiple head pitch angles to effectively avoid recognition errors caused by different head pitch angles of different users in their normal state.

[0047] The following is a further exemplary introduction to determining the angle reference value. In an optional embodiment, as shown in FIG. Figure 4 As shown, step S302 may include: Step S400: Acquire a maximum head pitch angle and a minimum head pitch angle from the multiple head pitch angles.

[0048] Step S402 : averaging the maximum head pitch angle and the minimum head pitch angle to dynamically update the angle reference value.

[0049] The maximum head pitch angle and the minimum head pitch angle, as the extreme values ​​of the head pitch angle, can reflect the upper and lower limits of the target object's head rotation. The angle reference value can then be determined by taking the average of the maximum and minimum head pitch angles. Depending on actual needs, the angle reference value can be determined by taking the average of multiple head pitch angles, or by taking the median of multiple head pitch angles as the angle reference value. In practical applications, the angle reference value can be dynamically updated using a sliding time window. A sliding time window is a data processing strategy in time series analysis that is used to perform real-time analysis on continuously arriving data.

[0050] In this embodiment, since different subjects may have different natural postures, and the same subject may shift position due to changes in device height or posture, in this embodiment, the angle reference value is adjusted in real time using the current maximum and minimum head pitch angles to effectively avoid errors in target action recognition.

[0051] The target actions may include nodding up and nodding down, etc. The determination conditions of these two actions will be further explained below.

[0052] Regarding the upward nod, in an optional embodiment, step S306 may include: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be an upward nodding action: (1) a first head pitch angle in the angle sequence is higher than a first preset threshold value of the upper limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

[0053] As the head posture of the target object changes, the head pitch angle will also change. When a head pitch angle is higher than the angle reference range, it means that the target object's head has turned upward. Furthermore, in the process of the head turning upward, when there is a first head pitch angle that is higher than a first preset threshold value of the upper limit of the angle reference range, it can be determined that the target object has completed the head upward movement. When a second head pitch angle after the first head pitch angle is within the angle reference range, it means that the target object has completed the upward nodding movement. For example, in the process of the user turning his head upward, when the head pitch angle exceeds the angle reference value of 7°, it is determined that the user has completed the head upward movement in the upward nodding. When the user's head pitch angle returns to the range of ±5° of the angle reference value (i.e., the angle reference range), it is determined that the user has completed the complete upward nodding movement.

[0054] In this embodiment, the upward nod is an action in which the head first rotates upward by a certain angle and then returns to the initial head position. Therefore, in this embodiment, the sequence of changes in the head pitch angle over time and its relationship to the reference angle range are used to accurately identify the upward nod action and effectively avoid recognition failures caused by device shaking.

[0055] Regarding the nod, in an optional embodiment, step S306 may further include: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be a downward nodding action: (1) The first head pitch angle in the angle sequence is lower than a second preset threshold value of the lower limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

[0056] When a head pitch angle is lower than the angle reference range, it indicates that the target object's head has turned downward. Furthermore, in the process of the head turning downward, when there is a first head pitch angle that is lower than a second preset threshold value of the upper limit of the angle reference range, it can be determined that the target object has completed the head downward movement in the lower nod. When the second head pitch angle after the first head pitch angle is within the angle reference range, it indicates that the target object has completed the complete movement of the lower nod. For example, in the process of the user turning his head downward, when the head pitch angle is lower than the angle reference value of 6°, it is determined that the user has completed the head downward movement in the lower nod. When the user's head pitch angle returns to within the range of ±5° of the angle reference value, it is determined that the user has completed the complete movement of the lower nod.

[0057] In this embodiment, the downward nod is an action in which the head first rotates downward by a certain angle and then returns to the initial head position. Therefore, in this embodiment, the sequence of changes in the head pitch angle over time and its relationship to the angle reference range are used to accurately identify the downward nod action and effectively avoid recognition failures caused by device shaking.

[0058] Target actions can also include blinking.

[0059] In an optional embodiment, if Figure 5 As shown, step S202 may further include: Step S500: Acquire multiple eye position parameters based on the multiple head images; wherein the eye position parameters include the longitudinal distance and the lateral distance of the key points of the eyes.

[0060] Step S502: When the ratio of the longitudinal distance to the lateral distance of the eye position parameter is less than a third preset threshold, determine that the head image corresponding to the eye position parameter is in the eye-closed state; wherein one eye position parameter corresponds to one head image.

[0061] Step S504 : When a preset number of consecutive head images are all in the eyes-closed state, determining that the target action is a blink.

[0062] The corresponding eye key points can be obtained from the head image through a multimedia machine learning framework. The eye key points can be key coordinate points on the eye contour, such as the coordinates of the upper and lower eyelids and the coordinates of the left and right eye corners. The longitudinal distance of the eye key points can be the distance between the upper and lower eyelids, that is, the height of the eyes when they are open. The lateral distance of the eye key points can be the distance between the left and right eye corners, that is, the width of the eyes. When the eyes are open, the longitudinal distance is relatively large. When the eyes are closed, the longitudinal distance decreases or even approaches 0, while the lateral distance remains almost unchanged. Therefore, the ratio of the longitudinal distance of the eyes to the transverse distance can be used to determine whether the corresponding head image is in a closed-eye state. Since blinking is the action of opening and closing the eyelids, when the target subject is blinking, there may be multiple consecutive head images that all show the eyes closed. Therefore, it can be determined that the target subject is blinking when a preset number of consecutive head images show the eyes closed.

[0063] In this embodiment, accurate recognition of the target object's blinking action is achieved through the ratio of the longitudinal distance to the lateral distance of the eyes and the continuity of the eye closing state during the blinking process.

[0064] Step S204 , generating a target control instruction according to the target action.

[0065] The target action triggers the corresponding target control instruction. A control instruction can be a command that drives the execution of a certain operation or function. A control instruction can be a standardized signal, code, function call, message, string, event, or protocol instruction.

[0066] As mentioned above, the target action can be any action such as nodding up, nodding down, blinking (single or multiple times), etc.

[0067] In some embodiments, nodding up may correspond to a page-up control instruction, and nodding down may correspond to a page-down instruction. In other embodiments, nodding up or nodding down may correspond to other instructions, depending on user settings.

[0068] In some embodiments, a single blink corresponds to a page-up control instruction, and multiple blinks (e.g., three) correspond to a page-down control instruction. In other embodiments, one, two, three, four, etc. blinks within a unit of time may correspond to different instructions.

[0069] In an optional embodiment, if Figure 6 As shown, step S204 may include: Step S600: When a single blink occurs within a preset time range, a first control instruction is generated.

[0070] Step S602 : generating a second control instruction when there are a preset number of blinks within the preset time range.

[0071] A specific number of blinks can be set within a preset time range, and a corresponding target control instruction can be generated based on the specific number of blinks. In some embodiments, the first control instruction can be a page-up control instruction, and the second control instruction can be a page-down control instruction. For example, if there is only one blink within 600ms, the target action is determined to be a single blink, and a page-up control instruction is generated. If there are two consecutive blinks within 600ms, the target action is determined to be two blinks, and a page-down control instruction is generated.

[0072] In this embodiment, a specific number of blinks is set within a preset time range to effectively distinguish between a single blink and multiple blinks, thereby generating corresponding control instructions.

[0073] Step S206 , in response to the target control instruction, controlling the target application to perform a corresponding target operation.

[0074] In some embodiments, when the target application is a comic reading program, an e-book reading program, a slide show program, or other program that requires frequent page turning, the target control instruction may be a page up control instruction or a page down control instruction.

[0075] In an optional embodiment, the target application includes: a comic reading program; the target operation includes: turning the page up and turning the page down.

[0076] In order to make this application easier to understand, the following Figure 7 、 8 An exemplary application is provided.

[0077] 1. As Figure 7 As shown, it is taken as an example that the user controls the target application to turn pages by nodding up and down.

[0078] S1. With the user's authorization, obtain multiple head images of the user (i.e., the target object) to obtain multiple user face data.

[0079] S2. Calculate the user's head pitch angle (i.e., obtain multiple head pitch angles).

[0080] S3. Obtain the maximum / minimum value of the rotation pitch angle and take the average value as the pitch reference value.

[0081] That is, the maximum head pitch angle and the minimum head pitch angle are obtained, an average of the maximum head pitch angle and the minimum head pitch angle is taken, and the angle reference value is dynamically updated.

[0082] The following steps S4-S6 are met to determine that the target action is an upward nod action and issue a page-up instruction: S4. When the user's head is turned upward and the corresponding head pitch angle exceeds the Pitch reference value of 7°, it is determined that the user has completed the head-up action.

[0083] That is, when the first head pitch angle in the angle sequence is higher than the first preset threshold of the upper limit of the angle reference range, it is determined that the user has completed the head-up action.

[0084] S5. When the pitch angle of the user's head returns to the range of ±5° of the Pitch reference value, it is determined that the user has completed the upward nodding action.

[0085] That is, the second head pitch angle in the angle sequence is within the angle reference range, and it is determined that the user has completed the upward nodding action.

[0086] S6. Issue a page-up instruction.

[0087] The following steps S7-S9 are met to determine that the target action is to nod down and issue a next page turning instruction: S7. When the user's head is turned downward and the corresponding head pitch angle exceeds the Pitch reference value of 6°, it is determined that the user has completed the head-down action.

[0088] That is, when the first head pitch angle in the angle sequence is lower than the second preset threshold of the lower limit of the angle reference range, it is determined that the user has completed the head downward action.

[0089] S8. When the pitch angle of the user's head returns to the range of ±5° of the Pitch reference value, it is determined that the user has completed the nodding action.

[0090] That is, the second head pitch angle in the angle sequence is within the angle reference range, and it is determined that the user has completed the nodding action.

[0091] S9. Issue a page-down instruction.

[0092] 2. If Figure 8 As shown, it is taken as an example that the user controls the target application to turn pages by blinking.

[0093] S1. With the user's authorization, obtain multiple head images of the user (i.e., the target object) to obtain multiple user face data.

[0094] S2. Obtain the upper and lower eyelid distance and eye width corresponding to the head image.

[0095] That is, a plurality of eye position parameters are obtained, where the eye position parameters include the longitudinal distance and the lateral distance of the eye key points.

[0096] S3. When the ratio of the upper and lower eyelid distance to the eyelid width is less than 0.2, it is determined that the corresponding head image is eyes closed.

[0097] That is, when the ratio of the longitudinal distance to the lateral distance of the eye position parameter is less than the third preset threshold, it is determined that the head image corresponding to the eye position parameter is in the eye-closed state.

[0098] S4. After continuously detecting three frames of head images in the eyes-closed state, determine that the user is blinking.

[0099] That is, when a preset number of consecutive head images are all in the eyes-closed state, the target action is determined to be blinking.

[0100] The following steps S5-S6 are met to determine that the target action is a single blink to issue a page-up instruction: S5. If there is no blink within 600ms, a single blink is triggered (i.e., there is a single blink within the preset time range).

[0101] S6. Issue a page-up instruction.

[0102] The following steps S7-S8 are met to determine that the target action is two blinks and issue a page-down instruction: S7. If there is a blink within 600ms, two blinks are triggered (i.e., there are a preset number of multiple blinks within the preset time range).

[0103] S8. Issue a page-down instruction.

[0104] In this exemplary application, the identified target action can generate the expected target control instructions, so as to achieve accurate control of the target application (such as a comic reading program) without having to touch the device.

[0105] Example 2 Figure 9 The block diagram of the control device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 9 As shown, the apparatus 900 may include: an acquisition module 910, a determination module 920, a generation module 930, and a control module 940, wherein: An acquisition module 910 is configured to acquire a plurality of head images of a target object within a target time window while running the target application; A determination module 920, configured to determine a target action of the target object based on the multiple head images; A generating module 930, configured to generate a target control instruction according to the target action; and The control module 940 is configured to control the target application to execute a corresponding target operation in response to the target control instruction.

[0106] As an optional embodiment, the determining module 920 is further configured to: acquiring an angle sequence based on the multiple head images, wherein the head angle sequence includes multiple head pitch angles sorted in chronological order; determining an angle reference value according to the multiple head pitch angles; determining an angle reference range according to the angle reference value; A target action of the target object is determined according to the multiple head pitch angles and the angle reference range.

[0107] As an optional embodiment, the determining module 920 is further configured to: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be an upward nodding action: (1) a first head pitch angle in the angle sequence is higher than a first preset threshold value of the upper limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

[0108] As an optional embodiment, the determining module 920 is further configured to: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be a downward nodding action: (1) The first head pitch angle in the angle sequence is lower than a second preset threshold value of the lower limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

[0109] As an optional embodiment, the determining module 920 is further configured to: acquiring a maximum head pitch angle and a minimum head pitch angle from the plurality of head pitch angles; An average value is taken for the maximum head pitch angle and the minimum head pitch angle, and the angle reference value is dynamically updated.

[0110] As an optional embodiment, the determining module 920 is further configured to: Acquire multiple eye position parameters according to the multiple head images; wherein the eye position parameters include longitudinal distances and lateral distances of key points of the eyes; When the ratio of the longitudinal distance to the lateral distance of the eye position parameter is less than a third preset threshold, determining that the head image corresponding to the eye position parameter is in an eye-closed state; wherein one eye position parameter corresponds to one head image; When a preset number of consecutive head images are all in the eyes-closed state, the target action is determined to be blinking.

[0111] As an optional embodiment, the generating module 930 is further configured to: generating a first control instruction when a single blink occurs within a preset time range; When a preset number of blinks occur within the preset time range, a second control instruction is generated.

[0112] As an optional embodiment, the target application includes: a comic reading program; the target operation includes: turning the page up and turning the page down.

[0113] Example 3 Figure 10The following schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing the control method according to the third embodiment of the present application. In some embodiments, the computer device 10000 can be a terminal device such as a smart phone, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. Figure 10 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other via a system bus. Memory 10010 includes at least one type of computer-readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, and the like. In some embodiments, memory 10010 may be an internal storage module of computer device 10000, such as a hard disk or memory of computer device 10000. In other embodiments, memory 10010 may also be an external storage device of computer device 10000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, and the like equipped on computer device 10000. Of course, memory 10010 may also include both internal storage modules and external storage devices of computer device 10000. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed on the computer device 10000, such as program codes of control methods, etc. In addition, the memory 10010 can also be used to temporarily store various data that has been output or is to be output.

[0114] In some embodiments, processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data exchange or communication with computer device 10000. In this embodiment, processor 10020 is used to execute program code stored in memory 10010 or process data.

[0115] Network interface 10030 may include a wireless network interface or a wired network interface. Network interface 10030 is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and a communication link between computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, or Wi-Fi.

[0116] It should be pointed out that Figure 10 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementing all of the shown components is not a requirement, and more or fewer components may alternatively be implemented.

[0117] In this embodiment, the control method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0118] Example 4 An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the control method in the embodiment when executed by a processor.

[0119] In this embodiment, computer-readable storage media include flash memory, hard disks, multimedia cards, card-type memories (e.g., SD or DX memories), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disks, optical disks, and the like. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the computer device's hard disk or memory. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, and the like. Of course, the computer-readable storage medium may also include both the internal storage unit and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the control method described in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that has been output or is about to be output.

[0120] Example 5 An embodiment of the present application further provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0121] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented using general-purpose computer devices. They can be concentrated on a single computer device or distributed on a network composed of multiple computer devices. Alternatively, they can be implemented using program codes executable by the computer device, so that they can be stored in a storage device and executed by the computer device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0122] It should be noted that the above are only preferred embodiments of the present application and do not limit the scope of patent protection of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the scope of patent protection of the present application.

Claims

1. A control method, characterized in that: Used in a client having a target application installed, the method includes: While running the target application, acquiring multiple head images of a target object within a target time window; determining a target action of the target object based on the multiple head images; Generate a target control instruction according to the target action; and In response to the target control instruction, the target application is controlled to perform a corresponding target operation.

2. The method according to claim 1, characterized in that Determining a target action of the target object according to the multiple head images includes: acquiring an angle sequence based on the multiple head images, wherein the head angle sequence includes multiple head pitch angles sorted in chronological order; determining an angle reference value according to the multiple head pitch angles; determining an angle reference range according to the angle reference value; A target action of the target object is determined according to the multiple head pitch angles and the angle reference range.

3. The method according to claim 2, characterized in that Determining a target action of the target object according to the multiple head pitch angles and the angle reference range includes: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be an upward nodding action: (1) a first head pitch angle in the angle sequence is higher than a first preset threshold value of the upper limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

4. The method according to claim 2, characterized in that Determining a target action of the target object according to the multiple head pitch angles and the angle reference range includes: If the following conditions are simultaneously met at the multiple head pitch angles, the target action is determined to be a downward nodding action: (1) The first head pitch angle in the angle sequence is lower than a second preset threshold value of the lower limit of the angle reference range; (2) a second head pitch angle in the angle sequence is within the angle reference range; (3) In the angle sequence, the second head pitch angle is located after the first head pitch angle.

5. The method according to claim 2, characterized in that Determining an angle reference value according to the multiple head pitch angles includes: acquiring a maximum head pitch angle and a minimum head pitch angle from the plurality of head pitch angles; An average value is taken for the maximum head pitch angle and the minimum head pitch angle, and the angle reference value is dynamically updated.

6. The method according to claim 1, characterized in that Determining a target action of the target object according to the multiple head images includes: Acquire multiple eye position parameters according to the multiple head images; wherein the eye position parameters include longitudinal distances and lateral distances of key points of the eyes; When the ratio of the longitudinal distance to the lateral distance of the eye position parameter is less than a third preset threshold, determining that the head image corresponding to the eye position parameter is in an eye-closed state; wherein one eye position parameter corresponds to one head image; When a preset number of consecutive head images are all in the eyes-closed state, the target action is determined to be blinking.

7. The method according to claim 6, characterized in that Generating a target control instruction according to the target action includes: generating a first control instruction when a single blink occurs within a preset time range; When a preset number of blinks occur within the preset time range, a second control instruction is generated.

8. The method according to any one of claims 1 to 7, characterized in that: The target applications include: comic reading applications; The target operations include: turning the page up and turning the page down.

9. A control device, characterized in that: The device comprises: an acquisition module, configured to acquire a plurality of head images of a target object within a target time window while running the target application; a determination module, configured to determine a target action of the target object based on the multiple head images; A generating module, configured to generate a target control instruction according to the target action; and The control module is configured to control the target application to execute a corresponding target operation in response to the target control instruction.

10. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

Images