Drawing device control method, electronic drawing device, storage medium and computer program product

Abstract

The invention provides a drawing device control method, an electronic drawing device, a storage medium and a computer program product. The method comprises the following steps: starting a camera to shoot a face image; recognizing a cheek action in the face image and judging whether the cheek action is a preset cheek action or not; under the condition that the cheek action is a preset cheek action, determining a selection operation, and obtaining a shortcut key corresponding to the selection operation; and controlling a drawing program of the electronic drawing device to execute corresponding operation according to the shortcut key. By utilizing the method and the device, the dependence on entity hardware is reduced, the hand operation of the user during drawing can be reduced, and the efficient interaction between the user and the electronic drawing device is facilitated.

Description

Technical Field

[0001] The present application relates to the technical field of digital product interaction, and in particular to a painting device control method, an electronic painting device, a storage medium, and a computer program product. Background Art

[0002] When using existing electronic drawing devices, such as pen tablets and pen displays, users typically hold the pen in one hand while using the other to operate shortcut keys in the drawing software to zoom in, zoom out, rotate, and save their work. Consequently, existing electronic drawing devices typically include physical buttons, sliders, knobs, or touch screens. However, these hardware modules for user interaction require manual operation, increasing user inconvenience and hindering efficient interaction. Furthermore, the addition of hardware modules increases costs and operational complexity.

[0003] In addition, the relevant shortcut keys are bound to fixed actions and lack dynamic customization capabilities, making them unable to adapt to complex creative needs and personal habits.

[0004] Therefore, there is an urgent need to provide a drawing device control method, an electronic drawing device, a storage medium, and a computer program product to improve the convenience of users using the electronic drawing device to draw and / or reduce the cost of the electronic drawing device. Summary of the Invention

[0005] Embodiments of the present application provide a drawing device control method, an electronic drawing device, a storage medium, and a computer program product.

[0006] In a first aspect, an embodiment of the present application provides a method for controlling a painting device, the method comprising:

[0007] Enable the camera to capture facial images;

[0008] Recognizing cheek movements in a face image and determining whether the cheek movements are preset cheek movements;

[0009] In a case where the cheek movement is a preset cheek movement, determining a selected operation and obtaining a shortcut key corresponding to the selected operation;

[0010] The drawing program of the electronic drawing device is controlled to perform corresponding operations according to the shortcut keys.

[0011] In one embodiment, only when the duration of the cheek movement reaches a preset value is it determined whether the cheek movement is a preset cheek movement, and / or the preset cheek movement is used as the selected operation.

[0012] In one embodiment, when the cheek movement is a preset cheek movement, the duration of the preset cheek movement is further determined. When the duration of the preset cheek movement reaches a preset value, the selected operation is determined based on the preset cheek operation, and / or the preset cheek movement is used as the selected operation.

[0013] In one embodiment, the method further comprises:

[0014] Acquire the user's operations on the electronic drawing device in real time; and

[0015] determining whether the combination of the preset cheek operation and the operation performed by the user on the electronic painting device is a preset combination; and

[0016] The determining of the selected operation based on the preset cheek movement includes:

[0017] When the combination is a preset combination, the preset combination is used as the selected operation.

[0018] In one embodiment, the method further comprises:

[0019] Analyze the user's action sequence and predict the user's next intention;

[0020] A corresponding shortcut key is obtained according to the predicted user intention, and the painting program is controlled to execute a corresponding function according to the corresponding shortcut key.

[0021] In one embodiment, the method further comprises:

[0022] Identify the frequency and / or intensity of user actions; and

[0023] Adjusting a threshold for triggering recognition of the user operation according to the frequency and / or intensity of the user operation;

[0024] The user operation is one of cheek movements, operations performed by the user on the electronic painting device, and a preset combination.

[0025] In one embodiment, the method further comprises:

[0026] In response to two or more operations of a user obtained simultaneously, obtaining a preset priority rule, wherein the priority rule defines a priority order of the user operations; and

[0027] According to the priority order, the operation with the highest priority among the two or more operations is selected as the selected operation.

[0028] In one embodiment, the method further comprises:

[0029] In response to two or more operations of a user obtained simultaneously, obtaining a preset priority rule, wherein the priority rule defines a priority order of the user operations;

[0030] Determining whether a preset combination exists among the two or more operations;

[0031] When there are preset combinations in the multiple operations, determining whether the number of the preset combinations is more than one;

[0032] When the number of the preset combinations is more than one, determining whether there are other operations besides the preset combinations;

[0033] When the other operations still exist, the preset combination or operation with the highest priority is selected from the preset combinations and the other operations according to the priority order as the selected operation.

[0034] In one embodiment, the method further comprises:

[0035] When the other operations do not exist, the preset combination with the highest priority is selected from the preset combinations according to the priority order as the selected operation.

[0036] In one embodiment, the method further comprises:

[0037] When there is no preset combination among the two or more operations, the operation with the highest priority is selected from the two or more operations as the selected operation according to the priority order.

[0038] In one embodiment, the method further comprises:

[0039] When there is only one preset combination, determining whether there are other operations besides the preset combination;

[0040] When the other operations do not exist, the preset combination is used as the selected operation.

[0041] In one embodiment, the method further comprises:

[0042] When the other operations exist, the preset combination or operation with the highest priority is selected from the preset combination and the other operations according to the priority order as the selected operation.

[0043] In one embodiment, before recognizing the cheek movements in the face image, a current creative scene is obtained from a painting program, and the selected operation is related to the current creative scene.

[0044] In a second aspect, the present application further provides a painting device control device, the device comprising:

[0045] a memory configured to store computer-executable instructions; and

[0046] The controller is configured to run the computer executable instructions to perform the above-mentioned painting device control method.

[0047] In a third aspect, the present application further provides a storage medium storing computer-executable instructions, wherein the computer-executable instructions are executed by a processing unit to implement the above-mentioned painting device control method.

[0048] In a fourth aspect, the present application further provides a computer program product, comprising computer program instructions, which implement the above-mentioned painting device control method when executed by a processing unit.

[0049] Compared with the existing technology, the painting device control method, electronic painting device, storage medium and computer program product provided by this application reduce the dependence on physical hardware by setting cheek movements to correspond to shortcut keys, and can reduce the user's hand operations when painting, making it convenient for the user to achieve efficient interaction with the electronic painting device. In addition, the user can adapt to complex creative needs and adapt to the user's own habits by customizing the actions corresponding to shortcut keys. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0051] Figure 1 This is a schematic diagram of the hardware architecture of an electronic painting device provided in one embodiment of the present application;

[0052] Figure 2 This is a flow chart of a method for controlling a painting device provided in the first embodiment of the present application;

[0053] Figure 3 2 is a flow chart of a method for controlling a painting device according to a second embodiment of the present application;

[0054] Figure 4 2 is a flow chart of a method for controlling a painting device according to a third embodiment of the present application;

[0055] Figure 5A 、 Figure 5B 1 is a flow chart of the control method of the painting device provided in the fourth and fifth embodiments of the present application;

[0056] Figure 61 is a flow chart of a method for controlling a painting device according to a sixth embodiment of the present application;

[0057] Figure 7 2 is a flow chart of a method for controlling a painting device according to a seventh embodiment of the present application;

[0058] Figure 8 2 is a flow chart of a method for controlling a painting device according to an eighth embodiment of the present application;

[0059] Figure 9 It is a flowchart of a method for controlling a painting device provided in the ninth embodiment of the present application. DETAILED DESCRIPTION

[0060] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0061] See also Figure 1Figure 1 is a schematic diagram of the hardware architecture of an electronic drawing device according to a first embodiment of the present application. The electronic drawing device 1 includes a drawing screen 11, a camera 13, a processor 15, and a memory 17. The drawing screen 11 has both input and output functions, namely, receiving and displaying input from a user operating a paintbrush. The camera 13 is located on the frame of the electronic drawing device 1, facing the user. While the user is drawing, the camera 13 is activated to capture the user's face. In other embodiments, the camera 13 can be located elsewhere, such as on the drawing screen 11. The specific location of the camera 13 is not limited, as long as it can capture the user's face. The drawing screen 11, camera 13, and memory 17 are all connected to the processor 15. The memory 17 stores a drawing program 171 and a drawing device control program 173. The processor 15 runs the drawing program 171, receives input from the user operating a paintbrush via the drawing screen 11, processes the input, and displays it on the drawing screen 11, thereby implementing the drawing function. In this embodiment, the processor 15 simultaneously runs the drawing device control program 173, receives a facial image captured by the camera 13, identifies cheek movements of the face, and implements shortcut key functions in the drawing program based on the identified cheek movements. Specifically, in one embodiment, the processor 15 runs the drawing device control program 173, and after identifying cheek movements of the face, maps the identified cheek movements, or a preset combination of the identified cheek movements and other operations, to a shortcut key or shortcut key combination (which may be a virtual key) in the drawing program 171 according to a predefined correspondence. The processor 15 then sends the key value of the corresponding shortcut key or shortcut key combination to the drawing program 171, causing the drawing program 171 to implement the function of the shortcut key or shortcut key combination. In another embodiment, the processor 15 runs the drawing device control program 173, and after identifying cheek movements of the face, invokes a shortcut key remapping function in the drawing program, thereby implementing the function of the corresponding shortcut key or shortcut key combination through the shortcut key remapping function.

[0062] See also Figure 2 FIG. 1 is a flow chart of a method for controlling a painting device according to an embodiment of the present application.

[0063] In operation 201 , a camera is enabled to capture a face image.

[0064] In one embodiment, the camera is activated when the associated painting program is activated, and is deactivated when the associated painting program is deactivated. That is, when the associated painting program is detected to be activated, the camera is automatically activated, and when the associated painting program is detected to be deactivated, the camera is automatically deactivated. A digital painting device may have multiple painting programs installed. These multiple painting programs may all be applicable to the painting device control method provided in this application, i.e., their shortcut key functions can be controlled using the painting device control method provided in this application, or they may only partially be applicable to the painting device control method provided in this application. Therefore, painting programs whose shortcut key functions can be controlled using the painting device control method provided in this application are referred to as "associated painting programs" in this application.

[0065] In one embodiment, when it is detected that multiple associated painting programs are all opened, the camera is turned on following the opening of the first opened painting program and is turned off following the closing of the last closed painting program.

[0066] In one embodiment, the camera may be activated by a user via a physical button provided on the painting device or an icon provided on the interface of the painting device.

[0067] In operation 203 , cheek movements in the face image are identified and it is determined whether the cheek movements are preset cheek movements.

[0068] In one embodiment, the preset cheek movements include "puffing the left cheek," "puffing the right cheek," "puffing both cheeks simultaneously," "closing both cheeks simultaneously," "closing the left cheek," and "closing the right cheek." Multiple images of the user puffing the left cheek, puffing the right cheek, puffing both cheeks simultaneously, closing both cheeks simultaneously, closing the left cheek, and closing the right cheek are acquired in advance. A CNN artificial neural network is used to train a movement model, and deep learning methods are used to perform facial recognition to identify the preset cheek movements.

[0069] In one embodiment, facial key points are detected to identify whether a pre-set cheek movement is present in a facial image. Typically, cheek movement changes cause the shape and / or position of facial key points to change. Therefore, cheek movements can be identified by identifying changes in the shape and / or position of key points. For example, using the mouth as a key point, changes in the mouth's position can be used to determine whether the cheek is puffing out on the left or puffing out on the right. Changes in the mouth's shape can be used to determine whether the cheek is puffing out or puckering, thereby determining whether both cheeks are puffing out or puckering simultaneously. Combining changes in shape and position can determine whether the cheek is puffing out on the left or puckering on the left, or puffing out on the right or puckering on the right. In another embodiment, facial contour detection is used to identify whether a pre-set cheek movement is present in a facial image. Typically, cheek movement changes the facial contour. For example, puffing out the cheeks will cause the cheek contour to expand, while puckering will cause the cheek contour to shrink.

[0070] The following describes a specific implementation of how to identify whether a preset cheek movement exists in a facial image.

[0071] Use OpenCV's Haar cascades classifier to locate facial regions, compare the normal facial contour, and analyze cheek deformation. This can be determined by calculating the area between key regions. Specific image processing and recognition techniques include the following aspects.

[0072] Preprocessing

[0073] Facial images are preprocessed, including normalization, denoising, and posture correction. Normalization includes grayscaling and histogram equalization. Denoising uses Gaussian filtering to smooth the image, preserving edge features, such as the facial outline and key points of the mouth. Posture correction scales and rotates the image to remove excess parts, making each facial image uniform or uniform in size to facilitate subsequent similarity comparison. Afterwards, facial posture is unified through affine transformation. Specifically, correction can be performed within a ±15° pitch angle range to accommodate different facial angles during capture.

[0074] Edge extraction

[0075] Through differential operator extraction, such as using the Sobel algorithm to extract the outer contour of the face or the contour of key points, in some embodiments, initial calibration can be further performed, that is, removing the non-external contour part and dividing the face image after removing the non-external contour part into regions, and calculating the area of ​​the specific region.

[0076] Dynamic contour tracking

[0077] In one embodiment, after a predetermined number of sequentially captured facial photos undergo the aforementioned processing, the facial photos are compared to obtain changes in the positions and / or shapes of key points. When the changes reach or exceed a predetermined value, or when the differences between the changed positions and / or shapes and a predetermined position (or a position obtained through deep learning) and / or a predetermined shape (or a shape obtained through deep learning) are less than or equal to a predetermined value, corresponding cheek movements are identified based on the changes in the positions and / or shapes of the key points. For example, if the key point is the mouth, corresponding cheek movements are identified based on changes in the position and / or shape of the mouth.

[0078] In one embodiment, after a predetermined number of facial photos are sequentially taken and processed as described above, the area of ​​a specific region is compared to determine the change in the specific region's area. When the change reaches or exceeds a predetermined value, a corresponding cheek movement is identified based on the change in the specific region's area. For example, if the specific region is the entire face, when the entire facial area increases by a predetermined value (e.g., 10%), the cheek movement is determined to be simultaneous puffing of both cheeks; and when the entire facial area decreases by a predetermined value (e.g., 10%), the cheek movement is determined to be simultaneous retraction of both cheeks.

[0079] In the case that the cheek movement is a preset cheek movement, in operation 205 , a selected operation is determined, and a shortcut key corresponding to the selected operation is obtained.

[0080] In this embodiment, if the cheek movement is not a preset cheek movement, the process returns to operation 203 .

[0081] In one embodiment, the preset cheek movement is directly used as the selection operation, and the shortcut key corresponding to the selection operation is the shortcut key corresponding to the preset cheek movement.

[0082] In one embodiment, an interface is provided for the user to directly preset the correspondence between cheek movements and shortcut keys (shortcut keys and shortcut key combinations are collectively referred to as shortcut keys in this application), such as Figure 3 As shown, six corresponding relationships between cheek actions and shortcut keys are preset, including: "Puff out the left cheek" corresponds to shortcut key A, and the function of shortcut key A is to swipe the canvas to the left; "Puff out the right cheek" corresponds to shortcut key B, and the function of shortcut key B is to swipe the canvas to the right; "Puff out both cheeks at the same time" corresponds to shortcut key C, and the function of shortcut key C is to zoom in on the canvas; "Pinch in both cheeks at the same time" corresponds to shortcut key D, and the function of shortcut key D is to zoom out on the canvas; "Pinch in the left cheek" corresponds to shortcut key E, and the function of shortcut key E is to slide the canvas up; "Pinch in the right cheek" corresponds to shortcut key F, and the function of shortcut key F is to slide the canvas down.

[0083] In one embodiment, a machine learning algorithm automatically establishes a correspondence between a user's cheek movements and the shortcut keys they operate. For example, in the method provided herein, a learning phase is also predefined. After the user initiates the learning phase, the drawing device control program running on the device automatically establishes a correspondence between the cheek movements and the shortcut keys that correspond to the user's cheek movements in time. For example, during the learning phase, if the user is detected repeatedly pressing shortcut key A while puffing their left cheek, a correspondence between puffing their left cheek and shortcut key A is automatically established.

[0084] In operation 207 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0085] In one embodiment, the key value corresponding to the shortcut key is sent to a painting program so that the painting program executes the corresponding function.

[0086] In another embodiment, a remapping function of a shortcut key in a painting program is called, and the painting program is controlled to perform a corresponding operation through the remapping function.

[0087] In this implementation, using the user's cheek movements to execute shortcut functions within the painting program reduces the number of physical buttons, thereby reducing hardware costs and operational complexity. Furthermore, it prevents the user's hands from being occupied with tools, enabling efficient interaction. Furthermore, through machine learning, dynamic custom shortcut keys are enabled to accommodate complex creative needs.

[0088] See also Figure 3 FIG. 1 is a flow chart of a control method for a painting device according to a second embodiment of the present application.

[0089] In operation 301 , a camera is started to capture a face image.

[0090] In operation 303 , cheek movements in the face image are recognized and the duration of the cheek movements is detected.

[0091] In operation 305 , when the duration of the cheek movement reaches a preset value, it is determined whether the cheek movement is a preset cheek movement.

[0092] In this embodiment, the duration of the same cheek movement is determined based on a preset number of facial photos taken sequentially. The preset number can be 2 frames, or 3 frames, 4 frames or more frames. Only when the duration of the same cheek movement reaches a preset value (for example, the same cheek movement continues to appear in 2 frames or 3 frames or more frames taken sequentially), is it determined whether the cheek movement is the preset cheek movement.

[0093] When the cheek movement is a preset cheek movement, in operation 307 , a selected operation is determined, and a shortcut key corresponding to the selected operation is obtained.

[0094] In this embodiment, when the cheek movement is not a preset cheek movement, the process returns to operation 303 .

[0095] In operation 309 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0096] In this embodiment, the duration of the cheek movement is further considered. Only after the cheek movement lasts for a preset time is it determined whether the cheek movement is a preset movement. In this way, the user's inadvertent facial movements are avoided from being responded to and affecting the user's drawing.

[0097] For technical details not introduced in this embodiment, please refer to the introduction in the first embodiment and will not be repeated here.

[0098] See also Figure 4 FIG. 1 is a flow chart of a control method for a painting device according to a third embodiment of the present application.

[0099] In operation 401 , a camera is started to capture a face image.

[0100] In operation 403 , cheek movements in the face image are identified and it is determined whether the cheek movements are preset cheek movements.

[0101] When the cheek movement is a preset cheek movement, the process returns to operation 403 .

[0102] When the cheek movement is a preset cheek movement, in operation 405 , the duration of the preset cheek movement is detected to determine whether the duration of the preset cheek movement reaches a preset value.

[0103] When the duration of the preset cheek movement reaches a preset value, in operation 407 , a selected operation is determined and a shortcut key corresponding to the selected operation is obtained.

[0104] In the present embodiment, after only judging that cheek movement is the default cheek movement, just continue to monitor the duration of this cheek movement, only when the duration of this cheek movement reaches a preset value, just proceed to the operation of next step. When the duration of this cheek movement does not reach this preset value, process returns to operation 403.

[0105] In operation 409 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0106] In this embodiment, the duration of the preset cheek movement is further considered, and the shortcut key corresponding to the preset cheek movement is obtained only after the preset cheek movement continues for a preset time. In this way, the user's inadvertent facial movements are avoided from being responded to, which affects the user's painting.

[0107] For technical details not introduced in this embodiment, please refer to the introduction in the first embodiment and will not be repeated here.

[0108] See also Figure 5A and Figure 5B , which is a flow chart of the control method of the painting device provided in the fourth and fifth embodiments of the present application.

[0109] The fourth and fifth embodiments further include the following operations based on the preset cheek movements obtained in the above embodiments.

[0110] In operation 501 , operations performed by a user on an electronic painting device are acquired in real time.

[0111] In operation 503 , it is determined whether the combination of the preset cheek operation and the operation performed by the user on the electronic painting device is a preset combination.

[0112] When the combination is a preset combination, in operation 505 , the preset combination is used as a selected operation and a shortcut key corresponding to the selected operation is obtained.

[0113] In this embodiment, the operation performed by the user on the electronic drawing device may be any of the following: clicking, sliding, etc. on the drawing screen, and clicking, toggling, etc. on mechanical keys, which can be recognized by the electronic drawing device.

[0114] In the fourth and fifth embodiments, if the combination is not a preset combination, the process ends as shown in the fourth embodiment, or, as in operation 506 of the fifth embodiment, the operation with the higher priority is selected as the selected operation according to the preset priority order to obtain the corresponding shortcut key. For example, if the priority of the preset cheek movement is higher than the priority of the operation performed by the user on the electronic painting device, the preset cheek movement is selected as the selected operation and the response is to the preset cheek movement. Otherwise, the operation performed by the user on the electronic painting device is selected as the selected operation and the response is to the operation performed by the user on the electronic painting device.

[0115] In operation 507 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0116] In this embodiment, cheek movements are further combined with operations performed by the user on the electronic painting device. When a preset combination is detected, the shortcut key function corresponding to the preset combination is executed, thereby further increasing the number of shortcut operations and improving the user's drawing convenience. Examples of the preset combinations of cheek movements and operations performed by the user on the electronic painting device, as well as their corresponding shortcut keys and shortcut key functions, are as follows:

[0117] The combination of "pushing left cheek" and the user clicking the drawing screen of the electronic drawing device with a stylus corresponds to the shortcut key "Ctrl+P" to quickly switch the brush type;

[0118] The combination of "puffing the right cheek" and sliding the stylus on the drawing screen of the electronic drawing device corresponds to the shortcut key "Ctrl+[ / ]" to adjust the brush thickness;

[0119] The combination of "pinching the left and right cheeks" and the user hovering the stylus above the drawing screen of the electronic drawing device corresponds to the shortcut key "Ctrl+Z", which enables the quick undo function.

[0120] The recognition of the operation performed by the user on the electronic drawing device can be performed by relying on existing technologies, such as through camera recognition, through drawing screen sensing recognition, etc., which is not limited here.

[0121] See also Figure 6 , which is a flow chart of a method for controlling a painting device according to a sixth embodiment of the present application.

[0122] This embodiment further includes the following operations based on any of the above embodiments.

[0123] In operation 601 , the user's action sequence is analyzed to predict the user's next intention.

[0124] In operation 603 , a corresponding shortcut key is acquired according to the predicted user intention, and the painting program is controlled to execute a corresponding function according to the corresponding shortcut key.

[0125] In this embodiment, by predicting the user's next intention and automatically executing the next function, the intelligence of the electronic painting device is improved, further facilitating user convenience. Specifically, in this embodiment, an LSTM or Transformer model can be used to analyze the user's action sequence. For example, if the user puffs their left cheek three times in a row to enlarge the canvas, it is predicted that the user may need to further enlarge the canvas. The shortcut key value corresponding to the "enlarge canvas" function is automatically obtained and sent to the painting software. Based on the received shortcut key value, the painting software automatically loads the enlargement tool to further enlarge the canvas. Alternatively, the painting software directly further enlarges the canvas based on the received shortcut key value.

[0126] See also Figure 7 , which is a flow chart of a method for controlling a painting device according to the seventh embodiment of the present application.

[0127] This embodiment further includes the following operations based on any of the above embodiments.

[0128] In operation 701 , in response to two or more operations of a user obtained simultaneously, a preset priority rule is obtained, where the priority rule defines a priority order of the user operations.

[0129] In operation 703 , according to the priority order, the operation with the highest priority among the two or more operations is selected as the selected operation.

[0130] In operation 705 , a shortcut key corresponding to the selected operation is obtained.

[0131] In operation 707 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0132] In this embodiment, by pre-setting priority rules, when a user simultaneously performs multiple operations, the highest priority operation is responded to. This prevents the electronic drawing device from becoming unresponsive or causing system disruptions due to too many simultaneous operations. For example, if a user is using the stylus to perform a specific operation, and the electronic drawing device also detects the user performing a preset cheek puffing motion, the pre-set priority rules can be used to ensure that the electronic drawing device responds to the higher priority operation, thus preventing unresponsiveness or system disruptions.

[0133] In this embodiment, the priority rules can be pre-set by the user or automatically generated through machine deep learning.

[0134] In this embodiment, "acquired simultaneously" means acquired within a preset time interval.

[0135] See also Figure 8 , which is a flow chart of a method for controlling a painting device according to an eighth embodiment of the present application.

[0136] This embodiment further includes the following operations based on any of the above embodiments.

[0137] In operation 801 , in response to two or more operations of a user obtained simultaneously, a preset priority rule is obtained, where the priority rule defines a priority order of the user operations.

[0138] In this embodiment, the user operation may be a preset cheek movement, a preset user click, slide, etc. on the drawing screen, and a click, toggle, etc. on a mechanical button.

[0139] In operation 802 , it is determined whether there is a preset combination among the two or more operations.

[0140] When there is a preset combination in the two or more operations, the process proceeds to operation 803 to determine whether the number of the preset combination is more than one.

[0141] When the number of the preset combinations is greater than one, the process proceeds to operation 804 to determine whether there are other operations besides the preset combination.

[0142] When there are other operations, the process proceeds to operation 805 , and the preset combination or operation with the highest priority is selected from the preset combinations and the other operations as the selected operation according to the priority order, and the process proceeds to operation 811 .

[0143] When there are no other operations, the process proceeds to operation 806 , and the preset combination with the highest priority is selected from the preset combinations according to the priority order as the selected operation, and the process proceeds to operation 811 .

[0144] When there is no preset combination among the two or more operations, the process proceeds to operation 807 , and the operation with the highest priority is selected from the multiple operations as the selected operation according to the priority order, and the process proceeds to operation 811 .

[0145] When there is only one preset combination, the process proceeds to operation 808 to determine whether there are other operations besides the preset combination.

[0146] When there are other operations, the process proceeds to operation 809 , and the preset combination or operation with the highest priority is selected from the preset combinations and the other operations as the selected operation according to the priority order, and the process proceeds to operation 811 .

[0147] When there are no other operations, the process proceeds to operation 810 , the preset combination is selected as the selected operation, and the process proceeds to operation 811 .

[0148] In operation 811 , a shortcut key corresponding to the selected operation is obtained.

[0149] In operation 812 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0150] In this embodiment, by presetting priority rules, when the user applies multiple operations at the same time, the operation with the highest priority or the preset combination is responded to, thereby avoiding too many operations being applied at the same time, which may cause the electronic painting device to become unresponsive or the system to become disordered.

[0151] See also Figure 9 , which is a flow chart of a method for controlling a painting device according to a ninth embodiment of the present application.

[0152] In operation 901 , a current creative scene is obtained from a painting program.

[0153] In operation 903 , a selected operation is determined, where the selected operation is related to the current authoring scene.

[0154] In this embodiment, after the current creative scene is identified, the selected operation can be obtained using the method provided in any of the above embodiments.

[0155] In operation 905 , a shortcut key corresponding to the selected operation is obtained.

[0156] In operation 907 , the painting program is controlled to perform a corresponding operation according to the shortcut key.

[0157] In this embodiment, since different creative scenarios utilize different functions of the painting program, users can pre-set shortcut keys corresponding to different creative scenarios and the operations corresponding to these shortcut keys. These shortcut keys correspond to the functions of the painting device in that creative scenario. In other words, the operations correspond to the shortcut keys, and the shortcut keys correspond to the functions in the specific creative scenario, with a one-to-one correspondence between the three. For example, users can assign the same shortcut key to different functions in different creative scenarios, or set different shortcut keys for one creative scenario than for other creative scenarios. For example, in sketching mode, "simultaneously inflate both cheeks" corresponds to "eraser," and "simultaneously contract both cheeks" corresponds to "pencil hardness adjustment." In coloring mode, "simultaneously inflate both cheeks" corresponds to "fill color," and "simultaneously contract both cheeks" corresponds to "adjust transparency."

[0158] In one embodiment, the drawing device control method provided herein may further include: identifying the frequency and / or intensity of user operations, and adjusting a threshold for triggering the identification of the operation based on the frequency and / or intensity of the user operations. The user operation may be a cheek movement, an operation performed by the user on the electronic drawing device, or a preset combination.

[0159] For example, when a user frequently uses a certain cheek movement or hand operation to trigger a shortcut function, the electronic painting device, when executing the painting device control program, can lower the threshold for triggering the judgment of the operation. For example, according to the previous setting, only when the duration of a cheek movement exceeds a preset value (e.g., 2 seconds) will the judgment of whether the cheek movement is the preset cheek movement be triggered. However, when the user frequently uses the cheek movement to trigger a shortcut function, the electronic painting device, running the painting device control program, can automatically reduce the preset value, for example, from 2 seconds to 1 second. After resetting the preset value to 1 second, the judgment of whether the cheek movement is the preset cheek movement be triggered when the duration of the cheek movement exceeds 1 second. For another example, according to the previous setting, only when the force of the user's operation on the electronic painting device exceeds a preset value will the judgment of whether the operation is the preset operation be triggered. However, when the user frequently uses the operation to trigger a shortcut function, the electronic painting device, running the painting device control program, can automatically reduce the preset value, so that even when the user performs the operation with less force, the corresponding function of the operation can be triggered. Conversely, when certain cheek movements or operations are infrequently used, the threshold for triggering these actions can be raised. This prevents false triggering while also improving response sensitivity. Furthermore, the user can pre-set the criteria for determining whether an action is frequent or intense, or the electronic painting device's control program can automatically set the criteria based on historical records. For example, historical records from the past month can be compiled and the threshold adjusted accordingly to meet both anti-interference and sensitivity requirements.

[0160] The present application also provides a storage medium storing a computer-executable program code. When the program code is executed by a processor, at least one of the painting device control methods described in the above embodiments can be executed. The implementation principle and technical effect of the storage medium are similar to those of the painting device control method described in the above embodiments, and will not be repeated here.

[0161] The present application also provides a computer program product, which includes computer-executable program instructions, and the computer-executable program instructions can enable a processing unit to execute at least one of the painting device control methods described in the above embodiments. The implementation principle and technical effect of the computer program product are similar to the painting device control method described in the above embodiments, and will not be repeated here.

[0162] In summary, compared with the existing technology, the painting device control method, electronic painting device, storage medium and computer program product provided by this application reduce the dependence on physical hardware by setting cheek movements to correspond to shortcut keys, and can reduce the user's hand operations when painting, making it convenient for the user to achieve efficient interaction with the electronic painting device. In addition, the user can adapt to complex creative needs and adapt to the user's own habits by customizing the actions corresponding to shortcut keys.

[0163] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A drawing device control method, said method being applied to an electronic drawing device, characterized in that: The method comprises: Enable the camera to capture facial images; Recognizing cheek movements in a face image and determining whether the cheek movements are preset cheek movements; In a case where the cheek movement is a preset cheek movement, determining a selected operation and obtaining a shortcut key corresponding to the selected operation; The drawing program of the electronic drawing device is controlled to perform corresponding operations according to the shortcut keys.

2. The painting device control method according to claim 1, characterized in that: When the duration of the cheek movement reaches a preset value, it is determined whether the cheek movement is a preset cheek movement, and / or the preset cheek movement is used as the selected operation.

3. The method according to claim 1, characterized in that When the cheek movement is a preset cheek movement, the duration of the preset cheek movement is further determined. When the duration of the preset cheek movement reaches a preset value, the selected operation is determined, and / or the preset cheek movement is used as the selected operation.

4. The method according to claim 1, wherein Also includes: Acquire the user's operations on the electronic drawing device in real time; and determining whether the combination of the preset cheek operation and the operation performed by the user on the electronic painting device is a preset combination; and The determining and selecting operation includes: When the combination is the preset combination, the preset combination is used as the selected operation.

5. The method according to claim 1, wherein Also includes: Analyze the user's action sequence and predict the user's next intention; A corresponding shortcut key is obtained according to the predicted user intention, and the painting program is controlled to execute a corresponding function according to the corresponding shortcut key.

6. The method according to claim 1, characterized in that Also includes: Identify the frequency and / or intensity of user actions; and Adjusting a threshold for triggering recognition of the user operation according to the frequency and / or intensity of the user operation; The user operation is one of a cheek movement, an operation performed by the user on the electronic painting device, and a preset combination.

7. The method according to claim 1, characterized in that Also includes: In response to two or more operations of a user obtained simultaneously, obtaining a preset priority rule, wherein the priority rule defines a priority order of the user operations; and According to the priority order, the operation with the highest priority among the two or more operations is selected as the selected operation.

8. The method according to claim 1, characterized in that Also includes: In response to two or more operations of a user obtained simultaneously, obtaining a preset priority rule, wherein the priority rule defines a priority order of the user operations; Determining whether a preset combination exists among the two or more operations; When there are preset combinations in the multiple operations, determining whether the number of the preset combinations is more than one; When the number of the preset combinations is more than one, determining whether there are other operations besides the preset combinations; When the other operations still exist, the preset combination or operation with the highest priority is selected from the preset combinations and the other operations according to the priority order as the selected operation.

9. The method according to claim 8, characterized in that Also includes: When the other operations do not exist, the preset combination with the highest priority is selected from the preset combinations according to the priority order as the selected operation.

10. The method according to claim 9, characterized in that Also includes: When there is no preset combination among the two or more operations, the operation with the highest priority is selected from the two or more operations as the selected operation according to the priority order.

11. The method according to claim 10, characterized in that Also includes: When there is only one preset combination, determining whether there are other operations besides the preset combination; When the other operations do not exist, the preset combination is used as the selected operation.

12. The method according to claim 11, characterized in that Also includes: When the other operations exist, the preset combination or operation with the highest priority is selected from the preset combination and the other operations according to the priority order as the selected operation.

13. The method according to claim 1, wherein Before recognizing cheek movements in the face image, a current creative scene is obtained from a painting program, and the selected operation is related to the current creative scene.

14. A painting device control device, characterized in that: The device comprises: a memory configured to store computer-executable instructions; and A controller is configured to run the computer executable instructions to perform the painting device control method according to any one of claims 1 to 13.

15. A storage medium, characterized in that: The storage medium stores computer-executable instructions, and the computer-executable instructions, when executed by a processing unit, implement the painting device control method according to any one of claims 1 to 13.

16. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processing unit, implement the painting device control method according to any one of claims 1 to 13.

Images