Operation identification method and device, electronic equipment, storage medium and computer program product

Abstract

The invention discloses an operation recognition method and device, electronic equipment, a storage medium and a computer program product, and belongs to the technical field of electronics, and the method comprises the steps that under the condition that the electronic equipment detects a user part image through a camera, the electronic equipment displays first prompt information in a target area on a screen, the user part image represents relative position information of the user part and the camera, the relative position information of the user part image does not meet a preset condition, the preset condition is used for triggering execution of an operation associated with the user part in the user part image, and the first prompt information is used for prompting a first direction; the first direction is used for indicating the moving direction of the user part so as to update the relative position information; and when the electronic equipment detects that the relative position information of the updated user part image meets a preset condition, the electronic equipment executes an operation associated with the user part in the user part image.

Description

Technical Field

[0001] The present application belongs to the field of electronic technology, and specifically relates to an operation identification method, device, electronic device, storage medium and computer program product. Background Art

[0002] With the advancement of technology, the hardware performance of electronic device cameras has been continuously improved, and the functions have been intelligentized. For example, when the user's fingers are dirty or water-stained and it is inconvenient to touch the screen, the user can use air gesture input to trigger the electronic device to detect the user's hand movements and perform some simple operations accordingly. At the same time, the electronic device can also detect in real time whether the user is the owner through the camera to ensure the information security of the electronic device.

[0003] However, in the above method, when the user triggers the electronic device to perform a corresponding operation through air gesture input, since the front camera has a limited field of view and the electronic device does not display a preview interface, the user's gesture input may exceed the range of the shooting field of view, causing the electronic device to receive an incomplete air gesture input and fail to successfully recognize the air gesture input, thereby causing the electronic device to respond incorrectly or even not respond. This results in poor accuracy in the electronic device's recognition of operations. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an operation recognition method, device, electronic device, storage medium and computer program product, which can improve the accuracy of electronic device recognition operations.

[0005] In a first aspect, an embodiment of the present application provides an operation identification method, which includes: when an electronic device detects an image of a user part through a camera, the electronic device displays first prompt information in a target area on the screen, wherein the user part image represents relative position information between the user part and the camera, and the relative position information of the user part image does not meet a preset condition, and the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, and the first direction is used to indicate the direction in which the user part moves to update the relative position information; when the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device executes the operation associated with the user part in the user part image.

[0006] In a second aspect, an embodiment of the present application provides an operation identification device, which includes: a display module and an execution module; the display module is used to display a first prompt information in a target area on the screen when the electronic device detects a user part image through a camera, wherein the user part image represents relative position information between the user part and the camera, and the relative position information of the user part image does not meet a preset condition, and the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, and the first direction is used to indicate the direction in which the user part moves to update the relative position information; the execution module is used to execute the operation associated with the user part in the user part image when the electronic device detects that the relative position information of the updated user part image displayed by the display module meets the preset condition.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program / program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In an embodiment of the present application, when an electronic device detects a user part image through a camera, the electronic device displays a first prompt information in a target area on the screen, wherein the user part image represents the relative position information of the user part and the camera, the relative position information of the user part image does not meet the preset condition, the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, the first direction is used to indicate the direction in which the user part moves, so as to update the relative position information; when the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device executes the operation associated with the user part in the user part image. In this scheme, since the electronic device can display the first prompt information to inform the user of the body part to be moved and the direction to be moved when it is detected that the user part image does not meet the preset condition, after the user moves according to the first prompt information, if the relative position information of the user part image detected by the camera can meet the preset condition, the electronic device can perform the correct operation associated with the user part according to the user part image. In this way, the electronic device can detect the user part image more accurately and identify the user's operation according to the user part, thereby improving the accuracy of the electronic device in identifying the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is one of the flow charts of an operation identification method provided in an embodiment of the present application;

[0013] Figure 2A This is one of the example schematic diagrams of determining a center point based on an image provided by an embodiment of the present application;

[0014] Figure 2B This is a second example schematic diagram of determining a center point based on an image provided by an embodiment of the present application;

[0015] Figure 3A This is one of the example schematic diagrams of determining the direction according to the relative position relationship of the center points provided in the embodiment of the present application;

[0016] Figure 3B This is a second example schematic diagram of determining a direction according to a relative position relationship of a center point provided in an embodiment of the present application;

[0017] Figure 4A This is one of the example schematic diagrams of determining a direction based on an image provided by an embodiment of the present application;

[0018] Figure 4B This is a second example schematic diagram of determining a direction based on an image provided by an embodiment of the present application;

[0019] Figure 5AThis is one of the example schematic diagrams of a mobile phone display area provided in an embodiment of the present application;

[0020] Figure 5B This is one of the example schematic diagrams of displaying prompt information provided by the embodiment of the present application;

[0021] Figure 5C This is a second example schematic diagram of displaying prompt information provided by an embodiment of the present application;

[0022] Figure 5D This is a third example schematic diagram of displaying prompt information provided by an embodiment of the present application;

[0023] Figure 5E This is a fourth example schematic diagram of displaying prompt information provided by an embodiment of the present application;

[0024] Figure 6 This is a fifth example schematic diagram of displaying prompt information provided by an embodiment of the present application;

[0025] Figure 7 This is a sixth example schematic diagram of displaying prompt information provided by an embodiment of the present application;

[0026] Fig. 8A This is a second example schematic diagram of a mobile phone display area provided in an embodiment of the present application;

[0027] Figure 8B This is the seventh example schematic diagram of displaying prompt information provided by the embodiment of the present application;

[0028] Fig. 9 This is the second flowchart of an operation identification method provided in an embodiment of the present application;

[0029] Fig. 10A This is one of the example schematic diagrams of the area corresponding to the field of view angle range of a camera provided in an embodiment of the present application;

[0030] Fig. 10B This is a second example schematic diagram of an area corresponding to a camera field of view angle range provided in an embodiment of the present application;

[0031] Fig. 10C This is a third example schematic diagram of an area corresponding to a camera field of view angle range provided in an embodiment of the present application;

[0032] Fig.11A This is the third flowchart of an operation identification method provided in an embodiment of the present application;

[0033] Fig. 11B This is a fourth flowchart of an operation identification method provided in an embodiment of the present application;

[0034] Fig. 12A This is one of the structural schematic diagrams of an operation identification device provided in an embodiment of the present application;

[0035] Fig. 12B This is a second structural diagram of an operation identification device provided in an embodiment of the present application;

[0036] Fig.13 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0037] Fig.14 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0039] The terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0040] The terms "at least one (item)", "at least one of" and the like in the specification of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0041] The following is a detailed description of the operation identification method provided by the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0042] The operation recognition method provided in the embodiment of the present application can be applied to scenarios where a user inputs air gestures or triggers an electronic device to recognize the user's facial image to perform a corresponding operation.

[0043] For example, if the user's fingers are dirty or wet and it is inconvenient to touch the screen, the user can use air gesture input to trigger the electronic device to detect the user's hand movements to perform the corresponding operations. Or, if the electronic device is in the locked screen state, the user can trigger the electronic device to recognize the user's facial information to unlock the electronic device.

[0044] In the operation recognition method, device, electronic device, storage medium and computer program product provided by the embodiment of the present application, when the electronic device detects the user part image through the camera, the electronic device displays the first prompt information in the target area on the screen, wherein the user part image represents the relative position information of the user part and the camera, the relative position information of the user part image does not meet the preset condition, the preset condition is used to trigger the execution of the operation associated with the user part in the user part image, and the first prompt information is used to prompt the first direction, the first direction is used to indicate the direction in which the user part moves, so as to update the relative position information; when the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device performs the operation associated with the user part in the user part image. In this scheme, since the electronic device can display the first prompt information to inform the user of the body part to be moved and the direction to be moved when it is detected that the user part image does not meet the preset condition, therefore, after the user moves according to the first prompt information, if the relative position information of the user part image detected by the camera can meet the preset condition, the electronic device can perform the correct operation associated with the user part according to the user part image. In this way, the electronic device can detect the user part image more accurately and identify the user's operation according to the user part, thereby improving the accuracy of the electronic device in identifying the operation.

[0045] The execution subject of the operation identification method provided in the embodiment of the present application is an operation identification device, which can be an electronic device, or a functional module or entity in an electronic device, and the embodiment of the present application does not limit this. The operation identification method provided in the embodiment of the present application will be exemplarily described below using an electronic device as an example.

[0046] The present application embodiment provides an operation identification method. Figure 1 FIG. 1 is a flow chart of an operation identification method provided by an embodiment of the present application. Figure 1 As shown, the operation identification method provided in the embodiment of the present application may include the following steps 201 and 202.

[0047] Step 201: When the electronic device detects an image of a user's body part through a camera, the electronic device displays a first prompt message in a target area on the screen.

[0048] In the embodiment of the present application, the user part image may include a user gesture image or a user face image. The user part image may also include any other image of a user part that can represent the user input, which can be determined according to actual usage requirements and is not specifically limited in the embodiment of the present application.

[0049] In some embodiments of the present application, when the user part image is a user gesture image, the user part corresponding to the user part image is the user's hand.

[0050] In some embodiments of the present application, when the user part image is a user facial image, the user part corresponding to the above user part image is the head or face of the user.

[0051] In the embodiment of the present application, the above-mentioned user part image represents the relative position information of the user part and the camera.

[0052] In some embodiments of the present application, the above-mentioned relative position information may include but is not limited to at least one of the following: relative direction, relative distance, and relative angle.

[0053] In the embodiment of the present application, the relative position information of the user part image does not meet a preset condition, wherein the preset condition can be used to trigger the execution of an operation associated with the user part in the user part image.

[0054] In some embodiments of the present application, the above-mentioned preset conditions may include but are not limited to any of the following:

[0055] The area corresponding to the field of view of the camera includes a complete image of the user's body parts;

[0056] The area corresponding to the specified range in the field of view of the camera includes a complete user body part image;

[0057] The area corresponding to the field of view angle range of the camera includes a complete user part image, and the ratio of the user part image included in the area corresponding to the specified range to the complete user part image is greater than or equal to a preset threshold.

[0058] In some embodiments of the present application, the camera may be an always-on camera (AON).

[0059] In this way, due to the low power consumption of the AON camera, the AON camera of the electronic device can continuously perform detection-related behaviors, such as detecting the user's air gestures, detecting the parameters of the user's gaze on the screen, monitoring the user's fatigue level, etc.

[0060] In some embodiments of the present application, the above-mentioned camera can be a front camera of the electronic device, or a camera on the electronic device that can capture images of the user's body parts, and the embodiments of the present application do not specifically limit this.

[0061] In some embodiments of the present application, the camera may be located at any position in the top area of ​​the electronic device screen. For example, the center, left or right position. The camera may also be located at any position in the bottom area or side area of ​​the electronic device screen. The specific position may be determined according to actual use requirements, and the embodiments of the present application are not specifically limited.

[0062] In the embodiment of the present application, the first prompt information is used to prompt the first direction.

[0063] In some embodiments of the present application, the first direction is used to indicate the direction in which the user's body part moves, so as to update the relative position information.

[0064] In some embodiments of the present application, the first direction may be the direction of the user part corresponding to the user part image relative to the camera, or the direction in which the user part needs to move.

[0065] In some embodiments of the present application, the first direction may be any one of the following: the user's position is above the camera, the user's position is below the camera, the user's position is on the left side of the camera, or the user's position is on the right side of the camera.

[0066] In some embodiments of the present application, the first direction may also be any one of the following: the user's position is at the upper left of the camera, the user's position is at the lower left of the camera, the user's position is at the upper right of the camera, or the user's position is at the lower right of the camera.

[0067] In some embodiments of the present application, the first direction may be combined with an angle based on the direction to be described in a more specific form.

[0068] For example, the first direction may be described as follows: Direction 1 is biased toward Direction 2 by an angle. Direction 1 may be directly above or directly below, and Direction 2 may be left or right. For example, “directly above, 30° to the left”, “directly below, 60° to the right”.

[0069] In some embodiments of the present application, the above-mentioned first direction can also be combined with the distance on the basis of the direction to be described in a more specific form.

[0070] Exemplarily, the description of the first direction may be: the user part is forward relative to the shooting range of the camera, so the first prompt information may prompt the user to increase the distance between the user part and the camera, that is, the user part needs to move backward; or, the user part is backward relative to the shooting range of the camera, so the first prompt information may prompt the user to reduce the distance between the user part and the camera, that is, the user part needs to move forward.

[0071] In some embodiments of the present application, before "the electronic device displays the first prompt information in the target area on the screen" in the above step 201, the operation identification method provided by the embodiment of the present application may also include the following steps 301 and 302.

[0072] Step 301: The electronic device obtains a second direction based on the field of view angle range of the camera.

[0073] Step 302: The electronic device determines the first direction based on the second direction.

[0074] In some embodiments of the present application, the field of view of the camera may be: the angle formed by two edge lines of the maximum range of the camera through which the image of the measured object can pass, with the camera as the vertex. That is, the field of view range may be understood as: the range that the camera can capture.

[0075] For example, when the camera is a normal lens, the angle corresponding to the field of view angle range can be about 45 degrees; when the camera is a telephoto lens, the angle corresponding to the field of view angle range can be within 20 degrees; when the camera is a wide-angle lens, the angle corresponding to the field of view angle range can be 60 degrees to 90 degrees.

[0076] It should be noted that the field of view angle ranges of cameras of different electronic devices may be different.

[0077] In the embodiment of the present application, the second direction is the direction of the center point of the image of the user's body part relative to the center point of the area corresponding to the field of view angle range.

[0078] In some embodiments of the present application, before the electronic device determines the second direction, the electronic device can determine the display position of the user part image therein based on the image area captured by the camera, and determine the center point of the user part image.

[0079] In some embodiments of the present application, the electronic device can determine a graphic area based on the user's body part image, and then determine the center point of the graphic area based on the geometric figure corresponding to the graphic area. For example, the graphic area can be a circular area or a rectangular area, and the center point is the center point of the circular area or the rectangular area.

[0080] In some embodiments of the present application, the electronic device can draw a circle based on the two points with the longest distance in the detected user body part image as the diameter to determine the circular area corresponding to the user body part image. In addition, the electronic device can use the point of the circular area as the center point of the circular area. It can be understood that the center point is the midpoint of the line segment formed by the two points with the longest distance.

[0081] For example, Figure 2A As shown, when the user part image is a user face image, the electronic device can first determine the longest line segment in the user head image as the diameter, that is, determine the line segment AB formed by point A located at the top of the head and point B located at the chin as the diameter, then the electronic device can draw a circular area corresponding to the user head image with line segment AB as the diameter and the midpoint O of line segment AB as the center of the circle. Point O is the center point of the circular area, that is, the center of the circle.

[0082] In some embodiments of the present application, the electronic device may draw a rectangle based on the detected user body part image to determine the rectangular area corresponding to the user body part image, and then the electronic device may determine the intersection of the diagonal lines of the rectangular area as the center point of the rectangular area. Figure 2B In the rectangular area shown in FIG. 1 , the intersection point P of the diagonal lines of the rectangular area is the center point of the rectangular area.

[0083] For example, Figure 3A As shown, the rectangular area 10 is the field of view of the camera. When the user's head image is detected by the camera, the electronic device can determine the rectangular area 11 corresponding to the head image. Then, the electronic device can determine the center point of the rectangular area 10 corresponding to the field of view, that is, point R; and determine the center point of the rectangular area 11, that is, point Q. Therefore, the electronic device can determine that point Q is below point R based on the relative position relationship between the center point Q of the rectangular area 11 and the center point R of the rectangular area 10 (that is, the field of view of the camera), that is, the second direction is "down".

[0084] Again illustratively, in combination Figure 2B and Figure 3B ,like Figure 3B As shown, when a user gesture image is detected by a camera, the electronic device may first determine a rectangular area 12 corresponding to the user gesture image. Then, the electronic device may determine the center point of the rectangular area 12, that is, point P. Thus, the electronic device may determine that point P is to the right of point R, that is, the second direction is "right", based on the relative positional relationship between the center point P of the rectangular area 12 and the center point R of the rectangular area 10 (that is, the field of view of the camera).

[0085] It should be noted that, since the second direction is similar to the first direction, the description of the second direction may refer to the relevant description of the first direction in the above step 201, which will not be repeated here.

[0086] In some embodiments of the present application, the first direction may be the same as or opposite to the second direction.

[0087] In some embodiments of the present application, when the first direction is the direction of the user part corresponding to the user part image relative to the camera, the first direction may be the same as the second direction. Figure 3B , the second direction is "right", and the first direction is also "right".

[0088] In some embodiments of the present application, when the first direction is the direction in which the user's body part needs to move, the first direction may be opposite to the second direction. Figure 3B , the second direction is "right", then the first direction is "left".

[0089] In some embodiments of the present application, the electronic device can obtain a second direction based on the field of view angle range of the camera, and determine the first direction based on the second direction, so that the electronic device can display first prompt information to the user according to the determined first direction. In this way, since the first direction can indicate the direction of the user part corresponding to the user part image relative to the camera, or the direction in which the user part needs to move, displaying prompt information to the user according to the first direction can improve the accuracy of the prompt information displayed by the electronic device.

[0090] In some embodiments of the present application, when the user part image is a user facial image, if the user facial image detected by the electronic device is incomplete, the electronic device can also determine the first direction by comparing the detected user facial image with the user facial image stored in the electronic device.

[0091] For example, it is assumed that the user's facial image detected by the electronic device is as follows Figure 3A The user's facial image shown in the middle area 11, the user's facial image stored in the electronic device is as follows Figure 2A As shown in FIG4 , the electronic device can obtain a straight line L by comparing the user facial image, and the image above the straight line L is the user facial image detected by the electronic device, and the image below the straight line L is the user facial image not detected by the electronic device. Therefore, the electronic device can determine that the direction of the user part corresponding to the user facial image relative to the camera is "downward", that is, the first direction.

[0092] Furthermore, combined with Figure 4A ,like Figure 4BAs shown, the electronic device can also determine two intersection points of the straight line L and the user's facial image, namely point m and point n. Then, the electronic device can connect point O and point m, and extend them to intersect with the circular area at point M; connect point O and point n, and extend them to intersect with the circular area at point N. Therefore, the first direction can also be represented by the direction corresponding to ∠MON or arc MN.

[0093] In some embodiments of the present application, the above-mentioned “stored user facial image” may include but is not limited to any of the following:

[0094] The user's facial image obtained by the electronic device during the process of setting up the electronic device to start facial recognition;

[0095] The user's facial image detected by the electronic device in the past;

[0096] The electronic device obtains the user's facial image based on the images stored in the photo album.

[0097] In this way, the electronic device can more accurately determine the first direction by detecting the center point of the image of the user's body part, thereby facilitating the subsequent display of prompt information to the user, and further, can improve the accuracy of the electronic device's recognition operation.

[0098] In some embodiments of the present application, the screen of the above-mentioned electronic device may include but is not limited to any of the following: an extreme full screen, a water drop screen, and a notch screen.

[0099] It should be noted that the position where the electronic device can display the first prompt information may be different depending on the screen of the electronic device.

[0100] For example, when the screen of the electronic device is an extreme full screen, the electronic device can display the first prompt information in the area around the camera. When the screen of the electronic device is a notch screen, the electronic device can display the first prompt information in the side area of ​​the "notch area" where the camera is set.

[0101] In some embodiments of the present application, the screen of the electronic device may include but is not limited to any one of the following: a liquid crystal module (LCM), an organic light-emitting diode (OLED).

[0102] In some embodiments of the present application, the target area may include but is not limited to at least one of the following: a lens area on the screen corresponding to the camera, a first edge area of ​​the screen, and a second edge area of ​​the prompt mark displayed on the screen.

[0103] In some embodiments of the present application, the above-mentioned prompt mark is used to represent the first prompt information.

[0104] In some embodiments of the present application, the first prompt information includes: display attributes of the first display mark in the target area.

[0105] In some embodiments of the present application, the above-mentioned display attributes include at least one of the following: display position, display color, display shape, and display animation.

[0106] In some embodiments of the present application, the electronic device may also detect the distance between the center point of the image of the user's body part and the center point of the area corresponding to the field of view angle range, and dynamically adjust the first prompt information according to the distance.

[0107] In some embodiments of the present application, when the target area is the lens area corresponding to the camera, and the display shape of the first display identifier included in the first prompt information is a first graphic, the "electronic device displays the first prompt information in the target area on the screen" in the above step 201 can be specifically implemented through the following step 201a.

[0108] Step 201a: The electronic device displays a first graphic based on the lens area.

[0109] In the embodiment of the present application, the shape of the first graphic matches the shape of the lens area.

[0110] It can be understood that the above-mentioned lens area is a circular area formed with the camera as the center.

[0111] For example, the electronic device is a mobile phone. Figure 5A As shown, a camera 20 is arranged at the center of the top area of ​​the mobile phone, that is, Figure 5A Therefore, the “circular area formed by the camera as the center” can be the area between the solid circle of the camera 20 and the outer dotted circle, that is, the circular area 21.

[0112] It should be noted that the above Figure 5A The dotted line is only used to indicate the area range and will not be displayed during actual use.

[0113] In some embodiments of the present application, the first graphic includes first marking information.

[0114] In some embodiments of the present application, the position information of the first mark information in the first graphic corresponds to the first direction, so as to prompt the first direction.

[0115] For example, in combination Figure 3B and Figure 5A , when the mobile phone determines that the direction of the hand corresponding to the user's gesture image relative to the camera is "right", then Figure 5B As shown, the mobile phone can adjust the color of the right 1 / 4 area in the circular area 21 so that the color of 1 / 4 of the circular area 21 is different from the color of the screen, thereby prompting the user through the different colors.

[0116] In some embodiments of the present application, the display of the first marking information may be static or dynamic.

[0117] In some embodiments of the present application, the display form of the above-mentioned first mark information may include but is not limited to any of the following: adjusting the color of at least part of the first graphic, adjusting the size of at least part of the first graphic, and displaying a mark symbol at a position in the first graphic corresponding to the first direction.

[0118] For example, in combination Figure 3B and Figure 5A , when the mobile phone determines that the direction of the hand corresponding to the user's gesture image relative to the camera is "right", then Figure 5C As shown, the mobile phone can first display the first color, that is, the first graphic, in the circular area 21, and then the mobile phone can change the color of the right 1 / 4 area of ​​the circular area 21 to the second color to indicate that the direction of the user's hand relative to the camera is "right" and the user needs to move his hand to the left. Or, as Figure 5D As shown, the mobile phone can first adjust the circular area 21 to display the first color, and then adjust the color of the right 1 / 4 area in the circular area 21 to the second color, and dynamically adjust the color of the right 1 / 2 area to the second color, and repeat this adjustment to dynamically indicate that the direction of the user's hand relative to the camera is "right", that is, the user needs to move his hand to the left.

[0119] For example, in combination Figure 4B and Figure 5A , the mobile phone receives the user's facial image Figure 4B When the area is above the straight line L shown in FIG. 1 , the mobile phone can determine by calculation that the first direction is the direction corresponding to the arc MN, then Figure 5E As shown, the first color is first displayed in the circular area 21, and then the circular area 21 is Figure 4B The partial ring corresponding to the arc MN in is adjusted to the second color to indicate the direction of the user's face relative to the camera.

[0120] It should be noted that the first color and the second color are different. For example, the first color may be green, and the second color may be red.

[0121] In this way, the electronic device can display the first prompt information according to the user's body part image to inform the user of the body part to be moved and the direction to be moved. Therefore, after the user moves according to the first prompt information, the subsequent electronic device can perform the correct associated operation according to the user's body part image. In this way, the electronic device can detect the user's body part image more accurately and recognize the user's operation according to the user's body part, thereby improving the accuracy of the electronic device's recognition of the operation.

[0122] In some embodiments of the present application, when the target area is the first edge area of ​​the screen and the first prompt information includes the display color of the first display identifier, the "electronic device displays the first prompt information in the target area on the screen" in the above step 201 can be specifically implemented through the following step 201b.

[0123] Step 201b: The electronic device adjusts the display color of the first edge area.

[0124] In the embodiment of the present application, the first edge area is an edge area of ​​the screen associated with the first direction.

[0125] The first direction may be the direction of the user part corresponding to the user part image relative to the camera.

[0126] In some embodiments of the present application, the first edge region may include at least one of the following: an upper edge region, a lower edge region, a left edge region, and a right edge region.

[0127] Exemplarily, when the first direction is “left”, the first edge area may be a left edge area of ​​the electronic device.

[0128] Exemplarily, when the first direction is “upper left”, the first edge area may be the left edge area and the upper edge area of ​​the screen of the electronic device.

[0129] In some embodiments of the present application, the area width of the above-mentioned first edge area may include but is not limited to any of the following: a width set by default by the electronic device, a width automatically determined by the electronic device based on content displayed on the interface, and a width set by the user.

[0130] It should be noted that if the area width of the first edge area is "a width automatically determined by the electronic device according to the content displayed on the interface", it can be understood that the above-mentioned first edge area is an edge area that does not affect the display content in the electronic device interface.

[0131] For example, in combination Figure 3B , when the mobile phone determines that the direction of the hand corresponding to the user's gesture image relative to the camera is "right", such as Figure 6As shown, the mobile phone can adjust the display color of the right edge area to indicate that the direction of the user's hand relative to the camera is "right", that is, the user needs to move his hand to the left.

[0132] In this way, the electronic device can display the first prompt information according to the user's body part image to inform the user of the body part to be moved and the direction to be moved. Therefore, after the user moves according to the first prompt information, the subsequent electronic device can perform the correct associated operation according to the user's body part image. In this way, the electronic device can detect the user's body part image more accurately and recognize the user's operation according to the user's body part, thereby improving the accuracy of the electronic device's recognition of the operation.

[0133] In some embodiments of the present application, when the target area is the first edge area of ​​the screen and the display shape of the first display identifier included in the first prompt information is a second graphic, the "electronic device displays the first prompt information in the target area on the screen" in the above step 201 can be specifically implemented through the following step 201c.

[0134] Step 201c: The electronic device displays a second graphic based on the first edge area.

[0135] In the embodiment of the present application, the first edge area is an edge area of ​​the screen associated with the first direction.

[0136] The first direction may be the direction in which the user's body part needs to move.

[0137] In some embodiments of the present application, the first edge region may include at least one of the following: an upper edge region, a lower edge region, a left edge region, and a right edge region.

[0138] In some embodiments of the present application, the first edge area may be an edge area that does not affect display content in the interface of the electronic device.

[0139] In the embodiment of the present application, the second graphic is used to indicate the first direction.

[0140] In some embodiments of the present application, the second graphic may be an arrow, or other graphic that can indicate a direction.

[0141] For example, in combination Figure 3B , when the mobile phone determines that the direction of the hand corresponding to the user's gesture image relative to the camera is "right", the mobile phone can display an arrow pointing to the left in the upper edge area, or the mobile phone can display an arrow pointing to the left in the lower edge area, or Figure 7 As shown, the mobile phone can display an arrow pointing to the left in the upper edge area and the lower edge area respectively to indicate that the user's hand needs to move to the left.

[0142] Exemplarily, when the first direction is "upper left", that is, the user's part needs to move to the upper left, the electronic device can display an arrow pointing to the left in the right edge area and an arrow pointing upward in the lower edge area to indicate that the user's part needs to move to the upper left.

[0143] In this way, the electronic device can display the first prompt information according to the user's body part image to inform the user of the body part to be moved and the direction to be moved. Therefore, after the user moves according to the first prompt information, the subsequent electronic device can perform the correct associated operation according to the user's body part image. In this way, the electronic device can detect the user's body part image more accurately and recognize the user's operation according to the user's body part, thereby improving the accuracy of the electronic device's recognition of the operation.

[0144] In some embodiments of the present application, when the target area is the second edge area of ​​the prompt logo displayed on the screen, and the display shape of the first prompt information including the first display logo is a third graphic, the "electronic device displays the first prompt information in the target area on the screen" in the above step 201 can be specifically implemented through the following step 201d.

[0145] Step 201d: The electronic device displays a prompt mark on the screen, and displays a third graphic based on the second edge area of ​​the prompt mark.

[0146] In some embodiments of the present application, the display position of the prompt logo may include but is not limited to any of the following: the area below the camera, the area on the side of the camera, or other areas that will not block the displayed content on the screen.

[0147] For example, Fig. 8A As shown, the above-mentioned "area below the camera" can be the rectangular area 22 directly below the camera 20; the above-mentioned "area on the side of the camera" can be the side area on the same horizontal line as the camera 20, that is, the rectangular area 23a or the rectangular area 23b, and the prompt mark can be displayed in any one of the rectangular area 22, the rectangular area 23a, and the rectangular area 23b.

[0148] It should be noted that the above Fig. 8A The dotted line is only used to indicate the area range of the display area of ​​the prompt mark, and will not be displayed during actual use.

[0149] In some embodiments of the present application, the display shape of the above-mentioned prompt mark may include but is not limited to any of the following: circle, rectangle, triangle, or any other possible geometric shape.

[0150] In the embodiment of the present application, the shape of the third graphic matches the shape of the prompt mark.

[0151] It can be understood that, when the prompt mark is a circular mark, the second edge area can be a circular ring area formed with the prompt mark as the center.

[0152] In the embodiment of the present application, the third graphic includes second marking information.

[0153] In some embodiments of the present application, the position information of the second marking information in the third graphic corresponds to the first direction, so as to indicate the first direction.

[0154] In some embodiments of the present application, the second edge area is an edge area of ​​the screen associated with the first direction.

[0155] For example, in combination Figure 3B and Fig. 8A , taking the display position of the prompt mark as the rectangular area 22 as an example. When the mobile phone determines that the direction of the hand corresponding to the user's gesture image relative to the camera is "right", then Figure 8B As shown, the mobile phone can display a circular prompt mark 24 directly below the camera 20, and then the mobile phone can display a circular ring of the first color outside the circular mark 24, that is, the above-mentioned second edge area, and adjust the right 1 / 4 area in the circular ring to the second color to indicate that the direction of the user's hand relative to the camera is "right", that is, the user needs to move his hand to the left.

[0156] In this way, the electronic device can display the first prompt information according to the user's body part image to inform the user of the body part to be moved and the direction to be moved. Therefore, after the user moves according to the first prompt information, the subsequent electronic device can perform the correct associated operation according to the user's body part image. In this way, the electronic device can detect the user's body part image more accurately and recognize the user's operation according to the user's body part, thereby improving the accuracy of the electronic device's recognition of the operation.

[0157] In some embodiments of the present application, the electronic device can also map the camera's field of view angle range onto the screen to determine the corresponding position on the screen based on the position of the user's part image in the camera's field of view angle range, and then display an identifier at the position to prompt the user of the position of the user's part image relative to the camera, so that the user can move the part corresponding to the user's part image according to the display position of the identifier.

[0158] Step 202: When the electronic device detects that the relative position information of the updated user part image meets a preset condition, the electronic device executes an operation associated with the user part in the user part image.

[0159] In some embodiments of the present application, the above-mentioned preset conditions may include but are not limited to any of the following:

[0160] The area corresponding to the field of view of the camera includes a complete image of the user's body parts;

[0161] The area corresponding to the specified range in the field of view of the camera includes a complete user body part image;

[0162] The area corresponding to the field of view angle range of the camera includes a complete user part image, and the ratio of the user part image included in the area corresponding to the specified range to the complete user part image is greater than or equal to a preset threshold.

[0163] It should be noted that the area corresponding to the above-mentioned specified range can be: an area that is convenient for the camera to recognize the image of the user's body part. For example, the area corresponding to 80% of the range in the middle position of the camera's field of view. The specific area setting can be determined according to the hardware conditions of the electronic device, and the embodiments of the present application are not limited here.

[0164] Exemplarily, the above-mentioned preset threshold may be 75%. The specific setting of the preset threshold may be determined according to the hardware requirements of the electronic device, and the embodiment of the present application is not limited thereto.

[0165] In some embodiments of the present application, Figure 1 ,like Fig. 9 As shown, the above step 202 can be specifically implemented through the following step 202a.

[0166] Step 202a: When the relative position of the user part image is located within the area corresponding to the field of view angle range of the camera, the electronic device performs an operation associated with the user part in the user part image.

[0167] In some embodiments of the present application, the area corresponding to the above-mentioned field of view angle range may be: a complete area corresponding to the field of view angle range, or a designated area corresponding to the field of view angle range.

[0168] For example, in combination Figure 3A Since the user's facial image contained in the rectangular area 10 corresponding to the field of view angle range is incomplete, the electronic device can display the first prompt information according to the direction of the center point of the rectangular area 11 relative to the center point of the rectangular area 10. Then, as Fig. 10A As shown, when the user moves the face upward so that the complete facial image of the user is within the rectangular area 10, the electronic device can perform an operation associated with the facial image of the user.

[0169] For example, in combination Figure 3B ,like Fig. 10BAs shown, since the rectangular area 12 corresponding to the user gesture image is located in the rectangular area 10 corresponding to the field of view of the camera, but the rectangular area 12 is not entirely located in the rectangular area 30 in the field of view of the camera for the camera to obtain the user gesture image, the electronic device can display the first prompt information according to the direction of the center point of the rectangular area 12 relative to the center point of the rectangular area 10. Then, as Fig. 10C As shown, when the user moves his hand to the left so that the rectangular area 12 corresponding to the user gesture image is completely located in the rectangular area 30, the electronic device can perform the operation associated with the user gesture image.

[0170] In this way, the electronic device can detect the user's body part image more accurately, and thus can perform a correct association operation according to the user's body part image, thereby improving the accuracy of the electronic device in recognizing input.

[0171] In some embodiments of the present application, when the user part image meets a preset condition, the electronic device can determine its associated operation based on the user part image detected in real time, and execute the associated operation.

[0172] In some embodiments of the present application, the user body part image includes a user gesture image. The above step 202 can be specifically implemented by the following step 202b.

[0173] Step 202b: When the relative position of the user gesture image meets a preset condition, the electronic device performs an operation associated with the gesture input parameter based on the gesture input parameter corresponding to the user gesture image.

[0174] In some embodiments of the present application, the electronic device first determines the corresponding gesture based on the gesture input parameters corresponding to the user gesture image, and then determines the operation to be performed based on the determined gesture, thereby executing the operation associated with the gesture input parameters.

[0175] In some embodiments of the present application, the above-mentioned gesture input parameters may include but are not limited to any of the following: gesture input movement trajectory, gesture input movement direction, and the number of fingers used for gesture input.

[0176] In some embodiments of the present application, the operations associated with the above-mentioned gesture input parameters may include but are not limited to any of the following: screenshot operation, downward sliding operation, upward sliding operation, left sliding operation, right sliding operation, zoom in operation, zoom out operation, and image capture operation.

[0177] For example, assuming that the application program running on the electronic device is an e-book reading application, when the electronic device detects the user's hand in the central area of ​​the camera's field of view, the electronic device can determine the corresponding gesture input based on the user's gesture image. For example, if it is determined that the user's palm is facing the camera, the fingertips are facing right, and then the user swings his wrist to the left, the electronic device can determine the gesture input movement trajectory based on the user's gesture image to determine that the operation associated with the gesture input parameter is a left sliding operation, and thus the electronic device can display the "next page" of the e-book.

[0178] In this way, the electronic device can perform associated operations according to the detected user gesture image, thereby improving the flexibility of the electronic device in performing operations.

[0179] In some embodiments of the present application, the user body part image includes a user face image. The above step 202 can be specifically implemented by the following step 202c.

[0180] Step 202c: when the relative position of the user's facial image meets a preset condition, the electronic device performs an operation associated with the facial feature information based on the facial feature information corresponding to the user's facial image.

[0181] In some embodiments of the present application, the electronic device may pre-store facial feature information of at least one user, for example, facial feature information of the owner of the electronic device.

[0182] In some embodiments of the present application, the facial feature information may include but is not limited to any of the following: the shape of the facial part, the size of the facial part, and the position of the facial part.

[0183] In some embodiments of the present application, the above-mentioned facial parts may include but are not limited to at least one of the following: eyes, nose, mouth, eyebrows, and ears.

[0184] In some embodiments of the present application, after the electronic device obtains facial feature information, the electronic device may compare the obtained facial feature information with pre-stored facial feature information to determine the user corresponding to the facial image.

[0185] In some embodiments of the present application, the operations associated with the above-mentioned facial feature information may include but are not limited to any of the following: a screen unlocking operation and an operation of lowering the screen brightness.

[0186] For example, when the electronic device is in a locked screen state, if the electronic device detects that the user's facial image meets a preset condition, the electronic device can obtain corresponding facial feature information based on the detected user's facial image, and then determine whether the user's facial image is a facial image of a user who can unlock the screen based on the facial feature information. Therefore, when the user's facial image is a facial image of a user who can unlock the screen set by the electronic device, the electronic device can perform a screen unlocking operation.

[0187] In this way, the electronic device can perform associated operations based on the detected user facial image, thereby improving the flexibility of the electronic device in performing operations.

[0188] In the operation recognition method provided in the embodiment of the present application, since the electronic device can display the first prompt information to inform the user of the body part to be moved and the direction to be moved when it is detected that the user part image does not meet the preset conditions, after the user moves according to the first prompt information, if the relative position information of the user part image detected by the camera can meet the preset conditions, the electronic device can perform the correct operation associated with the user part according to the user part image. In this way, the electronic device can detect the user part image more accurately and recognize the user's operation according to the user part, thereby improving the accuracy of the electronic device in recognizing the operation.

[0189] The present application embodiment provides an operation identification method. Fig.11A The flowchart of an operation recognition provided by an embodiment of the present application is shown. Take the user part image as the user gesture image, the direction of the center point of the user gesture image relative to the center point of the area corresponding to the field of view angle range is "right", and the preset condition is that the user gesture image is in the center area of ​​the area corresponding to the camera field of view angle. Fig.11A As shown, the operation identification method provided in the embodiment of the present application may include the following steps 1 to 6.

[0190] Step 1: The electronic device detects a user gesture image, and the relative position of the user gesture image does not meet a preset condition.

[0191] Step 2: The electronic device determines the position of the center point of the user gesture image relative to the center point of the area corresponding to the field of view angle range based on the field of view angle range of the camera, and determines the second direction.

[0192] Step 3: The electronic device determines the first direction based on the second direction.

[0193] Step 4: The electronic device displays a green ring around the camera, and adjusts the corresponding right 1 / 4 of the green ring to red according to the first direction, that is, displays the first prompt information.

[0194] Step 5: When the user moves his hand according to the first prompt information, that is, the user gesture image is in the center area of ​​the area corresponding to the camera field of view, the electronic device determines the corresponding gesture input parameters according to the user gesture image detected in real time.

[0195] Step 6: The electronic device performs an operation associated with the gesture input parameter based on the gesture input parameter corresponding to the user gesture image.

[0196] The present application embodiment provides an operation identification method. Fig. 11B The flowchart of an operation recognition provided by an embodiment of the present application is shown. Assume that the user part image is a user facial image, the direction of the center point of the user facial image relative to the center point of the area corresponding to the field of view angle range is "down", and the preset condition is that the user facial image is completely in the area corresponding to the camera field of view angle. Fig. 11B As shown, the operation identification method provided in the embodiment of the present application may include the following steps 7 to 12.

[0197] Step 7: The electronic device detects a user facial image, and the relative position of the user facial image does not meet a preset condition.

[0198] Step 8: The electronic device determines the orientation of the center point of the user's facial image relative to the center point of the area corresponding to the field of view angle range based on the field of view angle range of the camera, and determines the second direction.

[0199] Step 9: The electronic device determines the first direction based on the second direction.

[0200] Step 10: The electronic device displays a green ring around the camera, and adjusts the corresponding lower 1 / 4 of the green ring to red according to the first direction, thereby displaying the first prompt information.

[0201] Step 11: When the user moves his face according to the first prompt information, that is, the user's facial image is in the area corresponding to the camera's field of view, the electronic device determines the corresponding facial feature information based on the user's facial image detected in real time.

[0202] Step 12: The electronic device performs an operation associated with the facial feature information based on the facial feature information corresponding to the user's facial image.

[0203] It should be noted that the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0204] It should be noted that the operation recognition method provided in the embodiment of the present application can be executed by an operation recognition device. In the embodiment of the present application, the operation recognition device provided in the embodiment of the present application is described by taking the operation recognition method executed by the operation recognition device as an example.

[0205] Fig. 12A A possible structural diagram of the operation identification device involved in the embodiment of the present application is shown. Fig. 12A As shown, the operation identification device 70 may include: a display module 71 and an execution module 72;

[0206] The display module 71 is used to display first prompt information in a target area on the screen when the electronic device detects a user part image through a camera, wherein the user part image represents relative position information between the user part and the camera, and the relative position information of the user part image does not meet a preset condition, and the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, and the first direction is used to indicate the direction in which the user part moves, so as to update the relative position information;

[0207] The execution module 72 is used to execute an operation associated with the user part in the user part image when the electronic device detects that the relative position information of the updated user part image displayed by the display module 71 meets a preset condition.

[0208] In a possible implementation, the target area includes at least one of the following: a lens area on the screen corresponding to the camera, a first edge area of ​​the screen, and a second edge area of ​​a prompt mark displayed on the screen, where the prompt mark is used to represent the first prompt information.

[0209] In a possible implementation, the first prompt information includes: display attributes of the first display identifier in the target area, where the display attributes include at least one of the following: display position, display color, display shape, and display animation.

[0210] In one possible implementation, when the target area is the lens area corresponding to the camera, and the display shape of the first display identifier included in the first prompt information is a first graphic, the above-mentioned display module 71 is specifically used to display the first graphic based on the lens area, wherein the shape of the first graphic matches the shape of the lens area, and the first graphic includes first marking information, and the position information of the first marking information in the first graphic corresponds to the first direction, so as to prompt the first direction.

[0211] In one possible implementation, when the target area is the first edge area of ​​the screen and the first prompt information includes the display color of the first display identifier, the above-mentioned display module 71 is specifically used to adjust the display color of the first edge area; wherein the first edge area is the edge area of ​​the screen associated with the first direction.

[0212] In one possible implementation, when the target area is the first edge area of ​​the screen and the display shape of the first display identifier included in the first prompt information is a second graphic, the above-mentioned display module 71 is specifically used to display the second graphic based on the first edge area, wherein the second graphic is used to prompt the first direction, and the first edge area is the edge area of ​​the screen associated with the first direction.

[0213] In one possible implementation, when the target area is the second edge area of ​​the prompt logo displayed on the screen, and the display shape of the first prompt information including the first display logo is a third graphic, the above-mentioned display module 71 is specifically used to display the prompt logo on the screen, and display the third graphic based on the second edge area of ​​the prompt logo, wherein the shape of the third graphic matches the shape of the prompt logo, the third graphic includes second mark information, and the position information of the second mark information in the third graphic corresponds to the first direction, so as to prompt the first direction, and the second edge area is the edge area of ​​the screen associated with the first direction.

[0214] In one possible implementation, combining Fig. 12A ,like Fig. 12B As shown, the operation identification device 70 provided in the embodiment of the present application also includes: an acquisition module 73 and a determination module 74; the above-mentioned acquisition module 73 is used to acquire a second direction based on the field of view angle range of the camera before the display module 71 displays the first prompt information in the target area on the screen, and the second direction is the direction of the center point of the user part image relative to the center point of the area corresponding to the field of view angle range; the above-mentioned determination module 74 is used to determine the first direction based on the second direction acquired by the acquisition module 73.

[0215] In a possible implementation, the execution module 72 is specifically configured to execute an operation associated with a user part in the user part image when the relative position of the user part image is within an area corresponding to the field of view of the camera.

[0216] In one possible implementation, the user part image includes a user gesture image; the execution module 72 is specifically used to execute operations associated with gesture input parameters based on gesture input parameters corresponding to the user gesture image when the relative position of the user gesture image meets preset conditions.

[0217] In a possible implementation, the user part image includes a user facial image; the execution module 72 is specifically used to execute operations associated with facial feature information based on facial feature information corresponding to the user facial image when the relative position of the user facial image meets a preset condition.

[0218] In the operation recognition device provided in the embodiment of the present application, since the operation recognition device can display the first prompt information to inform the user of the body part to be moved and the direction to be moved when it is detected that the user part image does not meet the preset conditions, after the user moves according to the first prompt information, if the relative position information of the user part image detected by the camera can meet the preset conditions, the operation recognition device can perform the correct operation associated with the user part according to the user part image. In this way, the operation recognition device can detect the user part image more accurately and recognize the user's operation according to the user part, thereby improving the accuracy of the operation recognition device in recognizing the operation.

[0219] The operation identification device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtualreality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0220] The operation identification device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0221] The operation identification device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0222] Alternatively, if Fig.13 As shown, an embodiment of the present application further provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned operation identification method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0223] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0224] Fig.14 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0225] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0226] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.14 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0227] The display unit 106 is used to display first prompt information in a target area on the screen when the electronic device detects a user part image through a camera, wherein the user part image represents relative position information between the user part and the camera, the relative position information of the user part image does not meet a preset condition, the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, and the first direction is used to indicate the direction in which the user part moves to update the relative position information;

[0228] The processor 110 executes an operation associated with the user part in the user part image when the electronic device detects that the relative position information of the updated user part image meets a preset condition.

[0229] Optionally, the target area includes at least one of the following: a lens area on the screen corresponding to the camera, a first edge area of ​​the screen, and a second edge area of ​​a prompt mark displayed on the screen, where the prompt mark is used to represent the first prompt information.

[0230] Optionally, the first prompt information includes: display attributes of the first display identifier in the target area, and the display attributes include at least one of the following: display position, display color, display shape, and display animation.

[0231] Optionally, when the target area is the lens area corresponding to the camera, and the display shape of the first display identifier included in the first prompt information is a first graphic, the display unit 106 is specifically used to display the first graphic based on the lens area, wherein the shape of the first graphic matches the shape of the lens area, and the first graphic includes first marking information, and the position information of the first marking information in the first graphic corresponds to the first direction, so as to prompt the first direction.

[0232] Optionally, when the target area is the first edge area of ​​the screen and the first prompt information includes the display color of the first display identifier, the processor 110 is specifically used to adjust the display color of the first edge area; wherein the first edge area is the edge area of ​​the screen associated with the first direction.

[0233] Optionally, when the target area is the first edge area of ​​the screen and the display shape of the first display identifier included in the first prompt information is a second graphic, the display unit 106 is specifically used to display the second graphic based on the first edge area, wherein the second graphic is used to prompt the first direction, and the first edge area is the edge area of ​​the screen associated with the first direction.

[0234] Optionally, when the target area is the second edge area of ​​the prompt logo displayed on the screen, and the display shape of the first prompt information including the first display logo is a third graphic, the display unit 106 is specifically used to display the prompt logo on the screen, and display the third graphic based on the second edge area of ​​the prompt logo, wherein the shape of the third graphic matches the shape of the prompt logo, the third graphic includes second mark information, and the position information of the second mark information in the third graphic corresponds to the first direction, so as to prompt the first direction, and the second edge area is the edge area of ​​the screen associated with the first direction.

[0235] Optionally, the processor 110 is also used to obtain a second direction based on the field of view angle range of the camera before displaying the first prompt information in the target area on the screen, the second direction being the direction of the center point of the user part image relative to the center point of the area corresponding to the field of view angle range; and determine the first direction based on the second direction.

[0236] Optionally, the processor 110 is specifically configured to execute an operation associated with a user part in the user part image when the relative position of the user part image is within an area corresponding to the field of view angle range of the camera.

[0237] Optionally, the user part image includes a user gesture image; the processor 110 is specifically configured to execute an operation associated with the gesture input parameter based on the gesture input parameter corresponding to the user gesture image when the relative position of the user gesture image meets a preset condition.

[0238] Optionally, the user part image includes a user facial image; the processor 110 is specifically configured to perform operations associated with facial feature information based on facial feature information corresponding to the user facial image when the relative position of the user facial image meets a preset condition.

[0239] In the electronic device provided in the embodiment of the present application, since the electronic device can display the first prompt information to inform the user of the body part to be moved and the direction to be moved when it is detected that the user part image does not meet the preset conditions, after the user moves according to the first prompt information, if the relative position information of the user part image detected by the camera can meet the preset conditions, the electronic device can perform the correct operation associated with the user part according to the user part image. In this way, the electronic device can detect the user part image more accurately and recognize the user's operation according to the user part, thereby improving the accuracy of the electronic device in recognizing the operation.

[0240] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0241] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0242] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0243] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0244] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0245] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0246] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0247] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0248] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0249] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0250] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0251] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0252] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An operation identification method, characterized in that: The method comprises: In the case where the electronic device detects a user part image through a camera, the electronic device displays first prompt information in a target area on the screen, wherein the user part image represents relative position information of the user part and the camera, the relative position information of the user part image does not meet a preset condition, the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, the first direction is used to indicate the direction in which the user part moves, so as to update the relative position information; When the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device performs an operation associated with the user part in the user part image.

2. The method according to claim 1, characterized in that The target area includes at least one of the following: The lens area on the screen corresponding to the camera, the first edge area of ​​the screen, and the second edge area of ​​the prompt mark displayed on the screen, wherein the prompt mark is used to represent the first prompt information.

3. The method according to claim 1 or 2, characterized in that: The first prompt information includes: display attributes of the first display mark in the target area, and the display attributes include at least one of the following: display position, display color, display shape, and display animation.

4. The method according to claim 3, characterized in that When the target area is a lens area corresponding to a camera, and the display shape of the first display identifier included in the first prompt information is a first graphic, the electronic device displays the first prompt information in the target area on the screen, including: The electronic device displays a first graphic based on the lens area, wherein a shape of the first graphic matches a shape of the lens area, and the first graphic includes first mark information, and position information of the first mark information in the first graphic corresponds to the first direction, so as to prompt the first direction.

5. The method according to claim 3, characterized in that: When the target area is a first edge area of ​​the screen and the first prompt information includes a display color of the first display identifier, the electronic device displays the first prompt information in the target area on the screen, including: The electronic device adjusts the display color of the first edge area; The first edge area is an edge area of ​​the screen associated with the first direction.

6. The method according to claim 3, characterized in that When the target area is a first edge area of ​​the screen, and the display shape of the first display identifier included in the first prompt information is a second graphic, the electronic device displays the first prompt information in the target area on the screen, including: The electronic device displays a second graphic based on the first edge area, wherein the second graphic is used to prompt the first direction, and the first edge area is an edge area of ​​the screen associated with the first direction.

7. The method according to claim 3, characterized in that When the target area is the second edge area of ​​the prompt mark displayed on the screen, and the display shape of the first prompt information including the first display mark is a third graphic, the electronic device displays the first prompt information in the target area on the screen, including: The electronic device displays a prompt logo on the screen and displays a third graphic based on the second edge area of ​​the prompt logo, wherein the shape of the third graphic matches the shape of the prompt logo, the third graphic includes second marking information, and the position information of the second marking information in the third graphic corresponds to the first direction to prompt the first direction, and the second edge area is the edge area of ​​the screen associated with the first direction.

8. The method according to claim 1, characterized in that Before the electronic device displays the first prompt information in the target area on the screen, the method further includes: The electronic device acquires a second direction based on the field of view angle range of the camera, where the second direction is the direction of the center point of the image of the user's body part relative to the center point of the area corresponding to the field of view angle range; The electronic device determines the first direction based on the second direction.

9. The method according to claim 1, characterized in that: When the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device performs an operation associated with the user part in the user part image, including: In a case where the relative position of the user part image is located within an area corresponding to the field of view angle range of the camera, the electronic device performs an operation associated with the user part in the user part image.

10. The method according to claim 1 or 9, characterized in that: The user part image includes the user gesture image; When the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device performs an operation associated with the user part in the user part image, including: When the relative position of the user gesture image meets the preset condition, the electronic device performs an operation associated with the gesture input parameter based on the gesture input parameter corresponding to the user gesture image.

11. The method according to claim 1 or 9, characterized in that: The user part image includes the user face image; When the electronic device detects that the relative position information of the updated user part image meets the preset condition, the electronic device performs an operation associated with the user part in the user part image, including: When the relative position of the user's facial image meets the preset condition, the electronic device performs an operation associated with the facial feature information based on the facial feature information corresponding to the user's facial image.

12. An operation recognition device, characterized in that: The operation identification device comprises: a display module and an execution module; The display module is used to display first prompt information in a target area on the screen when the electronic device detects a user part image through a camera, wherein the user part image represents relative position information of the user part and the camera, the relative position information of the user part image does not meet a preset condition, and the preset condition is used to trigger the execution of an operation associated with the user part in the user part image, and the first prompt information is used to prompt a first direction, and the first direction is used to indicate the direction in which the user part moves, so as to update the relative position information; The execution module is used to execute an operation associated with the user part in the user part image when the electronic device detects that the relative position information of the updated user part image displayed by the display module meets the preset condition.

13. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the operation identification method according to any one of claims 1 to 10 are implemented.

14. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the operation identification method according to any one of claims 1 to 10 are implemented.

15. A computer program product, characterized in that The computer program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the operation identification method according to any one of claims 1 to 10.

Images