Air gesture recognition method and electronic equipment

Abstract

The invention is suitable for the technical field of image processing, and provides an air gesture recognition method and electronic equipment. The air gesture recognition method comprises the following steps: acquiring information of a hand image acquired by a front camera at a first moment; acquiring a screen content display direction of the electronic equipment at the first moment; determining a target preprocessing operation corresponding to the screen content display direction; executing a target preprocessing operation on a hand in the hand image to obtain a to-be-recognized image; according to the technical scheme, the air gesture is recognized based on the multiple frames of to-be-recognized images, so that the finger orientation recognized by the electronic equipment in the air gesture recognition process is consistent with the finger orientation of a user in an actual physical space, air gesture recognition in various different screen content display directions is compatible, and the error recognition probability of the air gesture is reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for recognizing air gestures and an electronic device. Background Art

[0002] Currently, electronic devices such as mobile phones can support non-contact air gesture interaction technology. The air gesture interaction technology can achieve non-contact human-computer interaction in scenarios where it is inconvenient for users to touch the input devices of the electronic device (such as touch screens, keyboards, buttons, mice, or digitizing tablets, etc.), improving the convenience of users operating the electronic device.

[0003] The air gesture recognition methods in the related art are only applicable to the scenario where the screen content display direction of the electronic device is: starting from the side where the front camera is located and displaying along the long side of the electronic device, and are not applicable to scenarios where the screen content display direction is in other states. In scenarios where the screen content display direction is in other states, the probability of misrecognizing air gestures is relatively high. Summary of the Invention

[0004] Embodiments of this application provide a method for recognizing air gestures and an electronic device, which can be compatible with the recognition of air gestures in various different screen content display directions and reduce the probability of misrecognizing air gestures.

[0005] In a first aspect, embodiments of this application provide a method for recognizing air gestures, which is applied to an electronic device including a front camera. The method for recognizing air gestures includes: obtaining information of a human hand image collected by the front camera at a first moment; obtaining the screen content display direction of the electronic device at the first moment; determining a target preprocessing operation corresponding to the screen content display direction; performing the target preprocessing operation on the human hand in the human hand image to obtain an image to be recognized; and recognizing air gestures based on multiple frames of images to be recognized.

[0006] Herein, the first moment can be any moment.

[0007] The human hand image can refer to an image including a human hand collected by the AO camera.

[0008] The information of the human hand image can include the human hand image itself and the coordinates of the key points of the human hand corresponding to the human hand image.

[0009] According to the air gesture recognition method provided by the embodiments of the present application, by obtaining the information of the human hand image collected by the front camera at the first moment and the screen content display direction of the electronic device at the first moment, and determining the target preprocessing operation corresponding to the screen content display direction, performing the target preprocessing operation on the human hand in the human hand image to obtain the image to be recognized, and recognizing the air gesture based on the image to be recognized, it is possible to make the finger orientation recognized by the electronic device during the air gesture recognition process consistent with the finger orientation of the user in the actual physical space, thereby being compatible with the air gesture recognition under various different screen content display directions and reducing the misrecognition probability of the air gesture.

[0010] In an alternative implementation manner of the first aspect, the target preprocessing operation includes a rotation operation and a left - right flipping operation; determining the target preprocessing operation corresponding to the screen content display direction includes: determining the target rotation angle corresponding to the screen content display direction; performing the target preprocessing operation on the human hand in the human hand image includes: after rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand.

[0011] Among them, the left - right flipping operation refers to a left - right mirroring operation.

[0012] According to the air gesture recognition method provided by the embodiments of the present application, by determining the target rotation angle corresponding to the screen content display direction, and after rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand, it is possible to make the finger orientation of the human hand in the human hand image consistent with the finger orientation of the user in the actual physical space, facilitating the electronic device to accurately recognize the air gesture according to the finger orientation.

[0013] In an alternative implementation manner of the first aspect, after rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand includes: after rotating the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand image.

[0014] In an alternative implementation manner of the first aspect, after rotating the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand image includes: when the resolution of the human hand image is less than or equal to the first resolution and the aspect ratio of the width to height of the human hand image is 1:1, after rotating the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand image.

[0015] In an alternative implementation manner of the first aspect, the information of the human hand image includes the coordinates of the human hand key points corresponding to the human hand image; after rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand includes: determining the target coordinate conversion method corresponding to the target rotation angle and the left - right flipping operation; converting the coordinates of the human hand key points based on the target conversion method.

[0016] In an alternative implementation of the first aspect, the information of the human hand image includes the coordinates of the key points of the human hand corresponding to the human hand image; after rotating the human hand in the human hand image by a target rotation angle, performing a left-right flipping operation on the human hand includes: when the resolution of the human hand image is greater than the first resolution, or the aspect ratio of the human hand image is not 1:1, determining a target coordinate conversion method corresponding to the target rotation angle and the left-right flipping operation; and converting the coordinates of the key points of the human hand based on the target conversion method.

[0017] According to the air gesture recognition method provided by the embodiments of the present application, only when the resolution of the human hand image is less than or equal to the first resolution and the aspect ratio of the human hand image is 1:1, a rotation operation and a left-right flipping operation are performed on the entire human hand image. However, when the resolution of the human hand image is greater than the first resolution, or the aspect ratio of the human hand image is not 1:1, the rotation and flipping of the human hand are achieved only by performing coordinate conversion on the human hand in the human hand image. This can not only reduce the computational load of the electronic device, improve the efficiency of air gesture recognition, but also save the power consumption of the electronic device, and at the same time improve the accuracy of air gesture recognition.

[0018] In an alternative implementation of the first aspect, determining the target rotation angle corresponding to the screen content display direction includes: when the screen content display direction is the first display direction, determining the target rotation angle as -90 degrees; when the screen content display direction is the second display direction, determining the target rotation angle as 0 degrees; when the screen content display direction is the third display direction, determining the target rotation angle as 180 degrees; where the first display direction is used to indicate that the screen content starts from the side where the front camera is located and is displayed along the long side of the electronic device; the second display direction is used to indicate that the screen content is displayed along the short side of the electronic device and the front camera is located on the left side of the screen content display direction; the third display direction is used to indicate that the screen content is displayed along the short side of the electronic device and the front camera is located on the right side of the screen content display direction; a negative target rotation angle is used to indicate counterclockwise rotation, and a positive target rotation angle is used to indicate clockwise rotation.

[0019] In an alternative implementation of the first aspect, determining a target coordinate transformation method corresponding to a target rotation angle and a left-right flipping operation includes: when the target rotation angle is -90 degrees, determining the target coordinate transformation method as: x = h - 1 - y_raw, y = w - 1 - x_raw; when the target rotation angle is 0 degrees, determining the target coordinate transformation method as: x = w - 1 - x_raw, y = y_raw; when the target rotation angle is -180 degrees, determining the target coordinate transformation method as: x = w - 1 - x_raw, y = h - 1 - y_raw; where x_raw is used to represent the abscissa of the hand key point before coordinate transformation, y_raw is used to represent the ordinate of the hand key point before coordinate transformation, h is used to represent the height of the hand image, w is used to represent the width of the hand image, x is used to represent the abscissa of the hand key point after coordinate transformation, and y is used to represent the ordinate of the hand key point after coordinate transformation.

[0020] In a second aspect, an embodiment of the present application provides an electronic device, including: one or more processors, and a memory;

[0021] The memory is coupled to the one or more processors, and the memory is used to store computer program code. The computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the electronic device to execute the air gesture recognition method according to any implementation manner of the first aspect described above.

[0022] In a third aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium includes instructions that, when running on an electronic device, cause the electronic device to execute the air gesture recognition method according to any implementation manner of the first aspect described above.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-executable program product. When the computer-executable program product runs on an electronic device, it causes the electronic device to execute the air gesture recognition method according to any implementation manner of the first aspect described above.

[0024] In a fifth aspect, an embodiment of the present application provides a chip system. The chip system is applied to an electronic device, and the chip system includes one or more processors. The one or more processors are used to call computer instructions to cause the electronic device to execute the air gesture recognition method according to any implementation manner of the first aspect described above.

[0025] It can be understood that the beneficial effects of the second to fifth aspects described above can refer to the relevant descriptions in the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the placement posture of an electronic device provided by an embodiment of the present application;

[0027] Figure 2 Schematic diagram of the display direction of the screen content when the electronic device is in different placement postures when the screen auto-rotation function is turned on provided by an embodiment of the present application;

[0028] Figure 3 Schematic diagram of the display direction of the screen content when the electronic device is in different placement postures when the screen auto-rotation function is not turned on provided by an embodiment of the present application;

[0029] Figure 4 Schematic diagram of a scenario for controlling an electronic device through air gestures provided by an embodiment of the present application;

[0030] Figure 5 Schematic diagram of a scenario where a user inputs an air gesture in a screen content display direction provided by an embodiment of the present application;

[0031] Figure 6 Another schematic diagram of a scenario where a user inputs an air gesture in a screen content display direction provided by an embodiment of the present application;

[0032] Figure 7 Another schematic diagram of a scenario where a user inputs an air gesture in a screen content display direction provided by an embodiment of the present application;

[0033] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0034] Figure 9 Schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0035] Figure 10 Schematic flowchart of a method for recognizing air gestures provided by an embodiment of the present application;

[0036] Figure 11 Schematic diagram of the interaction timing between modules in the system architecture of an electronic device during the implementation process of a method for recognizing air gestures provided by an embodiment of the present application;

[0037] Figure 12 Schematic diagram of the interaction timing between modules in the system architecture of an electronic device during the implementation process of a method for recognizing air gestures provided by another embodiment of the present application. Detailed implementation manners

[0038] It should be noted that the terms used in the implementation part of the embodiments of the present application are only used to explain the specific embodiments of the present application, rather than to limit the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B; "and / or" herein is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more than two, and "at least one" and "one or more" mean one, two or more than two.

[0039] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0040] The reference to "one embodiment" or "some embodiments" etc. described in this specification means that a specific feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] For ease of understanding, the relevant concepts involved in the embodiments of the present application will be described first.

[0042] 1. Placement postures of electronic devices

[0043] The placement postures of electronic devices can be used to represent the ways in which users hold the electronic devices. Taking a mobile phone as an example of the electronic device, for example, please refer to Figure 1 which is a schematic diagram of the placement postures of an electronic device provided by an embodiment of the present application. The placement postures of the electronic device can include, for example: placed vertically with the front camera on the top, such as (a) in Figure 1 ; placed horizontally with the front camera on the left, such as (b) in Figure 1 ; placed horizontally with the front camera on the right, such as (c) in Figure 1 .

[0044] 2. Screen auto-rotation function

[0045] An electronic device can support the screen auto-rotation function. When the screen auto-rotation function is enabled, the electronic device can automatically adaptively rotate the display direction of the screen content according to its placement posture, so that the display direction of the screen content is the same as the user's face direction. Exemplarily, please refer to Figure 2 , which is a schematic diagram of the display direction of the screen content of the electronic device in different placement postures when the screen auto-rotation function is enabled provided by an embodiment of the present application.

[0046] When the screen auto-rotation function is enabled, as shown in (a) of Figure 2 , if the placement posture of the electronic device is vertically placed and the front camera is above, the display direction of the screen content can be: starting from the side where the front camera is located, and displaying along the long side of the electronic device. As shown in (b) of Figure 2 , if the placement posture of the electronic device is horizontally placed and the front camera is located on the left side of the display direction of the screen content, the display direction of the screen content can be: displaying along the short side of the electronic device and the front camera is located on the left side of the display direction of the screen content. As shown in (c) of Figure 2 , if the placement posture of the electronic device is horizontally placed and the front camera is located on the right side of the display direction of the screen content, the display direction of the screen content can be: displaying along the short side of the electronic device and the front camera is located on the right side of the display direction of the screen content.

[0047] Exemplarily, please refer to Figure 3 , which is a schematic diagram of the display direction of the screen content of the electronic device in different placement postures when the screen auto-rotation function is not enabled provided by an embodiment of the present application. When the screen auto-rotation function is not enabled, as shown in (a), (b), and (c) of Figure 3 , regardless of the placement posture of the electronic device, the display direction of the screen content is: starting from the side where the front camera is located, and displaying along the long side of the electronic device.

[0048] It can be seen that the display mode of the screen content of the electronic device can be determined by the placement posture of the electronic device and whether the electronic device enables the screen auto-rotation function.

[0049] Currently, electronic devices such as mobile phones can support non-contact air gesture interaction technology. The air gesture interaction technology can realize non-contact human-computer interaction in usage scenarios where it is inconvenient for users to touch the input devices (such as touch screens, keyboards, buttons, mice, or digitizing tablets, etc.) of the electronic device, and improve the convenience of users to control the electronic device.

[0050] Specifically, the electronic device can support the always-on (AO) function of the camera. When the AO function of the camera is turned on, the front camera of the electronic device is in an always-on state and can capture images in real time. The electronic device can identify the air gestures input by the user by analyzing multiple frames of images captured by the AO camera (i.e., the front camera in the always-on state), and can perform corresponding operations in response to the identified air gestures to achieve air control of the electronic device.

[0051] Exemplarily, please refer to Figure 4 , which is a schematic diagram of a scenario for controlling an electronic device by air gestures provided by an embodiment of the present application. Assume that Figure 4 the palm with fingers facing up shown in (a) in Figure 4 changes to the grasping gesture of a clenched fist shown in (b) in Figure 4 , and this grasping gesture corresponds to a screenshot operation. Then, the electronic device can perform a screenshot operation after recognizing the grasping gesture to obtain a screenshot image, thereby achieving air screenshot. Assume that Figure 4 the palm with fingers facing up shown in (c) in Figure 4 changes to the downward sliding gesture of the back of the hand with fingers facing down shown in (d) in Figure 4 , and this downward sliding gesture corresponds to a downward sliding screen operation or a downward page turning operation. Then, the electronic device can perform a downward sliding screen operation or a downward page turning operation after recognizing the downward sliding gesture, thereby achieving air downward sliding screen or air downward page turning. Assume that

[0052] the back of the hand with fingers facing down shown in (e) in

[0053] changes to the upward sliding gesture of the palm with fingers facing up shown in (f) in Figure 4 , Figure 4 and this upward sliding gesture corresponds to an upward sliding screen operation or an upward page turning operation. Then, the electronic device can perform an upward sliding screen operation or an upward page turning operation after recognizing the upward sliding gesture, thereby achieving air upward sliding screen or air upward page turning. Figure 4 The fist shown in (b) in Figure 4 can be the end gesture of the grasping gesture. Figure 4 The back of the hand with fingers facing down shown in (d) inFigure 4 The back of the hand with the fingers facing down as shown in (e) in [reference] can be the starting gesture of the upward swipe gesture, Figure 4 and the palm with the fingers facing up as shown in (f) in [reference] can be the ending gesture of the upward swipe gesture. It can be seen that at least one of the starting gesture and the ending gesture of different air gestures is different. Based on this, the electronic device can at least recognize the air gesture according to the starting gesture and the ending gesture. And to accurately recognize the air gesture, it is necessary to first accurately recognize the starting gesture and / or the ending gesture of the air gesture.

[0054] Generally, the electronic device can determine whether the gesture in the image is a starting gesture according to the human hand category corresponding to the image and the finger orientation. Exemplarily, as Figure 4 shown in (a) and (c) in [reference], when the human hand category corresponding to the image is the palm and the finger orientation is upward, it can be determined that the gesture in the image is the starting gesture of an air gesture such as a grasping gesture or a downward swipe gesture.

[0055] However, the user may input an air gesture when the screen content display direction of the electronic device is in different states. For example, please refer to Figure 5 , the user may input an air gesture when the screen content display direction is: starting from the side where the front camera is located and displaying along the long side of the electronic device. For another example, please refer to Figure 6 , the user may input an air gesture when the screen content display direction is: displaying along the short side of the electronic device and the front camera is located on the left side of the screen content display direction. For another example, please refer to Figure 7 , the user may input an air gesture when the screen content display direction is: displaying along the short side of the electronic device and the front camera is located on the right side of the screen content display direction.

[0056] When the screen content display direction of the electronic device is in different states, the AO camera will perform different processing on the captured image, resulting in the finger orientation in the image output by the AO camera being inconsistent with the finger orientation of the user in the actual physical space. For example, in the scenario shown in Figure 5 , assuming that the gesture input by the user is a palm with the fingers facing up, that is, the finger orientation of the user in the actual physical space is upward as shown in (e) in Figure 5 , however, the AO camera will output the image shown in (d) in Figure 5 . If the air gesture recognition is directly based on the image output by the AO camera, it is very easy to recognize the palm with the fingers facing up as the palm with the fingers facing left, resulting in misrecognition.

[0057] It can be seen that the gesture recognition method in the related art is only applicable to the scenario where the screen content display direction of the electronic device is: starting from the side where the front camera is located and displaying along the long side of the electronic device, and is not applicable to the scenario where the screen content display direction is in other states. In the scenario where the screen content display direction is in other states, the probability of false recognition of the gesture in the air is relatively high.

[0058] In view of this, the embodiments of the present application provide a gesture recognition method in the air and an electronic device. By obtaining the information of the human hand image collected by the front camera at the first moment and the screen content display direction of the electronic device at the first moment, and determining the target preprocessing operation corresponding to the screen content display direction, performing the target preprocessing operation on the human hand in the human hand image to obtain an image to be recognized, and recognizing the gesture in the air based on the image to be recognized, so that the electronic device can make the finger orientation recognized in the process of gesture recognition in the air consistent with the finger orientation of the user in the actual physical space, thereby being compatible with gesture recognition in the air in various different screen content display directions and reducing the probability of false recognition of the gesture in the air.

[0059] The gesture recognition method in the air provided by the embodiments of the present application can be applied to an electronic device including a front camera. The electronic device can include a mobile phone, a tablet computer, a wearable device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, and a personal digital assistant (PDA), etc. The embodiments of the present application do not limit the specific type of the electronic device.

[0060] Exemplarily, please refer to Figure 8 , which is a schematic structural diagram of an electronic device provided by the embodiments of the present application.

[0061] As Figure 8As shown in the figure, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0062] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0063] Exemplarily, the processor 110 may be used to execute the air gesture recognition method in the embodiments of the present application.

[0064] The controller may be the nerve center and command center of the electronic device. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0065] A memory for storing instructions and data may also be provided in the processor 110. For example, a first storage area and a second storage area may be provided in the memory. Among them, the first storage area may be used to store the frame information record queue involved in the air gesture recognition process; the second storage area may be used to store the recognition time, type, etc. of each valid air gesture recognized.

[0066] The camera 193 may be used to capture still images or videos. The electronic device may include one or N cameras 193, where N is a positive integer greater than 1. At least one of the N cameras 193 is a front camera.

[0067] The display screen 194 is used to display images, videos, hand icons involved in the air gesture recognition process, etc. The display screen 194 may include a display panel. The electronic device may implement the display function through the GPU, the display screen 194, and the application processor, etc.

[0068] It can be understood that the above is an exemplary description of the structure of the electronic device. It should be understood that in other embodiments, the electronic device may include more or fewer components than those shown in the figure, or may combine certain components, or split certain components, or may have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0069] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In this embodiment of the application, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software architecture of the electronic device.

[0070] Please refer to Figure 9 , which is a schematic diagram of the software architecture of an electronic device provided by this embodiment of the application.

[0071] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. For example, the Android system can be divided into four layers, from top to bottom, namely the application layer (application), the application framework layer (application framework), the system runtime library layer, and the kernel layer (kernel).

[0072] The application layer may include a series of application packages. For example, it may include application packages such as settings, camera, and intelligent perception. For the convenience of description, the application packages may be abbreviated as applications hereinafter.

[0073] The intelligent perception application can be used to support the air gesture service. The air gesture service can include a display service related to air gestures, such as displaying a hand icon corresponding to the starting gesture of the air gesture; or, the air gesture service can include a response service for responding to air gestures, such as a service for performing an operation corresponding to the air gesture.

[0074] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer can include some predefined functions.

[0075] Exemplarily, the application framework layer can include a window manager, a message manager, a sensor service, etc.

[0076] The window manager can be used to manage window programs. For example, the window manager can be used to obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0077] The message manager can be used to implement the transmission of messages between different components. Specifically, the message manager can achieve decoupling between different components through the message publishing and subscription mechanism, enabling a component to communicate with other components directly through message publishing without having to call another component.

[0078] Exemplarily, the intelligent perception application can subscribe to messages related to the air gesture service through the message manager.

[0079] The sensor service can be used to manage and provide sensor data. For example, it can be used to manage the data of acceleration sensors, magnetic sensors, and gyro sensors in the hardware layer and provide the data of each sensor to other modules.

[0080] The system runtime library layer can include an air gesture algorithm library, etc.

[0081] The air gesture algorithm library can be used to recognize air gestures based on a series of consecutive frames of images captured by the AO camera in the hardware layer. Exemplarily, the air gesture algorithm library can include a human hand feature acquisition module, a preprocessing module, and an air gesture recognition module.

[0082] The human hand feature acquisition module can include a human hand target detection unit, a human hand classification unit, and a human hand key point detection unit.

[0083] It should be noted that the specific functions of the human hand target detection unit, the human hand classification unit, the human hand key point detection unit, the preprocessing module, and the air gesture recognition module will be introduced in subsequent embodiments and will not be elaborated here.

[0084] The kernel layer is the layer between hardware and software. The kernel layer may include a camera driver, a sensor driver, etc.

[0085] It should be noted that Figure 9 Only the modules related to the embodiments of the present application are shown. In other embodiments, each layer may further include any other possible modules, and each module may further include one or more sub-modules, which are not limited in the present application.

[0086] Please refer to Figure 10 , which is a schematic flowchart of a gesture recognition method in the air provided by the embodiments of the present application. The gesture recognition method in the air may include S101 to S105, which are described in detail as follows:

[0087] S101, obtain the information of the hand image collected by the AO camera at the first moment.

[0088] Among them, the first moment can be any moment.

[0089] The hand image may refer to an image including a human hand collected by the AO camera.

[0090] The information of the hand image may include the hand image itself and the coordinates of the hand key points corresponding to the hand image.

[0091] S102, obtain the display direction of the screen content of the electronic device at the first moment.

[0092] Optionally, the electronic device may obtain the display direction of the screen content of the electronic device at the first moment from the message manager.

[0093] Exemplarily, the display direction of the screen content may include a first display direction, a second display direction, and a third display direction.

[0094] Combined with Figure 5 , the first display direction may be used to indicate that the screen content starts from the side where the front camera is located and is displayed along the long side of the electronic device. Combined with Figure 6 , the second display direction may be used to indicate that the screen content is displayed along the short side of the electronic device and the front camera is located on the left side of the display direction of the screen content. Combined with Figure 7 , the third display direction may be used to indicate that the screen content is displayed along the short side of the electronic device and the front camera is located on the right side of the display direction of the screen content.

[0095] S103, determine the target preprocessing operation corresponding to the display direction of the screen content.

[0096] Exemplarily, the target preprocessing operation may include a rotation operation and a left-right flipping operation.

[0097] Based on this, S103 may specifically include: determining a target rotation angle corresponding to the screen content display direction.

[0098] Optionally, the correspondence between the screen content display direction and the rotation angle may be stored in the electronic device. Exemplarily, the correspondence between the screen content display direction and the rotation angle may be as shown in Table 1.

[0099] Table 1

[0100]

[0101]

[0102] Among them, a negative rotation angle can be used to represent counterclockwise rotation, and a positive rotation angle can be used to represent clockwise rotation.

[0103] Based on this, the electronic device may determine the target rotation angle corresponding to the screen content display in the following manner:

[0104] When the screen content display direction is the first display direction, determine that the target rotation angle is -90 degrees;

[0105] When the screen content display direction is the second display direction, determine that the target rotation angle is 0 degrees;

[0106] When the screen content display direction is the third display direction, determine that the target rotation angle is -180 degrees.

[0107] S104, perform a target preprocessing operation on the hand in the hand image to obtain an image to be recognized.

[0108] Exemplarily, when the target processing operation includes a rotation operation and a left-right flipping operation, S104 may specifically include:

[0109] After rotating the hand in the hand image by the target rotation angle, perform a left-right flipping operation on the hand.

[0110] Optionally, the electronic device may implement the rotation and flipping of the hand in the hand image by directly rotating and flipping the entire hand image. Based on this, in an alternative implementation, after rotating the hand in the hand image by the target rotation angle and performing a left-right flipping operation on the hand, it may include:

[0111] After rotating the hand image by the target rotation angle, perform a left-right flipping operation on the hand image.

[0112] It can be understood that when the resolution of the hand image is relatively large, directly rotating and flipping the hand image will result in a large computational load on the electronic device, thereby reducing the efficiency of air gesture recognition and increasing the power consumption of the electronic device. In addition, when the aspect ratio of the hand image is not 1:1, directly rotating and flipping the hand image will cause the content of the hand image to change, thereby reducing the accuracy of air gesture recognition. Based on this, in order to improve the efficiency and accuracy of air gesture recognition, after rotating the hand image by a target rotation angle, a left-right flipping operation is performed on the hand image, which may specifically include:

[0113] When the resolution of the hand image is less than or equal to the first resolution and the aspect ratio of the hand image is 1:1, after rotating the hand target image by the target rotation angle, a left-right flipping operation is performed on the hand image.

[0114] This implementation method can be executed after the AO camera outputs the hand image.

[0115] Among them, the first resolution can be set according to actual needs and is not limited here.

[0116] Exemplarily, please continue to refer to Figure 5 , when the screen content display direction is the first display direction, assuming that the user inputs a palm with fingers facing up, the hand image output by the AO camera is the image shown in (d) in Figure 5 . After the electronic device rotates the hand image counterclockwise by 90 degrees and then performs a left-right flip, the image shown in (e) in Figure 5 can be obtained. The finger orientation in the image shown in (e) in Figure 5 is consistent with the finger orientation of the user in the actual physical space.

[0117] Exemplarily, please continue to refer to Figure 6 , when the screen content display direction is the second display direction, assuming that the user inputs a palm with fingers facing up, the hand image output by the AO camera is the image shown in (b) in Figure 6 . After the electronic device performs a left-right flip on the hand image, the image shown in (c) in Figure 6 can be obtained. The finger orientation in the image shown in (c) in Figure 6 is consistent with the finger orientation of the user in the actual physical space.

[0118] Exemplarily, please continue to refer to Figure 7 , when the screen content display direction is the third display direction, assuming that the user inputs a palm with fingers facing up, the hand image output by the AO camera is the image shown in (b) in Figure 7 . After the electronic device rotates the hand image counterclockwise by 180 degrees and then performs a left-right flip, it can obtainFigure 7 the image shown in (c) in Figure 7 the finger orientation in the image shown in (c) in is consistent with the finger orientation of the user in the actual physical space.

[0119] Optionally, when the resolution of the human hand image is greater than the first resolution, or the aspect ratio of the human hand image is not 1:1, the electronic device may only rotate and flip the human hand in the human hand image. Based on this, in another optional implementation manner, after rotating the human hand in the human hand image by a target rotation angle, performing a left - right flipping operation on the human hand may include:

[0120] determining a target coordinate conversion method corresponding to the target rotation angle and the left - right flipping operation;

[0121] converting the coordinates of the human hand key points based on the target conversion method.

[0122] This implementation manner can be correspondingly executed after obtaining the coordinates of the human hand key points corresponding to the human hand image.

[0123] Optionally, the electronic device may store the correspondence between the rotation angle and the left - right flipping operation and the coordinate conversion method. Exemplarily, the correspondence between the rotation angle and the left - right flipping operation and the coordinate conversion method can be shown in Table 2.

[0124] Table 2

[0125]

[0126] Among them, x_raw can be used to represent the abscissa of the human hand key point in the preset coordinate system, and y_raw can be used to represent the ordinate of the human hand key point in the preset coordinate system. Exemplarily, please refer to Figure 8 , which is a schematic diagram of a preset coordinate system provided by an embodiment of the present application. The preset coordinate system can be a plane rectangular coordinate system established with the upper - left vertex O of the human hand image 81 as the coordinate origin and the two intersecting sides at the upper - left vertex O as the x - axis and the y - axis respectively. Specifically, the straight line OL parallel to the horizontal plane among the two intersecting sides at the upper - left vertex can be used as the x - axis of the preset coordinate system, and the straight line OH perpendicular to the horizontal plane among the two intersecting sides at the upper - left vertex can be used as the y - axis of the preset coordinate system.

[0127] h can be used to represent the height of the human hand image, and w can be used to represent the width of the human hand image.

[0128] x can be used to represent the abscissa of the human hand key point after coordinate conversion, and y can be used to represent the ordinate of the human hand key point after coordinate conversion.

[0129] S105. Recognize an air gesture based on multiple frames of images to be recognized.

[0130] It should be noted that the specific method for recognizing an air gesture based on multiple frames of images to be recognized is a prior art. For details, please refer to the relevant descriptions in the prior art, which will not be elaborated here.

[0131] Please refer to Figure 11 , which is a schematic diagram of the interaction timing between modules in the system architecture of an electronic device during the implementation process of an air gesture recognition method provided in an embodiment of this application. Exemplarily, during the implementation process of the air gesture recognition method, interactions can occur between the AO camera, the preprocessing module, the human hand feature acquisition module, and the air gesture recognition module. The specific interaction process can include S111 to S118, which are described in detail as follows:

[0132] S111. The AO camera captures multiple consecutive frames of images and sequentially sends each frame of image to the preprocessing module according to the acquisition timing.

[0133] Among them, the acquisition timing can be used to represent the sequence of the AO camera capturing each frame of image.

[0134] S112. The preprocessing module obtains the display direction of the screen content.

[0135] The preprocessing module can obtain the display direction of the screen content from the message manager.

[0136] S113. The preprocessing module obtains the target rotation angle corresponding to the display direction of the screen content.

[0137] It should be noted that S113 is Figure 10 a specific implementation manner of S103 shown in Figure 10 . For the content of S113, please refer to the relevant descriptions in S103 shown in

[0138] S114. For each frame of the image, after rotating the image by the target rotation angle, the preprocessing module performs a left - right flipping operation on the image to obtain the image to be recognized, and sends the image to be recognized to the human hand target detection unit.

[0139] It should be noted that S114 is similar to the step in Figure 10 S104, where after rotating the human hand in the human hand image by the target rotation angle, a left - right flipping operation is performed on the human hand. For details, please refer to the relevant descriptions in S104, which will not be elaborated here.

[0140] S115. The human hand target detection unit performs human hand target detection on the image to be recognized, obtains the information of the human hand detection box corresponding to the image to be recognized, and sends the information of the human hand detection box corresponding to the image to be recognized to the human hand classification unit, the human hand key point detection unit, and the air gesture recognition module.

[0141] It can be understood that since it is uncertain whether there is a human hand in each frame of image collected by the AO camera, during the air gesture recognition process, after receiving each frame of image from the AO camera, the human hand feature acquisition module can first preliminarily identify the image with a human hand through the human hand target detection unit, and determine the information of the human hand detection box corresponding to the image with a human hand. It should be noted that the specific content of the information of the human hand detection box can refer to the relevant description in the foregoing embodiments and will not be elaborated here.

[0142] In this embodiment, the purpose of the human hand target detection unit outputting the information of the human hand detection box is to enable the human hand classification unit or the human hand key point detection unit, etc. to locate the human hand in the corresponding image based on the information of the human hand detection box, facilitating subsequent operations on the image by the human hand classification unit, the human hand key point detection unit, or the air gesture recognition module, etc.

[0143] Optionally, the human hand target detection unit can perform human hand target detection on the image to be recognized based on the object detection (OD) algorithm, and obtain the information of the human hand detection box corresponding to the image to be recognized.

[0144] S116. The human hand classification unit classifies the human hand in the image to be recognized based on the information of the human hand detection box corresponding to the image to be recognized, obtains the human hand category corresponding to the image to be recognized, and sends the human hand category corresponding to the image to be recognized to the air gesture recognition module.

[0145] Exemplarily, a pre-trained human hand classification model can be configured in the human hand classification unit. The human hand classification model can be used to classify the human hand to determine the human hand category. Based on this, for each frame of image, the human hand classification unit can determine the human hand category corresponding to the image to be recognized through the pre-trained human hand classification model.

[0146] In an optional implementation manner, in order to reduce the computational complexity of air gesture recognition and improve the efficiency of air gesture recognition, for each frame of image, the human hand classification unit can crop out the human hand local image including only the human hand from the image to be recognized based on the information of the human hand detection box corresponding to the image to be recognized, and input the human hand local image corresponding to the image to be recognized into the pre-trained human hand classification model to obtain the human hand category corresponding to the image to be recognized.

[0147] Exemplarily, the human hand classification model may be a neural network module trained based on a deep learning algorithm. The embodiments of the present application do not make any limitations on the specific type of the human hand classification model, the training method, etc.

[0148] S117. The human hand key point detection unit performs human hand key point detection on the image to be recognized based on the information of the human hand detection frame corresponding to the image to be recognized, obtains the coordinates of the human hand key points corresponding to the image to be recognized, and sends the coordinates of the human hand key points corresponding to the image to be recognized to the air gesture recognition module.

[0149] Exemplarily, a pre-trained human hand key point detection model may be configured in the human hand key point detection unit. The human hand key point detection model can be used to detect the human hand key points in an image and output the coordinates of the human hand key points. Based on this, for each frame of image, the human hand key point detection unit can obtain the coordinates of the human hand key points corresponding to the image to be recognized through the pre-trained human hand key point detection model.

[0150] In an alternative implementation, in order to reduce the computational complexity of air gesture recognition and improve the efficiency of air gesture recognition, for each frame of image, the human hand key point detection unit can crop out a human hand local image including only the human hand from the image to be recognized based on the information of the human hand detection frame corresponding to the image to be recognized, and input the human hand local image corresponding to the image to be recognized into the human hand key point detection model to obtain the coordinates of the human hand key points corresponding to the image to be recognized.

[0151] S118. The air gesture recognition module recognizes air gestures based on the information of multiple frames of images to be recognized.

[0152] Exemplarily, different air gestures can be configured with different recognition strategies. The recognition strategy can be used to represent the characteristics of the hand shape change of the air gesture.

[0153] Based on this, in a specific implementation, air gesture recognition can determine whether there are multiple consecutive frames of images to be recognized that conform to the characteristics of the hand shape change of any air gesture based on the recognition strategies of each air gesture.

[0154] Optionally, in the case where there are multiple consecutive frames of images to be recognized that conform to the characteristics of the hand shape change of the target air gesture, the air gesture recognition module can determine that there is an air gesture currently, and can determine the type of the target air gesture as the type of the currently recognized air gesture. For example, assuming that there are multiple consecutive frames of images to be recognized that conform to the characteristics of the hand shape change of the upward sliding gesture, the air gesture recognition module can determine that there is an upward sliding gesture currently.

[0155] Optionally, in the case where there are no consecutive multiple frames of to-be-recognized images that conform to the hand gesture change characteristics of any air gesture, the air gesture recognition module may determine that there is no air gesture currently.

[0156] Please refer to Figure 12 , which is a schematic diagram of the interaction timing between modules in the system architecture of an electronic device during the implementation process of an air gesture recognition method provided in another embodiment of this application. The difference between this embodiment and Figure 11 the corresponding embodiment is that Figure 11 in the corresponding embodiment, the target preprocessing operation (including rotation operation and left-right flipping operation) is performed on the entire image to implement the target preprocessing of the human hand in the image. In this embodiment, the target preprocessing of the human hand in the image is implemented by performing coordinate conversion on the coordinates of the key points of the human hand corresponding to the image, so as to save the computing resources of the electronic device, reduce the power consumption of the electronic device, and improve the efficiency and accuracy of air gesture recognition.

[0157] Specifically, as Figure 12 shown, during the implementation process of the air gesture recognition method, interactions can occur between the AO camera, the human hand feature acquisition module, the preprocessing module, and the air gesture recognition module. The specific interaction process may include S121 to S129, which are described in detail as follows:

[0158] S121, the AO camera captures consecutive multiple frames of images and sequentially sends each frame of image to the human hand target detection unit according to the acquisition timing.

[0159] It should be noted that S121 is similar to S111 in Figure 11 the corresponding embodiment, and specific reference can be made to Figure 11 the relevant description of S111 in the corresponding embodiment, which will not be elaborated here.

[0160] S122, for each frame of image in sequence according to the acquisition timing, the human hand target detection unit performs human hand target detection on the current frame of image, obtains the information of the human hand detection box corresponding to the current frame of image, and sends the information of the human hand detection box corresponding to the current frame of image to the human hand classification unit, the human hand key point detection unit, and the preprocessing module.

[0161] It should be noted that S122 is similar to S115 in Figure 11 the corresponding embodiment, and specific reference can be made to Figure 11 the relevant description of S115 in the corresponding embodiment, which will not be elaborated here.

[0162] S123. For each frame of image in sequence according to the acquisition timing, based on the information of the hand detection box corresponding to the current frame of image, the hand classification unit classifies the hand in the current frame of image to obtain the hand category corresponding to the current frame of image, and sends the hand category corresponding to the current frame of image to the preprocessing module.

[0163] It should be noted that S123 is similar to Figure 11 S116 in the corresponding embodiment. Specifically, reference can be made to Figure 11 the relevant description of S116 in the corresponding embodiment, which will not be elaborated here.

[0164] S124. For each frame of image in sequence according to the acquisition timing, based on the information of the hand detection box corresponding to the current frame of image, the hand key point detection unit performs hand key point detection on the current frame of image to obtain the coordinates of the hand key points corresponding to the current frame of image, and sends the coordinates of the hand key points corresponding to the current frame of image to the preprocessing module.

[0165] It should be noted that S124 is similar to Figure 11 S117 in the corresponding embodiment. Specifically, reference can be made to Figure 11 the relevant description of S117 in the corresponding embodiment, which will not be elaborated here.

[0166] S125. The preprocessing module obtains the display direction of the screen content.

[0167] S126. The preprocessing module obtains the target rotation angle corresponding to the display direction of the screen content.

[0168] S127. The preprocessing module determines the target coordinate conversion method corresponding to the target rotation angle and the left - right flipping operation.

[0169] S128. The preprocessing module performs coordinate conversion on the current frame of image based on the target coordinate conversion method to obtain the image to be recognized, and sends the information of the image to be recognized to the air gesture recognition module.

[0170] It should be noted that S127 - S128 is similar to Figure 10 In S104 shown, the steps of determining the target coordinate conversion method corresponding to the target rotation angle and the left - right flipping operation, and converting the coordinates of the hand key points based on the target conversion method are similar. Specifically, reference can be made to the relevant description in S104, which will not be elaborated here.

[0171] The information of the image to be recognized sent by the preprocessing module to the air gesture recognition module may include: the information of the hand detection box corresponding to the image to be recognized, the hand category, and the coordinates of the hand key points after coordinate conversion.

[0172] S129. The air gesture recognition module recognizes air gestures based on the information of multiple frames of images to be recognized.

[0173] It should be noted that S129 is similar to S118 in the Figure 11 corresponding embodiment. Specifically, reference can be made to Figure 11 the relevant description of the S118 part in the corresponding embodiment, which will not be elaborated here.

[0174] Based on the same technical concept, the embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program. When the computer-executable program is called by a computer, the computer is enabled to execute one or more steps in any of the above method embodiments.

[0175] Based on the same technical concept, the embodiment of the present application also provides a chip system, including a processor. The processor is coupled to a memory, and the processor executes the computer-executable program stored in the memory to implement one or more steps in any of the above method embodiments. The chip system can be a single chip or a chip module composed of multiple chips.

[0176] Based on the same technical concept, the embodiment of the present application also provides a computer-executable program product. When the computer-executable program product runs on an electronic device, the electronic device is enabled to execute one or more steps in any of the above method embodiments.

[0177] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0179] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware with a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The foregoing storage medium includes various media that can store program codes, such as ROM or random access memory RAM, magnetic disks, or optical discs.

[0180] As described above, the above are only specific implementation manners of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present application should be covered by the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A method for air gesture recognition, characterized in that: Applied to an electronic device including a front camera, the air gesture recognition method includes: Obtaining information of a human hand image collected by the front camera at a first moment; Obtaining the screen content display direction of the electronic device at the first moment; Determining a target preprocessing operation corresponding to the screen content display direction; Performing the target preprocessing operation on the human hand in the human hand image to obtain an image to be recognized; Recognizing an air gesture based on multiple frames of the image to be recognized.

2. The air gesture recognition method according to claim 1, characterized in that, The target preprocessing operation includes a rotation operation and a left - right flipping operation; Determining a target preprocessing operation corresponding to the screen content display direction includes: Determining a target rotation angle corresponding to the screen content display direction; Performing the target preprocessing operation on the human hand in the human hand image includes: After rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand.

3. The air gesture recognition method according to claim 2, characterized in that, After rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand includes: After rotating the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand image.

4. The air gesture recognition method according to claim 3, wherein After rotating the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand image includes: In the case where the resolution of the human hand image is less than or equal to a first resolution and the aspect ratio of the human hand image is 1:1, after rotating the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand image.

5. The air gesture recognition method according to claim 2, wherein The information of the human hand image includes the coordinates of the human hand key points corresponding to the human hand image; after rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand includes: Determining a target coordinate conversion method corresponding to the target rotation angle and the left - right flipping operation; Converting the coordinates of the human hand key points based on the target conversion method.

6. The air gesture recognition method according to claim 2, wherein The information of the human hand image includes the coordinates of the human hand key points corresponding to the human hand image; after rotating the human hand in the human hand image by the target rotation angle, performing a left - right flipping operation on the human hand includes: In the case where the resolution of the human hand image is greater than the first resolution or the aspect ratio of the human hand image is not 1:1, determining a target coordinate conversion method corresponding to the target rotation angle and the left - right flipping operation; Converting the coordinates of the human hand key points based on the target conversion method.

7. The air gesture recognition method according to any one of claims 2-6, characterized in that Determining a target rotation angle corresponding to the screen content display direction includes: In the case where the screen content display direction is a first display direction, determining the target rotation angle to be - 90 degrees; In the case where the screen content display direction is a second display direction, determining the target rotation angle to be 0 degrees; In the case where the screen content display direction is a third display direction, determining the target rotation angle to be 180 degrees; Among them, the first display direction is used to indicate that the screen content starts from the side where the front camera is located and is displayed along the long side of the electronic device; the second display direction is used to indicate that the screen content is displayed along the short side of the electronic device and the front camera is located on the left side of the display direction of the screen content; the third display direction is used to indicate that the screen content is displayed along the short side of the electronic device and the front camera is located on the right side of the display direction of the screen content; a negative target rotation angle is used to indicate counterclockwise rotation, and a positive target rotation angle is used to indicate clockwise rotation.

8. The air gesture recognition method according to claim 5 or 6, characterized in that, Determine the target coordinate conversion method corresponding to the target rotation angle and the left-right flip operation, including: When the target rotation angle is -90 degrees, determine that the target coordinate conversion method is: x = h - 1 - y_raw, y = w - 1 - x_raw; When the target rotation angle is 0 degrees, determine that the target coordinate conversion method is: x = w - 1 - x_raw, y = y_raw; When the target rotation angle is -180 degrees, determine that the target coordinate conversion method is: x = w - 1 - x_raw, y = h - 1 - y_raw; Among them, x_raw is used to represent the abscissa of the hand key point before coordinate conversion, y_raw is used to represent the ordinate of the hand key point before coordinate conversion, h is used to represent the height of the hand image, w is used to represent the width of the hand image, x is used to represent the abscissa of the hand key point after coordinate conversion, and y is used to represent the ordinate of the hand key point after coordinate conversion.

9. An electronic device, characterized in that, Include: One or more processors, and a memory; The memory is coupled to the one or more processors. The memory is used to store computer program code, and the computer program code includes computer instructions. The one or more processors call the computer instructions to cause the electronic device to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions. When the instructions run on the electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 8.

11. A chip system, characterized in that, The chip system is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to cause the electronic device to execute the method according to any one of claims 1 to 8.

Images