Image capture methods, systems, AR headsets, and storage media

Abstract

This invention provides a method, system, AR head-mounted device, and storage medium for image capture, belonging to the field of AR head-mounted devices. The method includes: displaying a virtual viewfinder within the field of view of a user wearing the AR head-mounted device, wherein the image area indicated by the virtual viewfinder changes with the field of view; acquiring the user's electromyography (EMG) signal data, and determining whether the user's hand movement is a preset hand movement based on the EMG signal data, wherein the EMG signal data is acquired by an EMG signal acquisition device worn by the user's hand; and, in response to the user's hand movement being a preset hand movement, capturing the image area indicated by the virtual viewfinder to obtain a target image. The technical solution of this invention improves the reliability of image capture in AR head-mounted devices.

Description

Technical Field

[0001] This invention relates to the field of AR head-mounted device technology, and in particular to a method, system, AR head-mounted device, and storage medium for capturing images. Background Technology

[0002] Augmented Reality (AR) technology provides users with an immersive interactive experience by overlaying virtual information onto the real world. In related technologies, AR headsets typically employ computer vision-based gesture tracking to capture screenshots of real-world scenes or objects. However, visual gesture recognition is susceptible to environmental lighting and occlusion. For example, in insufficient lighting, overexposure, or complex lighting conditions, the image quality captured by the camera deteriorates, leading to reduced gesture recognition accuracy or even failure. Hand occlusion (such as crossed fingers or part of the hand moving out of the field of view) can also cause tracking interruptions, resulting in low reliability. Therefore, improving the reliability of image capture in AR headsets is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This invention provides a method, system, AR head-mounted device, and storage medium for capturing images, aiming to improve the reliability of capturing images in AR head-mounted devices.

[0004] In a first aspect, embodiments of the present invention provide a method for capturing images, applied to an AR head-mounted device, the method comprising: A virtual viewfinder is displayed within the field of view of a user wearing an AR headset, wherein the area of ​​the screen indicated by the virtual viewfinder changes as the field of view changes; Acquire the user's electromyographic signal data, and determine whether the user's hand movement is a preset hand movement based on the electromyographic signal data. The electromyographic signal data is acquired by the electromyographic signal acquisition device worn by the user's hand. In response to the user's hand gesture being a preset hand gesture, the screen area indicated by the virtual viewfinder is captured to obtain the target image.

[0005] In a second aspect, embodiments of the present invention also provide an AR head-mounted device, the AR head-mounted device including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the screen capture method as described in the first aspect.

[0006] Thirdly, embodiments of the present invention also provide an image capture system, including an AR head-mounted device and an electromyography (EMG) signal acquisition device. The AR head-mounted device is communicatively connected to the EMG signal acquisition device. The EMG signal acquisition device is configured to acquire EMG signal data of a user wearing the AR head-mounted device. The AR head-mounted device is configured to implement the image capture method as described in the first aspect.

[0007] Fourthly, embodiments of the present invention also provide a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, which can be executed by one or more processors to implement the screen capture method as described in the first aspect.

[0008] This invention provides a method, system, AR head-mounted device, and storage medium for image capture. In this invention, a virtual viewfinder is displayed within the field of view of a user wearing the AR head-mounted device. The image area indicated by the virtual viewfinder changes with the user's field of view, allowing the user to select the image area to be captured by adjusting the virtual viewfinder through eye gaze. Then, in response to determining that the user's hand movement is a preset hand movement based on electromyography (EMG) signal data, the image area indicated by the virtual viewfinder is captured to obtain the target image. Compared to computer vision-based methods... Gesture tracking technology is used to control AR headsets to crop images. This invention uses electromyography (EMG) signals to control AR headsets to crop images. Even in low light, overexposure, or complex lighting conditions, or even when the camera is blocked, the user's intention can still be reliably recognized, improving the reliability of image cropping in AR headsets. Furthermore, the user only needs to turn their head or eyes to select the area to be cropped, and then control the AR headset to crop the image area through hand movements, improving the ease of use of image cropping in AR headsets. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of a scene in which the image capture method provided in the embodiments of the present invention is implemented; Figure 2 This is a flowchart illustrating a screen capture method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a virtual viewfinder identifier in an embodiment of the present invention; Figure 4 This is another schematic diagram of the virtual framing marker in an embodiment of the present invention; Figure 5 This is a schematic block diagram of the structure of an AR head-mounted device provided in an embodiment of the present invention; Figure 6 This is a schematic block diagram of a screen capture system provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0013] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] Augmented Reality (AR) technology provides users with an immersive interactive experience by overlaying virtual information onto the real world. Among related technologies, AR headsets typically employ computer vision-based gesture tracking to capture screenshots of real-world scenes or objects. However, visual gesture recognition is susceptible to environmental lighting and occlusion. For example, in insufficient lighting, overexposure, or complex lighting conditions, the image quality captured by the camera deteriorates, leading to reduced gesture recognition accuracy or even failure. Hand occlusion (such as crossed fingers or part of the hand moving out of the field of vision) can also cause tracking interruptions, resulting in low reliability.

[0015] To address the aforementioned issues, this invention provides a method, system, AR headset, and storage medium for image capture. In this invention, a virtual viewfinder is displayed within the user's field of view while wearing the AR headset. The area indicated by the virtual viewfinder changes with the user's field of view, allowing the user to select the desired area by adjusting the virtual viewfinder's gaze. Then, in response to determining that the user's hand movement is a preset hand movement based on electromyography (EMG) signal data, the area indicated by the virtual viewfinder is captured to obtain the target image. Compared to using computer vision-based gesture tracking technology to control AR headset image capture, this invention uses EMG signals to control image capture, ensuring reliable user intent recognition even under conditions of insufficient light, overexposure, complex lighting, or when the camera is obstructed, thus improving the reliability of image capture in the AR headset.

[0016] Furthermore, users can select the area to be captured simply by turning their head or eyes, and then control the AR headset to capture the area using hand gestures, thus improving the ease of use for capturing images in AR headsets. Moreover, electromyographic signals precede the muscle potentials generated by physical movements, resulting in more timely feedback, higher sensitivity, and lower latency compared to pure visual recognition. Even if the camera is temporarily obstructed, as long as the electromyographic signals are present, the AR headset can still maintain its judgment of the user's operational intentions, exhibiting higher resistance to interference.

[0017] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0018] Please see Figure 1 , Figure 1 This is a schematic diagram of a scene in which the image capture method provided in the embodiments of the present invention is implemented.

[0019] like Figure 1 As shown, the user wears an AR headset 100 on their head 10 and an electromyography (EMG) signal acquisition device 200 on their hand. The AR headset 100 and the EMG signal acquisition device 200 are communicatively connected. The EMG signal acquisition device 200 collects the user's EMG signal data and sends the collected EMG signal data to the AR headset 100. For example, the EMG signal acquisition device 200 includes an EMG wristband for collecting EMG signal data from the user's forearm flexor carpi radialis and flexor pollicis longus muscles.

[0020] In some embodiments, the AR headset 100 displays a virtual viewfinder within the field of view of a user wearing the AR headset, wherein the image area indicated by the virtual viewfinder changes as the user's field of view changes; it acquires electromyographic signal data of the user sent by the electromyographic signal acquisition device 200, and determines whether the user's hand movement is a preset hand movement based on the electromyographic signal data; in response to the user's hand movement being a preset hand movement, it crops the image area indicated by the virtual viewfinder to obtain the target image.

[0021] In some embodiments, the AR headset 100 is communicatively connected to a terminal device, and the terminal device is communicatively connected to an electromyography (EMG) signal acquisition device 200. The AR headset 100 displays a virtual viewfinder within the user's field of view, wherein the image area indicated by the virtual viewfinder changes with the user's field of view. The AR headset 100 acquires EMG signal data of the user collected by the EMG signal acquisition device 200, which is forwarded by the terminal device. The EMG signal acquisition device 200 collects the user's EMG signal data and sends the collected EMG signal data to the terminal device. Based on the EMG signal data, the AR headset determines whether the user's hand movement is a preset hand movement. In response to the user's hand movement being a preset hand movement, the AR headset captures the image area indicated by the virtual viewfinder to obtain the target image.

[0022] It should be noted that the AR headset 100 may include Augmented Reality (AR) glasses, AR helmets, Mixed Reality (MR) glasses, and MR helmets. Terminal devices may include mobile phones, tablets, laptops, desktop computers, personal digital assistants, and wearable devices, etc.

[0023] The following will combine Figure 1 The following describes in detail the image capture method provided by the embodiments of the present invention. It should be noted that... Figure 1 The scenarios described are only used to explain the image capture method provided in the embodiments of the present invention, but do not constitute a limitation on the application scenarios of the image capture method provided in the embodiments of the present invention.

[0024] Please see Figure 2 , Figure 2 This is a flowchart illustrating a screen capture method provided in an embodiment of the present invention.

[0025] like Figure 2 As shown, the screen capture method includes steps S101 to S103.

[0026] Step S101: Display a virtual viewfinder within the field of view of the user wearing the AR headset, wherein the area of ​​the screen indicated by the virtual viewfinder changes as the field of view changes.

[0027] In this embodiment, the user's field of view changes as the user's head or eyes rotate. The user's field of view can be determined based on the user's field of view center and a preset field of view range. The style of the virtual viewfinder can be set based on actual conditions, and this embodiment does not impose specific limitations on it. For example, the virtual viewfinder may include a virtual viewfinder frame, which can be rectangular, circular, elliptical, etc.

[0028] In some embodiments, displaying a virtual viewfinder icon within the field of view of a user wearing an AR headset includes: displaying the virtual viewfinder icon within the field of view of the user wearing an AR headset in response to a user-triggered command to enable the capture function. The command to enable the capture function can be triggered in several ways: in response to the user's voice data including information indicating that the capture function is enabled; or in response to the user's operation of turning on / off a capture function switch button on the AR headset; or in response to the user's hand-worn electromyography (EMG) signal acquisition device changing from a first posture to a second posture. In this embodiment, when the user enables the capture function, the virtual viewfinder icon is displayed within the field of view of the user wearing the AR headset; when the capture function is disabled, the virtual viewfinder icon is not displayed, thus allowing the user to intuitively know whether the capture function is enabled or not.

[0029] In some embodiments, displaying a virtual viewfinder marker within the field of view of a user wearing an AR headset includes: obtaining the center of the user's field of view; defining the area centered on the user's field of view and bounded by a preset field of view as the user's field of view region; and displaying a virtual viewfinder marker within the user's field of view region. The preset field of view can be obtained by measuring the user's field of view or it can be a default field of view set based on experience; this embodiment does not specifically limit this.

[0030] In some embodiments, obtaining the field of view center of a user wearing an AR headset includes: controlling at least one eye sensor of the AR headset to acquire an image of the user's eyes; determining the center coordinates of the user's pupil and the center coordinates of the reflected light spot on the cornea based on the eye image; determining the offset vector of the user's pupil center relative to the corneal reflected light spot based on the center coordinates of the user's pupil and the center coordinates of the reflected light spot on the cornea; calculating the optical axis direction of the eyeball based on the offset vector of the user's pupil center relative to the corneal reflected light spot and a preset eye physiological model; acquiring the user's head tracking data, and constructing a three-dimensional ray extending along the optical axis direction from the center of the user's eyeball based on the optical axis direction of the eyeball and the head tracking data; determining the intersection point of the three-dimensional camera and the virtual display plane of the AR headset to obtain the field of view center of the user wearing the AR headset.

[0031] In some embodiments, displaying a virtual viewfinder marker within the field of view of a user wearing an AR headset includes: displaying a virtual viewfinder marker on the target object in response to the presence of a target object within the user's field of view; and displaying a virtual viewfinder marker at a target distance from the user in response to the absence of a target object within the user's field of view. The target object includes objects with a certain volume, such as people, animals, plants, buildings, scenery, and items. The target distance can be set based on actual conditions, and this embodiment does not impose a specific limitation; for example, the target distance is 3 meters. In this embodiment, when a target object exists within the user's field of view, a virtual viewfinder marker is displayed on the target object, making the virtual viewfinder marker appear visually attached to the surface of the target object.

[0032] Step S102: Acquire the user's electromyographic signal data. Based on the electromyographic signal data, determine whether the user's hand movement is a preset hand movement. The electromyographic signal data is collected by the electromyographic signal acquisition device worn by the user's hand.

[0033] In this embodiment, an electromyography (EMG) signal acquisition device worn by the user collects EMG signal data from the flexor carpi radialis and flexor pollicis longus muscles of the user's forearm. This EMG signal acquisition device may include an EMG wristband. The preset hand movements can be set based on actual conditions, and this embodiment does not impose specific limitations on them. For example, the preset hand movements may include a pinching motion of the thumb and at least one other finger together and stretching in one direction.

[0034] In some embodiments, acquiring the user's electromyographic (EMG) signal data includes: acquiring the user's EMG signal data sent by an EMG signal acquisition device, wherein the EMG signal acquisition device is communicatively connected to the AR head-mounted device. The communication connection between the EMG signal acquisition device and the AR head-mounted device includes a Bluetooth connection or a WiFi connection.

[0035] In some embodiments, acquiring the user's electromyographic (EMG) signal data includes: acquiring the user's EMG signal data sent by the terminal device; the EMG signal acquisition device communicating with the terminal device; and the terminal device communicating with the AR head-mounted device. The communication connection between the EMG signal acquisition device and the terminal device includes a Bluetooth connection or a WiFi connection, and the communication connection between the terminal device and the AR head-mounted device includes a Bluetooth connection or a WiFi connection.

[0036] In some embodiments, determining whether a user's hand movement is a preset hand movement based on electromyographic signal data includes: performing a hand movement recognition operation based on the electromyographic signal data using a preset hand movement recognition model to obtain a hand movement recognition result, the hand movement recognition result including a hand movement label; determining that the user's hand movement is a preset hand movement if the hand movement label included in the hand movement recognition result is a first hand movement label; and determining that the user's hand movement is not a preset hand movement if the hand movement label included in the hand movement recognition result is a second hand movement label. This embodiment uses a preset hand movement recognition model to perform hand movement recognition on the user's electromyographic signal data, which can quickly and accurately determine whether the user's hand movement is a preset hand movement.

[0037] In some embodiments, the preset hand movement recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes positive sample data or negative sample data. The positive sample data includes first electromyography (EMG) signal data and a first hand movement label. The first movement label indicates that the hand movement corresponding to the first EMG signal data is a preset hand movement. The negative sample data includes second EMG signal data and a second hand movement label. The second hand movement label indicates that the hand movement corresponding to the second EMG signal data is not a preset hand movement.

[0038] In some embodiments, performing hand movement recognition operations based on electromyographic (EMG) signal data using a preset hand movement recognition model to obtain hand movement recognition results includes: performing hand movement recognition operations based on EMG signal data using a hand movement recognition model deployed on an AR head-mounted device to obtain hand movement recognition results; or sending EMG signal data to a cloud server, where the cloud server uses a hand movement recognition model deployed on the cloud server to perform hand movement recognition operations based on the EMG signal data to obtain hand movement recognition results; and obtaining the hand movement recognition results sent by the cloud server.

[0039] Step S103: In response to the user's hand gesture, which is a preset hand gesture, the screen area indicated by the virtual viewfinder is captured to obtain the target image.

[0040] In this embodiment, a virtual viewfinder is displayed within the user's field of view while wearing the AR headset. The area indicated by the virtual viewfinder changes with the user's field of view, allowing the user to select the area to be captured by adjusting the virtual viewfinder with their gaze. Then, in response to determining that the user's hand movement is a preset hand movement based on the user's electromyography (EMG) signal data, the area indicated by the virtual viewfinder is captured to obtain the target image. Compared to using computer vision-based gesture tracking technology to control the AR headset to capture the image area, this embodiment uses EMG signals to control the AR headset to capture the image area. Even under conditions of insufficient light, overexposure, or complex lighting, or even when the camera is obstructed, it can still reliably identify the user's operating intention, improving the reliability of image capture in the AR headset.

[0041] In some embodiments, the image capture method provided by this invention further includes: in response to a user's hand gesture being a preset hand gesture, changing the visual presentation of the virtual viewfinder and / or outputting a preset prompt audio, wherein the preset prompt audio is used to notify the user that the AR headset has received the hand gesture. This embodiment changes the visual presentation of the virtual viewfinder and / or outputs a preset prompt audio when the user's hand gesture is a preset hand gesture, thereby providing feedback to the user's AR headset that the intention to capture the image area indicated by the virtual viewfinder has been received, resulting in a better user experience.

[0042] In some embodiments, changing the visual presentation of the virtual viewfinder includes switching the visual presentation of the virtual viewfinder from a first visual presentation to a second visual presentation, wherein the first visual presentation and the second visual presentation are different. The first and second visual presentations can be set based on actual circumstances, and this embodiment of the invention does not specifically limit their selection. For example, as... Figure 3 As shown, before the user's hand has made the action of pinching together with at least one other finger and stretching in one direction, the virtual viewfinder is displayed in a first-person perspective, such as... Figure 4 As shown, after the user makes a pinching motion with their thumb and at least one other finger and stretches it in one direction, the virtual viewfinder is displayed in a second-view presentation mode.

[0043] In some embodiments, changing the visual presentation of the virtual viewfinder includes changing the display color of the virtual viewfinder. For example, in response to a user's hand gesture (a preset hand gesture), the display color of the virtual viewfinder is switched from white to green, or the corresponding border of the virtual viewfinder is highlighted.

[0044] In some embodiments, the image capture method provided by the present invention further includes: in response to a user's hand gesture being a preset hand gesture, rendering a shadow effect within the image area indicated by the virtual viewfinder, so that the image area indicated by the virtual viewfinder appears to float above the real background; performing a shrinking animation operation on the image area indicated by the virtual viewfinder to shrink the image area indicated by the virtual viewfinder to a thumbnail of a preset size; and controlling the thumbnail to smoothly move from the position of the virtual viewfinder to the target spatial position corresponding to the user's hand for display and / or controlling the thumbnail to move from the position of the virtual viewfinder to a preset favorites list. The preset size can be set based on actual conditions, and the present invention does not specifically limit this. The preset favorites list can be located in the left sidebar, right sidebar, top sidebar, or bottom sidebar of the virtual display plane of the AR head-mounted device. This embodiment visually achieves the animation effect of extracting the screen area indicated by the virtual viewfinder from the real world, shrinking it, and moving it to the target spatial location for display and / or preset favorites, solving the problem of the disconnect between virtual and reality in AR interaction and providing a better user experience.

[0045] In some embodiments, the target spatial location corresponding to the user's hand may include a spatial location corresponding to the center of the user's hand and a spatial location corresponding to the side of the user's hand. The distance between the spatial location corresponding to the side of the user's hand and the spatial location corresponding to the center of the user's hand is less than or equal to a preset distance. This preset distance can be set based on actual conditions, and this embodiment of the invention does not impose a specific limitation on it.

[0046] In some embodiments, after step S103, the method further includes: displaying a preset floating icon; and sending the target image to a contact selected by the user in response to a user's triggering operation on the preset floating icon. The contact can be pre-selected by the user or currently selected. For example, in response to a user's triggering operation on the preset floating icon, a contact list is displayed; and in response to a user's selection of a contact from the contact list, the target image is sent to the selected contact. This embodiment automatically displays the preset floating icon after capturing the target image, allowing the user to conveniently send the target image to the selected contact, thus improving the user experience. In one embodiment, the preset floating icon can be displayed in specific scenarios, such as when it is determined to be in a sports scenario and / or a social sharing scenario; and the target image can be sent to the user's selected contact in response to a user's triggering operation on the preset floating icon.

[0047] For example, the image capture method provided in this embodiment of the invention can be applied to industrial maintenance and digital inspection scenarios: a maintenance worker wearing AR glasses and an electromyography (EMG) bracelet discovers a damaged part; the worker looks at the part, and the virtual viewfinder automatically locks onto the damaged part; the worker only needs to make a pinching motion with one hand (even if the other hand is holding a tool) to capture the image area corresponding to the part locked by the virtual viewfinder and send it to the remote expert selected by the user, thereby solving the problem of low visual gesture recognition rate in complex industrial environments, and because it is EMG-controlled, the worker can operate accurately even when wearing gloves.

[0048] For example, the image capture method provided in this embodiment of the invention can be applied to interactive information collection scenarios in museums / exhibition halls (where visitors want to "collect" a detail while browsing artworks): The user wears AR glasses and an electromyography (EMG) bracelet; when viewing a famous painting, the user looks at a specific detail in the painting, and the virtual viewfinder is aligned with that detail; after the user makes a pinching motion with their hand, the AR glasses display the image corresponding to that detail "jumping out" of the painting, accompanied by a shrinking animation, and flying into the virtual collection on the left side of the virtual display plane of the AR glasses, thereby enhancing the sense of "ownership" and the fun of interaction, turning a simple photo-taking action into a virtual "grabbing" behavior.

[0049] For example, the image capture method provided in this embodiment of the invention can be applied to outdoor sports and social sharing scenarios (when cycling or climbing, the user's hands may be restricted (e.g., holding the handlebars)): The cyclist wears AR glasses and an electromyography (EMG) bracelet; the cyclist sees the roadside scenery through the AR glasses, and the head stabilizer ensures that the virtual viewfinder locks onto the scenery; the cyclist only needs to make a pinching motion on the handlebars (without releasing the handlebars), and the AR glasses capture the scenery selected by the virtual viewfinder to obtain a landscape image, and automatically display a floating "Send to a Friend" icon. The user can send the landscape image to the contact selected by looking at the floating icon, thereby improving security and realizing non-contact shutter control in specific sports states.

[0050] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of an AR head-mounted device provided in an embodiment of the present invention.

[0051] like Figure 5 As shown, the AR head-mounted device 100 includes a processor 101 and a memory 102, which are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0052] Specifically, processor 101 provides computing and control capabilities to support the operation of the entire AR headset. Processor 101 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0053] Specifically, the memory 102 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0054] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the embodiments of the present invention, and does not constitute a limitation on the AR head-mounted device to which the embodiments of the present invention are applied. A specific AR head-mounted device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0055] The processor 101 is used to run a computer program stored in the memory 102, and implements any of the screen capture methods provided in the embodiments of the present invention when executing the computer program.

[0056] In one embodiment, the processor 101 is configured to run a computer program stored in a memory, and when executing the computer program, to perform the following steps: A virtual viewfinder is displayed within the field of view of a user wearing an AR headset, wherein the area of ​​the screen indicated by the virtual viewfinder changes as the field of view changes; Acquire the user's electromyographic signal data, and determine whether the user's hand movement is a preset hand movement based on the electromyographic signal data. The electromyographic signal data is acquired by the electromyographic signal acquisition device worn by the user's hand. In response to the user's hand gesture being a preset hand gesture, the screen area indicated by the virtual viewfinder is captured to obtain the target image.

[0057] In some embodiments, when the processor 101 determines whether the user's hand movement is a preset hand movement based on the electromyographic signal data, it is configured to: A hand movement recognition operation is performed based on the electromyographic signal data using a preset hand movement recognition model to obtain hand movement recognition results, which include hand movement labels. In response to the hand movement recognition result including a first hand movement label, it is determined that the user's hand movement is a preset hand movement; and In response to the hand movement recognition result including a second hand movement label, it is determined that the user's hand movement is not a preset hand movement.

[0058] In some embodiments, the processor 101 is further configured to perform the following steps: In response to the user's hand gesture being a preset hand gesture, the visual presentation of the virtual viewfinder is changed and / or a preset prompt audio is output, the preset prompt audio being used to notify the user that the AR headset has received the hand gesture.

[0059] In some embodiments, when the processor 101 implements the change in the visual presentation of the virtual viewfinder, it is configured to: Change the display color of the virtual viewfinder.

[0060] In some embodiments, when the processor 101 displays a virtual viewfinder icon within the field of view of a user wearing the AR headset, it is configured to: In response to the presence of a target object within the user's field of view of the AR headset, the virtual viewfinder is displayed on the target object; In response to the absence of a target object within the user's field of view of the AR headset, the virtual viewfinder is displayed at a distance from the target object.

[0061] In some embodiments, the processor 101 is further configured to implement: A shadow effect is rendered within the screen area indicated by the virtual viewfinder, so that the screen area indicated by the virtual viewfinder appears to rise visually from the real background; Perform a zoom-out animation operation on the screen area indicated by the virtual viewfinder to shrink the screen area indicated by the virtual viewfinder to a thumbnail of a preset size; and Control the thumbnail to smoothly move from the position of the virtual viewfinder to the target spatial position corresponding to the user's hand for display and / or control the thumbnail to move from the position of the virtual viewfinder to a preset favorites.

[0062] In some embodiments, after the processor 101 implements a preset hand gesture in response to the user's hand gesture, and crops the screen area indicated by the virtual viewfinder to obtain the target image, it is further configured to: Displays a preset floating icon; In response to the user's triggering action on the preset floating icon, the target image is sent to the contact selected by the user.

[0063] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the AR head-mounted device described above can be referred to the corresponding process in the aforementioned image capture method embodiment, and will not be repeated here.

[0064] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of a screen capture system provided in an embodiment of the present invention.

[0065] like Figure 6 As shown, the image capture system 300 includes an AR head-mounted device 100 and an electromyography (EMG) signal acquisition device 200. The AR head-mounted device 100 is communicatively connected to the EMG signal acquisition device 200. The EMG signal acquisition device 200 is configured to acquire EMG signal data of a user wearing the AR head-mounted device 100. The AR head-mounted device 100 is configured to implement any of the image capture methods provided in the embodiments of the present invention.

[0066] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described image capture system can be referred to the corresponding process in the aforementioned image capture method embodiments, and will not be repeated here.

[0067] This invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement any of the screen capture methods provided in the specification of this invention.

[0068] The storage medium can be volatile or non-volatile. It can be an internal storage unit of the AR headset described in the foregoing embodiments, such as the hard drive or memory of the AR headset. Alternatively, it can be an external storage device of the AR headset, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the AR headset.

[0069] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0070] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0071] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The above descriptions are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for capturing an image, characterized in that, Applied to AR head-mounted devices, the method includes: A virtual viewfinder is displayed within the field of view of a user wearing an AR headset, wherein the area of ​​the screen indicated by the virtual viewfinder changes as the field of view changes; Acquire the user's electromyographic signal data, and determine whether the user's hand movement is a preset hand movement based on the electromyographic signal data. The electromyographic signal data is acquired by the electromyographic signal acquisition device worn by the user's hand. In response to the user's hand gesture being a preset hand gesture, the screen area indicated by the virtual viewfinder is captured to obtain the target image.

2. The image capture method according to claim 1, characterized in that, The step of determining whether the user's hand movement is a preset hand movement based on the electromyographic signal data includes: A hand movement recognition operation is performed based on the electromyographic signal data using a preset hand movement recognition model to obtain hand movement recognition results, which include hand movement labels. In response to the hand movement recognition result including a first hand movement label, it is determined that the user's hand movement is a preset hand movement; and In response to the hand movement recognition result including a second hand movement label, it is determined that the user's hand movement is not a preset hand movement.

3. The image capture method according to claim 1, characterized in that, The method further includes: In response to the user's hand gesture being a preset hand gesture, the visual presentation of the virtual viewfinder is changed and / or a preset prompt audio is output, the preset prompt audio being used to notify the user that the AR headset has received the hand gesture.

4. The image capture method according to claim 3, characterized in that, The change in the visual presentation of the virtual viewfinder includes: Change the display color of the virtual viewfinder.

5. The image capture method according to claim 1, characterized in that, The step of displaying virtual viewfinder markers within the field of view of the user wearing the AR headset includes: In response to the presence of a target object within the user's field of view of the AR headset, the virtual viewfinder is displayed on the target object; In response to the absence of a target object within the user's field of view of the AR headset, the virtual viewfinder is displayed at a distance from the target object.

6. The image capture method according to claim 1, characterized in that, The method further includes: In response to the user's hand gesture being a preset hand gesture, a shadow effect is rendered within the screen area indicated by the virtual viewfinder, so that the screen area indicated by the virtual viewfinder appears to float up from the real background. Perform a zoom-out animation operation on the screen area indicated by the virtual viewfinder to shrink the screen area indicated by the virtual viewfinder to a thumbnail of a preset size; and Control the thumbnail to smoothly move from the position of the virtual viewfinder to the target spatial position corresponding to the user's hand for display and / or control the thumbnail to move from the position of the virtual viewfinder to a preset favorites.

7. The image capture method according to any one of claims 1-6, characterized in that, The response to the user's hand gesture is a preset hand gesture. After cropping the screen area indicated by the virtual viewfinder to obtain the target image, the process further includes: Displays a preset floating icon; In response to the user's triggering action on the preset floating icon, the target image is sent to the contact selected by the user.

8. An AR head-mounted device, characterized in that, The AR head-mounted device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for establishing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of the screen capture method as described in any one of claims 1 to 7.

9. A screen capture system, characterized in that, The device includes an AR headset and an electromyography (EMG) signal acquisition device. The AR headset is communicatively connected to the EMG signal acquisition device. The EMG signal acquisition device is configured to acquire EMG signal data of a user wearing the AR headset. The AR headset is configured to implement the steps of the image capture method according to any one of claims 1 to 7.

10. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the screen capture method according to any one of claims 1 to 7.

Images