Image capture eyewear with automatic sending of images based on environmental determination of recipient

Abstract

Systems, methods, and instructions on a non-transitory computer-readable medium for automatically sending an image to a designated recipient based on a background selection criteria (e.g., one or more of location, content, or quality) are disclosed. The system includes a camera and a user interface that triggers the camera to capture an image. The method includes identifying a background selection criteria, identifying a designated recipient, receiving the image captured by the camera, determining image data of the captured image, comparing the determined image data to the identified background selection criteria to identify a match, and sending the captured image to the group of designated recipients in response to the identified match.

Description

[0001] This application is a divisional application of the invention patent entitled "Image Capture Eyewear with Recipient Determination Based on Environment for Automatic Image Transmission", filed on September 23, 2021, with application number 2021800667367.

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 085,296, filed September 30, 2020, and U.S. Patent Application Serial No. 17 / 147,872, filed January 31, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This topic relates to image capture eyewear, such as smart glasses, and more specifically, to image capture eyewear systems that automatically send captured images to a receiver based on image background selection criteria. Background Technology

[0004] Today's available image-capturing eyewear, such as smart glasses, headbands, and headsets, integrates lenses, cameras, and wireless network transceiver devices. Users of such eyewear expect increased functionality to improve convenience and efficiency. Attached Figure Description

[0005] The accompanying drawings illustrate specific embodiments by way of example only and not by way of limitation. In the drawings, the same reference numerals denote the same or similar elements. When multiple similar elements exist, a single reference numeral may be assigned to multiple similar elements, where the letter designation refers to a specific element. Lowercase letter designations may be omitted when elements are mentioned jointly or when referring to one or more non-specific elements within the element set.

[0006] The features of the various examples described will be readily understood from the following detailed embodiments with reference to the accompanying drawings. Unless otherwise indicated, the various elements shown in the drawings are not drawn to scale. The dimensions of the individual elements may be enlarged or reduced for clarity. The following figures are included in the drawings: Figure 1A This is a side view (right) of an exemplary hardware configuration suitable for use in an image capture eyewear with a context transmission system. Figure 1B yes Figure 1A A partial cross-sectional perspective view of the right corner of the eye-wearing device, depicting the right visible light camera and circuit board; Figure 1C yes Figure 1A A side view (left) of an exemplary hardware configuration for an eye-wearing device, showing the left visible light camera; Figure 1D yes Figure 1CA partial cross-sectional perspective view of the left corner of the eye-wearing device, depicting the left visible light camera and circuit board; Figure 2A and Figure 2B This is a rear view of an exemplary hardware configuration of an eyewear device utilized in an image capture eyewear device with a context transmission system; Figure 3 It is a graphical depiction of a 3D scene, the left raw image captured by the left visible light camera, and the right raw image captured by the right visible light camera; Figure 4 It is a functional block diagram of an exemplary image capture eyewear in a context-based transmission system, which includes mobile devices (e.g., eyewear devices) and server systems connected via various networks; Figure 5 It has Figure 4 A graphical representation of an exemplary hardware configuration of a mobile device for image capture of an eye-wearing device in a scenario transmission system; Figure 6A and Figure 6B These are example graphical user interfaces used to specify the recipient and background selection criteria, respectively; Figure 7A , Figure 7B , Figure 7C , Figure 7D , Figure 7E , Figure 7F and Figure 7G This is a flowchart outlining exemplary steps for implementing an image capture eyewear with a context transmission system. Detailed Implementation

[0007] The examples described herein involve automatically sending an image captured using an eye-wearing device to designated recipients in response to a match of image data associated with a captured image based on background selection criteria (e.g., location, environment, or quality). The eye-wearing device includes a camera and a user interface. The captured image is sent by: identifying background selection criteria, identifying designated recipients, receiving the image captured by the camera, determining the image data of the captured image, comparing the determined image data with the identified background selection criteria to identify a match, and automatically sending the captured image to the designated group of recipients in response to the identified match.

[0008] Although this article describes various systems and methods by referring to the automatic transmission of images captured by eye-wearing devices, the techniques described can be applied to other mobile devices, such as tablets, watches, or cellular phones.

[0009] The following detailed description includes systems, methods, techniques, instruction sequences, and computer program products illustrating the examples set forth in this disclosure. Numerous details and examples are included to provide a thorough understanding of the disclosed subject matter and its related teachings. However, those skilled in the art will understand how to apply the related teachings without such details. The aspects of the disclosed subject matter are not limited to the specific devices, systems, and methods described, as the related teachings can be applied or practiced in various ways. The terminology and naming used herein are for descriptive purposes only and are not intended to be limiting. Typically, well-known examples of instructions, protocols, structures, and techniques are not necessarily shown in detail.

[0010] As used herein, the terms “coupled” or “connected” refer to any logical, optical, physical, or electrical connection, including links, through which electrical or magnetic signals generated or provided by one system element are transmitted to another coupled or connected system element. Unless otherwise stated, coupled or connected elements or devices are not necessarily directly connected to each other and may be separated by intermediate components, elements, or communication media, one or more of which may modify, manipulate, or carry electrical signals. The term “on” means directly supported by an element or indirectly supported by an element through another element integrated into or supported by that element.

[0011] For purposes of illustration and discussion, the orientation of eye-wearing devices, other mobile devices, associated components, and any other device including a camera, inertial measurement unit, or both shown in any of the accompanying figures is given by way of example only. In operation, the eye-wearing device may be oriented in any other direction suitable for the particular application of the eye-wearing device, such as up, down, sideways, or any other orientation. Furthermore, for the purposes of this document, any directional terms such as front, back, inside, outside, towards, left, right, sideways, longitudinal, up, down, high, low, top, bottom, side, horizontal, vertical, and diagonal are used by way of example only and do not limit the orientation or orientation of any camera or inertial measurement unit as constructed or otherwise described herein.

[0012] Other objects, advantages, and novel features of the examples will be set forth in part in the detailed description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings, or may be learned by means of production or operation of the examples. The objects and advantages of this subject matter may be realized and achieved by means of the methods, means, and combinations particularly pointed out in the appended claims.

[0013] Now refer in detail to the accompanying drawings and the examples discussed below.

[0014] Figure 1AThis is a side view (right) of an exemplary hardware configuration of an eye-wearing device 100 including a touch-sensitive input device or touchpad 181. As shown, the touchpad 181 may have a subtle and barely perceptible boundary; alternatively, the boundary may be clearly visible or include raised or otherwise tactile edges that provide feedback to the user about the position and boundary of the touchpad 181. In other embodiments, the eye-wearing device 100 may include a touchpad on the left side.

[0015] The surface of touchpad 181 is configured to detect finger touches, taps, and other gestures (e.g., motion touches) for use with the GUI displayed on the image display of the eye-wearing device, allowing users to navigate and select menu options intuitively, thus improving and simplifying the user experience. 。

[0016] Detection of finger input on touchpad 181 enables several functions. For example, touching anywhere on touchpad 181 can cause the GUI to be displayed or an item to be highlighted on an image display, which may be projected onto at least one of optical components 180A, 180B. Double-clicking on touchpad 181 selects an item or icon. Sliding or swiping a finger in a specific direction (e.g., from front to back, from back to front, from top to bottom, or from bottom to top) causes an item or icon to slide or scroll in that direction; for example, to move to the next item, icon, video, image, page, or slideshow. Sliding a finger in another direction allows sliding or scrolling in the opposite direction; for example, to move to the previous item, icon, video, image, page, or slideshow. Touchpad 181 can be located virtually anywhere on the eye-wearing device 100.

[0017] In one example, a finger gesture recognized on touchpad 181 initiates image capture by eyewear device 100, image capture using automatic transmission, and selection or pressing of graphical user interface elements in an image displayed on the optical components 180A, 180B. A single-click finger gesture can be set to trigger image capture without transmission; and a double-click finger gesture or a tap and hold can be set to trigger image capture and automatic transmission in response to a previously defined contextual selection criterion. Although the user interface is shown and described as a touchpad, the user interface may include other components besides the touchpad or in place of the touchpad, such as buttons.

[0018] As shown in the figure, the eye-wearing device 100 includes a right visible light camera 114B. As further described herein, two cameras 114A and 114B capture image information of the scene from two different viewpoints. The two captured images can be used to project a 3D display onto an image display for viewing using 3D glasses.

[0019] The eye-worn device 100 includes a right optical component 180B, which has an image display for presenting images, such as depth images. Figure 1A and Figure 1B As shown, the eye-wearing device 100 includes a right visible light camera 114B. The eye-wearing device 100 may include multiple visible light cameras 114A, 114B forming a passive three-dimensional camera, such as a stereo camera, wherein the right visible light camera 114B is located at the right corner 110B. Figures 1C to 1D As shown, the eye-wearing device 100 also includes a left visible light camera 114A.

[0020] Left and right visible light cameras 114A and 114B are sensitive to wavelengths within the visible light range. Each visible light camera 114A and 114B has a different forward field of view, which overlaps to enable the generation of a three-dimensional depth image; for example, the right visible light camera 114B depicts a right field of view 111B. Typically, a "field of view" is a portion of a scene in space that is visible to a camera at a specific location and orientation. Fields of view 111A and 111B have an overlapping field of view 304 (…). Figure 3 When a visible light camera captures an image, objects or object features outside the field of view 111A, 111B are not recorded in the original image (e.g., photograph or picture). 。 The field of view describes the angular range or amplitude of electromagnetic radiation picked up by the image sensor of the visible light camera 114A, 114B in an image of a given scene. The field of view can be expressed as the angular size of the view frustum; that is, the viewing angle. The viewing angle can be measured horizontally, vertically, or diagonally.

[0021] In the exemplary configuration, one or both of the visible light cameras 114A and 114B have a 100° field of view and a resolution of 480×480 pixels. "Coverage angle" describes the visible light camera 114A, 114B, or infrared camera 410 that can effectively image (see...). Figure 2A The angular range of the lens. Typically, camera lenses produce an image circle large enough to completely cover the camera's film or sensor, which may include some vignetting (e.g., the image darkens towards the edges compared to the center). If the camera lens's coverage angle does not extend across the sensor, the image circle will be visible, typically with strong vignetting towards the edges, and the effective viewing angle will be limited to the coverage angle.

[0022] Examples of such visible light cameras 114A and 114B include high-resolution complementary metal-oxide-semiconductor (CMOS) image sensors and digital VGA cameras (video graphics arrays) capable of having resolutions of 640p (e.g., 640 × 480 pixels, totaling 0.3 megapixels), 720p, or 1080p. 。Other examples of visible light cameras 114A and 114B include those capable of capturing high-definition (HD) still images and storing these images at a resolution of 1642 × 1642 pixels (or greater); or recording high-definition video at a high frame rate (e.g., thirty to sixty frames per second or more) and storing the recording at a resolution of 1216 × 1216 pixels (or greater). 。

[0023] The eye-wearing device 100 can capture image sensor data from visible light cameras 114A and 114B, as well as geolocation data digitized by an image processor, for storage in memory. The visible light cameras 114A and 114B capture corresponding left and right raw images in a two-dimensional spatial domain. These raw images include a pixel matrix in a two-dimensional coordinate system, which includes an X-axis for horizontal positioning and a Y-axis for vertical positioning. Each pixel includes color attribute values ​​(e.g., red pixel light value, green pixel light value, or blue pixel light value); and positioning attributes (e.g., X-axis coordinates and Y-axis coordinates).

[0024] In order to capture stereoscopic images for later display as a 3D projection, image processor 412 (in...) Figure 4 (As shown in the diagram) Visible light cameras 114A and 114B can be coupled to receive and store visual image information. Image processor 412 or another processor controls the operation of visible light cameras 114A and 114B to act as stereo cameras simulating human binocular vision and can add timestamps to each image. The timestamps on each pair of images allow the images to be displayed together as part of a 3D projection. The 3D projection produces an immersive and realistic experience, which is desired in various scenarios including virtual reality (VR) and video games.

[0025] Figure 1B yes Figure 1A A cross-sectional perspective view of the right corner portion 110B of the eye-wearing device 100, which depicts the right visible light camera 114B and the circuit board of the camera system. Figure 1C yes Figure 1A A side view (left) of an exemplary hardware configuration of an eye-wearing device 100, showing the left visible light camera 114A of the camera system. Figure 1D yes Figure 1C A cross-sectional perspective view of the left corner portion 110A of the eye-wearing device, which depicts the left visible light camera 114A of the three-dimensional camera and the circuit board.

[0026] Except for the connection and coupling located on the left side 170A, the structure and arrangement of the left visible light camera 114A are similar to those of the right visible light camera 114B. For example... Figure 1BAs shown in the example, the eye-wearing device 100 includes a right visible light camera 114B and a circuit board 140B, which may be a flexible printed circuit board (PCB). A right hinge 126B connects the right corner portion 110B to the right temple 125B of the eye-wearing device 100. In some examples, components such as the right visible light camera 114B, the flexible PCB 140B, or other electrical connectors or contacts may be located on the right temple 125B or the right hinge 126B. A left hinge 126A connects the left corner portion 110A to the left temple 125A of the eye-wearing device 100. In some examples, components such as the left visible light camera 114A, the flexible PCB 140A, or other electrical connectors or contacts may be located on the left temple 125A or the left hinge 126A.

[0027] The right corner portion 110B includes a corner body 190 and a corner cover. Figure 1B The corner caps are omitted in the cross-section. Inside the right corner 110B are various interconnected circuit boards, such as PCBs or flexible PCBs, including controller circuitry for the right visible light camera 114B, a microphone, low-power wireless circuitry (e.g., for short-range wireless network communication via Bluetooth™), and high-speed wireless circuitry (e.g., for wireless LAN communication via Wi-Fi). 。

[0028] The right visible light camera 114B is coupled to or disposed on the flexible PCB 140B and is covered by a visible light camera overlay lens, which is aimed through an opening formed in the frame 105. For example, the right edge 107B of the frame 105, as... Figure 2A As shown, it connects to the right corner 110B and includes an opening for a visible light camera cover lens. Frame 105 includes a front side configured to face outwards and away from the user's eye. The opening for the visible light camera cover lens is formed on and extends through the front or outer side of frame 105. In the example, the right visible light camera 114B has an outward-facing field of view 111B (in... Figure 3 (As shown in the diagram), its line of sight or viewing angle is related to the right eye of the user of the eye-wearing device 100. The visible light camera cover lens may also be adhered to the front side or outward-facing surface of the right corner portion 110B, wherein the opening forms an outward-facing coverage angle, but in a different outward direction. Coupling may also be achieved indirectly via an intermediary member.

[0029] like Figure 1B As shown, the flexible PCB 140B is disposed within the right corner portion 110B and coupled to one or more other components housed in the right corner portion 110B. Although shown as being formed on a circuit board on the right corner portion 110B, the right visible light camera 114B may be formed on a circuit board on the left corner portion 110A, temples 125A, 125B, or frame 105.

[0030] Figure 2A and Figure 2B This is a rear perspective view of an exemplary hardware configuration of an eye-wearing device 100 that includes two different types of image displays. The size and shape of the eye-wearing device 100 are designed to be configured for wear by a user; in this example, it is in the form of eyeglasses. The eye-wearing device 100 may take other forms and may be combined with other types of frames, such as headbands, headphones, or helmets.

[0031] In the example of eyeglasses, the eye-wearing device 100 includes a frame 105 comprising a left edge 107A connected to the right edge 107B via a nose bridge 106 adapted for support by the user's nose. The left and right edges 107A, 107B include corresponding apertures 175A, 175B that hold corresponding optical elements 180A, 180B, such as lenses and display devices. As used herein, the term "lens" is intended to include a sheet of transparent or translucent glass or plastic having a curved or flat surface that causes light to converge / diverge or to cause little or no convergence or divergence.

[0032] Although shown as having two optical elements 180A, 180B, the eyewear device 100 may include other arrangements, such as a single optical element (or it may not include any optical elements 180A, 180B), depending on the application or intended use of the eyewear device 100. As further shown, the eyewear device 100 includes a left corner portion 110A adjacent to the left side face 170A of the frame 105 and a right corner portion 110B adjacent to the right side face 170B of the frame 105. The corner portions 110A, 110B may be integrated into the corresponding sides 170A, 170B of the frame 105 (as shown) or implemented as separate components attached to the corresponding sides 170A, 170B of the frame 105. Alternatively, the corner portions 110A, 110B may be integrated into temples (not shown) attached to the frame 105.

[0033] In one example, the image display of optical components 180A and 180B includes an integrated image display. For example... Figure 2A As shown, each optical component 180A, 180B includes a suitable display matrix 177, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, or any other such display. Each optical component 180A, 180B also includes one or more optical layers 176, which may include any combination of lenses, optical coatings, prisms, mirrors, waveguides, optical strips, and other optical components. Optical layers 176A, 176B, ..., 176N (in... Figure 2AThe optical layer 176A-N (shown herein as 176A-N) may include a prism having suitable dimensions and construction and including a first surface for receiving light from a display matrix and a second surface for emitting light toward a user's eye. The prism of the optical layer 176A-N extends over all or at least a portion of corresponding apertures 175A, 175B formed in the left and right edges 107A, 107B, to allow the user to see the second surface of the prism when viewing through the corresponding left and right edges 107A, 107B. The first surface of the prism of the optical layer 176A-N faces upward from the frame 105, and the display matrix 177 covers the prism such that photons and rays emitted by the display matrix 177 illuminate the first surface. The prism is sized and shaped such that light is refracted within the prism and directed to the user's eye by the second surface of the prism of the optical layer 176A-N. In this respect, the second surface of the prism of the optical layer 176A-N may be convex to direct light toward the center of the eye. The size and shape of the prism can be optionally designed to magnify the image projected by the display matrix 177, and the light travels through the prism such that the image viewed from the second surface is larger than the image emitted from the display matrix 177 in one or more dimensions.

[0034] In one example, optical layers 176A-N may include a transparent LCD layer (keeping the lens open) unless and until a voltage is applied that makes the layer opaque (closing or blocking the lens). An image processor 412 on the eyewear device 100 may execute a program to apply voltage to the LCD layer to create an active shutter system, thereby adapting the eyewear device 100 for viewing visual content displayed as a three-dimensional projection. Technologies other than LCDs may be used in the active shutter mode, including other types of reactive layers that respond to voltage or another type of input.

[0035] In another example, the image display device with optical components 180A and 180B includes, for example... Figure 2B The projected image display shown. Each optical component 180A, 180B includes a laser projector 150, which is a three-color laser projector using a scanning mirror or galvanometer. During operation, a light source such as the laser projector 150 is positioned within or above one of the temples 125A, 125B of the eyewear device 100. In this example, optical component 180B includes one or more optical strips 155A, 155B, ... 155N (in... Figure 2B (shown as 155A-N), which are spaced apart on the width of the lens of each optical component 180A, 180B, or on the depth of the lens between the front and rear surfaces of the lens.

[0036] As photons projected by the laser projector 150 travel through the lenses of each optical component 180A, 180B, they encounter optical strips 155A-N. When a particular photon encounters a particular optical strip, it is either redirected toward the user's eye or propagated to the next optical strip. A combination of modulation of the laser projector 150 and modulation of the optical strips can control a particular photon or beam of light. In the example, the processor controls the optical strips 155A-N by emitting mechanical, acoustic, or electromagnetic signals. Although shown as having two optical components 180A, 180B, the eye-wearing device 100 may include other arrangements, such as single or three optical components, or each optical component 180A, 180B may be arranged in a different configuration, depending on the application of the eye-wearing device 100 or the intended user.

[0037] In another example, Figure 2B The eye-wearing device 100 shown may include two projectors, a left projector 150A (not shown) and a right projector 150B (shown as projector 150). The left optical assembly 180A may include a left display matrix 177A (not shown) or left optical strips 155'A, 155'B, ..., 155'N (155', A to N, not shown), configured to interact with light from the left projector 150A. Similarly, the right optical assembly 180B may include a right display matrix 177B (not shown) or right optical strips 155''A, 155''B, ..., 155''N (155'', A to N, not shown), configured to interact with light from the right projector 150B. In this example, the eye-wearing device 100 includes a left display and a right display.

[0038] Figure 3 This is a graphical depiction of a 3D scene 306, a left raw image 302A captured by a left visible light camera 114A, and a right raw image 302B captured by a right visible light camera 114B. As shown, the left field of view 111A may overlap with the right field of view 111B. The overlapping field of view 304 represents the portion captured by the two cameras 114A and 114B in the image. The term "overlapping" in relation to field of view means that the pixel matrix in the generated raw image overlaps by thirty percent (30%) or more. "Substantially overlapping" means that the pixel matrix in the generated raw image or the pixel matrix in the infrared image of the scene overlaps by fifty percent (50%) or more. As described herein, the two raw images 302A and 302B may be processed to include a timestamp that allows the images to be displayed together as part of a 3D projection.

[0039] To capture stereoscopic images, such as Figure 3As shown, a pair of raw red-green-blue (RGB) images of a real scene 306 are captured at a given time—a left raw image 302A captured by the left camera 114A and a right raw image 302B captured by the right camera 114B. When these raw images 302A and 302B are processed (e.g., by the image processor 412), a depth image is generated. 。 The generated depth image can be viewed on the optical components 180A, 180B of the eye-wearing device, on another display (e.g., image display 580 on the mobile device 401), or on a screen. 。

[0040] The generated depth image is in a three-dimensional spatial domain and may include a vertex matrix in a three-dimensional positional coordinate system, which includes an X-axis for horizontal positioning (e.g., length), a Y-axis for vertical positioning (e.g., height), and a Z-axis for depth (e.g., distance). Each vertex may include color attributes (e.g., red pixel light value, green pixel light value, or blue pixel light value); positional attributes (e.g., X-coordinate, Y-coordinate, and Z-coordinate); texture attributes; reflectance attributes; or combinations thereof. Texture attributes quantify the perceptual texture of the depth image, such as the spatial arrangement of colors or intensities in the vertex regions of the depth image.

[0041] In one example, there is a scenario sending system 400 ( Figure 4 The image-capturing eyewear includes an eyewear device 100, which includes a frame 105, a left temple 125A extending from the left side 170A of the frame 105, and a right temple 125B extending from the right side 170B of the frame 105. The eyewear device 100 may also include at least two visible light cameras 114A, 114B having overlapping fields of view. In one example, the eyewear device 100 includes a left visible light camera 114A having a left field of view 111A, such as... Figure 3 As shown. The left camera 114A is attached to the frame 105 or the left temple 125A to capture a left raw image 302A from the left side of scene 306. The eye-wearing device 100 also includes a right visible light camera 114B with a right field of view 111B. The right camera 114B is attached to the frame 105 or the right temple 125B to capture a right raw image 302B from the right side of scene 306.

[0042] Figure 4 This is a functional block diagram of an exemplary image capture eyewear with a scene transmission system 400, which includes wearable devices (e.g., eyewear 100), mobile devices 401, and server systems 499 connected via various networks 495 such as the Internet. 。The image capture eyewear with scene transmission system 400 includes a low-power wireless connection 425 and a high-speed wireless connection 437 between the eyewear device 100 and the mobile device 401.

[0043] like Figure 4 As shown and described herein, the eye-wearing device 100 includes one or more visible light cameras 114A, 114B that capture still images, video images, or both. Cameras 114A, 114B may have direct memory access (DMA) to high-speed circuitry 430 and function as stereo cameras. Cameras 114A, 114B can be used to capture initial depth images, which can be rendered into three-dimensional (3D) models, which are texture-mapped images of a red-green-blue (RGB) imaged scene. Device 100 may also include a depth sensor 213 that uses infrared signals to estimate the location of an object relative to device 100. In some examples, depth sensor 213 includes one or more infrared emitters 215 and an infrared camera 410.

[0044] The eye-wear device 100 also includes two image displays for each optical component 180A, 180B (one associated with the left side 170A and one associated with the right side 170B). The eye-wear device 100 also includes an image display driver 442, an image processor 412, low-power circuitry 420, and high-speed circuitry 430. The image displays for each optical component 180A, 180B are used to present images, including still images, video images, or a combination of still and video images. The image display driver 442 is coupled to the image displays for each optical component 180A, 180B to control the display of the images.

[0045] The eye-wearing device 100 also includes one or more speakers 440 (e.g., one associated with the left side of the eye-wearing device and another associated with the right side of the eye-wearing device). The speakers 440 may be embedded in the frame 105, temple 125, or corner 110 of the eye-wearing device 100. The one or more speakers 440 are driven by an audio processor 443 under the control of a low-power circuit 420, a high-speed circuit 430, or both. The speakers 440 are used to present audio signals, including, for example, a beat track. The audio processor 443 is coupled to the speakers 440 to control the presentation of sound.

[0046] Figure 4The components shown for the eye-wearing device 100 are located on one or more circuit boards, such as printed circuit boards (PCBs) or flexible printed circuit boards (FPCs) located in the edges or temples. Alternatively or additionally, the depicted components may be located in the corners, frames, hinges, or bridge of the eye-wearing device 100. The left and right visible light cameras 114A, 114B may include digital camera elements, such as complementary metal-oxide-semiconductor (CMOS) image sensors, charge-coupled devices, lenses, or any other corresponding visible or light-capturing elements that can be used to capture data, including still images or videos of scenes with unknown objects.

[0047] like Figure 4 As shown, the high-speed circuit 430 includes a high-speed processor 432, a memory 434, and a high-speed wireless circuit 436. In this example, an image display driver 442 is coupled to the high-speed circuit 430 and is operated by the high-speed processor 432 to drive the left and right image displays of each optical component 180A, 180B. The high-speed processor 432 can be any processor capable of managing the high-speed communication and operation of any general-purpose computing system required by the eye-wear device 100. The high-speed processor 432 includes the processing resources required to transmit high-speed data from the high-speed wireless connection 437 to a wireless local area network (WLAN) using the high-speed wireless circuit 436.

[0048] In some examples, the high-speed processor 432 executes an operating system, such as the LINUX operating system or other such operating system of the eye-wear device 100, and the operating system is stored in memory 434 for execution. Among other duties, the high-speed processor 432, which executes the software architecture of the eye-wear device 100, also manages data transmissions utilizing the high-speed wireless circuit 436. In some examples, the high-speed wireless circuit 436 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, also referred to herein as Wi-Fi. In other examples, the high-speed wireless circuit 436 may implement other high-speed communication standards.

[0049] The low-power circuitry 420 includes a low-power processor 422 and a low-power wireless circuitry 424. The low-power wireless circuitry 424 and high-speed wireless circuitry 436 of the eye-wear device 100 may include short-range transceivers (Bluetooth™ or Bluetooth Low Energy (BLE)) and wireless wide-area network, local area network, or wide-area network transceivers (e.g., cellular or Wi-Fi). 。 Mobile device 401, including transceivers communicating via low-power wireless connection 425 and high-speed wireless connection 437, can be implemented using the architectural details of eye-wearing device 100, just like other components of network 495.

[0050] Memory 434 includes any storage device capable of storing various data and applications, including, among other things, camera data generated by the left and right visible light cameras 114A, 114B, the infrared camera 410, and the image processor 412, as well as images generated for display on the image display of each optical component 180A, 180B by the image display driver 442. While memory 434 is shown as integrated with high-speed circuitry 430, in other examples, memory 434 may be a separate, independent component of the eye-wearing device 100. In some such examples, electrical wiring provides a connection from the image processor 412 or the low-power processor 422 to memory 434 via a chip including the high-speed processor 432. In other examples, the high-speed processor 432 may manage addressing of memory 434 such that the low-power processor 422 will initiate the high-speed processor 432 whenever a read or write operation involving memory 434 is required.

[0051] like Figure 4 As shown, the high-speed processor 432 of the eye-wearing device 100 can be coupled to a camera system (visible light cameras 114A, 114B), an image display driver 442, a user input device 491, and a memory 434. Figure 5 As shown, the CPU 530 of the mobile device 401 can be coupled to the camera system 570, IMU 572, mobile display driver 582, user input layer 591 and memory 540A.

[0052] Server system 499 may be one or more computing devices as part of a service or network computing system, including, for example, a processor, memory, and a network communication interface for communicating with eye-wearing device 100 and mobile device 401 via network 495.

[0053] The output components of the eye-wearing device 100 include visual elements, such as left and right image displays associated with each lens or optical assembly 180A, 180B, as... Figure 2A and Figure 2BThe device 100 may include, for example, a display such as a liquid crystal display (LCD), a plasma display panel (PDP), a light-emitting diode (LED) display, a projector, or a waveguide. The eye-wearing device 100 may include user-facing indicators (e.g., LEDs, speakers, or vibration actuators) or outward-facing signals (e.g., LEDs, speakers). The image display of each optical component 180A, 180B is driven by an image display driver 442. In some exemplary configurations, the output components of the eye-wearing device 100 may also include additional indicators, such as audible elements (e.g., speakers), tactile elements (e.g., actuators, such as vibration motors for generating tactile feedback), and other signal generators. For example, the device 100 may include a set of user-facing indicators and a set of outward-facing signals. The user-facing set of indicators is configured to be seen or otherwise perceived by the user of the device 100. For example, the device 100 may include an LED display positioned so that the user can see it, one or more speakers positioned to generate sounds the user can hear, or actuators providing tactile feedback the user can feel. A set of outward-facing signals is configured to be seen or otherwise perceived by an observer near device 100. Similarly, device 100 may include LEDs, speakers, or actuators configured and positioned to be perceived by an observer.

[0054] The input components of the eye-wearing device 100 may include alphanumeric input components (e.g., a touchscreen or touchpad configured to receive alphanumeric input, a photographic optical keyboard, or other alphanumeric-configured elements), point-based input components (e.g., a mouse, touchpad, trackball, joystick, motion sensor, or other pointing instrument), haptic input components (e.g., a push-button switch, a touchscreen or touchpad that senses the position, force, or position and force of a touch or touch gesture, or other haptic-configured elements), and audio input components (e.g., a microphone). The mobile device 401 and the server system 499 may include alphanumeric, point-based, haptic, audio, and other input components.

[0055] In some examples, the eye-wearing device 100 includes a collection of motion-sensing components referred to as an inertial measurement unit (IMU) 472. These motion-sensing components can be microelectromechanical systems (MEMS) with micro-moving parts, typically small enough to be part of a microchip. In some exemplary configurations, the IMU 472 includes an accelerometer, a gyroscope, and a magnetometer. The accelerometer senses the linear acceleration (including acceleration due to gravity) of the device 100 relative to three orthogonal axes (x, y, z). The gyroscope senses the angular velocity of the device 100 about three rotational axes (pitch, roll, yaw). Together, the accelerometer and gyroscope provide positioning, orientation, and motion data about the device relative to six axes (x, y, z, pitch, roll, yaw). If a magnetometer is present, it senses the heading of the device 100 relative to magnetic north. The positioning of device 100 can be determined by position sensors such as GPS unit 473, one or more transceivers for generating relative positioning coordinates, altitude sensors or barometers, and other orientation sensors. Such positioning system coordinates can also be received from mobile device 401 via low-power wireless circuit 424 or high-speed wireless circuit 436 through wireless connections 425 and 437.

[0056] IMU 472 may include, or cooperate with, a digital motion processor or program that acquires raw data from components and calculates multiple useful values ​​regarding the positioning, orientation, and motion of device 100. For example, acceleration data acquired from an accelerometer may be integrated to obtain velocity relative to each axis (x, y, z); and integrated again to obtain the positioning of device 100 (represented in linear coordinates x, y, and z). Angular velocity data from a gyroscope may be integrated to obtain the positioning of device 100 (represented in spherical coordinates). The program used to calculate these effective values ​​may be stored in memory 434 and executed by the high-speed processor 432 of the eye-wearing device 100.

[0057] The eye-worn device 100 may optionally include additional peripheral sensors, such as biometric sensors, characteristic sensors, or display elements integrated with the eye-worn device 100. For example, peripheral device elements may include any I / O components, including output components, motion components, positioning components, or any other such components described herein. For example, biometric sensors may include components that detect facial expressions (e.g., gestures, facial expressions, vocal expressions, body posture, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), or identify a person (e.g., identification based on voice, retina, facial features, fingerprints, or electrophysiological signals such as electroencephalogram data).

[0058] Mobile device 401 may be a smartphone, tablet, laptop, access point, or any other such device capable of connecting to eye-wearing device 100 using both low-power wireless connection 425 and high-speed wireless connection 437. Mobile device 401 connects to server system 499 and network 495. Network 495 may include any combination of wired and wireless connections.

[0059] like Figure 4 As shown, the image capture eyewear with scene transmission system 400 includes a computing device, such as mobile device 401, coupled via a network to eyewear device 100. The image capture eyewear with scene transmission system 400 includes a memory for storing instructions and a processor for executing the instructions. The execution of instructions by processor 432 on the image capture eyewear with scene transmission system 400 can configure eyewear device 100 to cooperate with mobile device 401. The image capture eyewear with scene transmission system 400 can utilize the memory 434 of eyewear device 100 or the memory elements 540A, 540B, 540C of mobile device 401. Figure 5 Furthermore, the image capture eyewear with the scene transmission system 400 can utilize the processor elements 432, 422 of the eyewear device 100 or the central processing unit (CPU) 530 of the mobile device 401. Figure 5 Additionally, the image capture eyewear with the scene transmission system 400 can also utilize the memory and processor elements of the server system 499. In this respect, the memory and processing capabilities of the image capture eyewear with the scene transmission system 400 can be shared or distributed across the eyewear device 100, the mobile device 401, and the server system 499.

[0060] In some exemplary embodiments, memory 434 includes or is coupled to feature model 480, coordinate database 482, background selection criteria 484, and designated receivers 486. Feature model 480 is a CNN model trained to recognize, for example, landmarks and iconic characters (e.g., the Eiffel Tower and Mickey Mouse). Coordinate database 482 includes location coordinates. Location coordinates may include the location coordinates of eye-wearing device 100 (e.g., to determine when it is in a new area), the location coordinates of images captured by eye-wearing device 100 at the time of capture, and the location coordinates of images from other devices used to identify popular image capture locations. Location coordinates are stored in one or more databases in memory 434 and accessed by processor 432. Background selection criteria 484 includes a series of selections made by the user / wearer of eye-wearing device 100 to determine when images are automatically sent. Designated receivers 486 include one or more groups of receivers to whom the eye-wearing device automatically sends images. In one example, a receiver is a person or group of people with electronic devices for viewing images. In another example, additionally or alternatively, the recipient is a wearer / user's primary social media platform (e.g., Snapchat Stories available through Snap Inc. in Santa Monica, California). Background selection criteria and the designated recipient may be stored in one or more databases in memory 434 and accessible by processor 432.

[0061] The memory 434 also includes a background selection engine 492, a receiver designation engine 494, an image data generation engine 496, and a selection engine 498, all executed by the processor 432. The background selection engine 492 includes instructions for selecting the background of an image upon which automatic transmission is based. The receiver designation engine 494 includes instructions for specifying the receiver of the automatically transmitted image. The image data generation engine 496 includes instructions for generating image information (e.g., location information and presence of landmarks or iconic figures) for comparison with the image background. The selection engine 498 includes instructions for selecting an image to be automatically transmitted based on a matching comparison of image data with background selection criteria established by the user of the eye-wearing device 100.

[0062] In one example, server system 499 receives images from eye-wearing device 100, mobile device 401, and other devices via network 395 through mobile device 401 for training a feature model 480 by programming a neural network. Server system 499 sends the trained feature model to eye-wearing device 100 or mobile device 401 for landmark and iconic character recognition. Suitable neural networks are convolutional neural networks (CNNs) based on one of the following architectures: VGG16, VGG19, ResNet50, Inception V3, and Xception, or other CNN architectures.

[0063] In one example, machine learning techniques (e.g., deep learning) are used to identify objects in an image, such as specific landmarks or iconic figures (e.g., the Eiffel Tower, Mickey Mouse, etc.) and the presence of people or animals. Deep learning is a subset of machine learning that uses a set of algorithms and depth maps with multiple processing layers, including linear and nonlinear transformations, to model high-level abstractions in data. While many machine learning systems are embedded with initial features and network weights that will be modified through the learning and updating of the machine learning network, deep learning networks train themselves to identify “good” features for analysis. Using a multi-layered architecture, machines employing deep learning techniques can process raw data better than those using conventional machine learning techniques. It is convenient to use different evaluation or abstraction layers to examine highly correlated groups of values ​​or data on different topics.

[0064] CNNs are biologically inspired interconnected data networks used for the detection, segmentation, and recognition of related objects and regions in a dataset through deep learning. CNNs evaluate raw data as multiple arrays, breaking down the data in a series of stages to examine the learned features of the data.

[0065] In one example, a CNN is used to perform image analysis. The CNN receives an input image and abstracts it in convolutional layers to identify learned features (e.g., landmark structures and iconic characters). In a second convolutional layer, the image is transformed into multiple images, where the learned features are each emphasized in their respective sub-images. These images are further processed to focus on the features of interest within the image. The resulting images are then processed by pooling layers, which reduce the image size to separate the image portions containing the features of interest. The output of the convolutional neural network receives values ​​from the final non-output layer and classifies the image based on the data received from that final non-output layer.

[0066] Figure 5 This is a high-level functional block diagram of an exemplary mobile device 401. Mobile device 401 includes flash memory 540A storing programs to be executed by CPU 530 to run all or a subset of the functions described herein.

[0067] The mobile device 401 may include a camera 570, which includes one or more visible light cameras (first and second visible light cameras with overlapping fields of view) or at least one visible light camera with substantially overlapping fields of view and a depth sensor. The flash memory 540A may also include a plurality of images or videos generated via the camera 570.

[0068] As shown in the figure, the mobile device 401 includes an image display 580, a mobile display driver 582 for controlling the image display 580, and a display controller 584. Figure 5 In one example, the image display 580 includes a user input layer 591 (e.g., a touchscreen) that is overlaid on top of the screen used by the image display 580 or otherwise integrated into the screen.

[0069] Examples of usable touchscreen mobile devices include (but are not limited to) smartphones, personal digital assistants (PDAs), tablets, laptops, or other portable devices. However, the structure and operation of touchscreen devices are provided by way of example; the subject matter described herein is not intended to be limited thereto. For ease of discussion, Figure 5 Therefore, a block diagram illustration of an exemplary mobile device 401 with a user interface is provided, the user interface including a touch screen input layer 591 for receiving input (touch via hand, stylus or other tool, multi-touch or gesture, etc.) and an image display 580 for displaying a background.

[0070] like Figure 5 As shown, mobile device 401 includes at least one digital transceiver (XCVR) 510 for digital wireless communication via a wide-area wireless mobile communication network, shown as a WWAN XCVR. Mobile device 401 also includes additional digital or analog transceivers, such as a short-range transceiver (XCVR) 520 for short-range network communication such as via NFC, VLC, DECT, ZigBee, Bluetooth™, or Wi-Fi. For example, the short-range XCVR 520 may take the form of any available bidirectional wireless local area network (WLAN) transceiver compatible with one or more standard communication protocols implemented in a wireless local area network, such as the Wi-Fi standard conforming to IEEE 802.11.

[0071] To generate location coordinates for locating mobile device 401, mobile device 401 may include a Global Positioning System (GPS) receiver. Alternatively or additionally, mobile device 401 may utilize either or both of a short-range XCVR 520 and a WWAN XCVR 510 to generate location coordinates for positioning. For example, positioning systems based on cellular networks, Wi-Fi, or Bluetooth™ can generate very accurate location coordinates, especially when used in combination. Such location coordinates can be transmitted to the eye-wearing device via one or more network connections through XCVRs 510, 520.

[0072] Transceivers 510 and 520 (i.e., network communication interfaces) conform to one or more of the various digital wireless communication standards used in modern mobile networks. 。 Examples of WWAN transceivers 510 include (but are not limited to) transceivers configured to operate according to Code Division Multiple Access (CDMA) and 3rd Generation Partnership Project (3GPP) network technologies, including, but not limited to, 3GPP Type 2 (or 3GPP2) and LTE, sometimes referred to as "4G". For example, transceivers 510, 520 provide bidirectional wireless communication of information including digitized audio signals, still images and video signals, web page information for display and web-related input, and various types of mobile messaging communications to / from mobile device 401.

[0073] Mobile device 401 also includes a microprocessor used as a central processing unit (CPU); such as Figure 4 The CPU 530 is shown in the figure. A processor is a circuit having elements constructed and arranged to perform one or more processing functions, typically various data processing functions. Although discrete logic components can be used, these examples utilize components that form a programmable CPU. A microprocessor includes, for example, one or more integrated circuit (IC) chips that incorporate electronic components that perform the functions of the CPU. For example, the CPU 530 may be based on any known or available microprocessor architecture, such as Reduced Instruction Set Computing (RISC) using the ARM architecture, as is commonly used today in mobile devices and other portable electronic devices. Of course, other arrangements of the processor circuitry can be used to form the CPU 530 or processor hardware in smartphones, laptops, and tablets.

[0074] By configuring the mobile device 401 to perform various operations, such as instructions or programs executable by the CPU 530, the CPU 530 acts as a programmable host controller for the mobile device 401. Such operations may include, for example, various general operations of the mobile device, as well as operations related to applications used on the mobile device. Although the processor can be configured using hardwired logic, a typical processor in a mobile device is a general-purpose processing circuit configured by executing programs.

[0075] Mobile device 401 includes a memory or storage system for storing programs and data. In this example, the memory system may include, as needed, flash memory 540A, random access memory (RAM) 540B, and other memory components 540C. RAM 540B serves as a short-term storage device for instructions and data processed by CPU 530, for example, as working data processing memory. 。 The 540A flash memory typically provides long-term storage.

[0076] Therefore, in the example of mobile device 401, flash memory 540A is used to store programs or instructions executed by CPU 530. Depending on the type of device, mobile device 401 stores and runs a mobile operating system, through which specific applications are executed. Examples of mobile operating systems include Google Android, Apple iOS (for iPhone or iPad devices), Windows Mobile, Amazon Fire OS, RIM BlackBerry OS, etc.

[0077] Figures 7A to 7G This is a flowchart outlining the steps in an exemplary method for sending a scenario. Reference is made below to eye-worn device 100 and GUI 602 for specifying the receiver. Figure 6A ) and GUI 610 for specifying background selection criteria ( Figure 6B These steps are described using the terms . Although these steps are described herein with reference to eye-worn device 100, those skilled in the art will understand from the description herein that the described steps are for other specific implementations of other types of mobile devices. Additionally, it is conceivable that one or more steps shown in the figures and described herein may be omitted, performed simultaneously or sequentially, performed in a different order than shown and described, or performed in combination with additional steps.

[0078] These steps are described with reference to an eye-wearing device 100, including a camera 114, a processor 432, and a memory 434. Consistent with aspects of some example implementations, the eye-wearing device 100 initiates and runs a background search engine 492, a receiver specification engine 494, an image data generation engine 496, and a selection engine 498. In other exemplary embodiments, another device, such as a mobile device 401 or a server system 499, performs some or all of the functions of the eye-wearing device 100, or performs some functions in conjunction with the eye-wearing device 100.

[0079] Figure 7A A flowchart 700 is depicted for implementing an automatic image transmission system based on a background selection criterion applied to an image. At block 702, an eye-wearing device 100 captures an image. The processor 432 of the eye-wearing device 100 can capture an image using a visible light camera 114 in response to a gesture on a user input device 491.

[0080] At box 704, the eye-wearing device 100 stores images. In one example, processor 432 stores the captured images in memory 434. In another example, processor 432 additionally or alternatively sends the captured images to another device (e.g., mobile device 401) for storage (e.g., by processor 530 in memory 540).

[0081] At box 706, the eye-wearing device 100 uses a context selection engine 492 to identify context selection criteria. In one example, processor 432 presents a graphical user interface on display 180 for the wearer to use when identifying context selection criteria (see GUI 610). Figure 6B In another example, the processor 530 of the relevant mobile device 401 presents a graphical user interface on a display 580 for identifying context selection criteria. Context selection criteria include one or more of the following: image capture location, image content (e.g., landmarks or people), image quality, or number of images.

[0082] At box 708, the eye-wearing device 100 uses receiver designation engine 494 to identify a designated receiver. In one example, processor 432 presents a graphical user interface on display 180 for the wearer to use when identifying background selection criteria (see GUI 602). Figure 6AIn another example, the processor 530 of the relevant mobile device 401 presents a graphical user interface on display 580 for identifying background selection criteria. The processor may retrieve a list 604 of known contact names, present a list with checkboxes 606a-d next to each name to the wearer / user, receive checkbox selections, and identify the contacts associated with the checkboxes as designated recipients. The processor may additionally or alternatively retrieve a list of the wearer / user's social media platforms, present a list with checkboxes next to each platform to the wearer / user, receive checkbox selections, and identify the platforms associated with the checkboxes as designated recipients. The processor may additionally or alternatively provide the user with the option to select all contacts of the wearer / user with a specific identifier or associated status (e.g., friend or close friend) or such contacts within a geographic location associated with the wearer / user's current location.

[0083] At frame 710, the eye-wearing device 100 receives an image. In one example, processor 432 receives an image from camera 114, memory 434, or an associated mobile device 401.

[0084] In box 712, the eye-wearing device 100 uses an image data generation engine 496 to determine image data. In one example, processor 432 determines the image data. In another example, processor 530 of the associated mobile device 401 determines the image data. For location-based image data, processor 432 may retrieve location coordinates from GPS 473 when the image is captured. For content-based image data, processor 432 may apply a feature model 480 (e.g., trained using images of landmarks and people) to the captured image, or apply a known image recognition program (e.g., Watson from IBM in Armonk, NY). For image quality, processor 432 may apply image quality metrics to measure image quality (e.g., using algorithms such as BRISQUE or NIQE, available from MathWorks in Natick, Massachusetts, to measure one or more of sharpness, artifacts, and distortion).

[0085] At box 714, the eye-wearing device 100 compares the image data with background selection criteria. The processor 432 compares the image data with the background selection criteria by sequentially analyzing each of the identified criteria and comparing it with the image data. Additionally, if the processor 432 determines that the maximum number of images has been reached (e.g., based on a counter that increments each time an image to be transmitted is identified), the processor may stop performing further comparisons.

[0086] At decision box 716, the eye-wearing device 100 uses selection engine 498 to determine whether a match exists between the image data and background selection criteria. In one example, processor 432 compares the image data with the background selection criteria and identifies one or more images that match precisely. In another example, processor 432 compares the image data with contextual selection criteria and identifies one or more images that include matching criteria exceeding a predefined threshold level (e.g., 90% match). For the background selection criteria selection depicted in GUI 610, the matching image would be located near the Eiffel Tower in Paris (e.g., within 300 feet of the Eiffel Tower), would include both the Eiffel Tower and people, would have a high level of sharpness and low level of artifacts without distortion, and would be one of the top 10 images matching all criteria.

[0087] At box 718, if a match is found, the condition is met, and the eye-wearing device 100 automatically sends the image to the designated recipient. The processor 432 can send the image directly via network 495 through wireless circuits 424 / 436 or indirectly via mobile device 401. At box 720, if no match is found, the condition is met, and the eye-wearing device 100 does not automatically send the image to the designated recipient.

[0088] Figure 7B A flowchart 721 depicts exemplary steps of an exemplary implementation of receiver-specified engine 494 when multiple sets of specified receivers with different selection criteria exist. At block 722, the eye-wearing device 100 identifies a first criterion. Processor 432 can identify the first selection criterion as described above with reference to block 706. At block 724, the eye-wearing device 100 identifies a second criterion. Processor 432 can identify the second selection criterion as described above with reference to block 706.

[0089] At box 726, the eye-wearing device 100 identifies a first designated receiver for a first selection criterion. Processor 432 can identify the first designated receiver as described above with reference to box 708. At box 728, the eye-wearing device 100 identifies a second designated receiver for a second selection criterion. Processor 432 can identify the second designated receiver as described above with reference to box 708.

[0090] At box 730, the eye-wearing device 100 sends an image to a first designated recipient whose image content matches a first selection criterion. The processor 432 may send the image to the first designated recipient as described above with reference to box 718. At box 732, the eye-wearing device 100 sends an image to a second designated recipient whose image content matches a second selection criterion. The processor 432 may send the image to the second designated recipient as described above with reference to box 718.

[0091] Figure 7C A flowchart 740 depicts exemplary steps for identifying background selection criteria according to an exemplary specific implementation of background selection engine 492. At decision box 742, eye-wearing device 100 determines whether the background selection criteria include a specified capture location (e.g., using GUI 610, which includes location selection 612a, content selection 612b, quality selection 612c, and quantity selection 612d). Processor 432 may present GUI 610, including location 614 and corresponding checkboxes 616 (e.g., Paris 616a and Los Angeles 616b), to the wearer on display 180 via image display driver 442. Processor 432 may use location 614 or destination (e.g., determined by processing the wearer's calendar) within a predefined radius of eye-wearing device 100 (e.g., within 25 miles) to populate text for the checkboxes.

[0092] At box 743, if the wearer wishes to include the capture location as one of the background selection criteria, then the eye-wearing device 100 receives and stores the location parameter. The wearer can indicate that they want to include the capture location parameter by checking one or more boxes within GUI 610 (e.g., Paris 616a). In the example, selecting a specific location such as Paris generates more specific location information 618 for selection (e.g., the Eiffel Tower 620a and the Louvre 620b) to further refine the capture location. In the illustrated GUI 610, the location selection criterion is the Eiffel Tower within Paris. If no selection is made, processing continues at box 744.

[0093] At decision box 746, eye-wearing device 100 determines whether the background selection criteria include specified content (e.g., using GUI 610). Processor 432 may present GUI 610, including content 622 and corresponding checkboxes 624 (e.g., Eiffel Tower 624a and person 624b), to the wearer on display 180 via image display driver 442. Processor 432 may fill the checkboxes with text associated with the selected location using content 622, such as local landmarks within a predefined radius of the specified location (e.g., Eiffel Tower 624a near Eiffel Tower 620a in Paris 616a) and general information not specific to the location, such as the image must include person 624.

[0094] At box 747, if the wearer desires to include content as one of the background selection criteria, then the eye-wearing device 100 receives and stores the position parameter. The wearer can indicate the content parameter they want to include by checking one or more boxes within GUI 610 (e.g., Eiffel Tower 624a and Person 624b). In the illustrated GUI 610, the content selection criteria are Eiffel Tower 624a and Person 624b. If no selection is made, processing continues at box 748.

[0095] At decision box 750, eye-wearing device 100 determines whether the background selection criteria include a specified quality (e.g., using GUI 610). Processor 432 may present GUI 610 to the wearer on display 180 via image display driver 442, including image quality 626 (e.g., sharpness 628a, artifacts 628b, and distortion 628c) and corresponding input boxes (e.g., drop-down number selection; for example, a range from 1 to 10, where 1 equals low priority and 10 equals high priority). Image quality may be predefined.

[0096] At box 751, if the wearer wishes to include image quality as one of the background selection criteria, then the eye-wearing device 100 receives and stores the quality parameter. The wearer can indicate which quality parameter they want to include by adjusting it (e.g., setting Sharpness 628a to "8" for a desired high level of sharpness; setting Artifacts 628b to "9" for a very high level of artifact detection requirements; and setting Distortion to "10" for removing all images with any detected distortion). If no setting is made, processing continues at box 752.

[0097] At decision box 754, eye-wearing device 100 determines whether the background selection criteria include a specified quantity (e.g., using GUI 610). Processor 432 may present GUI 610 to the wearer on display 180 via image display driver 442, including quantity selection 630 and a corresponding input box for specifying the maximum number of images to be sent during travel.

[0098] At box 755, if the wearer wishes to include quantity as one of the background selection criteria, then the eye-wearing device 100 receives and stores the quantity parameter. The wearer can indicate the quantity parameter they want to include by entering a value (e.g., 10 images) within the GUI 610. If not set, processing continues at box 756.

[0099] Figure 7DA flowchart 760 depicts exemplary steps for generating image data according to an exemplary implementation of an image data generation engine 496. At block 762, the eye-wearing device 100 receives location information of a captured image. In this example, processor 432 receives location information from GPS 473 when capturing an image. Processor 432 may periodically query GPS 473 for location coordinates or request when to capture an image.

[0100] At box 764, the eye-wearing device 100 analyzes the image. In this example, processor 432 analyzes the image by applying a feature model 480 trained using images of known landmarks and iconic figures. The feature model 480 can be additionally trained to detect the presence of people in the image.

[0101] At box 766, the eye-wearing device 100 generates image data. In this example, processor 432 generates image data in response to identifying matches during image analysis. The image data may include strings associated with identified landmarks and iconic figures (and the presence of people) in the image.

[0102] At box 768, the eye-wearing device 100 associates image data with the image and stores the image data. In the example, processor 432 adds the image data to metadata stored along with the captured image.

[0103] Figure 7E A flowchart 770 depicts exemplary steps for determining whether to establish automatic transmission. At block 772, the eye-wearing device 100 monitors its location coordinates. The processor 432 monitors the location coordinates by periodically querying the GPS 473 for these coordinates.

[0104] At frame 774, the eye-worn device 100 determines the range of past location coordinates. The processor 432 can determine the range of past locations (e.g., the range of all locations within 25 miles of each other).

[0105] At frame 776, the eye-wearing device 100 compares the latest location coordinates with the range of past location coordinates. The processor 432 can determine the range of past locations (e.g., including the range of all locations within 25 miles of each other).

[0106] At frame 778, the eye-worn device 100 asks the wearer if they want automatic transmission when outside a defined range. The processor 432 can compare the defined range of past locations with the current location and ask the wearer if the current location is outside the defined range. For example, when the current location is not within the defined range (or a predefined distance, such as a 10-mile range), the processor 432 can display the query on display 180.

[0107] Figure 7F A flowchart 780 depicts exemplary steps for specifying selection criteria based on images taken by others (e.g., popular images). At box 782, the eye-wearing device 100 receives location information of a remote image captured by another device. In the example, server system 499 monitors and stores images provided by other devices and their corresponding image location coordinates.

[0108] At box 784, the eye-wearing device 100 groups the remote image locations. In one example, processor 432 receives the remote image locations from server system 499 and groups them into multiple sets of adjacent location coordinates. In another example, server system 499 groups the image location coordinates.

[0109] At box 786, the eye-wearing device 100 identifies groups that exceed a predefined threshold. In this example, processor 432 identifies groups with more than a threshold number of images (e.g., 100 images) and identifies those groups as popular image capture areas. In another example, server system 499 identifies these groups.

[0110] At box 788, the eye-wearing device 100 specifies a region containing groups exceeding a threshold as a selection criterion. In the example, processor 432 specifies a region as a selection criterion (e.g., for display in the location section 612a of GUI 610). Figure 6B In another example, when the server system 499 identifies a group, the processor 432 first receives the identified group from the server system 499 (and optionally the mobile device 401) via the network 495.

[0111] Figure 7G A flowchart 790 depicts exemplary steps for automatically sending an image in response to a gesture. At box 792, the eye-wearing device 100 recognizes a first input gesture. In this example, processor 432 recognizes the first input gesture (e.g., a click on user input device 491). At box 794, the eye-wearing device 100 recognizes a second input gesture. In this example, processor 432 recognizes the second input gesture (e.g., a double-click or tap and hold on user input device 491).

[0112] At box 796, the eye-wearing device 100 captures and stores a first image in response to a first input gesture, without sending it automatically. In the example, the processor 432 captures an image with the camera 114 in response to the first input gesture (e.g., a click on the user input device 491) and stores the image in memory 434 (e.g., after processing by the image processor 412).

[0113] At box 798, the eye-wearing device 100 captures, stores, and automatically sends a second image in response to a second input gesture. In the example, the processor 432 captures an image with the camera 114 in response to the second input gesture (e.g., a double-tap or tap and hold on the user input device 491), stores the image in memory 434, and automatically sends the image (e.g., after processing by the image processor 412).

[0114] Figures 7A to 7G The steps described herein may be executed by one or more of the processor 432 of the eye-wearing device 100, the processor 530 of the mobile device 401, or the processor of the server system 499 when loading and executing software code or instructions tangibly stored on a tangible (i.e., non-transitory) computer-readable medium, such as on a magnetic medium such as a computer hard disk drive, an optical medium such as an optical disk, a solid-state memory such as a flash memory, or other storage media known in the art. Therefore, any function executed by the processor 432 of the eye-wearing device 100, the processor 530 of the mobile device 401, or the processor of the server system 499 described herein, such as those in Figure [figure number missing], will be executed. Figures 7A to 7G The steps described herein can be implemented by software code or instructions tangibly stored on a tangible computer-readable medium. When a processor loads and executes such software code or instructions, a device including the processor can perform any of the functions of the device described herein, including those described herein. Figures 7A to 7G The steps in the process.

[0115] It should be understood that, unless otherwise specified herein, the terms and expressions used herein have the general meaning consistent with those in the corresponding fields of investigation and research. Relational terms such as “first” and “second” are used only to distinguish one entity or action from another, and do not necessarily require or imply any actual such relationship or order between these entities or actions. The terms “comprising,” “including,” “containing,” “having,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes or comprises a list of elements or steps includes not only those elements or steps, but may also include other elements or steps not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element prefixed with “a” or “an” does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element.

[0116] Unless otherwise stated, any and all measurements, values, ratings, positions, quantities, dimensions, and other specifications set forth in this specification, including those in the appended claims, are approximate, not precise. Such quantities are intended to have a reasonable range consistent with the functions they relate to and the conventions in the fields to which they pertain. For example, unless otherwise expressly stated, parameter values, etc., can vary from said quantities by up to ±10%.

[0117] Furthermore, as can be seen in the foregoing specific embodiments, various features have been combined in various examples for the purpose of simplifying this disclosure. The disclosed method should not be construed as reflecting an intention to require more features than expressly recited in each claim in the claimed examples. Rather, as reflected in the following claims, the claimed subject matter lies in fewer features than in any single disclosed example. Therefore, the following claims are hereby incorporated into the specific embodiments, wherein each claim exists independently as a separately claimed subject matter.

[0118] While examples considered to be best practices and other examples have been described above, it should be understood that various modifications may be made therein, and the subject matter disclosed herein can be implemented in various forms and examples, and is applicable to many applications, of which only some have been described herein. The appended claims are intended to claim protection for any and all modifications and variations falling within the true scope of the inventive concept.

Claims

1. A system comprising: Image capture device, including: Support structure; A position sensor is used to monitor the position of the image capture device; monitor; A camera, connected to the support structure, is used to capture multiple images; and A user interface, connected to the camera and the support structure, configured to trigger the camera; A processor, coupled to the image capture device, is configured to: Identify contextual selection criteria based on one or more user choices; Identify a group of one or more designated recipients, which are associated with the context selection criteria; The position of the image capture device is monitored using a position sensor; Compare the current position of the image capture device with its past position; If the current position of the image capture device exceeds the range of its past position, the user is asked via the display of the image capture device whether they wish to send an image automatically. Configure automatic sending in response to the user's request for automatic image sending in response to the inquiry; Use the camera to capture an image; Determine the image data of the captured image; The determined image data is compared with the identified context selection criteria to identify matches, and When automatic sending is set, the captured image is automatically sent to the set of designated receivers in response to a recognized match.

2. The system according to claim 1, wherein, The image capture device is an eye-worn device.

3. The system according to claim 1, wherein, The position sensor includes: A Global Positioning System (GPS) coupled to the processor, the GPS being configured to generate location coordinates; The processor determines the location in response to the location coordinates received from the GPS.

4. The system according to claim 3, wherein, The processor is further configured to: The range of past locations is determined in response to the generated position coordinates; and Determine whether the current position coordinates are outside the range of the determined past position coordinates, so as to determine whether the current position of the image capture device is outside the range of the past position.

5. The system according to claim 4, wherein, The range of past locations refers to all locations within a first preset distance range from each other.

6. The system according to claim 5, wherein, When the distance between the current position coordinates and the coordinates of the past position range is greater than a second preset distance, the processor determines that the position coordinates are outside the determined range of the past position.

7. The system according to claim 6, wherein, The first preset distance and the second preset distance are different.

8. The system according to claim 1, wherein, The processor is further configured to: Receive remote image location information of remote images captured by other devices within a predefined range of the image capture device; The remote image locations are grouped using remote image location information; Identify groups associated with remote image location information that exceed a predefined threshold; as well as Specify a region containing each of the identified groups as a contextual selection criterion.

9. A method for an image capture device, the image capture device including a camera configured to capture an image, a position sensor, a display, and a user interface configured to trigger the camera to capture an image, the method comprising the steps of: Identify contextual selection criteria based on one or more user choices; Identify a group of one or more designated receivers, the designated receivers being associated with the context selection criteria; The position of the image capture device is monitored using a position sensor; Compare the current position of the image capture device with its past position; If the current position of the image capture device exceeds the range of its past position, the user is asked via the display of the image capture device whether they wish to send an image automatically. Configure automatic sending in response to the user's request for automatic image sending in response to the inquiry; Use the camera to capture an image; Determine the image data of the captured image; The determined image data is compared with the identified context selection criteria to identify matches, and When automatic sending is set, the captured image is automatically sent to the set of designated recipients in response to a recognized match.

10. The method according to claim 9, wherein, The location sensor is a Global Positioning System (GPS), the monitoring includes monitoring location coordinates received from the Global Positioning System (GPS), and the method further includes: In response to the location coordinates, the range of the past location is determined; and Determine whether the current position coordinates are outside the range of the determined past position coordinates, so as to determine whether the current position of the image capture device is outside the range of the past position.

11. The method according to claim 10, wherein, The range of past locations refers to all locations within a first preset distance range from each other.

12. The method according to claim 11, wherein, The determination that the position coordinates are outside the range of the determined past positions includes: determining this when the distance between the current position coordinates and the coordinates of the past position range is greater than a second preset distance.

13. The method according to claim 12, wherein, The first preset distance and the second preset distance are different.

14. The method of claim 9, further comprising: Receive remote image location information of remote images captured by other devices within a predefined range of the image capture device; The remote image locations are grouped using remote image location information; Identify groups associated with remote image location information that exceed a predefined threshold; as well as Specify a region containing each of the identified groups as a contextual selection criterion.

15. A non-transitory computer-readable medium comprising instructions for use with an image capture device, the image capture device including a camera configured to capture images, a position sensor for monitoring the position of the image capture device, a display, and a user interface configured to trigger the camera to capture images; wherein, when a processor executes the instructions, the image capture device is configured as follows: Identify contextual selection criteria based on one or more user choices; Identify a group of one or more designated receivers, the designated receivers being associated with the context selection criteria; The position of the image capture device is monitored using a position sensor; Compare the current position of the image capture device with its past position; If the current position of the image capture device exceeds the range of its past position, the user is asked via the display of the image capture device whether they wish to send an image automatically. Configure automatic sending in response to the user's request for automatic image sending in response to the inquiry; Use the camera to capture an image; Determine the image data of the captured image; The determined image data is compared with the identified context selection criteria to identify matches, and When automatic sending is set, the captured image is automatically sent to the set of designated recipients in response to a recognized match.

16. The non-transitory computer-readable medium according to claim 15, wherein, The position sensor is a Global Positioning System (GPS), and the monitoring includes monitoring the position coordinates received from the Global Positioning System (GPS). When the instruction is executed by the processor, the image capture device is further configured to: In response to the location coordinates, the range of the past location is determined; and Determine whether the current position coordinates are outside the range of the determined past position coordinates, in order to determine whether the current position of the image capture device is outside the range of the past positions.

17. The non-transitory computer-readable medium according to claim 16, wherein, The range of past locations refers to all locations within a first preset distance range from each other.

18. The non-transitory computer-readable medium according to claim 17, wherein, The determined location coordinates are outside the determined past location range, including when the instruction is executed by the processor, further configuring the image capture device as follows: This is determined when the distance between the current location coordinates and the coordinates of the past location range is greater than the second preset distance.

19. The non-transitory computer-readable medium according to claim 18, wherein, The first preset distance and the second preset distance are different.

20. The non-transitory computer-readable medium according to claim 15, wherein, When the processor executes the instruction, the image capture device is further configured as follows: Receive remote image location information of remote images captured by other devices within a predefined range of the image capture device; The remote image locations are grouped using remote image location information; Identify groups associated with remote image location information that exceed a predefined threshold; as well as Specify a region containing each of the identified groups as a contextual selection criterion.

Images