Ar-assisted safety criding

Abstract

A method of providing proximity alerts using a head-mounted device is disclosed. Image processing techniques are used to determine a distance between a head-mounted device and a related object (e.g., a bicycle rider, jogger, or vehicle). The speed of the head-mounted device is determined using a GPS receiver or other location component located in the head-mounted device or an associated user device. A braking distance of the head-mounted device is determined based on the speed of the head-mounted device, and the braking distance is compared to the distance to the related object. If the distance to the related object is less than the braking distance, the head-mounted device provides a warning notification.

Description

[0001] Related application data

[0002] This application claims priority to U.S. patent application serial number 17 / 840,346 filed on June 14, 2022. Technical Field

[0003] The present disclosure relates generally to augmented reality devices and interfaces, and more particularly to providing proximity warnings using head-mounted AR devices. Background Art

[0004] A head mounted device may be implemented with a transparent or translucent display through which a user of the head mounted device can view the surrounding environment. Such a device enables the user to view through the transparent or translucent display to view the surrounding environment, and also to view objects (e.g., virtual objects such as 3D renderings, images, videos, text, etc.) generated for the display to appear as part of the surrounding environment and / or superimposed on the surrounding environment. This is generally referred to as "augmented reality."

[0005] The head mounted device may also completely block the user's field of view and display a virtual environment through which the user can move or be moved. This is often referred to as "virtual reality". As used herein, unless the context indicates otherwise, the term "augmented reality" or "AR" refers to both augmented reality and virtual reality as traditionally understood.

[0006] A user of the head mounted device may access a messaging application or a social networking application to view content or share content with other users of the application. In some cases, real-time content or stored content may be viewed and enhanced or modified by the user. That is, images, videos, or other media for enhancement may be captured from a real-time camera or retrieved from a local or remote data storage device.

[0007] As referred to herein, the phrase "augmented reality experience" includes or refers to various image processing operations corresponding to image modification, filtering, media overlay, transformation, etc., as further described herein. In some examples, these image processing operations provide an interactive experience of a real-world environment, in which objects, surfaces, backgrounds, lighting, etc. in the real world are enhanced by computer-generated perceptual information. In this context, an "augmented reality effect" includes a collection of data, parameters, and other assets required to apply a selected augmented reality experience to an image or video feed. In some examples, augmented reality effects are provided by Snap Inc. under the registered trademark LENSES.

[0008] Thus, AR effects and associated systems and modules for modifying content using AR effects may involve: detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.), tracking of such objects as they leave, enter, and move around the field of view in a video frame, and modification or transformation of such objects as they are tracked. In various examples, different methods for implementing such transformations may be used. For example, some examples may involve generating a 3D mesh model of one or more objects, and using transformations of the model and animated textures within a video to implement the transformations. In other examples, tracking of points on an object may be used to place an image or texture at the tracked location, which image or texture may be two-dimensional or three-dimensional. In yet another example, neural network analysis of a video frame may be used to place an image, model, or texture in content (e.g., an image or video frame). Thus, AR effect data may include both: images, models, and textures used to create transformations in content; and additional modeling and analysis information required to implement such transformations using object detection, tracking, and placement. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To easily identify the discussion of any particular element or act, the highest digit or digits in a reference number refer to the figure number in which the element is first introduced.

[0010] Figure 1 is a stereoscopic diagram of a head mounted device according to some examples.

[0011] Figure 2 According to some examples Figure 1 Additional views of the head-mounted device.

[0012] Figure 3 It is shown that according to some examples, Figure 1 A block diagram of a networking system 300 showing details of a head mounted device.

[0013] Figure 4 is an architecture 400 of a proximity warning system according to some examples.

[0014] Figure 5 A flow chart is shown that is performed by a proximity module according to some examples.

[0015] Figure 6 A flowchart 600 is shown that is performed by the application 402 according to some examples.

[0016] Figure 7 The field of view 700 as seen by a wearer of glasses 100 according to some examples is shown.

[0017] Figure 8is a diagrammatic representation of a networked environment in which the present disclosure may be deployed, according to some examples.

[0018] Fig. 9 is a block diagram illustrating a software architecture within which the present disclosure may be implemented according to some examples.

[0019] Fig.10 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed, for causing the machine to perform any one or more of the methodologies discussed. DETAILED DESCRIPTION

[0020] Known head-mounted devices such as AR glasses include a transparent or translucent display, which enables the user to look through the transparent or translucent display to view the surrounding environment. Additional information or objects (e.g., virtual objects such as 3D renderings, images, videos, text, etc.) can be displayed on the display and appear as part of the surrounding environment and / or superimposed on the surrounding environment to provide an augmented reality experience for the user. The display can, for example, include a waveguide that receives a beam from a projector, but any suitable display can be used to present augmented content or virtual content to the wearer.

[0021] A rider of a bicycle, motorcycle, or other powered or non-powered vehicle may not be aware of an unsafe distance to follow other vehicles. The head mounted device, with its image processing and other tracking capabilities, can monitor the speed of the bicycle and the distance to other cyclists riding ahead. If the distance between the rider and the cyclist ahead becomes less than the expected braking distance based on the surface and terrain, the glasses 100 can provide a warning prompt to the rider. The warning can be visual, audible, or a combination of both.

[0022] In some examples, a method for providing a proximity warning using a head-mounted device including a camera and one or more display devices is provided. The method includes: using a camera of the head-mounted device to capture an image; using an image processing technique performed on the captured image to determine a distance between the head-mounted device and an object of interest; using a location component located in the head-mounted device or an associated user device to determine a speed of the head-mounted device; determining a braking distance of the head-mounted device based on the speed of the head-mounted device; comparing the distance to the object of interest with the braking distance; and providing a warning notification based on the distance to the object of interest being less than the braking distance.

[0023] The speed may be determined using data received from a GPS receiver located in the head mounted device or an associated user device.The warning notification may include a visual notification displayed on one or more of the display devices of the head mounted device.

[0024] The braking distance may also be determined based on user-specific parameters such as user weight or bicycle type. The braking distance may also be determined based on environmental parameters such as surface type or incline.

[0025] In some examples, a head-mounted device system including one or more camera devices, one or more display devices, and one or more processors is provided. The head-mounted device system also includes a memory storing instructions that, when executed by one or more processors, configure the system to perform operations for providing proximity warnings according to any of the methods and limitations described above, including but not limited to: using a camera device of the head-mounted device to capture an image; using image processing techniques performed on the captured image to determine the distance between the head-mounted device and the relevant object; using a location component located in the head-mounted device or an associated user device to determine the speed of the head-mounted device; determining a braking distance of the head-mounted device based on the speed of the head-mounted device; comparing the distance to the relevant object with the braking distance; and providing a warning notification based on the distance to the relevant object being less than the braking distance.

[0026] In some examples, a non-transitory computer-readable storage medium is provided, which includes instructions that, when executed by a head-mounted device system including one or more display devices and a camera device, cause the head-mounted device system to perform operations for providing proximity warnings according to any of the methods and limitations described above, including but not limited to: using a camera device of the head-mounted device to capture an image; using image processing techniques performed on the captured image to determine the distance between the head-mounted device and a related object; using a location component located in the head-mounted device or an associated user device to determine the speed of the head-mounted device; determining a braking distance of the head-mounted device based on the speed of the head-mounted device; comparing the distance to the related object with the braking distance; and providing a warning notification based on the distance to the related object being less than the braking distance.

[0027] Other technical features may be readily apparent to those skilled in the art from the accompanying drawings, descriptions and claims.

[0028] Figure 11 is a stereogram of a head mounted device (e.g., glasses 100) according to some examples. Glasses 100 may include a frame 102, which is made of any suitable material, such as plastic or metal, including any suitable shape memory alloy. In one or more examples, the frame 102 includes a first optical element holder or a left optical element holder 104 (e.g., a display or lens holder) and a second optical element holder or a right optical element holder 106 connected by a bridge 112. A first optical element or a left optical element 108 and a second optical element or a right optical element 110 may be disposed in the left optical element holder 104 and the right optical element holder 106, respectively. Each of the right optical element 110 and the left optical element 108 may be a lens, a display, a display assembly, or a combination of the foregoing. Any suitable display assembly may be disposed in the glasses 100.

[0029] The frame 102 additionally includes a left arm or temple piece 120 and a right arm or temple piece 122. In some examples, the frame 102 can be formed from a single piece of material to have a unitary or unitary construction.

[0030] The glasses 100 may include a computing device such as a computer 118, which may be of any suitable type so as to be carried by the frame 102, and in one or more examples, the computing device may be of a suitable size and shape to be at least partially disposed in one of the temple pieces 120 or temple pieces 122. The computer 118 may include one or more processors with memory, wireless communication circuitry, and a power source. As discussed below, the computer 118 includes low-power circuitry, high-speed circuitry, and a display processor. Various other examples may include these elements in different configurations or integrated together in different ways. Additional details of various aspects of the computer 118 may be implemented as shown in the data processor 302 discussed below.

[0031] The computer 118 additionally includes a battery 116 or other suitable portable power supply. In some examples, the battery 116 is disposed in the left temple piece 120 and is electrically coupled to the computer 118 disposed in the right temple piece 122. The glasses 100 may include a connector or port (not shown) suitable for charging the battery 116, a wireless receiver, transmitter or transceiver (not shown), or a combination of such devices.

[0032] The glasses 100 include a camera 114. Although two cameras are depicted, other examples contemplate the use of a single or additional (i.e., more than two) cameras. In one or more examples, the glasses 100 include any number of input sensors or other input / output devices in addition to the camera 114. Such sensors or input / output devices may additionally include biometric sensors, positioning sensors, motion sensors, etc.

[0033] The glasses 100 may also include a touchpad 124 mounted to or integrated with one or both of the left temple piece 120 and the right temple piece 122. In some examples, the touchpad 124 is generally arranged vertically, approximately parallel to the temple of the user. As used herein, generally vertically aligned means that the touchpad is at least more vertical than horizontal, although potentially more vertical than the vertical. Additional user input can be provided by one or more buttons 126, which, in the example shown, are disposed on the outer upper edges of the left optical element holder 104 and the right optical element holder 106. One or more touchpads 124 and buttons 126 provide a device by which the glasses 100 can receive input from the user of the glasses 100.

[0034] Figure 2 The eyeglasses 100 are shown from the wearer's perspective. Figure 1 Several elements shown in FIG. Figure 1 As described, Figure 2 The illustrated eyeglasses 100 include left and right optical elements 108, 110 secured within each of left and right optical element holders 104, 106, respectively.

[0035] The glasses 100 include a forward optical assembly 202 including a right projector 204 and a right near-eye display 206 , and a forward optical assembly 210 including a left projector 212 and a left near-eye display 216 .

[0036] In some examples, the near-eye display is a waveguide. The waveguide includes a reflective structure or a diffractive structure (e.g., a grating and / or an optical element such as a mirror, lens, or prism). Light 208 emitted by projector 204 encounters the diffractive structure of the waveguide of near-eye display 206, which directs the light toward the right eye of the user to provide an image on or in the right optical element 110 that is superimposed on the view of the real world seen by the user. Similarly, light 214 emitted by projector 212 encounters the diffractive structure of the waveguide of near-eye display 216, which directs the light toward the left eye of the user to provide an image on or in the left optical element 108 that is superimposed on the view of the real world seen by the user.

[0037] However, it will be appreciated that other display technologies or configurations may be provided that can display images to a user in a forward field of view. For example, instead of providing a projector 204 and a waveguide, an LCD, LED or other display panel or surface may be provided instead.

[0038] In use, the wearer of the glasses 100 will be presented with information, content, and various user interfaces on the near-eye display. As described in more detail below, in addition to the associated devices such as Figure 3 In addition to providing voice input or touch input on the user device 328 shown, the user can then interact with the glasses 100 using the touchpad 124 and / or buttons 126.

[0039] Figure 3 is a block diagram illustrating a networked system 300 including details of glasses 100 according to some examples.

[0040] The networked system 300 includes the glasses 100, a user device 328, and a server system 332. The user device 328 may be a smartphone, a tablet computer, a tablet phone, a laptop, an access point, or any other such device capable of connecting to the glasses 100 using both a low power wireless connection 336 and a high speed wireless connection 334. The user device 328 is connected to the server system 332 via a network 330. The network 330 may include any combination of wired and wireless connections. The server system 332 may be one or more computing devices that are part of a service or network computing system. The glasses 100, the user device 328, and any elements of the server system 332 and the network 330 may use Fig. 9 and Fig.10 The details of the software architecture 904 or machine 1000 described in the description are implemented.

[0041] The glasses 100 include a data processor 302, a display 310, one or more cameras 308, and additional input / output elements 316. The input / output elements 316 may include microphones, audio speakers, biometric sensors, additional sensors, or additional display elements integrated with the data processor 302. Fig. 9 and Fig.10 Examples of input / output elements 316 are further discussed. For example, input / output elements 316 may include any I / O components 1006, including output components 1028, motion components 1036, etc. Figure 2 Examples of display 310 are discussed in . In the particular examples described herein, display 310 includes a display for each of the user's left eye and right eye.

[0042] Data processor 302 includes image processor 306 (eg, video processor), GPU and display driver 338, tracking module 340, interface 312, low power circuitry 304, and high speed circuitry 320. Components of data processor 302 are interconnected by bus 342.

[0043] Interface 312 refers to any source of user commands provided to data processor 302. In one or more examples, interface 312 is a physical button that, when pressed, sends a user input signal from interface 312 to low-power processor 314. Low-power processor 314 can process pressing such a button and then immediately releasing it as a request to capture a single image, and vice versa. Low-power processor 314 can process pressing such a button within a first time period as a request to capture video data when the button is pressed and stop video capture when the button is released, wherein the video captured when the button is pressed is stored as a single video file. Alternatively, pressing a button for a long period of time can capture a still image. In some examples, interface 312 can be any mechanical switch or physical interface capable of accepting user input associated with a data request from camera 308. In other examples, interface 312 can have a software component, or can be associated with a command received wirelessly from another source, such as from user device 328.

[0044] Image processor 306 includes circuitry for receiving signals from camera 308 and processing those signals from camera 308 into a format suitable for storage in memory 324 or for transmission to user device 328. In one or more examples, image processor 306 (e.g., a video processor) includes a microprocessor integrated circuit (IC) customized for processing sensor data from camera 308, and volatile memory used by the microprocessor in operation.

[0045] The low power circuit system 304 includes a low power processor 314 and a low power wireless circuit system 318. These elements of the low power circuit system 304 can be implemented as separate elements or can be implemented on a single IC as part of a single system on a chip. The low power processor 314 includes logic for managing other elements of the glasses 100. As described above, for example, the low power processor 314 can accept user input signals from the interface 312. The low power processor 314 can also be configured to receive input signals or command communications from the user device 328 via the low power wireless connection 336. The low power wireless circuit system 318 includes circuit elements for implementing a low power wireless communication system. Bluetooth TM Smart, also known as Bluetooth TM Low power consumption is a standard implementation of a low power wireless communication system that may be used to implement the low power wireless circuitry 318. In other examples, other low power communication systems may be used.

[0046] High-speed circuit system 320 includes a high-speed processor 322, a memory 324, and a high-speed wireless circuit system 326. High-speed processor 322 can be any processor capable of managing high-speed communications and operations of any general-purpose computing system required by data processor 302. High-speed processor 322 includes the processing resources required to manage high-speed data transfer over high-speed wireless connection 334 using high-speed wireless circuit system 326. In some examples, high-speed processor 322 executes an operating system such as a LINUX operating system or a program such as a Fig. 9 The high-speed processor 322, which executes the software architecture of the data processor 302, manages data transmission with the high-speed wireless circuit system 326, in addition to any other duties. In some examples, the high-speed wireless circuit system 326 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, which is also referred to herein as Wi-Fi. In other examples, other high-speed communication standards can be implemented by the high-speed wireless circuit system 326.

[0047] Memory 324 includes any storage device capable of storing camera data generated by camera 308 and image processor 306. Although memory 324 is shown as being integrated with high-speed circuitry 320, in other examples, memory 324 may be a separate, stand-alone element of data processor 302. In some such examples, electrical wiring may provide a connection from image processor 306 or low-power processor 314 to memory 324 through a chip including high-speed processor 322. In other examples, high-speed processor 322 may manage addressing of memory 324 so that low-power processor 314 will direct high-speed processor 322 any time a read or write operation involving memory 324 is needed.

[0048] The tracking module 340 provides various position, orientation, and object tracking functions. For example, the tracking module 340 uses image data and corresponding inertial data from the camera device 308 and the position component 1040, as well as GPS data, to track the location and determine the pose of the glasses 100 relative to a reference frame (e.g., a real-world environment). The tracking module 340 continuously collects and uses updated sensor data describing the movement of the glasses 100 to determine an updated three-dimensional pose of the glasses 100, which indicates changes in relative position and orientation relative to physical objects in the real-world environment, allowing the glasses 100 to visually place virtual objects relative to physical objects within the user's field of view via the display 310.

[0049] The tracking module 340 also uses object recognition techniques to identify and track objects of interest within the field of view of the camera 308 of the glasses 100 , and uses known image processing techniques to determine the depth of the objects of interest within the field of view of the glasses 100 .

[0050] The GPU and display driver 338 may use the pose of the glasses 100 to generate frames of virtual content or other content to be presented on the display 310 when the glasses 100 are operating in a conventional augmented reality mode. In this mode, the GPU and display driver 338 generate updated frames of virtual content based on the updated three-dimensional pose of the glasses 100, which reflects changes in the position and orientation of the user relative to physical objects in the user's real-world environment.

[0051] One or more functions or operations described herein may also be performed in an application resident on the glasses 100 or on the user device 328 or on a remote server. For example, one or more functions or operations described herein may be performed by one of the applications 906, such as the messaging application 946.

[0052] Figure 44 is an architecture 400 of a proximity warning system according to some examples. The illustrated architecture 400 is embodied in a combination of an application 402 running on the glasses 100, the user device 328, or a combination thereof, and the tracking module 340.

[0053] The application 402 includes a navigation module 404 and a proximity module 406. The application 402 uses services provided by the tracking module 340. The tracking module 340 includes a location service 408 and a scanning service 410 that provide information about the location of the glasses 100 or the user device 328. The scanning service 410 provides object detection, object tracking, and depth determination from an object detection service 412, an object tracking service 414, and a depth determination service 416, respectively. The scanning service 410 performs object detection, object tracking, and depth determination using known image processing techniques performed on images captured by one or more of the cameras 114 in the glasses 100.

[0054] In some examples, application 402 is a messaging client 814 that is part of a social networking or messaging system 800 provided by a single entity (see Figure 8 ). In other examples, the application 402 is a third-party application designed to work with the glasses 100 or the user device 328, and the proximity module 406 is a plug-in provided by the provider of the messaging system 800 to enable developers to access the features and capabilities of the glasses 100, the user device 328, or the messaging system 800. The proximity module 406 can be provided, for example, as part of a software development kit (SDK).

[0055] In some examples, such as Figure 4 As shown, application 402 includes a navigation module 404 that provides navigation prompts and instructions to allow a user to navigate from one location to another, but it should be understood that many different modules and types of applications 402 may be used, such as sports or fitness applications that may benefit from using proximity module 406.

[0056] While using application 402, navigation module 404 provides navigation information to the user via glasses 100, while receiving any warning notifications from proximity module 406 when the user is in close proximity to an object, such as a jogger, a vehicle, or a person on a bicycle riding in front of the user.

[0057] The proximity module 406 receives positioning and scanning service information from the tracking module 340. The safe minimum braking distance is determined using the speed of the glasses 100 or the user device 328 (as a proxy for the speed of the user). The safe minimum braking distance is compared to the distance of the user from an object of interest that has been detected and tracked by the scanning service 410. The relevant detected objects may include cyclists, runners, cars, etc. For the purpose of this comparison, the depth from the glasses 100 to the object as determined by the depth determination service 416 is used as the distance to the detected object. If the distance to the object of interest is less than the minimum safe braking distance, the glasses 100 provide a warning notification.

[0058] Figure 5 Flowchart 500 is shown that is executed by proximity module 406 in some examples. As described above, proximity module 406 executes flowchart 500 based on positioning and object tracking information received from tracking module 340 to determine a safe minimum braking distance to a person or vehicle being followed.

[0059] In some examples, the relevant parameters for use in determining the braking acceleration or in determining the braking distance are associated with the user's profile (e.g., the user's mass, the type and mass of the bicycle), or are received as user input to the application 402 when starting navigation or an activity. The braking acceleration or braking distance will also depend on environmental parameters, such as the type of surface (asphalt or gravel or dirt) and the inclination of the surface. The type of surface can be received as user input or determined based on positioning information received from the tracking module 340 in combination with map data stored locally or accessed from a remote server. The inclination can be determined based on the positioning information, the direction of travel, and the contour of the route determined by the navigation module 404 in combination with the map data.

[0060] For example, a heavier rider riding a mountain bike downhill on a gravel trail will have a smaller safe maximum braking acceleration based on their heavier mass and surface type compared to a lighter rider riding a road bike on a level asphalt trail, which combined with the downhill travel will translate into a longer safe minimum braking distance.

[0061] Flowchart 500 begins at operation 502, where relevant braking parameters (in the case of parameters associated with a user profile) are accessed, for example by accessing map data based on the location of the glasses 100 (and therefore the user), received as user input in response to a prompt, or otherwise determined.

[0062] In operation 504, the proximity module 406 determines whether proximity detection is active. The proximity module 406 determines whether proximity detection is active based on, for example, a notification received from the navigation module 404 or from another aspect or function executed by the application 402 to activate or deactivate proximity detection. If proximity detection is not active, the method returns to operation 502.

[0063] If proximity detection is active, then in operation 506, the proximity module 406 determines the speed of the glasses 100 or the user device 328 (as a proxy for the user's speed) based on the positioning information received from the tracking module 340. In some examples, the positioning information is based on a GPS receiver or other location component 1040 (see FIG. 104 ) in the user device 328 or the glasses 100. Fig.10 ) The reception of repeated updates of the location of the glasses 100 or the user device 328 reported by the user. Alternatively or in addition to GPS data, the positioning information can be provided or supplemented by data received from inertial sensors such as accelerometers and rotation sensors.

[0064] In operation 508, the proximity module 406 determines a braking distance for a bicycle or other personal mobility device or vehicle ridden by the user. The braking distance depends on the available retardation and the speed of the user as determined in operation 506. The retardation may be a default known safe maximum braking acceleration for the activity in question (e.g., biking), but the retardation may be determined for a specific individual, activity, equipment, and terrain to provide a more accurate estimate.

[0065] In some examples, relevant parameters (user weight, device type, surface type, etc.) are used to determine a maximum safe retardation force, which can be used with speed to determine a minimum safe braking distance using known equations for speed, acceleration, and distance. Alternatively, an adjustment factor can be used to adjust a default braking distance determined based on the user's speed to account for relevant parameters that differ from those associated with the standard braking distance.

[0066] In operation 510, the braking distance determined in operation 508 is compared to the distance to the object or person being followed. In some examples, the distance to the object being followed is depth information received from the depth determination service 416 in the tracking module 340. As described above, the tracking module 340 utilizes the images captured by the camera 114 of the glasses 100: has detected one or more objects of interest using the object detection service 412; has tracked the objects of interest using the object tracking service 414; and has determined the distance (depth) from the glasses 100 to one or more objects of interest using the depth determination service 416. If the braking distance is greater than the distance from the glasses 100 to the nearest object of interest as determined by the depth determination service 416, the method returns to operation 502 and continues from there.

[0067] If the distance from the glasses 100 to the nearest relevant object is less than or equal to the minimum safe braking distance, a warning notification is displayed by the glasses 100 in operation 512. Alternatively or additionally, the glasses 100 may play a warning sound. The method then returns to operation 504 and continues from there.

[0068] Figure 6 Flowchart 600 is shown as being executed by application 402 in some examples. Application 402 executes flowchart 600 in conjunction with proximity module 406, which is described above in Figure 5 Operate as described in .

[0069] Flowchart 600 begins at operation 602, where user input is received to enable application 402 or to enable a navigation function (e.g., a navigation applet) within application 402. In operation 604, application 402 receives user input for route information, including at least a destination and any optional selected navigation items (e.g., a selection between available routes) to avoid highways, etc. Alternatively, a saved or known route may be selected. After completing route entry in operation 604, application 402 begins navigation in operation 606 and provides navigation guidance to the user through visual and / or auditory prompts provided via glasses 100.

[0070] In operation 608, the application 402 instructs the proximity module 406 to begin the Figure 5 The proximity module 406 then executes the flowchart 500 to provide any triggered alerts to the application 402 as shown by arrow 616. For example, Figure 7 As shown, any alerts received from the proximity module 406 are presented by the glasses.

[0071] Navigation then continues until the destination is reached in operation 610, at which point the application 402 notifies the proximity module 406 to cease proximity detection in operation 612. Navigation also stops in operation 614, at which point the application 402 returns to the higher level user interface.

[0072] For purposes of illustration, the operations of flowchart 500 and flowchart 600 are described herein as occurring serially or linearly. However, multiple operations of the flowcharts may occur in parallel and need not be performed in the order shown and / or may be replaced by other operations. The operations in the flowcharts will typically be performed on the related hardware and data processor 302 in or associated with the glasses 100.

[0073] Various implementations are of course possible, where some of the operations described occur in an application such as a messaging application 946 in the user device 328, on the server system 332, or where one application on the user device 328 calls another application or SDK to obtain the desired functionality. In some examples, the operations are performed jointly between the messaging application 946 running on the user device 328 and the related hardware and data processor 302 in or associated with the glasses 100.

[0074] Figure 7 A field of view 700 is shown as seen by a wearer of glasses 100 according to some examples. Shown in the field of view are items in the real world visible through optical elements 108 and 110, including a bicycle 702 being ridden by the user along a gravel path 706. The user is following a cyclist 704 along path 706.

[0075] The user of the glasses 100 is traveling along a gravel path 706 at a speed at a following distance from the cyclist 704, as determined above by the glasses 100. The following distance has been determined to be less than the minimum safe braking distance in this situation. The glasses 100 have presented a dialog box 708 in the field of view 700, the dialog box 708 including a warning prompt 710, the warning prompt 710 including a warning symbol 712 and a warning text 714.

[0076] Figure 88 is a block diagram illustrating an example messaging system 800 for exchanging data (e.g., messages and associated content) over a network. The messaging system 800 includes multiple instances of a user device 328, each of which hosts a number of applications including a messaging client 814 and other applications 802. Each messaging client 814 is communicatively coupled to other instances of the messaging client 814 (e.g., hosted on respective other user devices 328), a messaging server system 804, and a third-party server 806 via a network 330 (e.g., the Internet). The messaging client 814 may also communicate with the locally hosted application 802 using an application program interface (API).

[0077] The messaging clients 814 are able to communicate and exchange data with other messaging clients 814 and with the messaging server system 804 via the network 330. The data exchanged between the messaging clients 814 and between the messaging clients 814 and the messaging server system 804 includes functions (e.g., commands for activating functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0078] The messaging server system 804 provides server-side functionality to specific messaging clients 814 via the network 330. Although certain functions of the messaging system 800 are described herein as being performed by either the messaging client 814 or the messaging server system 804, the location of certain functions within the messaging client 814 or within the messaging server system 804 may be a design choice. For example, it may be technically preferable to initially deploy certain technologies and functions within the messaging server system 804, but later migrate the technologies and functions to the messaging client 814 of the user device 328 that has sufficient processing power.

[0079] The messaging server system 804 supports various services and operations provided to the messaging clients 814. Such operations include sending data to the messaging clients 814, receiving data from the messaging clients 814, and processing data generated by the messaging clients 814. As examples, the data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. The data exchange within the messaging system 800 is activated and controlled by functions available via the user interface (UI) of the messaging client 814.

[0080] Turning now specifically to the messaging server system 804, an application program interface (API) server 808 is coupled to and provides a programming interface to an application server 812. The application server 812 is communicatively coupled to a database server 816, which facilitates access to a database 820 that stores data associated with messages processed by the application server 812. Similarly, a web server 824 is coupled to and provides a web-based interface to the application server 812. To this end, the web server 824 handles incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0081] The application program interface (API) server 808 receives and transmits message data (e.g., commands and message payloads) between the user device 328 and the application server 812. Specifically, the application program interface (API) server 808 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the messaging client 814 to activate the functions of the application server 812. The application program interface (API) server 808 exposes various functions supported by the application server 812, including: account registration; login functionality; sending messages from a particular messaging client 814 to another messaging client 814 via the application server 812, sending media files (e.g., images or videos) from the messaging client 814 to the messaging server 810, and possible access by another messaging client 814; setting up a collection of media data (e.g., a story); retrieving a friend list of a user of the user device 328; retrieving such a collection; retrieving messages and content; adding and removing entities (e.g., friends) to an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to the messaging client 814).

[0082] The application server 812 hosts several server applications and subsystems, including, for example, a messaging server 810, an image processing server 818, and a social network server 822. The messaging server 810 implements several message processing technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 814. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories or galleries). These collections are then made available to the messaging client 814. In view of the hardware requirements for such processing, other processor- and memory-intensive processing of data can also be performed on the server side by the messaging server 810.

[0083] The application server 812 also includes an image processing server 818 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from or received at the messaging server 810.

[0084] The social network server 822 supports various social networking functions and services and makes these functions and services available to the messaging server 810. To this end, the social network server 822 maintains and accesses an entity graph within the database 820. Examples of functions and services supported by the social network server 822 include identifying other users in the messaging system 800 with whom a particular user has relationships or that the particular user is "following", and also includes identifying interests and other entities of a particular user.

[0085] Returning to the messaging client 814, the features and functions of the external resource (e.g., application 802 or applet) are available to the user via the interface of the messaging client 814. In this context, "external" refers to the fact that the application 802 or applet is outside the messaging client 814. External resources are usually provided by a third party, but can also be provided by the creator or provider of the messaging client 814. The messaging client 814 receives a user selection of an option for launching or accessing the features of such an external source. The external resource can be an application 802 installed on the user system 328 (e.g., a "local app"), or a small-scale version of an application hosted on the user device 328 or located at the far end of the user device 328 (e.g., on a third-party server 806) (e.g., a "applet"). The small-scale version of the application includes a subset of the features and functions of the application (e.g., a full-scale local version of the application) and is implemented using a markup language document. In some examples, a small-scale version of an application (e.g., a "mini program") is a web-based markup language version of the application and is embedded in the messaging client 814. In addition to using markup language documents (e.g., .*ml files), mini programs can include scripting languages ​​(e.g., .*js files or .json files) and style sheets (e.g., *ss files).

[0086] In response to receiving a user selection of an option for launching or accessing a feature of an external resource, the messaging client 814 determines whether the selected external resource is a web-based external resource or a locally installed application 802. In some cases, the application 802 locally installed on the user device 328 can be independently of the messaging client 814 and separately launched from the messaging client 814, for example, by selecting an icon corresponding to the application 802 on the home screen of the user device 328. As used herein, an icon can include one or both of a text element and a graphic element. A small-scale version of such an application can be launched or accessed via the messaging client 814, and in some examples, any part of the small-scale application cannot (or a limited part can) be accessed outside the messaging client 814. The small-scale application can be launched by receiving a markup language document associated with the small-scale application from a third-party server 806 and processing such a document, for example, from the messaging client 814.

[0087] In response to determining that the external resource is a locally installed application 802, the messaging client 814 instructs the user device 328 to launch the external resource by executing the locally stored code corresponding to the external resource. In response to determining that the external resource is a web-based resource, the messaging client 814 communicates with (for example) a third-party server 806 to obtain a markup language document corresponding to the selected external resource. The messaging client 814 then processes the obtained markup language document to present the web-based external resource within the user interface of the messaging client 814.

[0088] Fig. 9 900 is a block diagram illustrating a software architecture 904 that may be installed on any one or more of the devices described herein. The software architecture 904 is supported by hardware, such as a machine 902 including a processor 920, a memory 926, and I / O components 938. In this example, the software architecture 904 may be conceptualized as a stack of layers, each of which provides specific functionality. The software architecture 904 includes layers such as an operating system 912, a library 908, a framework 910, and an application 906. In operation, the application 906 invokes an API call 950 through the software stack and receives a message 952 in response to the API call 950.

[0089] The operating system 912 manages hardware resources and provides public services. The operating system 912 includes, for example, a kernel 914, a service 916, and a driver 922. The kernel 914 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 914 provides memory management, processor management (e.g., scheduling), component management, networking and security settings, and other functions. The service 916 can provide other public services for other software layers. The driver 922 is responsible for controlling or interfacing with the underlying hardware. For example, the driver 922 may include a display driver, a camera driver, or Low-power drivers, Flash drivers, Serial communications drivers (e.g., Universal Serial Bus (USB) drivers), Drivers, audio drivers, power management drivers, etc.

[0090] The library 908 provides a low-level common infrastructure used by the application 906. The library 908 may include a system library 918 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 908 may include an API library 924, such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for rendering in two dimensions (2D) and three dimensions (3D) in graphics content on a display), a database library (e.g., SQLite providing various relational database functions), a web library (e.g., WebKit providing web browsing functions), etc. The library 908 may also include various other libraries 928 to provide many other APIs to the application 906.

[0091] The framework 910 provides a high-level common infrastructure used by the applications 906. For example, the framework 910 provides various graphical user interface (GUI) functions, advanced resource management, and advanced positioning services. The framework 910 can provide a wide range of other APIs that can be used by the applications 906, some of which can be specific to a particular operating system or platform.

[0092] In an example, applications 906 may include a home application 936, a contacts application 930, a browser application 932, a book reader application 934, a positioning application 942, a media application 944, a messaging application 946, a game application 948, and a variety of other applications such as third-party applications 940. Application 906 is a program that executes the functions defined in the program. Various programming languages ​​can be used to create one or more of the applications 906 constructed in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C language or assembly language). In a specific example, third-party applications 940 (e.g., those used by entities other than the vendor of a particular platform using ANDROID TM or IOS TM Software Development Kit (SDK) can be used to develop applications on platforms such as IOS TM ANDROID TM , Mobile software running on the mobile operating system of the phone or another mobile operating system. In this example, third-party applications 940 can activate API calls 950 provided by the operating system 912 to facilitate the functions described herein.

[0093] Fig.101000 in which instructions 1010 (e.g., software, programs, applications, applet, app, or other executable code) may be executed to cause the machine 1000 to perform any one or more of the methods discussed herein. For example, the instructions 1010 may cause the machine 1000 to perform any one or more of the methods described herein. The instructions 1010 transform a general purpose, unprogrammed machine 1000 into a specific machine 1000 that is programmed to perform the functions described and illustrated in the manner described. The machine 1000 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular phone, a smart phone, a mobile device, a head-mounted device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web device, a network router, a network switch, a network bridge, or any machine capable of executing instructions 1010 that specify actions to be taken by the machine 1000, either sequentially or otherwise. In addition, while only a single machine 1000 is shown, the term "machine" should also be taken to include a collection of machines that individually or jointly execute instructions 1010 to perform any one or more of the methodologies discussed herein.

[0094] The machine 1000 may include a processor 1002, a memory 1004, and an I / O component 1006, which may be configured to communicate with each other via a bus 1044. In an example, the processor 1002 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1008 that executes instructions 1010 and a processor 1012. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although Fig.10 Multiple processors 1002 are shown, but the machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0095] The memory 1004 includes a main memory 1014, a static memory 1016, and a storage unit 1018, which are all accessible by the processor 1002 via the bus 1044. The main memory 1004, the static memory 1016, and the storage unit 1018 store instructions 1010 that implement any one or more of the methods or functions described herein. During execution of the instructions 1010 by the networked system 300, the instructions 1010 may also reside, in whole or in part, within the main memory 1014, within the static memory 1016, within the machine-readable medium 1020, within the storage unit 1018, within at least one of the processors 1002 (e.g., within a cache memory of the processor), or within any suitable combination thereof.

[0096] I / O components 1006 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O components 1006 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O components 1006 may include Fig.10 1006. In various examples, the I / O components 1006 may include output components 1028 and input components 1032. The output components 1028 may include visual components (e.g., displays such as plasma display panels (PDPs), light emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. The input components 1032 may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric input, optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing instruments), tactile input components (e.g., physical buttons, touch screens that provide location and / or force of touch or touch gestures, or other tactile input components), audio input components (e.g., microphones), etc.

[0097] In another example, the I / O component 1006 may include: a biometric component 1034, a motion component 1036, an environment component 1038, or a position component 1040, as well as various other components. For example, the biometric component 1034 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 1036 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The environment component 1038 includes, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects concentrations of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The location component 1040 includes a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure that can obtain altitude), an orientation sensor component (e.g., a magnetometer), etc.

[0098] A variety of technologies may be used to achieve communication. I / O components 1006 also include communication components 1042 that are operable to couple networked system 300 to network 1022 or device 1024 via coupling 1030 and coupling 1026, respectively. For example, communication components 1042 may include a network interface component or another suitable device that interfaces with network 1022. In other examples, communication components 1042 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Parts (e.g. Low power consumption), Device 1024 may be another machine or any of a variety of peripheral devices (eg, a peripheral device coupled via USB).

[0099] In addition, the communication component 1042 can detect the identifier or include a component operable to detect the identifier. For example, the communication component 1042 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting the following: a one-dimensional barcode, such as a universal product code (UPC) barcode; a multi-dimensional barcode, such as a Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Hypercode, UCC RSS-2D barcode and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be obtained via the communication component 1042, such as positioning via Internet Protocol (IP) geolocation, ... Positioning via signal triangulation, positioning via detection of NFC beacon signals that can indicate a specific position, etc.

[0100] Various memories (e.g., memory 1004, main memory 1014, static memory 1016, and / or memory of processor 1002) and / or storage unit 1018 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instructions 1010) when executed by processor 1002 cause various operations to implement the disclosed examples.

[0101] The instructions 1010 may be sent or received over the network 1022 via a network interface device (e.g., a network interface component included in the communications component 1042) using a transmission medium and using any of a number of well-known transmission protocols (e.g., the Hypertext Transfer Protocol (HTTP)). Similarly, the instructions 1010 may be sent or received via a coupling 1026 (e.g., a peer-to-peer coupling) to the device 1024 using a transmission medium.

[0102] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media to facilitate communication of such instructions. Instructions may be sent or received over a network using a transmission medium via a network interface device.

[0103] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communications device that a user may use to access a network.

[0104] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, another type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.

[0105] "Component" refers to a device, physical entity or logic with boundaries defined by function or subroutine calls, branch points, APIs or other technologies provided for partitioning or modularizing specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be encapsulated functional hardware units designed for use with other components, and are usually part of a program that performs a specific function of a related function. Components can constitute software components (e.g., codes implemented on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., independent computer systems, client computer systems or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured to operate to perform certain operations as described herein by software (e.g., applications or application parts). Hardware components can also be implemented mechanically, electronically or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic that is permanently configured to perform certain operations. The hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component may also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component may include software executed by a general-purpose processor or other programmable processor. Once the hardware component is configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It will be understood that the decision to mechanically implement the hardware component in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., configured by software) circuit system can be driven by cost and time considerations. Therefore, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an example where a hardware component is temporarily configured (e.g., programmed), each of the hardware components does not need to be configured or instantiated at any time. For example, where a hardware component includes a general purpose processor that is configured by software to be a special purpose processor, the general purpose processor may be configured as different special purpose processors (e.g., including different hardware components) at different times. Software configures one or more specific processors accordingly, such as to constitute a specific hardware component at one time and to constitute different hardware components at different times. Hardware components may provide information to other hardware components and receive information from other hardware components. Thus, the described hardware components may be considered to be communicatively coupled.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission (e.g., by appropriate circuits and buses) between two or more hardware components or among two or more hardware components. In examples where multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessed by multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device coupled to it in communication. Then, other hardware components can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., a collection of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Whether it is a temporary configuration or a permanent configuration, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operation of the method can be performed by one or more processors or the parts implemented by the processor. In addition, one or more processors can also operate to support the execution of related operations in a "cloud computing" environment or operate as "software as a service" (SaaS). For example, at least some of the operations in the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., API). The execution of some operations can be distributed between processors, not only resident in a single machine, but deployed on multiple machines. In some examples, a processor or a part implemented by a processor can be located in a single geographic location (e.g., in a home environment, an office environment, or a server cluster). In other examples, a processor or a part implemented by a processor can be distributed across multiple geographic locations.

[0106] “Computer-readable medium” refers to non-transitory machine storage media. The terms “machine-readable medium,” “computer-readable medium,” and “device-readable medium” mean the same thing and may be used interchangeably in this disclosure.

[0107] "Ephemeral messages" are messages that are accessible for a limited duration of time. Ephemeral messages can be text, images, videos, etc. The access time for ephemeral messages can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is ephemeral.

[0108] "Machine storage medium" refers to a single or multiple storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions, routines, and / or data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium", "computer storage medium", and "device storage medium" explicitly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal media".

[0109] A "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and produces corresponding output signals that are applied to operate a machine. For example, a processor may be a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor may also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously.

[0110] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions executed by a machine, and "signal medium" includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be deemed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0111] Changes and modifications may be made to the disclosed examples without departing from the scope of the present disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure as expressed in the appended claims.

Claims

1. A method for providing a proximity warning using a head mounted device including a camera device and one or more display devices, comprising: capturing an image using a camera of the head mounted device; determining a distance between the head mounted device and an object of interest using image processing techniques performed on the captured images; determining a velocity of the head mounted device using a location component located in the head mounted device or an associated user device; determining a braking distance of the head mounted device based on the speed of the head mounted device; comparing the distance to the associated object with the braking distance; as well as Based on the distance to the object of interest being less than the braking distance, a warning notification is provided.

2. The method according to claim 1, further comprising: The speed is determined using data received from a GPS receiver located in the head mounted device or the associated user device.

3. The method according to claim 1, wherein: The warning notification includes a visual notification displayed on one or more of the display devices of the head mounted device.

4. The method according to claim 1, wherein: The braking distance is also determined based on user-specific parameters.

5. The method according to claim 4, wherein: The user specific parameters include the user's weight or the type of bicycle.

6. The method according to claim 1, wherein: The braking distance is also determined based on environmental parameters.

7. The method according to claim 6, wherein: The environmental parameters include surface type or slope.

8. A head mounted device system, comprising: one or more camera devices; one or more display devices; one or more processors; as well as a memory storing instructions that, when executed by the one or more processors, configure the system to perform operations for providing a proximity warning, the operations comprising: capturing images using a camera device of the head mounted device; determining a distance between the head mounted device and an object of interest using image processing techniques performed on the captured images; determining a velocity of the head mounted device using a location component located in the head mounted device or an associated user device; determining a braking distance of the head mounted device based on the speed of the head mounted device; comparing the distance to the object of interest to the braking distance; and Based on the distance to the object of interest being less than the braking distance, a warning notification is provided.

9. The head mounted device system according to claim 8, wherein: The warning notification includes a visual notification displayed on one or more of the display devices of the head mounted device.

10. The head mounted device system according to claim 8, wherein: The operations also include: The speed is determined using data received from a GPS receiver located in the head mounted device or the associated user device.

11. The head mounted device system according to claim 8, wherein: The braking distance is also determined based on user-specific parameters.

12. The head mounted device system of claim 11, wherein: The user specific parameters include the user's weight or the type of bicycle.

13. The head mounted device system of claim 11, wherein: The braking distance is also determined based on environmental parameters.

14. The head mounted device system of claim 13, wherein: The environmental parameters include surface type or slope.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed by a head mounted device system comprising one or more display devices and a camera, cause the head mounted device system to perform operations for providing a proximity warning, the operations comprising: capturing an image using a camera of the head mounted device; determining a distance between the head mounted device and an object of interest using image processing techniques performed on the captured images; determining a velocity of the head mounted device using a location component located in the head mounted device or an associated user device; determining a braking distance of the head mounted device based on the speed of the head mounted device; comparing the distance to the associated object with the braking distance; as well as Based on the distance to the object of interest being less than the braking distance, a warning notification is provided.

16. The non-transitory computer readable storage medium of claim 15, wherein: The operations also include: The speed is determined using data received from a GPS receiver located in the head mounted device or the associated user device.

17. The non-transitory computer-readable storage medium of claim 15, wherein: The braking distance is also determined based on user-specific parameters.

18. The non-transitory computer readable storage medium of claim 17, wherein: The user specific parameters include the user's weight or the type of bicycle.

19. The non-transitory computer-readable storage medium of claim 15, wherein: The braking distance is also determined based on environmental parameters.

20. The non-transitory computer readable storage medium of claim 19, wherein: The environmental parameters include surface type or slope.

Images