Non-contact wrist measurement

Abstract

This invention is entitled "Non-touch Wrist Measurement". Various embodiments disclosed herein include devices, systems, and methods for determining wrist measurements or watch band sizes using depth data captured by a depth sensor from one or more rotational orientations of the wrist. In some embodiments, depth data captured by a depth sensor is obtained, comprising at least two depth map images of the wrist from different angles. In some embodiments, an output corresponding to the circumference of the wrist or the watch band size is generated based on inputting the depth data into a machine learning model. A watch band size recommendation is then provided based on the output.

Description

Technical Field

[0001] This disclosure generally relates to electronic devices that perform non-touch wrist measurements, and more particularly to systems, methods, and apparatus for determining wrist measurements or watchband sizes using depth data captured by a depth sensor of the electronic device. Background Technology

[0002] Existing measurement techniques have various drawbacks when measuring the human wrist. Such techniques may require specialized measuring equipment, such as body measuring rulers or tape measures that require contact with the wrist and can be difficult to use and prone to errors. Summary of the Invention

[0003] The various embodiments disclosed herein include devices, systems, and methods for determining wrist measurements or watchband sizes using depth data captured by a depth sensor. In some embodiments, the wrist measurement technique is non-touch. In some embodiments, the wrist measurement technique is performed by a trained technician in a physical retail store. Alternatively, the wrist measurement technique is performed by a person using an electronic device, such as a smartphone that includes a depth sensor. For example, the electronic device is placed on a surface with the depth sensor facing upwards, and the user rotates their hand / wrist above the electronic device while at least two depth map images of the wrist are captured during the wrist scanning process. In some embodiments, the depth data includes at least two depth map images of the wrist, taken from sufficiently separated different angles to accurately represent the circumference of the wrist. For example, one of the user's wrist depth map images may be captured with the palm facing the depth sensor, and another of the user's wrist depth map images may be captured with the palm facing one side (e.g., approximately 90° rotation). In some embodiments, the depth data is fed into a machine learning (ML) model that outputs a measurement corresponding to the wrist circumference and / or watchband size among multiple band sizes. In some implementations, the ML model is a convolutional neural network (CNN) regressor.

[0004] In some implementations, at an electronic device with a processor, one method includes acquiring depth data captured by a depth sensor, which includes at least two depth map images of the wrist from different angles. In some implementations, an output corresponding to wrist circumference or wristband size is generated based on inputting the depth data into a machine learning model, and a wristband size recommendation is provided based on the output. Attached Figure Description

[0005] Therefore, this disclosure will be understood by those skilled in the art, and a more detailed description can be made with reference to some exemplary embodiments, some of which are shown in the accompanying drawings.

[0006] Figures 1 to 5Exemplary wrist measurement techniques according to some specific implementations are shown.

[0007] Figure 6 Examples of candidate locations along the forearm for wrist measurements are shown according to some specific implementations.

[0008] Figure 7 Examples of candidate tightness levels for wrist measurements are shown based on some specific implementations.

[0009] Figure 8 This is a flowchart illustrating an exemplary method for determining wrist measurements or watch band size based on depth data captured by a depth sensor in some specific implementations.

[0010] Figure 9 An exemplary operating environment according to some specific implementations is shown.

[0011] Figure 10 An exemplary electronic device according to some specific implementations is shown.

[0012] As is customary, the various features shown in the accompanying drawings may not be drawn to scale. Therefore, for clarity, the dimensions of various features may be arbitrarily expanded or reduced. Additionally, some drawings may not depict all components of a given system, method, or apparatus. Finally, similar reference numerals may be used throughout the specification and drawings to denote similar features. Detailed Implementation

[0013] Numerous details have been described to provide a thorough understanding of the exemplary embodiments illustrated in the accompanying drawings. However, the drawings illustrate only some exemplary aspects of this disclosure and should not be considered limiting. Those skilled in the art will recognize that other effective aspects or variations do not include all the specific details set forth herein. Furthermore, well-known systems, methods, components, devices, and circuits have not been described exhaustively so as not to obscure further relevant aspects of the exemplary embodiments described herein.

[0014] The various embodiments disclosed herein include devices, systems, and methods for determining wrist measurements and / or watchband size using depth data captured by a depth sensor. In some embodiments, the wrist measurement technique is non-touch. In some embodiments, the wrist measurement technique is performed by a personal electronic device (such as a smartphone including a depth sensor). The electronic device may be placed on a surface with the depth sensor facing upwards, and the user may rotate their hand / wrist above the electronic device while at least two depth map images of the wrist are captured during the wrist scanning process. In some embodiments, one of the user's wrist depth map images is captured with the palm facing the depth sensor, and another of the user's wrist depth map images is captured with the palm facing one side. In some embodiments, the depth data is fed into a machine learning (ML) model that outputs measurements corresponding to wrist circumference and / or watchband size among multiple band sizes. Optionally, the depth map images may include depth values ​​of portions of the hand, the entire hand, and / or portions of the arm above the wrist. In some embodiments, guidance on wrist positioning is provided while the depth data is obtained by the depth sensor.

[0015] In some implementations, the watch strap comes in multiple sizes. Furthermore, based on wrist circumference, a first watch type (e.g., wide) can have a first set of fixed sizes, and a second watch type (e.g., narrow) can have a second set of fixed sizes. However, strap sizes can overlap between the first and second watch types. In some implementations, the strap is non-adjustable (e.g., without clasps, loops, notches, etc.), although a non-adjustable strap can be slightly stretched to slide from above the hand onto the wrist. Accurate wrist measurement is particularly important for non-adjustable watches.

[0016] Figures 1 to 5 Exemplary wrist measurement techniques according to some specific implementations are illustrated. For example... Figure 1 As shown, wrist measurement technology can be implemented in electronic device 120. Figure 1 In this device, electronic device 120 includes a depth sensor 122 on the wrist 110 facing the user 100. In some specific embodiments, two depth images of the wrist 110 captured by the depth sensor 122 are used to determine wrist measurements and / or watchband size.

[0017] In some implementations, a set of more than two depth map images is obtained, and two depth map images are selected from the set based on determining that the two depth map images correspond to at least a threshold difference (e.g., 50°, 60°, 70°, 80°, 90°) at a viewpoint sufficient to provide a level of accuracy for the intended use case. In one example, the threshold difference at the viewpoint is selected based on determining that the difference is sufficient to identify two distinct elliptic parameters representing the shape and size of the wrist, where the wrist circumference can be determined according to the elliptic parameters. Identifying the elliptic parameters may require the threshold difference at the viewpoint. Optionally, the depth map images may include depth values ​​for portions of the hand, the hand as a whole, and / or other parts of the arm (e.g., the arm extending up to the elbow). In some implementations, guidance regarding the positioning of the wrist or depth sensor is provided when acquiring depth data.

[0018] When initializing the wrist measurement technology in the electronic device 120, an initialization screen 202 can be provided at the electronic device 120. For example... Figure 2 As shown, the first image 202a of the initialization screen 202 includes a wrist selection indicator 212 and a wrist selection actuator 216. In some embodiments, when user 100 selects to measure the right wrist at the wrist selection actuator 216, a wrist selection confirmation 218 in the second image 202b of the initialization screen 202 indicates the right wrist selection in the wrist selection image 202b. In some embodiments, the start selection 210 is continuously available throughout the wrist measurement technique.

[0019] When selecting the wrist to be measured, a first wrist measurement screen 204 can be provided at the electronic device 120. For example... Figure 3 As shown, the first wrist measurement screen 204 includes an instruction area 310, a fixed indicator 312a, a variable indicator 312b, and a first wrist measurement completion indicator 312c. For example, the instruction area 310 might say, "Place your right wrist palm down in the center of the lower view" and "Position the solid dot inside the hollow circle to fill it" (e.g., moving the variable indicator 312b relative to the fixed indicator 312a). In some embodiments, the fixed indicator 312a represents a preset height above the depth sensor 122. Furthermore, the variable indicator 312b moves with the wrist 110 and increases in size as it approaches the depth sensor 122. In some embodiments, the first wrist measurement screen 204 includes a view of the area above the depth sensor 122 (e.g., the area where the first depth measurement of the right wrist 110 was performed). Figure 3As shown, a series of exemplary images 204a, 204b, 204c of the first measurement screen 204 include the wrist 110, a fixed indicator 312a, and a variable indicator 312b, until the first wrist measurement completion indicator 312c is displayed in exemplary image 204d.

[0020] After the first wrist measurement is completed, a second wrist measurement screen 206 can be provided at the electronic device. For example... Figure 4 As shown, the second wrist measurement screen 206 includes an instruction area 410, a fixed indicator 412a, a variable indicator 412b, and a second wrist measurement completion indicator 412c. For example, the instruction area 410 might say “Turn your hand to one side and center your wrist in the lower view” and “Position the dot inside the circle to fill it” (e.g., moving the variable indicator 412b to fill the fixed indicator 412a). In some embodiments, the fixed indicator 412a represents a preset height above the depth sensor 122, and the variable indicator 412b moves with the wrist 110 to indicate the distance from the wrist 110 to the depth sensor 122. For example, the variable indicator 412b might increase in size as it approaches the depth sensor 122. In some embodiments, the second wrist measurement screen 206 includes a view of the area above the depth sensor 122 (e.g., the area where the second depth measurement of the right wrist 110 was performed). Figure 4 As shown, a series of exemplary images 206a, 206b, 206c of the second measurement screen 206 include the wrist 110, a fixed indicator 412a, and a variable indicator 412b, until the second wrist measurement completion indicator 412c is displayed in exemplary image 206d.

[0021] In some embodiments, two (or more) depth measurements or images of the wrist 110 captured by the depth sensor 122 are used to determine the wrist measurement or watch band size. In some embodiments, the measurement output screen 208 includes a measurement indicator. Figure 5 As shown, the measurement value indicator 512a provides a strap size of "Size 5". In some specific implementations, the measurement value output screen 508 includes a measurement completeness indicator 510 and a measurement confidence indicator 512b. For example... Figure 5 As shown, the measurement indicator 512a provides the strap size as "size 5" and the measurement confidence indicator 512b as "high".

[0022] In some alternative implementations, wrist circumference is the output of wrist measurement techniques. For example... Figure 5As shown, the measurement indicator 512a' provides the wrist circumference, which can be graphically displayed on a view of the user 100's wrist 110. In other embodiments, the band size or wrist circumference can be displayed as a 3D virtual object (e.g., using a table and / or wrist).

[0023] In some implementations, the measurement confidence indicator 512b outputs a relative value (e.g., high, good, average, poor), or simply an indicator when the confidence level for a wrist measurement technique is a recommendation to redo the wrist measurement for the user 100 times. In some implementations, the measurement confidence indicator 512b outputs a value between zero and 100.

[0024] In some implementations, two depth images of the wrist are input into a machine learning (ML) model (e.g., a neural network), which is trained to output a wrist measurement or one of multiple wristband sizes using the two input depth images (e.g., outputting two sizes when confidence is low or the circumference measurement is equal / close between the two sizes). For example, the two input depth images are a palm-facing depth map image and a lateral depth map image of the wrist. In some implementations, the ML model is trained on a dataset of left and right arms generated from real people, a dataset generated from a database of synthetic arm models that includes the wrist (e.g., forearm, wrist, or hand), or a combination thereof. For example, the ground-based true wrist circumference from a living subject can be provided by the data subject themselves or an assistive technician. In an example from a synthetic arm model database, the ground-based true wrist circumference (e.g., a cross-section) can be provided as the minimum circumference of the forearm or within a range along the longitudinal axis of the arm model. In some implementations, a convolutional neural network (CNN) is trained to regress wrist size based on two or more (e.g., five) views of the wrist or a 3D mesh of the forearm within a predetermined distance or range (e.g., 10 cm) around the styloid process. In some implementations, the ML model can use RGB data aligned with a depth map image (e.g., an image captured during the scanning process). In some implementations, a segmentation mask identifying portions of the wrist corresponding to the depth data is input into the ML model, where the segmentation mask is generated based on light intensity sensor data (e.g., RGB data).

[0025] In various implementations, the ML model can be, but is not limited to, a deep neural network (DNN), an encoder / decoder neural network, a convolutional neural network (CNN), or a generative adversarial neural network (GANN). In some implementations, the image data used to measure the wrist is designated for automated processing (e.g., machine viewing rather than human viewing) to address privacy concerns.

[0026] As described in this paper, an ML network can be trained using a dataset that includes possible obstacles such as shirt sleeves, accessories (e.g., rings, bracelets, etc.) or both. In some specific implementations, the ML network is trained to output a band size from multiple band sizes. Alternatively, the ML network is trained to output the wrist circumference and then mapped (e.g., automatically) to output the band size.

[0027] In some implementations, the watch is worn at different locations along the forearm (e.g., multiple watch / wrist positions). In some implementations, the wrist position is the input to the ML network, and the ML network is further trained based on the user's selection of one of multiple (e.g., three, four, five) positions along the forearm for wrist measurements (e.g., circumference). Figure 6 As shown, several possible positions can be defined relative to the user's styloid process. For example, position A is located in front of the styloid process, position B is located above the styloid process, position C is adjacent to and below the styloid process, and position D is away from and below the styloid process. In another example, the minimum circumference of the forearm is the wrist position (e.g., typically located below the styloid process).

[0028] In some implementations, the watch is worn on the forearm with different levels of tightness (e.g., comfort) (e.g., multiple tightness levels). In some implementations, the tightness level is the input to the ML network, and the ML network is further trained based on the user selecting more than one tightness level for wrist measurements (e.g., circumference). Figure 7 As shown, multiple levels of tightness can be: Level A: snug and deforms the skin around the wrist; Level B: comfortable and in contact with the skin around the wrist; or Level C: loose and does not contact the skin along the entire circumference of the wrist. Alternatively, the tightness can be adjusted by a preset distance such as 2mm to modify the wrist circumference.

[0029] In some implementations, the wrist can be represented as an ellipse with two orthogonal variables (e.g., width and length). However, the width and length measurements can change independently of each other, and therefore, a single view of the wrist is insufficient to accurately determine its circumference. Therefore, at least two views of the wrist are required, and these two views must be sufficiently separated during rotation to determine the wrist circumference (e.g., controlling both the independent width and length variables that determine the ellipse). In some implementations, the rotation between the two different views of the wrist is at least 45°, 75°, or 90°.

[0030] In some specific implementations, additional views taken before the first depth measurement, between the first and second depth measurements, or after the second depth measurement can be captured and used as input to the ML network.

[0031] In some implementations, the two depth images of the wrist include depth data for the fingers and the portion of the hand or forearm extending up to the elbow. In one example, including hand depth measurements (e.g., fist or fingers) in the two depth images of the wrist helps determine rotation and accurately model the ellipse representing the wrist. In another example, including forearm measurements extending up to the elbow in the two depth images of the wrist does not appear to significantly improve the accuracy of wrist measurements.

[0032] In some implementations, the ML network is adjusted to focus on the wrist portion of the two depth images. For example, input data within a predetermined distance from the styloid process or the minimum circumference of the forearm can be weighted to have a greater impact on the ML network. In another example, input depth data from the hand or elbow region of the depth images can be weighted to have a smaller impact on the ML network.

[0033] In some specific implementations, the depth image corresponds to and is registered with an image from an RGB image sensor. For example, each pixel in an RGB image has a corresponding depth value, and the combined color and depth image (e.g., RGB-D) is the image used for each depth image.

[0034] In some implementations, wrist measurement technology provides stationary measurements accurate to within + / -1 mm, + / -3 mm, or + / -5 mm. In some implementations, wrist measurement technology reduces the number of incorrectly sized watch straps subsequently returned or replaced. In one study, when wrists were measured using printable measuring tools or household items, there was an average of at least seven millimeters (too large or too small) in the wrist measurements.

[0035] In some specific implementations, the wrist measurement techniques described herein are implemented using a single wrist measurement (e.g., depth measurement, depth image, or depth data) of the wrist 110 captured by the depth sensor 122. In one example, a single wrist measurement of the wrist 110 can be captured at the depth sensor 122 with the palm facing down (e.g., see [link to documentation]). Figure 2 In another example, a single wrist measurement of the wrist 110 can be captured at a position with the palm facing 45°–90° above the depth sensor 122 (see, for example, [link to relevant documentation]). Figure 3In one implementation, a single wrist measurement uses a combined color and depth image (e.g., RGB-D). In these implementations, a single depth measurement of the wrist is input into a corresponding ML model (e.g., as described herein), which is trained to output a wrist measurement value or one of multiple watch band sizes using a single depth measurement input (e.g., a wrist depth image). In these implementations, the accuracy of determining a single wrist measurement is sufficient to identify a single watch band size (e.g., + / - 5 mm), but less accurate than implementations using two depth measurements from different angles of the wrist. Furthermore, the guidance used for locating the wrist 110 relative to the depth sensor 122 is significantly reduced (e.g., at least 50%). Therefore, the single wrist measurement technique can be implemented in a second operating mode (e.g., an accessible mode) for a subset of users (e.g., blind users, young children) because it is easier to perform.

[0036] like Figures 1 to 5 As shown, the user guidance used to collect two depth images of the wrist is visual guidance; however, the application is not intended to be so limited. In some examples, the user guidance used to collect two depth measurements of the wrist is audio guidance. In some implementations, the audio guidance varies depending on the spatial distance from alignment (e.g., x / y and depth). In one example, the frequency of a beep may change (e.g., increase, decrease) as the wrist gets closer to alignment with one of the depth images, and then remain stable (e.g., continuous or no beep) during alignment. In another example, the pitch of the audio guidance signal changes as the wrist gets closer to alignment with one of the depth images. In some implementations, the audio guidance includes text and direction, such as “Slowly move your wrist to the right, …stop now…, You are aligned.” In some implementations, guidance is provided by tactile signals or feedback that create a tactile experience by applying force, vibration, or movement to guide the wrist toward alignment with one of the depth images.

[0037] In some implementations, the depth sensor 122 is calibrated prior to wrist measurement techniques. For example, temperature fluctuations or physical impacts can affect the depth sensor and introduce bias into the depth measurements taken by the depth sensor 122. Therefore, the depth sensor can be recalibrated by holding a calibration target at a set distance and orientation from the depth sensor during the calibration process. Thus, in some implementations, the calibration target is either specifically designed for the depth sensor 122 or a common household item of known size, such as a credit card.

[0038] In some implementations, the depth sensor 122 is an active light stereo detection system. Alternatively, the depth sensor 122 may be multiple depth sensors. In some implementations, the depth sensor 122 detects the depth of the wrist based on known 2D or 3D object detection and localization algorithms, VIO information, infrared data, depth detection data, RGB-D data, other information, or some combination thereof.

[0039] Figure 8 This is a flowchart illustrating an exemplary method for determining wrist measurements or watch band size using depth data captured by a depth sensor. In some embodiments, the depth data includes at least two depth map images of the wrist, taken from sufficiently separated different angles to accurately represent the wrist circumference. In some embodiments, the measurement experience is non-touch. In some embodiments, the measurement experience is performed by a person using an electronic device. In some embodiments, the watch band is non-adjustable, and accurate wrist measurement is particularly useful. In some embodiments, method 800 is performed by a device (e.g., Figure 9 and Figure 10 The method 800 is executed by electronic devices 920, 1000. The method 800 can be executed using electronic devices or multiple devices communicating with each other. In some embodiments, the method 800 is executed by processing logic components (including hardware, firmware, software, or a combination thereof). In some embodiments, the method 800 is executed by a processor that executes code stored in a non-transitory computer-readable medium (e.g., memory). In some embodiments, the method 800 is executed by electronic devices having a processor.

[0040] At block 810, method 800 obtains depth data captured by a depth sensor, which includes at least two depth map images of the wrist from different angles. In some embodiments, the depth sensor is located in a portable electronic device (e.g., a smartphone, tablet, etc.). In some embodiments, the electronic device is placed on a surface with the depth sensor facing upwards, and a user can rotate their hand / wrist above the electronic device during the wrist scanning process. In some embodiments, the two depth map images are selected based on determining that the two depth map images correspond to at least a threshold difference (e.g., 90°) at a viewpoint, said threshold difference being sufficient to capture two different ellipses representing the circumference of the wrist. For example, one of the user's wrist depth map images is captured when the palm faces the depth sensor, and the other is captured when the palm faces one side. Optionally, the depth map images may include depth values ​​of a portion of the hand, the entire hand, and / or a portion of the arm up to the elbow. In some embodiments, guidance regarding the positioning of the wrist or depth sensor is provided when obtaining the depth data.

[0041] The depth sensor may be one or more depth sensors (e.g., structured light, time-of-flight, etc.). Alternatively, the depth data may include a two-view depth pair of two depth map images. In some embodiments, the depth data includes additional depth map images captured before, between, and after the two depth map images. In one embodiment, two cameras and stereo imaging techniques (e.g., triangulation) may be used. In one embodiment, a projector and an image sensor use structured light imaging techniques to determine the depth data.

[0042] At box 820, method 800 generates an output based on inputting depth data into a machine learning (ML) model, the output corresponding to the wrist circumference or wrist strap size. In some embodiments, the ML model is a convolutional neural network (CNN) regressor. In some embodiments, the ML model can use RGB data aligned with a depth map image. In some embodiments, a segmentation mask identifying portions of the depth data corresponding to the wrist is input into the ML model, wherein the segmentation mask is generated based on light intensity sensor data (e.g., RGB data). In some embodiments, the ML model is trained using real ground real data and / or synthetic ground real data based on user measurements of the real wrist, which identifies the wrist circumference at multiple arm locations along the longitudinal axis of the forearm (e.g., near the styloid process or the minimum circumference along the forearm). Furthermore, the ML model can be trained using training data corresponding to strap tightness (e.g., not touching a portion of the skin along the circumference, touching the wrist along the entire circumference, or partially deforming the skin). Additionally, the ML model can be trained based on weighted forearm measurements, which can increase the relative significance of measurements around the wrist (e.g., along the longitudinal axis of the forearm). In some implementations, the ML model also outputs a confidence value, below which a remeasurement of the wrist is recommended. Alternatively, the ML model outputs a confidence value corresponding to the circumference of the wrist or the size of the wrist strap.

[0043] At box 830, method 800 provides a watch band size recommendation based on this output. In some implementations, the watch band size recommendation is a watch band size selected from multiple watch band sizes (e.g., 5 sizes, 12 sizes, 15 sizes) or an actual wrist circumference measurement mapped to the wrist band size (e.g., 154 mm). In one example, the watch band size recommendation is provided to the user in a visual, textual, auditory, or a combination thereof.

[0044] In some implementations, method 800 uses a single depth measurement captured by a depth sensor to determine a wrist measurement or watch band size. In these implementations, a single depth map image of the wrist is obtained at box 810, the single depth measurement is input into an ML model at box 820, the ML model generates an output corresponding to the wrist circumference or watch band size, and a watch band size recommendation is provided based on the output at box 830. In one implementation, at box 810, a single wrist measurement is captured with the palm facing the depth sensor (e.g., see...). Figure 2 The depth sensor captures a combination of color and depth data (e.g., RGB-D) as a single depth measurement.

[0045] In some embodiments, the electronic device is a portable electronic device, such as a smartphone or tablet. In some embodiments, the electronic device is an HMD (Head-Down Display). For example, the techniques disclosed herein can be implemented on an HMD that provides an optical or video perspective view of the surrounding physical environment.

[0046] Figure 9 An exemplary operating environment 900 is illustrated, in which electronic devices 920 are used in a physical environment 905. A physical environment refers to the physical world that people can interact with and / or sense without the assistance of electronic systems. A physical environment may include physical features, such as physical surfaces or physical objects. For example, a physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and / or interact with a physical environment, such as through sight, touch, hearing, taste, and smell. Conversely, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment may include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person's physical motion or a representation thereof is tracked, and in response, one or more features of one or more virtual objects simulated in the XR system are adjusted in a manner consistent with at least one physical law. For example, an XR system can detect head movement and, in response, adjust the graphical content and sound field presented to the user in a manner similar to how such views and sounds change in a physical environment. Similarly, an XR system can detect movement of the electronic device (e.g., mobile phone, tablet, laptop, etc.) presenting the XR environment and, in response, adjust the graphical content and sound field presented to the user in a manner similar to how such views and sounds would change in a physical environment. In some cases (e.g., for accessibility reasons), an XR system may adjust the characteristics of the graphical content in the XR environment in response to a representation of physical motion (e.g., a voice command).

[0047] Many different types of electronic systems enable people to sense and / or interact with a variety of XR environments. Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays shaped like lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. Head-mounted systems may have an integrated opaque display and one or more speakers. Alternatively, head-mounted systems may be configured to receive external opaque displays (e.g., smartphones). Head-mounted systems may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Head-mounted systems may have transparent or semi-transparent displays instead of opaque displays. Transparent or semi-transparent displays may have a medium through which light representing the image is directed to the person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, holographic medium, optical combiner, optical reflector, or any combination thereof. In some implementations, transparent or translucent displays can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology, which projects graphic images onto the human retina. Projection systems can also be configured to project virtual objects onto a physical environment, such as as holograms or on a physical surface.

[0048] exist Figure 9 In the example, device 920 is shown as a single device. Some embodiments of device 920 are handheld. For example, device 920 could be a mobile phone, tablet, laptop, etc. In some embodiments, device 920 is worn by user 915. For example, device 920 could be a watch, head-mounted display (HMD), etc. In some embodiments, the functionality of device 920 is implemented via two or more devices (e.g., additionally including optional base stations). Other examples include laptops, desktop computers, servers, or other such devices that include additional capabilities in terms of power, CPU capacity, GPU capacity, storage capacity, memory capacity, etc. Multiple devices that can be used to implement the functionality of device 920 can communicate with each other via wired or wireless communication.

[0049] Figure 10This is a block diagram of an exemplary device 1000. Device 1000 illustrates an exemplary device configuration of device 920. Although some specific features are shown, those skilled in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure further relevant aspects of the specific implementations disclosed herein. Therefore, as a non-limiting example, in some specific implementations, electronic device 1000 includes one or more processing units 1002 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, etc.), one or more input / output (I / O) devices and sensors 1006, one or more communication interfaces 1008 (e.g., USB, Firewire, Thunderbolt, IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, GSM, CDMA, TDMA, GPS, IR, BlueTooth, ZigBee, SPI, I2C, or similar interfaces), one or more programming (e.g., I / O) interfaces 1010, one or more displays 1012, one or more internal or external sensor systems 1014, memory 1020, and one or more communication buses 1004 for interconnecting these components and various other components.

[0050] In some embodiments, the one or more communication buses 1004 include circuitry for interconnecting system components and controlling communication between system components. In some embodiments, the one or more I / O devices and sensors 1006 include at least one of the following: an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, or one or more depth sensors (e.g., structured light, time-of-flight, etc.) or similar devices.

[0051] In some embodiments, one or more displays 1012 are configured to present content to a user. In some embodiments, one or more displays 1012 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light-emitting field-effect transistor (OLET), organic light-emitting diode (OLED), surface-conducting electron emitter display (SED), field emission display (FED), quantum dot light-emitting diode (QD-LED), microelectromechanical system (MEMS), or similar display types. In some embodiments, one or more displays 1012 correspond to waveguide displays such as diffraction, reflection, polarization, and holography. For example, electronic device 1000 may include a single display. As another example, electronic device 1000 may include displays for each of the user's eyes.

[0052] In some embodiments, one or more internally or externally oriented sensor systems 1014 include an image capture device or array that captures image data or an audio capture device or array (e.g., a microphone) that captures audio data. The one or more image sensor systems 1014 may include one or more RGB cameras (e.g., those with a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor), monochrome cameras, IR cameras, etc. In various embodiments, the one or more image sensor systems 1014 also include an illumination source that emits light, such as a flash. In some embodiments, the one or more image sensor systems 1014 also include an on-camera image signal processor (ISP) configured to perform multiple processing operations on the image data.

[0053] Memory 1020 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices. In some embodiments, memory 1020 includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 1020 optionally includes one or more storage devices remotely located to one or more processing units 1002. Memory 1020 includes a non-transitory computer-readable storage medium.

[0054] In some embodiments, memory 1020 or a non-transitory computer-readable storage medium of memory 1020 stores an optional operating system 1030 and one or more instruction sets 1040. Operating system 1030 includes procedures for handling various basic system services and for performing hardware-related tasks. In some embodiments, instruction set 1040 includes executable software defined by binary information stored in charge. In some embodiments, instruction set 1040 is software executable by one or more processing units 1002 to implement one or more of the techniques described herein.

[0055] In some embodiments, instruction set 1040 includes a depth measurement generator 1042, which can be executed by processing unit 1002 to capture at least two depth measurements of the wrist from different angles sufficient to accurately represent the wrist circumference, according to one or more of the techniques disclosed herein. In some embodiments, instruction set 1040 includes an ML model 1044, which can be executed by processing unit 1002 to output wrist measurements based on one or more of the techniques disclosed herein and based on at least two depth measurements of the wrist.

[0056] Although instruction set 1040 is shown as residing on a single device, it should be understood that in other specific implementations, any combination of elements may reside in a separate computing device. Figure 10 This is used more as a functional description of various features present in a specific implementation, and differs from the structural diagrams of the specific implementations described herein. As those skilled in the art will recognize, items shown individually can be combined, and some items can be separated. For example, the actual number of instruction sets and the division of specific functions, and how features are allocated therein, will vary depending on the specific implementation, and in some implementations, depend in part on the specific combination of hardware, software, or firmware chosen for that particular implementation.

[0057] It should be understood that the specific embodiments described above are cited by way of example, and this disclosure is not limited to what has been specifically shown and described above. Rather, the scope includes both combinations and sub-combinations of the various features described above, as well as variations and modifications of the various features that would occur to those skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.

[0058] Those skilled in the art will recognize that well-known systems, methods, components, devices, and circuits have not been described exhaustively so as not to obscure further relevant aspects of the exemplary embodiments described herein. Furthermore, other effective aspects and / or variations do not include all the specific details described herein. Therefore, several details are described to provide a thorough understanding of the exemplary aspects illustrated in the accompanying drawings. Moreover, the drawings illustrate only some exemplary embodiments of this disclosure and should not be considered limiting.

[0059] While this specification contains numerous specific implementation details, these details should not be construed as limiting the scope of any invention or potentially claimed content, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of different embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, while certain features may be described above as functioning in certain combinations and even initially claimed in this manner, one or more features of a claimed combination may be removed from that combination in certain circumstances, and the claimed combination may involve sub-combinations or variations thereof.

[0060] Similarly, although operations are shown in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in a sequential order or the specific order shown, or requiring all shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the division of various system components in the above embodiments should not be construed as requiring such division in all embodiments, and it should be understood that the program components and the system may generally be integrated together in a single software product or packaged into multiple software products.

[0061] Therefore, specific embodiments of the subject matter have been described. Other embodiments are also within the scope of the following claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes shown in the accompanying drawings do not necessarily require a specific order or sequence to achieve the desired result. In some specific embodiments, multitasking and parallel processing may be advantageous.

[0062] The embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuits or in computer software, firmware, or hardware (including the structures disclosed in this specification and their equivalents) or in a combination thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by or control of the operation of a data processing device. Alternatively or otherwise, the program instructions may be encoded on artificially generated propagating signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof, or may be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device. Furthermore, although the computer storage medium is not a propagating signal, it may be a source or destination of computer program instructions encoded in artificially generated propagating signals. Computer storage media can also be one or more separate physical components or media (e.g., multiple CDs, disks or other storage devices), or included in one or more separate physical components or media.

[0063] The term "data processing apparatus" encompasses all kinds of devices, apparatuses, and machines for processing data, including programmable processors, computers, systems-on-a-chip, or multiple or combinations thereof. The apparatus may include special-purpose logic circuitry (e.g., FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits)). In addition to hardware, the apparatus may include code that creates an execution environment for the computer program under consideration, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. Unless otherwise specifically stated, it should be understood that throughout this specification, discussions using terms such as "processing," "computing," "calculating," "determining," and "identifying" refer to the actions or processes of computing devices, such as one or more computers or similar electronic computing devices, that manipulate or convert data represented as physical electronic or magnetic quantities within the memory, registers, or other information storage devices, transmission devices, or display devices of a computing platform.

[0064] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide results conditioned on one or more inputs. Suitable computing devices include computer systems based on multi-purpose microprocessors that access stored software that programs or configures the computing system from a general-purpose computing device to a special-purpose computing device that implements one or more specific embodiments of the subject matter of this invention. The teachings contained herein can be implemented in the software used for programming or configuring the computing device using any suitable programming, scripting, or other type of language or combination of languages.

[0065] Specific implementations of the methods disclosed herein can be performed in the operation of such a computing device. The order of the boxes presented in the above examples can be varied; for example, the boxes can be reordered, combined, and / or divided into sub-blocks. Certain boxes or processes can be executed in parallel. The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0066] The use of “applies to” or “configured to” in this document implies open and inclusive language, which does not exclude applicability to or configuration to devices performing additional tasks or steps. Similarly, the use of “based on” implies openness and inclusivity, as processes, steps, calculations, or other actions “based on” one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated. The headings, lists, and numbering included in this document are for illustrative purposes only and are not intended to be restrictive.

[0067] It will also be understood that while terms such as "first," "second," etc., may be used in this document to describe various elements, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first node can be called a second node, and similarly, a second node can be called a first node, changing the meaning of the description, provided that all occurrences of "first node" are consistently renamed and all occurrences of "second node" are consistently renamed. First nodes and second nodes are both nodes, but they are not the same node.

[0068] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of these embodiments and the appended claims, the singular forms “a” and “the” are intended to also cover the plural forms unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the term “comprising” as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0069] As used herein, the term "if" can be interpreted as meaning "when the prerequisite is true" or "when the prerequisite is true" or "in response to determination" or "according to determination" or "in response to detection" that the prerequisite is true, depending on the context. Similarly, the phrases "if it is determined [the prerequisite is true]" or "if [the prerequisite is true]" or "when [the prerequisite is true]" are interpreted as meaning "when it is determined that the prerequisite is true" or "in response to determination" or "according to determination" that the prerequisite is true or "when the prerequisite is detected" or "in response to detection" that the prerequisite is true, depending on the context.

Claims

1. A method for measuring the wrist, the method comprising: In electronic devices with processors: Depth data captured by a depth sensor is obtained, the depth data including a set of depth map images of the wrist from different angles, wherein the set of depth map images is captured for different portions of the wrist facing the depth sensor; The two depth map images are selected from the set based on the fact that the angles of the two captured depth map images differ from each other by at least a threshold amount. The output is generated by inputting the depth data into a machine learning model, and the output corresponds to the circumference of the wrist or the size of the wrist strap. as well as The output is used to provide a recommended watch band size.

2. The method according to claim 1, wherein the machine learning model is a convolutional neural network regressor.

3. The method of claim 1, wherein a segmentation mask for identifying portions of the depth data corresponding to the wrist is input into the machine learning model, wherein the segmentation mask is generated based on light intensity sensor data.

4. The method of claim 1, wherein the machine learning model is trained using real training data and synthetic training data.

5. The method of claim 1, wherein the machine learning model is trained using training data that identifies wrist circumference at multiple arm locations.

6. The method of claim 1, wherein the machine learning model is trained using training data corresponding to the tightness of the table.

7. The method of claim 1, wherein the machine learning model is trained based on weighted forearm measurements, wherein the weights of the weighted forearm measurements correspond to the relative significance of the weighted forearm measurements along the axis of the arm.

8. The method of claim 1, wherein the machine learning model further outputs a confidence value corresponding to the circumference of the wrist or the size of the watch strap of the wrist.

9. The method of claim 1, wherein the depth data comprises a dual-view depth pair of two depth map images.

10. The method of claim 9, wherein the depth data includes additional depth map images before, between, and after the two depth map images.

11. The method of claim 1, wherein the two depth map images are captured when the wrist rotates in front of the depth sensor.

12. The method of claim 1, wherein, when the depth data is obtained, guidance is provided regarding the positioning of the wrist or depth sensor.

13. The method of claim 1, wherein each of the two depth map images includes a depth value of a portion of the hand.

14. The method of claim 1, wherein the electronic device is a mobile phone or a tablet computer.

15. A system for measuring the wrist, the system comprising: Memory; and One or more processors, at a device coupled to the memory, wherein the memory includes program instructions that, when executed on the one or more processors, cause the system to perform operations including: Depth data captured by a depth sensor is obtained, the depth data including a set of depth map images of the wrist from different angles, wherein the set of depth map images is captured for different portions of the wrist facing the depth sensor; The two depth map images are selected from the set based on the fact that the angles of the two captured depth map images differ from each other by at least a threshold amount. The output is generated by inputting the depth data into a machine learning model, and the output corresponds to the circumference of the wrist or the size of the wrist strap. as well as The output is used to provide a recommended watch band size.

16. The system of claim 15, wherein the two depth map images are selected based on determining that the rotation of the wrist about the longitudinal axis of the arm differs from each other by at least a threshold amount when the two depth map images are captured.

17. A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations including: Depth data captured by a depth sensor is obtained, the depth data including a set of depth map images of the wrist from different angles, wherein the set of depth map images is captured for different portions of the wrist facing the depth sensor; The two depth map images are selected from the set based on the fact that the angles of the two captured depth map images differ from each other by at least a threshold amount. The output is generated by inputting the depth data into a machine learning model, and the output corresponds to the circumference of the wrist or the size of the wrist strap. as well as The output is used to provide a recommended watch band size.

18. The non-transitory computer-readable storage medium of claim 17, wherein the depth data comprises a dual-view depth pair of two depth map images.

Images