Systems and methods for improving radar scanning coverage and efficiency

Abstract

A system and method for obtaining the dimensions of an irregularly shaped object is disclosed. The method includes: displaying a 3D optical image of an object by a display of a handheld scanner system; displaying a grid superimposed on the 3D optical image of the object, the grid including a plurality of markers arranged in rows and columns; generating, by a radar module of the handheld scanner system, range data indicating a sensed distance between the radar module and at least a surface of a first portion of the irregularly shaped object covered by at least one item; receiving, by a processor of the handheld scanner system, the range data from the radar module when the object is scanned; and causing at least one first visual feature of a first marker of the plurality of markers to change when the processor successfully receives the corresponding range data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 874,618 filed on July 16, 2019. The contents of this U.S. Provisional Patent Application are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to data processing systems and more particularly to implementing systems and methods for improving radar scanning coverage and efficiency. Background Art

[0004] The dilemma facing today's clothing shoppers is the sheer number of choices in clothing styles, cuts and sizes, and the lack of information about how their size and their unique body proportions will fit into current styles. Summary of the invention

[0005] The present disclosure relates to an implementation system and method for obtaining the dimensions of an irregularly shaped object (e.g., a person). The method includes: displaying a 3D optical image of an object via a display of a handheld scanner system; displaying a grid superimposed on the 3D optical image of the object, the grid comprising a plurality of markers arranged in rows and columns; generating, via a radar module of the handheld scanner system, range data indicating a sensed distance between the radar module and at least a surface of a first portion of the irregularly shaped object covered by at least one item (e.g., a piece of clothing worn by the person); receiving the range data from the radar module by a processor of the handheld scanner system as the object is scanned; and causing at least one first visual feature (e.g., color, shape, or size) of a first marker of the plurality of markers to change when the processor successfully receives the corresponding range data.

[0006] The method may also include: generating 3D point cloud data by the optical module of the handheld scanner system, the 3D point cloud data representing the surface of at least one item covering at least a first portion of the irregularly shaped object, and the surface of an exposed second portion of the surface of the irregularly shaped object; receiving the 3D point cloud data from the optical module by the processor when the object is scanned; and using the 3D point cloud data and the radar data by the processor to create a map representing the dimensions of the irregularly shaped object.

[0007] In some cases, distance information is also superimposed on the 3D optical graphic indicating the distance from the handheld scanner system to the object. Additionally or alternatively, the processor causes at least one second visual feature of a first or second marker of the plurality of markers to change to indicate a pointing direction of the handheld scanner system relative to the scanned object. The second visual feature is different from the at least one first visual feature. The processor may also cause the at least one second visual feature of the first or second marker to change to a previous setting when the pointing direction of the handheld scanner system changes. When range data is successfully received during scanning of the object for a corresponding area of ​​the object, a new marker may be additionally superimposed on top of the 3D optical graphic.

[0008] The implementation system includes: a processor; and a non-transitory computer-readable storage medium containing programming instructions, the programming instructions being configured to cause the processor to implement a method for obtaining the size of an irregularly shaped object (e.g., a person) using a handheld scanner system. The programming instructions include instructions to: display a 3D optical image of the object; display a grid superimposed on the 3D optical image of the object, the grid comprising a plurality of markers arranged in rows and columns; cause a radar module to generate range data indicating a sensed distance between the radar module and at least a surface of a first portion of the irregularly shaped object covered by at least one item (e.g., a piece of clothing worn by the person); and / or cause at least one first visual feature (e.g., color, shape, or size) of a first marker of the plurality of markers to change when the radar module successfully generates corresponding range data.

[0009] The programming instructions may also include instructions for causing the optical module to generate 3D point cloud data representing a surface of at least one item covering at least a first portion of the irregularly shaped object and a surface of an exposed second portion of the irregularly shaped object; and / or causing the processor to use the 3D point cloud data and the radar data to create a map representing the dimensions of the irregularly shaped object.

[0010] The programming instructions may also include instructions for causing distance information to be also superimposed on the 3D optical pattern, the distance information indicating the distance from the handheld scanner system to the object, causing at least one second visual feature of a first or second marker of the plurality of markers to change to indicate the pointing direction of the handheld scanner system relative to the scanned object, causing the at least one second visual feature of the first or second marker to change to a previous setting when the pointing direction of the handheld scanner system changes, and / or causing a new marker to be additionally superimposed on top of the 3D optical pattern when range data is successfully received during scanning of the object for the corresponding area of ​​the object. The second visual feature may be different from the at least one first visual feature.

[0011] The present disclosure also relates to a handheld scanner system for obtaining the dimensions of an irregularly shaped object (e.g., a person). The handheld scanner system includes: a housing; an optical module including at least one 3D optical sensor, the at least one 3D optical sensor configured to generate 3D point cloud data, the 3D point cloud data representing the surface of at least one item covering at least a first portion of the irregularly shaped object and the surface of an exposed second portion of the surface of the irregularly shaped object; a radar module including at least one radar sensor, the at least one radar sensor configured to generate range data, the range data indicating a sensed distance between the housing and at least the surface of a first portion of the irregularly shaped object covered by at least one item (e.g., a piece of clothing worn by the person); a display configured to display a grid superimposed on the 3D optical pattern of the object while scanning the object using the optical module and the radar module, the grid including a plurality of markers arranged in rows and columns; and a processor. The processor is configured to: receive the 3D point cloud data from the optical module and the range data from the radar module when the object is scanned; change at least one first visual feature (e.g., color, shape, or size) of each of the markers when corresponding range data is successfully received during scanning of the object; and use the 3D point cloud data and the radar data to create a map representing the bulk dimensions of the irregularly shaped object.

[0012] In some cases, distance information is also superimposed on the 3D optical graphic, the distance information indicating the distance from the handheld scanner system to the object. Additionally or alternatively, the processor is further configured to: cause at least one second visual feature of one of the plurality of markers to change to indicate the pointing direction of the handheld scanner system relative to the scanned object; cause at least one second visual feature of the marker to change to a previous setting when the pointing direction of the handheld scanner system changes; and / or cause a new marker to be additionally superimposed on top of the 3D optical graphic when range data is successfully received during scanning of the object for a corresponding area of ​​the object. The second visual feature may be different from the at least one first visual feature. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present solution will be described with reference to the drawings, wherein like numerals refer to like items throughout.

[0014] Figure 1 is a perspective view of an exemplary system implementing the present solution.

[0015] Figure 2 is a front perspective view of an exemplary handheld scanner system.

[0016] Figure 3 is a rear perspective view of the handheld scanner system with the housing shown transparently.

[0017] Figure 4 is a block diagram of an exemplary handheld scanner system.

[0018] Figure 5 is a block diagram of an exemplary waveform radar unit.

[0019] Figure 6 is a system diagram of an exemplary radar processor.

[0020] Figure 7 is a schematic diagram of a multilateration process showing antenna geometry relative to a target used when simultaneously capturing range information using multiple antenna apertures.

[0021] Figure 8 is a perspective view showing a wire mesh mannequin and an overlay map.

[0022] Fig. 9 yes Figure 1 Illustration of an exemplary architecture for a computing device is shown.

[0023] Fig.10 It is used to understand Figure 1 Diagram of the operation of the system shown.

[0024] Figure 11A-11B(collectively referred to herein as "FIG. 11") is a flow chart of an exemplary method for generating a 3D model using a handheld scanner system.

[0025] Fig.12 is an illustration of an exemplary parametric 3D model.

[0026] Fig.13 is an illustration of an exemplary object point cloud.

[0027] Fig.14 An exemplary process of deforming a reference point cloud into a sensed object point cloud using a parametric 3D model and setting vectors is shown.

[0028] Fig.15 is an illustration of an exemplary synthetic surface model.

[0029] Fig.16 is a flow chart of an exemplary method for scanning an object.

[0030] Figure 17-Figure 19 Each of which provides an image showing a grid superimposed on top of an optical image of an object.

[0031] Fig. 20 A diagram is provided showing a first point cloud representing information created from optical data, a second point cloud representing information created from a radar sensor, and a third point cloud representing information created from both optical data and radar data. DETAILED DESCRIPTION

[0032] It will be readily understood that the components of the embodiments generally described herein and illustrated in the accompanying drawings may be arranged and designed into a variety of different configurations. Therefore, the following more detailed description of the various embodiments as illustrated in the drawings is not intended to limit the scope of the present disclosure, but merely represents various embodiments. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0033] Without departing from the spirit and essential features of the invention, the solution of the present invention may be implemented in other specific forms. The described embodiments should be considered to be illustrative in all respects only and not restrictive. Therefore, the scope of protection of the present solution is defined by the appended claims rather than by this specification. All changes within the equivalent meaning and scope of the claims are included within their scope.

[0034] References throughout this specification to features, advantages, or similar language do not imply that all features and advantages that can be achieved with the present solution should be included in any single embodiment of the present solution. Rather, language referring to features and advantages is understood to mean that a particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present solution. Thus, discussion of features and advantages and similar language throughout this specification may, but does not necessarily, refer to the same embodiment.

[0035] Furthermore, in one or more embodiments, the described features, advantages, and characteristics of the present solution may be combined in any suitable manner. Based on the description herein, one skilled in the relevant art will recognize that the present solution may be implemented without one or more specific features or advantages of a particular embodiment. In other cases, additional features and advantages may be recognized in certain embodiments, but may not be present in all embodiments of the present solution.

[0036] Reference in this specification to "one embodiment," "an embodiment," or similar language means that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present solution. Thus, the phrases "in one embodiment," "in an embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0037] As used in this document, the singular forms "a", "an", and "the" include plural references unless the context clearly dictates otherwise. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. As used in this document, the term "comprising" means "including but not limited to."

[0038] There are many systems for generating 3D surface scans or point clouds from complex real-world objects. Unlabeled 3D surfaces can be used to render static objects, but are less useful for measurement and interactive display of complex objects consisting of specific regions or sub-objects with known functions and meanings. The present solution generally relates to systems and methods for interactively capturing 3D data using a handheld scanning device with multiple sensors and providing semantic labeling of regions and sub-objects. The resulting model can then be used to derive improved measurements of the scanned object, display functional aspects of the scanned object, and / or determine the geometric fit of the scanned object with other objects based on functional aspects.

[0039] In a retail application, the purpose of this solution is to create a set of body measurements by creating a 3D point cloud representation of an individual wearing clothing using optical and radar technology. The clothing is transparent to radar signals operating at GHz frequencies, so the incident radar signal reflects from the body and provides range measurements to the inner surface. This feature of the radar is used to determine additional distance information so that it is possible to obtain a set of accurate measurements of the person. If the distance to the body and the distance to the clothing are known, a set of accurate measurements can be obtained by taking into account the range difference.

[0040] Exemplary Systems

[0041] Now refer to Figure 1-Figure 4 , provides an illustration of a system 150 that is useful for understanding the implementation of the present solution. The system 150 includes a handheld scanner system 100. The present solution is not limited to Figure 2-Figure 3 The shapes, sizes and other geometric features of the handheld scanner systems shown. In other cases, the handheld scanner systems have different overall designs.

[0042] exist Figure 1 , a handheld scanner system 100 is shown positioned relative to an irregularly shaped object 102 (a person in the illustrated application). The handheld scanner system 100 includes a housing 104 in which the various components described below are housed. The housing 104 can have a variety of configurations and is configured to fit comfortably in the hand of an operator. A support or bracket (not shown) optionally extends from the housing 104 to assist the operator in supporting the handheld scanner system 100 relative to the object 102. During operation, the handheld scanner system 100 is moved around the object 102 while being in close proximity to the object (e.g., 12-25 inches from the object). The housing 104 can be made of a durable plastic material. Portions of the housing located near the antenna element are transparent to the radar operating frequency.

[0043] Figure 4-Figure 6 An exemplary architecture of a handheld scanner system 100 is shown. The present solution is not limited to this architecture. Other handheld scanner architectures may be used without limitation herein.

[0044] like Figure 4As shown, a small display 402 is built into the housing 104 or on the exterior of the housing 104 and is visible to the operator during scanning. The display 402 can also be configured to perform basic data entry tasks (e.g., responding to prompts, entering customer information, and receiving diagnostic information about the status of the handheld scanner system 100). In addition, the handheld scanner system 100 can also provide feedback (tactile, auditory, visual, sensory, etc.) to the operator, for example, to guide the operator to the location of the customer who needs to be scanned. In some cases, the feedback (e.g., tactile, auditory, visual and / or sensory) can prompt the operator that the handheld scanner system 100 is too close or too far from the object being scanned.

[0045] The handheld scanner system 100 is powered by a rechargeable battery 406 or other power source 408. The rechargeable battery 406 may include, but is not limited to, a high energy density and / or lightweight battery (e.g., a lithium polymer battery). The rechargeable battery 406 may be interchangeable to support long-term or continuous operation. The handheld scanner system 100 may be docked in a cradle (not shown) when not in use. When docked, the cradle recharges the rechargeable battery 406 and provides an interface for a wired connection to an external computer device (e.g., computing device 106). The handheld scanner system 100 supports wired and wireless interfaces 410, 412. The housing 104 includes a physical interface 410 that allows power, high-speed transmission of data over a network 108 (e.g., the Internet or an intranet), and device programming or updating. The wireless interface 412 may include, but is not limited to, an 802.11 interface. The wireless interface 412 provides a general purpose operational communication link to exchange measurement data (radar and image data) to an auxiliary computer device 106 (e.g., an external host device) for rendering images to a display of an operator terminal. For manufacturing and testing purposes, an RF test port may be included for calibrating the RF circuitry.

[0046] The handheld scanner system 100 uses two measurement modes, namely the optical module 414 measurement mode and the radar module 416 measurement mode. Data from both modules 414, 416 flow into a central processing unit ("CPU") 418. On the CPU 418 or camera module, the optical and radar data streams are processed and synchronized together. The results of the co-processing are transmitted to the mobile computing device or other auxiliary computer device 106 (e.g., for display) via the network 108. A digital signal processor ("DSP") 420 may also be included in the handheld scanner system 100.

[0047] Subsequent measurement extraction can operate on the 3D data, and the results of the extraction can be provided to the fitting engine. Alternatively, the optical data is sent to the radar module 416. The radar module 416 interleaves the optical data with the radar data and provides a single USB connection to the auxiliary computer device 106. The optical data and radar data can also be written to the external data storage 110 to buffer the optical data frames.

[0048] Electronic memory 422 temporarily stores range information from previous scans. When radar module 416 and optical module 414 move around an object to obtain an improved representation of a body and determine physical features, the stored data from previous scans can increase the processing of the current sample. In some cases, Doppler signal processing or a moving target indicator (Moving Target Indicator, "MTI") algorithm is used to obtain an improved representation of a body and determine physical features. Doppler signal processing and MTI algorithms are well known in the art and will not be described here. Any known Doppler signal processing technology and / or MTI algorithms can be used here. The solution of the present invention is not limited to this. The handheld scanner system 100 allows the main platform to use optics and radar systems to determine two surfaces of an individual (e.g., clothing surface and the body surface of the wearer). Radar module 416 can also analyze optical range data and use this information to solve the range solution and eliminate ghosting or range ambiguity.

[0049] The optical module 414 is coupled to one or more cameras 424 and / or sensors 450. Cameras include, but are not limited to, 3D cameras, RGB cameras, and / or tracking cameras. Sensors include, but are not limited to, lasers and / or infrared systems. Each of the listed types of cameras and / or sensors is known in the art and will not be described in detail herein.

[0050] The 3D camera is configured such that the integrated 3D data structure provides a 3D point cloud (clothing and body), a volumetric parallax region (specified by the operator), and a statistical representation of the two surfaces (e.g., clothing surface and body surface). Such 3D cameras are well known in the art and are widely available from many manufacturers (e.g., Intel RealSense TM 3D optical camera scanner system).

[0051] The optical module 414 can maintain an inertial state vector relative to a fixed coordinate reference frame and relative to the body. The state information (including the orientation, translation, and rotation of the unit) is used together with the known physical offset of the antenna elements relative to the center of gravity of the unit to provide corrections and updated range estimates for each virtual antenna and the optical module. The inertial state vector can be obtained using sensor data from an optional on-board inertial measurement unit ("IMU") 426, or by a multi-antenna radar array (e.g., having 3 or more elements). IMUs and multi-antenna radar arrays are well known in the art and will not be described here. Any known or to be known IMU and / or multi-antenna radar array may be used here without limitation.

[0052] The capabilities of such systems typically achieve millimeter accuracy and resolution at close range, and improve to centimeter resolution at greater distances. Despite their excellent resolution, acquiring the physical dimensions of a clothed individual is limited by any obstructions (e.g., clothing). Camera systems that project patterns onto an object provide adequate performance for this application.

[0053] like Figure 5 As shown, the radar module 416 generally includes: a waveform generator 502, which is capable of generating an appropriate waveform for range determination; at least one antenna assembly 504 having at least one transmitting element (transmitter) 506 and at least one receiving element (receiver) 508; a frequency multiplier 510; a transmit selection switch 512; and a down converter (expansion processor), which is a matched filter configured to compare the instantaneous phase of the received target waveform with the instantaneous phase of the transmitted signal replica via the orthogonal output 514 to provide a beat frequency. An SSB mixer 516 may be included to perform up-conversion to impart a constant frequency shift. This functional block is not mandatory, but is a design enhancement to overcome the problem of feedthrough.

[0054] Note that the desired waveform is a linear frequency modulated ("LFM") chirped pulse. However, other waveforms may also be used. In order to obtain high range resolution, the radar module 416 is a broadband system. In this regard, the radar module 416 may include, but is not limited to, a radar module with X / Ku band operation. The LFM system includes a delayed copy of the transmission burst to compare with the return pulse. Since an operator using the handheld scanner system 100 cannot reliably maintain a fixed interval with the object, a laser rangefinder, optical system or other close-range sensor can help track the interval with the outer clothing of the object. This information can be used to confirm the radar measurement made using the LFM system and compensate for the delay parameters accordingly. Because the optical 3D camera 424 or laser 450 cannot measure a body covered by clothing, the ultra-wideband ("UWB") radar module 416 is responsible for making this measurement.

[0055] For the UWB radar module 416 shown, the waveform generator 502 transmits a low-power non-ionizing millimeter wave (e.g., operating between 58-64 GHz), which passes through clothing, reflects from the body, and returns a scattered response to the radar receiving aperture. In order to address this range, the UWB radar module 416 consists of two or more antenna elements 428, 430 with known spatial spacing. In this case, one or more antenna pairs are used. The aperture 204 is used together with the associated transmitting element 428. However, different configurations can meet both geometric and cost targets. In the case of multiple transmitting apertures 204, each element takes turns as a transmitter, while the other elements are receivers.

[0056] A single aperture 204 may be used for both transmission and reception. Dual apertures may be used to achieve high isolation between the transmit and receive elements for a given channel. Additionally, antenna 206 may be configured to transmit at a specific wave polarization to achieve additional isolation or to be more sensitive to a given polarization determined by the target.

[0057] The waveform transmitted in the direction of the body is an LFM ramp that sweeps through a bandwidth of several GHz. The waveform can be the same for all antenna pairs, or the waveform can be varied to express the characteristics of the reflecting surface. The bandwidth determines the explicit spatial resolution that can be achieved by radar module 416. Other radar waveforms and examples can be used, but in this case an LFM triangle waveform is used.

[0058] Now refer to Figure 6, the DSP 420 generally includes: a clock source 604 for providing a precise time base to operate the sampling clock of the processor, memory circuits and ADC, and to align the radar data with the optical frame time; a CPU (or other processor) 418, which is responsible for the configuration of the radar module 416, processing the raw radar data and calculating the range solution; an external memory 602, which stores the raw radar waveform for processing, and also stores calibration information and waveform corrections; an analog-to-digital converter ("ADC") 606; an anti-aliasing filter 608, which is used to filter the analog signal for low-pass (i.e., first Nyquist zone) or band-pass (i.e., intermediate frequency ("IF")) sampling; digital and analog multiplexing electronics 610, and a complex programmable logic device ("CPLD") or field programmable gate array ("FPGA") 612, which coordinate the timing of events. One of the quadrature channels can also be eliminated, thus simplifying the receiver hardware and eliminating a single ADC converter chain (elements 606, 608, 610, and 614) if a Hilbert transformer is used to apply a phase shift to the preserved signal chain and obtain the quadrature components required for complex signal processing. These processing techniques are well known and applicable to radar processing.

[0059] For all combinations of antenna pairs 428, 430, the range determination of the object can be performed through a process of trilateralization (for a pair) or multilateralization (for a group) of the elements. Figure 7 , when the target is located at an unknown distance in front of the handheld scanner system 100, the reflected waveform is mixed with a copy of the transmitted waveform and a beat frequency is generated. The beat frequency is directly mapped to the propagation delay of the ramp waveform. The total path length is resolved by performing a Fourier transform on the output of the LFM radar to extract the spectral frequency content. Optional analysis techniques may include the Prony method. The output of the Prony method can extract frequency, amplitude, phase, and damping parameters from the uniformly sampled signal. The utility of the Prony method analysis allows parameter extraction in the presence of noise. Prominent spectral peaks represent the round-trip distance to various scattering surfaces. How to perform this processing is well determined in the art. Other methods may also be used. For example, the radar module may also use a side scanning radar ("SSR") algorithm to determine range information to the target. The SSR algorithm may be used alone or in combination with trilateration.

[0060] like Figure 8 As shown, as the operator scans the person, a display 800 indicating the coverage area is updated. The operator will see real-time updates of the scans acquired with screen indications 802, which areas of the body have been scanned, and which areas of the body may still need to be scanned. The displayed information helps to assist the operator of the device in ensuring that all surfaces of the body have been scanned. A simple illustration of this concept is to display the outline of the body in black and white or grayscale to indicate the areas of the body that have been scanned.

[0061] In an exemplary application, the handheld scanner system 100 would allow a large number of fully clothed customers to be quickly scanned. A significant advantage of this technology is that the handheld scanner system 100 is not limited to a fixed orientation relative to the subject, and therefore can make challenging measurements of body areas that would otherwise be difficult to make with fixed structures. Additionally, the combination of two spatial measurement systems working in tandem can provide a higher fidelity reproduction of an individual's dimensions.

[0062] Although the handheld scanner system 100 is described in an exemplary garment fitting application, it will be appreciated that the handheld scanner system 100 may be used to determine dimensional measurements of other irregularly shaped objects and in other applications utilizing dimensional measurements of irregularly shaped objects.

[0063] Now refer to Fig. 9 , provides a detailed block diagram of an exemplary architecture of the computing device 106. The computing device 106 includes, but is not limited to, a personal computer, a desktop computer, a laptop computer, or a smart device (eg, a smart phone).

[0064] The computing device 106 may include Fig. 9 More or fewer components may be shown. However, the components shown are sufficient to disclose an embodiment of implementing the present solution. Fig. 9 The hardware structure of represents one embodiment of a typical computing device configured to facilitate the generation of a 3D model using data acquired by the handheld scanner system 100. Likewise, Fig. 9 The computing device 106 implements at least a portion of the methods described herein in connection with the present solution.

[0065] Some or all of the components of computing device 106 may be implemented as hardware, software, and / or a combination of hardware and software. Hardware includes, but is not limited to, one or more electronic circuits. Electronic circuits may include, but are not limited to, passive components (e.g., resistors and capacitors) and / or active components (e.g., amplifiers and / or microprocessors). Passive and / or active components may be suitably arranged and / or programmed to perform one or more methods, processes, or functions described herein.

[0066] like Fig. 9 As shown, the computing device 106 includes a user interface 902, a CPU 906, a system bus 910, a memory 912, which is connected to and accessible to other parts of the computing device 106 via the system bus 910, and a hardware entity 914, which is connected to the system bus 910. The user interface may include input devices (e.g., keys 950) and output devices (e.g., speakers 952, displays 954, and / or light emitting diodes 956) that facilitate user-software interaction for controlling the operation of the computing device 106.

[0067] At least some of the hardware entities 914 perform actions involving accessing and using a memory 912, which may be a random access memory and / or a disk drive. The hardware entity 914 may include a disk drive unit 916, which includes a computer-readable storage medium 918 on which one or more sets of instructions 920 (e.g., software code) are stored, which are configured to implement one or more methods, processes, or functions described herein. During the execution of the instructions 920 by the computing device 106, the instructions 920 may also reside completely or at least partially in the memory 912 and / or in the CPU 906. The memory 912 and the CPU 906 may also constitute a machine-readable medium. The term "machine-readable medium" as used herein refers to a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) storing one or more sets of instructions 920. As used herein, the term "machine-readable medium" also refers to any medium capable of storing, encoding, or carrying a set of instructions 920 for execution by the computing device 106, and causing the computing device 106 to perform any one or more methods of the present disclosure.

[0068] In some cases, the hardware entity 914 includes electronic circuitry (e.g., a processor) that is programmed to facilitate the generation of a 3D model using data acquired by the handheld scanner system 100. In this regard, it should be understood that the electronic circuitry can access and run one or more applications 924 installed on the computing device 106. The software applications 924 are typically used to: facilitate parametric 3D model building; store the parametric 3D model; obtain acquired data from the handheld scanner system 100; use the acquired data to identify a general object category or class to which the scanned object belongs; identify 3D models of objects that belong to the same category or class as the scanned object; generate a point cloud; modify the point clouds; compare the point clouds; identify a first point cloud that best fits a second point cloud; obtain a setup vector associated with the first point cloud; set the 3D model using the setup vector; obtain a complete 3D surface model and region markers from the set 3D model; obtain measurements of certain features of the scanned object from the setup vector; improve the measurements; and / or present information to a user. Fig.10 An illustration of these operations of a computing device is provided in . Fig.10 Also shown is the operational relationship between the handheld scanner system and the computing device.Other functionality of the software application 924 will become apparent as the discussion proceeds.

[0069] Exemplary method for generating improved 3D models

[0070] Referring now to FIG. 11 , a method of using a handheld scanner system (eg Figure 1Flowchart of an exemplary method 1100 for generating an improved 3D model using data acquired by a handheld scanner system 100. The 3D model has multiple meaningful sub-regions. The object (e.g., Figure 1 Each meaningful sub-region of the irregular shaped object 102) is identified by a key value and an associated value (semantic tag).

[0071] Method 1100 includes a plurality of operations shown in blocks 1102-1144. These operations may be performed in the same or different order as shown in FIG. 11. For example, operations 1116-1118 may be performed before operations 1109-1114, rather than before operations 1116-1118. Fig.11A The process is shown as being executed after 1109 to 1114. The present solution is not limited thereto.

[0072] like Fig.11A As shown, method 1100 starts at 1102 and continues at 1104, where a plurality of composite 3D models are created. The composite 3D models may be generated using a computing device external to and / or remote from the handheld scanner system (e.g., Figure 1 and 9 The computing device 106 is constructed. The computing device includes but is not limited to a personal computer, a desktop computer, a laptop computer, a smart device and / or a server. Each of the listed computing devices is well known in the art and will not be described here.

[0073] The synthetic 3D model represents different objects (e.g., living things (e.g., people (male and / or female) or animals), clothes, cars, etc.) sensed using the handheld scanner system. Methods for constructing synthetic 3D models are well known in the art. Any known or to-be-known method for building a synthetic 3D model may be used herein without limitation. For example, in some cases, the present solution employs a computer-aided design ("CAD") software program. CAD software programs are well known in the art. Additionally or alternatively, the present solution uses 3D modeling techniques described in a document entitled "A Morphable Model For The Synthesis Of 3D Faces" by Blanz et al. ("Blanz"). Fig.12 An exemplary synthesized 3D model of a human 1200 generated according to the 3D modeling technique is shown. The solution of the present invention is not limited to the details of this embodiment. In the next 1106, the plurality of synthesized 3D models are stored in a data storage (e.g., Figure 4 Memory 422 and / or Figure 6 external storage 602) for later use.

[0074] Each synthetic 3D model has a plurality of variable features. These variable features include, but are not limited to, appearance, shape, direction and / or size. These features can be changed by setting or adjusting parameter values ​​(e.g., height parameter value, weight parameter value, chest girth parameter value, waist girth parameter value, etc.), which are common to all objects in a given general object category or kind (e.g., human category or kind). In some cases, each parameter value can be adjusted to any value from -1.0 to 1.0. The set of discrete parameter value settings for synthetic 3D models is referred to as "setting vectors" here. The synthetic 3D model is designed so that for each perceptually different shape and appearance of a given general object category or kind, there is a setting vector that best describes the shape and appearance. The set of setting vectors for synthetic 3D models describes the distribution of possible shapes and appearances, which represents the distribution of those appearances and shapes that may be found in the real world. The set of setting vectors is referred to as "setting space" here.

[0075] The composite 3D model may also include multiple sub-objects or regions (e.g., forehead, ears, eyes, mouth, nose, chin, arms, legs, chest, hands, feet, neck, head, waist, etc.). Each meaningful sub-object or region is labeled with a key-value pair. A key-value pair is a set of two associated data items: a key (which is a unique identifier for some data item); and a value (the value is the identified data, or a pointer to the storage location of the data in a data store). The key may include, but is not limited to, a sequence of numbers, a sequence of letters, an alphanumeric sequence, or a sequence of numbers, letters, and / or other symbols.

[0076] Refer again Fig.11A , the method 1100 continues at 1107 , where each synthesized 3D model is combined with a dataset of actual 3D scan point clouds to generate a parametric 3D model. In one example, (some) datasets of 3D scan point clouds of actual (real) objects can be obtained by the handheld scanner system 100 .

[0077] Next, in 1108, an operator of the handheld scanner system scans an object (e.g., a person) in a continuous motion. Prior to scanning, additional physical features may be added to the real surface of the object to facilitate better and faster creation of a point cloud of the object using the scanned data later. For example, the object may wear a band having a given visible pattern formed thereon. The band is designed to circumscribe or extend around the object. The band has a width that allows (a) a portion of the object located above the band to be visible and (b) a portion of the object located below the band to be visible. The visible pattern of the band (e.g., a belt) facilitates faster and more accurate object recognition in point cloud data, shape registration, and 3D pose estimation. Thus, the additional physical features improve the functionality of the implementation device.

[0078] As a result of the scan, the handheld scanner system performs operations to acquire radar depth data (e.g., via Figure 4 radar module 416), optical camera depth data (e.g., via Figure 4 camera 424), color data (e.g., via Figure 4 camera 424) and / or spectral data (e.g., via Figure 4 The acquired data is stored in a data storage local to the handheld scanner system or remote from the handheld scanner system (e.g., Figure 1 In this regard, 1108 may also involve communicating via a network (e.g., Figure 1 The acquired data is transmitted to a remote computing device (e.g., Figure 1 and Fig. 9 computing device 106) to be stored in a remote data store.

[0079] The acquired data is used in 1109 to construct a 3D point cloud of the object. The term "point cloud" used here refers to a set of data points in a certain coordinate system. In a 3D coordinate system, these points can be defined by x, y and z coordinates. More specifically, optical key points are extracted from the optical data. In one case, these key points can be extracted using, but not limited to, ORB features. The key points are indexed and used to cascade all scan data into a 3D point cloud of the object. These key points are also used to (continuously) track the position (rotation and translation) of the handheld scanner system 100 relative to the fixed coordinate frame. Since the number and quality of key points depend on the scanning environment, it may happen that the handheld scanner system 100 cannot collect enough key points and cannot construct a 3D point cloud. To solve this problem, an additional layer can be applied to the actual surface of the object to ensure the number and type of key points. This layer can be some patterns worn by the object or projected on the object (on a belt).

[0080] In 1110, the acquired data is used to identify the general object category or class (e.g., a person) to which the scanned object belongs. For example, image processing is used to identify objects in the image, and feature information of the identified object is extracted from the image (e.g., the shape of all or part of the identified object and / or the color of the identified object). Object recognition and feature extraction techniques are well known in the art. Any known or later known object recognition and feature extraction techniques may be used herein without limitation. The extracted feature information is then compared with stored feature information, which defines multiple general object categories or classes (e.g., a person, an animal, a vehicle, etc.). When there is a match (e.g., to a certain extent), the corresponding general object category or class is identified in 1110. The solution of the present invention is not limited to the details of this embodiment. The operation of 1110 can be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 The computing device 106) is executed.

[0081] Next, at 1112, an initial 3D model is identified from a plurality of parametric 3D models representing objects that belong to the same general object class or category (e.g., a person) as the scanned object. Methods for identifying a 3D model from a plurality of 3D models based on various types of queries are well known in the art and will not be described herein. Any known or yet to be known method for identifying a 3D model may be used herein without limitation. One such method is described in a document entitled “ASearchEngine For 3D Models” by Funkhouser et al. (“Funkhouser”). The operations of 1112 may be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 The computing device 106) is executed.

[0082] In 1114, multiple reference point clouds are generated using the initial 3D model and multiple different setting vectors. Methods for generating point clouds are well known in the art. Any known or future known method for generating point clouds can be used here without limitation. Fig.12 An exemplary reference point cloud 1200 is shown. The operation of 1114 may be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 The computing device 106) is executed.

[0083] Thereafter, at 1116 , the acquired optical camera depth data is used to construct a partial or complete point cloud (“object point cloud”) of the scanned object. Fig.14 An exemplary object point cloud is shown.

[0084] The object point cloud may optionally be modified using radar depth data shown at 1118, color data shown at 1120, and / or spectral data shown at 1120. Methods of modifying point clouds using various types of camera-based data are well known in the art and will not be described herein. Any known or later known method of modifying a point cloud using camera-based data may be used herein without limitation. The operations of 1116-1120 may be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 The computing device 106) is executed.

[0085] 1118 is an important operation because it improves the accuracy of the object point cloud. As described above, radar can "see through" clothing because GHz frequencies reflect moisture in the skin. The radar depth measurement is registered with the optical camera data to identify a point or area on the scanned object and measure the radar distance from the device to the point or area. As a result, the final scan point cloud includes such identified radar points in addition to the optical point cloud. The radar point cloud can be used to reconstruct a 3D radar representation of the body of the object. These radar points can then also be used to modify the optical point cloud obtained by covering at least a portion of the object with something (e.g., clothing) and / or modify the reference point cloud (e.g., a human body model) to best match the true shape of the object. Since the modified cloud better represents the true shape of the object (e.g., an unclothed human body) compared to the optical cloud (wearing clothes), it improves the process of identifying and deforming the reference point cloud to obtain a better improved 3D model as shown in 1122-1132, thereby better matching the true shape of the object. As shown at 1134, the difference between the optical range and the radar range in the marking area can be further utilized to obtain a more accurate measurement of specific features (eg, chest circumference and / or waist circumference) of the scanned object.

[0086] At 1122, the object point cloud is compared to a plurality of reference point clouds. Based on the results of these comparisons, a reference point cloud is identified at 1124. The identified reference point cloud (e.g., Fig.12The reference point cloud 1200 is the reference point cloud that best matches the object point cloud. Best matching methods for point clouds are well known in the art. Any known or later known best matching method for point clouds may be used herein without limitation. Exemplary best matching methods are described in a document entitled “Computer Vision and Image Understanding” by Bardinet et al. and a document entitled “Use of Active Shape Models for Locating Structures in Medical Images” by Cootes. The operations of 1122-1124 may be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 In addition, the operations of 1122-1124 may be optionally performed before the radar modification of the object point cloud in 1118.

[0087] Once the best matching reference point cloud is identified, method 1100 continues Fig. 11B 1126. Fig. 11B As shown, 1126 includes obtaining a setup vector associated with the best matching reference point cloud. The initial 3D model is then converted to an improved 3D model at 1128. The improved 3D model is an instance of a parameterized 3D model that better matches the object point cloud. The conversion is achieved by setting the variable parameter values ​​of the initial 3D model to the parameter values ​​specified in the setup vector. The set initial 3D model describes the appearance and shape changes of the setup vector. The set initial 3D model also includes markers for meaningful sub-objects or areas. The operations of 1126-1128 can be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 The computing device 106) is executed.

[0088] In 1130, a complete 3D surface model and region labels are obtained from the improved 3D model. Surface modeling techniques are well known in the art and will not be described herein. Any known or to-be-known surface modeling techniques may be used herein without limitation. In some cases, a CAD software program with surface modeling capabilities is used in 1130. Fig.15 An exemplary surface model is shown. The operation of 1130 may be performed by the handheld scanner system 100 and / or a remote computing device (e.g., Figure 1 and Fig. 9 The computing device 106) is executed.

[0089] In 1132, the appearance of the scanned object is synthesized. The synthesis is achieved by fitting and mapping a 3D surface model to optical data and optional radar data from an object point cloud (optical and / or radar cloud, 3D radar reconstruction cloud). The phrase "fitting and mapping" as used herein means: finding the correspondence between points in the 3D surface model and points in the object point cloud, and optionally modifying the 3D surface model to best fit the object point cloud. In 1134, measurements of certain features of the scanned object (e.g., height, weight, chest circumference and / or waist circumference) are obtained from the setting vectors of the improved 3D model. As shown in 1136, the measurement is optionally (a) improved using radar data associated with marked areas of the improved 3D model, and / or (b) improved based on the geometry of the areas marked in the improved 3D model. As shown in 1138, the synthesized appearance of the scanned object is presented to an operator of the handheld scanner system and / or another computing device (e.g., Figure 1 and Fig. 9 In some cases, the composite appearance of the scanned object is similar to Fig.15 The surface models shown are identical.

[0090] The appearance and / or metrics (unmodified and / or modified) of the resultant scanned object may optionally be used at 940 to determine a geometric fit of the scanned object with at least one other object (e.g., a shirt or pants) or to identify at least one other object that fits the scanned object. At optional 1142, further information is presented to an operator of the handheld scanner system and / or another computing device (e.g., Figure 1 and Fig. 9 The further information indicates the determined geometric fit of the scanned object with the at least one other object and / or includes identification information of the at least one other object. For example, clothing suggestions for the scanned individual are output from the handheld scanner system and / or another computing device (e.g., a personal computer, a smart phone, and / or a remote server). Subsequently, execution 1144 occurs, where the method 1100 ends or performs other processing (e.g., returns to 1104).

[0091] The present solution is not limited to the details of FIG. 11 . For example, in some cases, a neighborhood of a reference point cloud is identified. Interpolated reference vectors are accumulated from the contributions of associated neighboring reference vectors. An initial 3D model is set using the interpolated reference vectors. The set initial 3D model describes the appearance and shape changes of the interpolated reference vectors. The region labels still refer to the transformed sub-regions. A complete 3D surface model and region labels are then obtained from the set initial 3D model.

[0092] The following operations of the above-mentioned method 1100 are considered novel: selecting a 3D parameter model from multiple 3D parameter models using object categories; using setting vectors associated with the reference point cloud found by spatial hashing to find setting vectors that fit the 3D parameter model to the sensed unstructured data; using the neighborhood of the found setting vectors to further improve the shape and appearance of the 3D parameter model; and / or returning an appearance model and semantic information of marked areas in the 3D parameter model.

[0093] Exemplary methods for improving scanning coverage and efficiency

[0094] As described above, the handheld scanner system 100 utilizes optical and radar technology to scan the clothed object 102. An optical camera (e.g., Figure 4 An optical camera 424) is used to determine the basic shape of the clothed object 102. The shape information is then used to optimize an algorithm for the shape of the radar sensor behind the garment so that measurements can be determined more accurately.

[0095] As the operator scans the object, the optical information can be seen on the screen. The operator uses the optical information to move the scanner over the object so that a 3D image of the object can be rendered. However, although the operator can view the optical information in the display, the operator has no information about the quality of the radar sensor data. Without visual assistance, it is difficult to determine whether the operator has captured a sufficient amount of radar data over the entire object. Therefore, the present solution uses augmented reality to add information to the optical display to assist the operator by improving the quality of the radar data.

[0096] Now refer to Fig.16 , an improved method 1600 for scanning an object or subject is provided. The method 1600 may be performed in 1108 of FIG. 11 . The method 1600 may include Fig.16 Some or all of the operations shown. These operations can be performed in the same Fig.16 In method 1600, a handheld scanner system (e.g., Figure 1 The handheld scanner system 100) performs operations to obtain radar depth data, optical camera depth data, color data and / or spectral data of the scanned object.

[0097] like Fig.16As shown, method 1600 begins at 1602 and continues at 1604, where physical features are optionally added to an object or a real surface of an object. Physical features can be added to facilitate the subsequent use of scan data to create an object point cloud better and faster. For example, an object can wear a belt with a given visible pattern formed thereon. The belt is designed to circumscribe or extend around the object. The width of the belt allows (a) a portion of the object located above the belt to be visible and (b) a portion of the object located below the belt to be visible. The visible pattern of the belt (e.g., a belt) contributes to faster and more accurate object recognition in point cloud data, shape registration, and 3D pose estimation. Therefore, the additional physical features improve the functionality of the implementation device.

[0098] At 1606, a scanning operation of the handheld scanner system is initiated (e.g., via a physical or virtual button). As shown at 1608, the handheld scanner system displays an optical pattern of the object to be scanned. At 1610, a grid is superimposed on top of the displayed optical pattern. The optical pattern may be a 3D optical pattern (e.g., a 3D optical pattern representing a person).

[0099] exist Fig.17 1700 is provided in an image of an exemplary grid 1700 superimposed on top of a displayed optical graphic 1720. The grid 1700 includes a plurality of rows 1702 and a plurality of columns 1704 defining a grid region 1710. The grid lines defining the cells in the rows and columns are shown in a transparent format. A plurality of marks 1712 are provided in each row, column, and / or grid region of the grid 1700. Although the marks 1712 are displayed as colored dots, the present solution is not limited thereto. The marks 1712 may have any shape and / or color. When the radar data of the corresponding region of the object is successfully acquired, the shape and / or color of the marks 1712 may be selectively changed. For example, when the radar information of the corresponding position on the object has not yet been acquired, the marks are displayed in a dark color (e.g., black). When the radar information of the corresponding position on the object is successfully acquired, the marks are displayed in a medium or light color (e.g., green or white). The solution of the present invention is not limited to the details of the present embodiment.

[0100] Optionally, a reference point cloud can be identified from multiple reference point clouds based on basic information of the scanned object (e.g., height and weight for human body scans); the accumulated optical cloud is then aligned with the reference point cloud to provide feedback and information to help guide the operator, including but not limited to information about the area of ​​interest (area that needs to be scanned by the radar), guidance for scanning direction and device orientation, etc. Methods for aligning a 3D (optical) point cloud with a reference point cloud are well known in the art. One such method is the Iterative Closest Point ("ICP") algorithm.

[0101] Refer again Fig.16 Information is also optionally superimposed on top of the optical pattern in 1612 to indicate the distance between the handheld scanner system and the object. Fig.17 An image of exemplary distance information 1708 superimposed on top of the optical pattern 1720 is provided in FIG. An operator of the handheld scanner system 100 uses the displayed optical pattern 1720, grid 1700, and / or distance information 1708 to assist in the scanning process.

[0102] Next, at 1614, operations are performed by the handheld scanner system to determine a location on the object that the handheld scanner system is pointing at for scanning purposes. The physical features added at 1604 may facilitate this determination. Once this determination is made, a first marker of the grid is identified as corresponding to the location on the object that the handheld scanner system is currently pointing at, as shown at 1616. At least one visual feature (e.g., size and / or color) of the first marker is modified at 1618.

[0103] Fig.17 An image is provided showing a marker 1706 that has been modified to show the direction in which the handheld scanner system is pointing relative to the object. Fig.17 As shown, the mark 1706 is enlarged and displayed in a relatively lighter color (e.g., green or white) so that it is visually distinguishable from other marks of the grid 1700. The solution of the present invention is not limited to the details of this embodiment. The mark 1706 can be modified selectively or additionally so as to have a different color and / or shape from other marks.

[0104] Refer again Fig.16 , method 1600 continues at 1620, where the handheld scanner system acquires radar depth data, optical camera depth data, color data, and / or spectral data of the object while the object is scanned. The radar data is analyzed at 1622 to identify a second marker of the grid corresponding to a location on the object for which radar depth data was successfully acquired at 1620. At least one visual feature (e.g., color) of the second marker is modified at 1624. Fig.17 An image of a modified second marker 1714 is provided. Fig.17 As shown, the color of the second marker 1714 has changed from a dark color (e.g., black) to a light color (e.g., gray or green). Additional markers 1716 are also added to the grid to illustrate the successful acquisition of radar depth data for a given area of ​​the object. The present solution is not limited to Fig.17 details.

[0105] Next, in Fig.16In 1626, it is determined whether a sufficient amount of radar depth data has been obtained. If yes [1626: yes], 1630 is executed, and the method 1600 ends or performs other processing.

[0106] If not [1626: No], then 1628 is performed, where the visual characteristics of the first marker are modified to their previous settings. For example, the size of the first marker is reduced and / or the color of the first marker is changed back to a darker color so that the first marker can no longer be distinguished from the other markers. There may be a time delay when the handheld scanner system changes its pointing direction and when the visual characteristics of the first marker are modified in 1628. This is useful for showing the operator the movement of the handheld scanner system within a given time period. Subsequently, method 1600 returns to 1614 so that the scanning process can be repeated from another area of ​​the object.

[0107] Reference Figure 18-Figure 19 , a diagram is provided showing a grid 1700 having an increased number of modified markers to show radar data successfully acquired for corresponding locations on an object.

[0108] Once the object is fully scanned, the optical data is used to create a 3D image of the wearer as discussed with respect to FIG. 11 . This optical 3D reconstruction does not rely on grid information. The radar information is then used to create a new 3D image within the contours of the garment (behind). The grid-assisted scan generates a denser and more consistent radar pattern. The radar pattern is used to generate a more accurate body image, which significantly improves measurement accuracy.

[0109] Fig. 20 3D point clouds are shown. The first point cloud 2000 represents information created from optical data. The second point cloud 2002 represents information created from a radar sensor. The third point cloud 2004 represents information created from both optical data and radar data. The spacing between the first and third point clouds clearly shows clothing lines and skin lines.

[0110] Although the present solution has been shown and described with respect to one or more embodiments, other technicians in the field will think of equivalent changes and modifications after reading and understanding this specification and drawings. In addition, although a particular feature of the present solution may be disclosed only for one of several embodiments, this feature can be combined with one or more other features of other embodiments, which is desirable and advantageous for any given or specific application. Therefore, the breadth and scope of the present solution should not be limited by any of the above-mentioned embodiments. On the contrary, the scope of the present solution should be defined according to the attached claims and their equivalents.

Claims

1. A method for obtaining the size of an irregularly shaped object, the method comprising: A 3D optical image of the object is displayed by a display of the handheld scanner system; superimposing a grid on the 3D optical pattern of the object, the grid comprising lines defining a plurality of cells arranged in rows and columns; providing indicia within each of the plurality of cells, the indicia being superimposed atop a display of data defining the 3D optical pattern; generating, by a radar module of the handheld scanner system, range data indicating a sensed distance between the radar module and at least a surface of a first portion of the irregularly shaped object covered by at least one item; receiving, by a processor of the handheld scanner system, the range data from the radar module when the object is scanned; analyzing, by the processor, the range data to identify at least one first marker among a plurality of markers provided within the plurality of cells, the at least one first marker corresponding to a location on the object for which radar depth data was received; as well as The radar module is caused, by the processor, to generate an optimized radar pattern based on a change in at least one first visual feature of the first marker.

2. The method according to claim 1, further comprising: 3D point cloud data is generated by an optical module of the handheld scanner system, the 3D point cloud data representing a surface of at least one item covering at least a first portion of the irregularly shaped object and a surface of an exposed second portion of the irregularly shaped object surface.

3. The method according to claim 2, further comprising: The 3D point cloud data is received by the processor from the optical module while the object is scanned.

4. The method according to claim 3, further comprising: The 3D point cloud data and the radar data are used by the processor to create a map representing dimensions of a body of the irregularly shaped object.

5. The method of claim 1, wherein the at least one first visual characteristic comprises color, shape, or size.

6. The method of claim 1, wherein distance information indicating the distance from the handheld scanner system to the object is also superimposed on the 3D optical pattern.

7. The method according to claim 1, further comprising: The processor causes at least one second visual feature of a first marker or a second marker of the plurality of markers to change to indicate a pointing direction of the handheld scanner system relative to an object being scanned.

8. The method of claim 7, wherein the at least one second visual characteristic is different from the at least one first visual characteristic.

9. The method according to claim 7, further comprising: When the pointing direction of the handheld scanner system changes, the processor causes the at least one second visual feature of the first marker or the second marker to change to a previous setting.

10. The method according to claim 1, further comprising: When range data is successfully received during scanning of the object for a corresponding area of ​​the object, a new marker is additionally superimposed on top of the 3D optical pattern by the processor.

11. The method of claim 1 , wherein the irregularly shaped object is a person and the at least one item is a garment worn by the person.

12. A system comprising: processor; A non-transitory computer-readable storage medium comprising programming instructions configured to cause the processor to implement a method for obtaining the dimensions of an irregularly shaped object using a handheld scanner system, wherein the programming instructions include the following instructions: Display 3D optical graphics of objects; superimposing a grid on the 3D optical pattern of the object, the grid comprising lines defining a plurality of cells arranged in rows and columns; providing indicia within each of the plurality of cells, the indicia being superimposed atop a display of data defining the 3D optical pattern; causing a radar module to generate range data indicating a sensed separation between the radar module and at least a surface of a first portion of the irregularly shaped object that is covered by at least one item; analyzing the range data to identify at least one first marker among a plurality of markers provided in the plurality of units, the at least one first marker corresponding to a location on the object for which radar depth data was received; as well as The radar module is caused to generate an optimized radar pattern based on a change in at least one first visual characteristic of the first marker.

13. The system of claim 12, wherein the programming instructions further comprise: Instructions for causing an optical module to generate 3D point cloud data representing a surface of at least one item covering at least a first portion of the irregularly shaped object and a surface of an exposed second portion of the irregularly shaped object surface.

14. The system of claim 13, wherein the programming instructions further comprise: Instructions cause a processor to create a map representing dimensions of a body of the irregularly shaped object using the 3D point cloud data and the radar data.

15. The system of claim 12, wherein the at least one first visual characteristic comprises color, shape, or size.

16. The system of claim 12, wherein distance information indicating a distance from the handheld scanner system to the object is also superimposed on the 3D optical graphic.

17. The system of claim 12, wherein the programming instructions further comprise: Instructions for causing at least one second visual feature of a first marker or a second marker of the plurality of markers to change to indicate a pointing direction of the handheld scanner system relative to an object being scanned.

18. The system of claim 17, wherein the at least one second visual characteristic is different from the at least one first visual characteristic.

19. The system of claim 17, wherein the programming instructions further comprise: When the pointing direction of the handheld scanner system changes, the at least one second visual feature of the first mark or the second mark is changed to the previously set instructions.

20. The system of claim 12, wherein the programming instructions further comprise: Instructions for causing new markings to be additionally superimposed on top of the 3D optical pattern when range data is successfully received during scanning of the object for corresponding areas of the object.

21. The system of claim 12, wherein the irregularly shaped object is a person and the at least one item is clothing worn by the person.

22. A handheld scanner system for obtaining dimensions of an irregularly shaped object, the handheld scanner system comprising: case; an optical module comprising at least one 3D optical sensor configured to generate 3D point cloud data representing (A) a surface of at least one item covering at least a first portion of the irregularly shaped object, and (B) a surface of an exposed second portion of the surface of the irregularly shaped object; a radar module comprising at least one radar sensor configured to generate range data indicative of a sensed separation between the housing and at least a surface of a first portion of the irregularly shaped object covered by the at least one item; a display configured to superimpose a grid on a 3D optical pattern of the object while scanning the object using the optical module and the radar module, the grid comprising a plurality of cells arranged in rows and columns; as well as A processor configured to: providing indicia within each of the plurality of cells, the indicia being superimposed atop a display of data defining the 3D optical pattern, receiving the 3D point cloud data from the optical module and the range data from the radar module while the object is being scanned, analyzing the range data to identify at least one first marker among a plurality of markers provided in the plurality of units, the at least one first marker corresponding to a location on the irregularly shaped object for which radar depth data is received, causing the radar module to generate an optimized radar pattern based on changes in at least one first visual feature of the at least one first marker, and A map representing dimensions of a body of the irregularly shaped object is created using the 3D point cloud data and the radar data.

23. The handheld scanner system of claim 22, wherein the at least one first visual characteristic comprises color, shape, or size.

24. The handheld scanner system of claim 22, wherein distance information indicating the distance from the handheld scanner system to the object is also superimposed on the 3D optical pattern.

25. The handheld scanner system of claim 22, wherein the processor is further configured to cause at least one second visual feature of a marker of the plurality of markers to change to indicate a pointing direction of the handheld scanner system relative to the object being scanned.

26. The handheld scanner system of claim 25, wherein the at least one second visual characteristic is different from the at least one first visual characteristic.

27. The handheld scanner system of claim 25, wherein the processor is further configured to cause the at least one second visual feature of the marker to change to a previous setting when the pointing direction of the handheld scanner system changes.

28. The handheld scanner system of claim 22, wherein the processor is further configured to cause a new marker to be additionally superimposed on top of the 3D optical pattern when range data is successfully received during scanning of the object for a corresponding area of ​​the object.

29. The handheld scanner system of claim 22, wherein the irregularly shaped object is a person and the at least one item is a piece of clothing worn by the person.

30. The handheld scanner system of claim 22, wherein the grid provides feedback to a user of the handheld scanner system regarding an area of ​​the object that needs to be scanned using the radar module.

Images