PORTABLE DEVICE FOR ANTHROPOMETRIC DATA COLLECTION AND METHOD FOR COLLECTING ANTHROPOMETRIC DATA

Description

TECHNICAL FIELD

[0001] The present invention relates to an anthropometric data portable acquisition device.BACKGROUND ART

[0002] As known, collection of anthropometric measures may be of interest in many fields, ranging from commerce to healthcare. For example, e-commerce systems dealing in garment may assist customers by providing suggestion on size selection based on personal data set including relevant body measures. Personal data may be fed to an automatic categorization system and size may be selected based on generally best fit criteria. This kind of service is not only convenient to the customers, but can dramatically reduce the rate of returned articles and is therefore beneficial also for the providers in terms of cost saving. As another example, in dietetics monitoring of body measures may help patient follow up and tailoring of treatments based on response.

[0003] It is apparent that advantages of use of anthropometric data would be greatly increased if data could be collected on the field directly by customers or patients using simple tools that do not require special training or skill to provide sufficiently accurate results. Anthropometric data are generally gathered through image acquisition and analysis and data collection may take advantage from the widespread diffusion of extremely versatile devices such as smartphones. Image capture tools have become so friendly that end users are nearly always expected to be able to frame and acquire images of sufficient quality to perform desired analysis procedures.

[0004] Rather, delegating the task of image acquisition to end users involves other issues, that need to be taken into consideration. On the one side, image processing, which is in itself demanding in terms of requirement of resources, must also cope with the fact that acquisition on the field normally suffers from poor conditions in terms of lighting, contrast against background, user posture and the like. Such conditions must be compensated by refining processing techniques, that normally result in increased computational load. In order to provide sufficient processing capacity, only basic capturing functions are devoted to end user devices image analysis, whereas main analysis procedures are carried out at remote sites reached through internet connections. On the other side, sharing images with remote sites leads to concerns regarding privacy and protection of personal data, because sensitive information is involved and law requirements are becoming increasingly strict. As a matter of fact, information relating to personal aspect is sent out of the control of the end user. Clearly, treatment of information is critical especially for medical applications.

[0005] Eugenio Alessandro Canepa, "ISizeYou - human measure capture by 2 mobile pictures + size suggestion for e-commerce", (20171009), Youtube, URL: https: / / www.youtube.com / watch?v=s5S0GzTE0ns discloses an anthropometric data portable acquisition device comprising an image sensor and a processing unit. The processing unit comprises an acquisition module, configured to receive first images of a person from the image sensor; an image processor, configured to provide second images from respective first images by applying edge detection procedure; a communication module, configured to send out the second images.

[0006] Other examples of known anthropometric data portable acquisition devices are disclosed in US 2016 / 275702 A1 and in ANG YOUBAO ET AL, "Scene Text Detection via Edge Cue and Multi-features", 2014 14TH INTERNATIONAL CONFERENCE ON FRONTIERS IN HANDWRITING RECOGNITION, IEEE, doi:10.1109 / ICFHR.2016.0040, ISSN 2167-6445, (20161023), pages 156 - 161.DISCLOSURE OF INVENTION

[0007] It is thus an object of the present invention to provide an anthropometric data portable acquisition device which allows to overcome or at least mitigate the limitations described.

[0008] According to the present invention, there are provided an anthropometric data portable acquisition device as defined in claims 1.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present invention will now be described with reference to the accompanying drawings, which illustrate some non-limitative embodiments thereof, in which: figure 1 is a simplified block diagram of an anthropometric data acquisition system; Figure 2 is a more detailed block diagram of an anthropometric data portable acquisition devices included in the system of figure 2 and made in accordance with an embodiment of the present invention; figures 3a and 3b show acquisition masks used in the device of figure 2; figure 4 is a simplified flow chart of steps of a method of collecting anthropometric data; figures 5a and 5b show images produced in the device of figure 2; figure 6 is more detailed flow chart of part of the method steps of figure 4; figure 7 is a more detailed block diagram of a component of the device of figure 2; figure 8 is a more detailed block diagram of a component of the system of figure 1; and figures 9 and 10 show images produced in the component of figure 8. BEST MODE FOR CARRYING OUT THE INVENTION

[0010] In figure 1, an anthropometric data acquisition system is indicated as a whole by reference number 1 and comprises one or more anthropometric data portable acquisition devices 2 in accordance with an embodiment of the present invention and a server 3. The portable acquisition devices 2 and the server 3 are coupled in communication over a wide area network 4, e.g. the internet. The portable acquisition devices 2 are configured to capture images of individuals and to perform preliminary image processing steps, which are identified as frontend processing in what follows, whereby data sent to the server for completing extraction of relevant features (backend processing) are cleared of identity information.

[0011] One exemplary portable acquisition device 2 is shown in figure 2 and will be referred to hereinafter, being it understood that the other portable acquisition devices 2 include the same components. The portable acquisition device 2 comprises a processing unit 5, a display 6 and an image sensor 7. In one embodiment, the portable acquisition device 2 may be integrated in a smartphone, a tablet computer or a laptop computer, advantageously, with integrated touchscreen as the display 6 and photo camera as the image sensor 7. The display 6 need not be a touchscreen, though.

[0012] The processing unit 5 is configured to implement an acquisition module 8, an image processor 10 and a communication module 11.

[0013] The acquisition module 8 activates the display 6 in live view mode during acquisition and captures images from the image sensor 7 in response to shooting commands sent by a user through an interface, which may include virtual or hard buttons (not shown) . In live view mode, output signals of the image sensor 7 are continuously displayed on the display 6. The acquisition module 8 also sends information to the display 6 for visualization to the user, for the purpose of aiding correct acquisition.

[0014] Images captured by the acquisition module 8 are supplied to the image processor 10. The acquisition module 8 may be provided with some processing capability to perform basic actions such as white balance, brightness and contrast adjustment and conversion into compressed format. As an alternative, images in raw format are sent to the image processor 10, that carries out all desired processing steps.

[0015] The image processor 10 extracts a modified frontend images from received images. The extracted frontend images are sent to the server 3 through the communication module 11 for backend processing, that may therefore rely on higher level resources.

[0016] Image acquisition is assisted by the acquisition module 8, that provides information to lead the framed person to assume a posture in a range of admissible postures, while an operator takes a photo from the back and a photo on one side. Standard posture remarkably reduces computational load associated with image processing. More specifically (figure 3a), the acquisition module 8 starts acquisition of a back image by superimposing a back mask 13 to images shown on the display 6 in live view. The person can thus be guided to take a posture that fits into the back mask 13, which in one embodiment may require a standing position with legs apart and arms away from the torso. When the posture is correct, shooting commands are activated by the operator and a back image IMG B (figure 2) is captured by the acquisition module 8 and sent to the image processor 10. The back image IMG B may be temporarily stored in a memory unit incorporated either in the acquisition module 8 or in the image processor 10 or in a general purpose memory unit of the portable acquisition device 2.

[0017] Once the back image IMG B has been captured and stored or sent to the image processor 10, the acquisition module 8 starts acquisition of a side image and superimposes a side mask 14 (figure 3b) to the images shown on the display 6 in live view. Again, the person is guided to take a posture that fits into the side mask 13, which requires a standing position with arms lying along the torso. When the posture is correct, the shooting command is activated by the operator and a side image IMGs is captured and sent to the image processor 10 or stored like the back image IMG B .

[0018] Each of the back image IMG B and side image IMGs is then processed by the image processor 10 for edge detection and contour reconstruction substantially through the same step. For the sake of simplicity, reference will be made in what follows to the back image IMG B and it will be understood that the same applies also to the side image IMGs, unless otherwise specified.

[0019] With reference to figure 4, after the side image IMGs has been received in a format suitable for processing, the image processor 10 clears portions of the side image IMGs corresponding to locations outside the back mask 13 (block 100), thereby reducing noise. Then, edge detection (block 110) and contour reconstruction (block 120) are performed. A frontend back image FIMG B including only body contour and possibly residual background noise is thus obtained and transferred to the server 3. Examples of a frontend back image FIMG B and of a frontend side image FIMGs are shown in figures 5a and 5b, respectively. During the step of edge detection, body features which may possibly allow identification of the portrayed person are lost.

[0020] Edge detection may be carried out in several manners, but it is generally preferred that contour thickness is fairly constant, because it may be a critical parameter in subsequent backend processing. The image processor 10 may be configured to combine different edge detection procedures, in order to enhance performance.

[0021] In one embodiment (figure 6), the image processor 10 preliminary applies a noise reduction process to the back image IMG B to mitigate effects of high-frequency noise (block 112). The noise reduction process may be median filtering. It has been found that a size of the kernel of 15 pixel in the median filtering operator provides acceptable filtering performance without affecting precision of subsequent edge detection.

[0022] Then (block 114), contour of the back image IMG B is first extracted using a Canny edge detector process, which is described in Canny, J., "A Computational Approach To Edge Detection", IEEE Trans. Pattern Analysis and Machine Intelligence, 8(6):679-698, 1986. The Canny edge detector is very sensitive to image regions of low contrast which may be actually present. This aspect may be critical, because in image acquisition on the field it is not always possible to choose background providing adequate contrast. In order to avoid loss of information, contour of the back image IMG B is second extracted by the image processor 10 using another edge detector process (block 116) and the result of second contour extraction is combined with the result of the first contour extraction by the Canny edge detector in block 120 of figure 4. In one embodiment, the image processor 10 uses a structured random forest for edge detection, which is also referred to as structured edge detector. The structured edge detector is less sensitive to local regions of low contrast and is also more robust to noise than the Canny edge detector, thus curing weaknesses of the latter.

[0023] Figure 7 shows an exemplary structure of the image detector 10 implementing the process of figures 4 and 6. In the embodiment illustrated therein, the image detector 10 comprises a noise reduction filter 15, receiving the back image IMG B (and the side image IMGs) and feeding in parallel the Canny edge detector and the structured edge detector, here indicated by 16 and 17, respectively. A first output I 1 of the Canny edge detector 16 and a second output I 2 of the structured edge detector 17 are then combined in a contour reconstruction module 18 which combines the results of the first and second contour extraction and supplies the frontend back image FIMG B and the frontend side image FIMGs.

[0024] Probabilistic models may be exploited, when sufficient computational capacity is available in the portable acquisition devices 2 to have acceptable response delay for users.

[0025] A label l∈[L1, L2] IMGB< that minimizes an energy function E(l) is assigned to each location in the image IMG B (or IMGs). The energy function has a first component and a second component as follows: E l = E UNARY l + E PAIRWISE l

[0026] The first component E UNARY (l) is a function which tends to assign to each pixel a value corresponding to the output of the edge detectors with relative probabilities w 1 and w 2 , respectively E UNARY l = w 1 ∑ x g I 1 x , l + w 2 ∑ x g I 2 x , l where g y , l = y if l = 0 1 − y if l = 1 and w 1 and w 2 are weight parameters that determine to what extent the Canny edge detector 16 and the structured edge detector 17 will affect the result.

[0027] The second component E PAIRWISE (l) seeks to make neighboring pixels agree on a label, adding a constant penalty for disagreement, and is defined by: E PAIRWISE l = w P ∑ i ,j ∈ PAIRS I Ψ i , j , l where Ψ i , j , l = 0 if l i = l j 1 otherwise

[0028] and PAIRS(I) includes all pairs of neighboring image points.

[0029] The weight parameter w P determines how strong the agreement between neighboring pixels should be. The optimal solution l * = arg min l E l is approximated using the tree reweighted belief propagation algorithm.

[0030] In this manner, frontend back image FIMG B and frontend side image FIMGs are obtained from the back image IMG B and from the side image IMGs, respectively, and are sent to the server 3 for backend processing. Frontend back image FIMG B and frontend side image FIMGs include complete and reliable body contour of the framed person and no personal identity information is transferred out of user's control. Personal information (e.g. face features) is in fact deleted during the edge detection and contour reconstruction steps without the need of any special dedicated processing and body contour is substantially anonymous. As already mentioned, exemplary frontend back image FIMG B and frontend side image FIMGs are shown in figures 5a and 5b, respectively, where also the back mask 13 and the side mask 14 are depicted to intuitively suggest how image acquisition and the frontend processing are carried out. However the back mask 13 and the side mask 14 in themselves need not be part of the frontend back image FIMG B and frontend side image FIMGs.

[0031] As illustrated in figure 8, the server 3 comprises a communication module 20, which receives the frontend back image FIMG B and the frontend side image FIMGs, and components for backend processing, including a key point detector 21, a projection measure extractor 22 and a 3D measure predictor 23.

[0032] The key point detector 21 is configured to identify key points based on contour line thickness, since frontend images may contain noise, i.e. points that do not correspond to body contour. More precisely, a scan by horizontal lines (e.g. from left to right) is firstly carried out on the frontend back image FIMG B and pairs of subsequent lines of equal thickness are considered as left and right borders of a respective body portion. For example, if the horizontal scan reveals three pairs of subsequent lines of equal thickness, three body portions are identified, which may be left and right border of the left arm for the first pair, left and right border of the torso for the second pair and left and right border of the right arm for the third pair.

[0033] By the scan of the frontend back image FIMG B , the key point detector 21 identifies head tip, heels, armpits and crotch. Then head tip and heels are identified by a second scan of the frontend side image FIMGs.

[0034] The projection measure extractor 22 is configured to identify the contour points belonging to the torso, based on armpits and crotch positions as determined by key point detector 21 (windows 30, 31 in figures 5a and 5b). In order to reduce the unevenness which may be introduced by the previous processing steps, the projection measure extractor 22 approximates torso lines (32, figure 9) by polynomial functions (33, figure 10). Given the nature of the torso side, and in particular the expected number of maxima and minima, a fourth degree polynomial is sufficient and at the same time not too complex for the fitting. Also possible missing parts of the contour may be estimated by the projection measure extractor 22 by use of polynomial functions, such as Bezier polynomials.

[0035] Then, the projection measure extractor 22 takes measures of the projections of main body dimensions on the image plane or on plane perpendicular thereto. The main body dimensions may include, for example, neck, chest, waist, hips, inseam. Measures of chest, waist and hips may be taken at fixed and predetermined vertical coordinates (Y-coordinate), e.g. as follows: Yhips = Ycrotch + 4 33 CA Ywaist = Ycrotch + 16 33 CA Ychest = Ycrotch + 28 33 CA where CA is the distance between crotch and armpits.

[0036] In another embodiment, Ychest is taken at the widest part of upper third of the torso as derived from the frontend side image FIMGs; Ywaist is taken at the smallest part of the central third of the torso as derived from the frontend back image FIMG B ; and Yhips is taken at the widest part of the lower third of the torso as derived from the frontend side image FIMGs. Other measures may be added according to design preferences.

[0037] The 3D measure predictor 23 is configured to provide estimates or predictions of actual body measures starting from the measures o projection measure provided by the projection measure extractor 22. To this end, 3D measure predictor 23 may use either a simplified model such as a linear model or a probabilistic model, e.g. a Gaussian Process model. While linear models are more simple and yield widely satisfactory results in line with industry standards in terms of errors, probabilistic models allow to complement the prediction for each measurement with a measure of its uncertainty. Uncertainty can be used to predict estimation errors and trigger appropriate corrective actions to avoid providing the users with wrong output and predictions affected by very high variance.

[0038] Gaussian Processes can effectively deal with noise and this is important in order to allow collection on data form images taken on the filed by users without specific training.

[0039] Typically, Gaussian Processes may indicated as f x ∼ GP m x , k x , x ′ where m(x)=E[f(x)] is the expected value and k(x,x')=E[(f(x)-m(x))(f(x')-m(x'))] is the covariance function.

[0040] The function modelled by the Gaussian Process is the one predicting 3D body measurements starting from the length of their planar projections as provided by the projection measure extractor 22. Covariance function allows us to obtain a non-linear mapping and give flexibility in the specific form of non-linearity which is allowed. Due to their mathematical properties, covariance functions can be combined together to obtain new ones showing the desired characteristics. In the one embodiment, the sum of 3 different covariance functions: a constant covariance function used to model non-zero means for the different variables k c = σ 0 2 a linear covariance function k l x , x ′ = ∑ d = 1 D σ d 2 x d x d ′ where x and x' represent two generic points and index d indexes the D dimensions of the input space a squared exponential covariance function which allows to obtain non-linear smooth mappings k SE x ,x ′ = exp − x − x ′ 2 2 h 2 where x and x' represent two generic points and h is a characteristic length-scale working as a scaling factor.

[0041] On its output, the 3D measure predictor 23 provides a data set DS which includes estimates of the actual (3D) size of relevant parts of the body of the framed person.

[0042] It is finally apparent that changes and variations may be made to the device described and illustrated without departing from the scope of protection of the accompanying claims.

Claims

1. An anthropometric data portable acquisition device comprising an image sensor (7) and a processing unit (5), the processing unit (5) comprising: an acquisition module (8), configured to receive first images (IMGB, IMGs) of a person from the image sensor (7) ; an image processor (10), configured to provide second images (FIMGB, FIMGS) from respective first images (IMGB, IMGs) by applying edge detection procedure; a communication module (11), configured to send out the second images (FIMGB, FIMGS); characterized in that the image processor (10) comprises a first edge detector (16), configured to perform a first contour extraction, a second edge detector (17), configured to perform a second contour extraction, and a contour reconstruction module (18), configured to combine results of the first contour extraction performed by the first edge detector (16) and of the second contour extraction performed by the second edge detector (17) and to provide the second images (FIMGB, FIMGs); wherein the first edge detector (16) is a Canny edge detector and the second edge detector (17) is a structured edge detector; wherein the first images (IMGB, IMGs) are fed in parallel to the first edge detector (16) and to the second edge detector (17); and wherein the contour reconstruction module (18) is configured to assign a label (1) that minimizes an energy function E(l) to each location in the first images (IMGB, IMGs) and wherein the energy function has a first component EUNARY(l) and a second component EPAIRWISE(l) and is defined as follows: E 1 = E UNARY 1 + E PAIRWISE 1 E UNARY l = w 1 ∑ g I 1 x , l + w 2 ∑ g I 2 x , l where g y , l = y if l = 0 1 − y if l = 1 I1 is a first output of the first edge detector (16), I2 is a second output of the second edge detector (17) and w1 and w2 are weight parameters E PAIRWISE l = w P ∑ i ,j ∈ PAIRS I Ψ i , j , l where Ψ i , j , l = 0 if l i = l j 1 otherwise And wP is a weight parameter.

2. The device according to claim 1, comprising a display (6) operable in live view mode, whereby output signals of the image sensor (7) are continuously displayed on the display (6), the acquisition module (8) being further configured to superimpose visual information to live view images on the display (7) for guiding the person to assume a posture in an range of acceptable postures.

3. The device according to claim 2, wherein the visual information include a first mask (13) and a second mask (14) and the range of acceptable postures includes first postures of the person that fit into the first mask (13) and second postures of the person that fit into the second mask (14), the first mask (13) requiring the person to take a standing position with legs apart and arms away from the torso and the second mask (14) requiring the person to take a standing position with arms lying along the torso.

4. The device according to claim 3, wherein the image processor (10) is configured to clear portions of the first images (IMGB, IMGs) corresponding to locations outside one of the first mask (13) and the second mask (14) .

5. An anthropometric data acquisition system comprising a server (3) and at least one anthropometric data portable acquisition device (2) according to any one of the foregoing claims coupled in communication to the server (3).

6. The system according to claim 5, wherein the server (3) comprises a communication module (20), configured to receive the second images (FIMGB, FIMGs) from the anthropometric data portable acquisition device (2), a key point detector (21) configured to identify key points in the second images (FIMGB, FIMGs) and respective positions thereof based on contour line thickness, the key points including head tip, heels, armpits, crotch.

7. The system according to claim 6, wherein the server (3) comprises a projection measure extractor (22), configured to identify body contour points belonging to a torso, based on armpits and crotch positions as determined by key point detector (21).

8. The system according to claim 7, wherein the projection measure extractor (22) is configure to approximate torso lines by polynomial functions.

9. The system according to claim 7 or 8, wherein the projection measure extractor (22) is further configured to take measures of projections of selected main body dimensions on an image plane or on a plane perpendicular thereto.

10. The system according to claim 7 or 8, wherein the projection measure extractor (22) is further configured to take measures of chest, waist and hips at fixed and predetermined vertical coordinates or to take measures of chest at a widest part of an upper third of the torso, measures of waist at a smallest part of a central third of the torso and measures of hips at a widest part of the lower third of the torso.

11. The system according to any one of claims 6 to 10, wherein the server (3) comprises a 3D measure predictor (23) configured to provide estimates or predictions of actual body measures from measures provided by the projection measure extractor (22) using a linear model or a probabilistic model.