Method and apparatus for obtaining foot measurement data

Abstract

The application provides a method and device for obtaining foot measurement data, the method comprising: receiving a real-time foot image taken by a user, determining whether the shooting of the foot image meets preset shooting conditions through image analysis of the foot image, wherein the preset shooting conditions include at least one of a preset shooting angle, a shooting distance and shooting content; guiding the user to shoot by sending shooting prompt information, obtaining a foot target image meeting the preset shooting conditions from the real-time foot image, obtaining two-dimensional detection data and three-dimensional detection data of the foot based on the foot target image, and finally obtaining foot measurement data according to the two-dimensional detection data and the three-dimensional detection data of the foot. The above method constrains the user shooting, only needs to obtain a single foot image meeting the preset shooting conditions, improves the accuracy of foot measurement, simplifies the user operation and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and device for acquiring foot measurement data. Background Technology

[0002] Augmented reality (AR) measurement is a fundamental technology in the metaverse. It combines AR imaging technology with real-world applications to achieve functions such as angle measurement, length measurement, distance measurement, and area measurement. In e-commerce, for example, in furniture selection scenarios, AR furniture matching can be achieved through indoor measurements and virtual furniture placement; similarly, in clothing selection scenarios, AR can measure users' body data and recommend suitable sizes.

[0003] AR measurement of users' body data includes foot data. Existing AR foot measurement solutions usually require the use of a reference object of known size, such as an A4 sheet of paper, to take pictures of the feet from multiple perspectives, construct a three-dimensional model of the feet, and extract foot measurement data such as foot length, foot width, and foot circumference based on the three-dimensional model of the feet.

[0004] The above solution has several drawbacks. First, it requires shooting from multiple angles, which places high demands on the user's shooting skills. It also requires the use of reference objects of known size, resulting in a large amount of backend data calculation and low measurement efficiency. Second, because it uses multiple images, motion blur, flatness of the reference object, changes in lighting, and shooting angles can all introduce errors, making the measurement accuracy uncontrollable. Summary of the Invention

[0005] This application provides a method and device for acquiring foot measurement data, which improves the accuracy of foot measurement data by guiding users to take a single high-quality foot measurement image.

[0006] A first aspect of this application provides a method for acquiring foot measurement data, applied to a terminal device, the method comprising:

[0007] Receives real-time foot images captured by the camera of the terminal device;

[0008] Determine whether the shooting of the foot image meets the preset shooting conditions, which include at least one of the preset shooting angle, shooting distance, and shooting content;

[0009] The user is guided to take a picture by shooting prompts. After obtaining a target image of the foot that meets the preset shooting conditions, two-dimensional detection data and three-dimensional detection data of the foot are obtained based on the target image of the foot.

[0010] Foot measurement data is obtained based on the two-dimensional and three-dimensional detection data of the foot.

[0011] In an optional embodiment of the first aspect of this application, the preset shooting conditions include a preset shooting angle and / or shooting distance, and the foot image includes a color image and a depth image; determining whether the shooting of the foot image meets the preset shooting conditions includes:

[0012] Determine foot posture information based on the color map;

[0013] Determine ground location information based on the depth map;

[0014] Based on the foot pose information and the ground position information, it is determined whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance; wherein, the foot pose information includes the position information and posture information of the foot in the camera coordinate system, and the ground position information includes the position information of the ground in the camera coordinate system.

[0015] In an optional embodiment of the first aspect of this application, determining the foot pose information based on the color map includes:

[0016] The key point data of the foot in the color map is extracted using the foot key point detection model;

[0017] The key point data is projected onto the camera coordinate system to obtain the three-dimensional projection points of the feet.

[0018] The foot pose information is determined based on the three-dimensional projection points of the foot and the key points of the standard three-dimensional foot model.

[0019] In an optional embodiment of the first aspect of this application, determining the ground location information based on the depth map includes:

[0020] Based on the depth map and camera parameters, obtain the point cloud data of the depth map;

[0021] The ground location information is determined based on the point cloud data.

[0022] In an optional embodiment of the first aspect of this application, the method further includes:

[0023] Acquire gyroscope data from the terminal device, the gyroscope data being used to indicate the attitude information of the terminal device relative to the ground;

[0024] Determining the ground location information based on the point cloud data includes:

[0025] Based on the point cloud data and the gyroscope data, the ground location information is determined.

[0026] In an optional embodiment of the first aspect of this application, determining whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance based on the foot pose information and the ground position information includes:

[0027] Based on the foot pose information and the ground position information, the relative pose data of the camera, the ground, and the foot of the terminal device are determined; based on the relative pose data, it is determined whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance.

[0028] In an optional embodiment of the first aspect of this application, the preset shooting conditions include a preset shooting distance and / or shooting content; determining whether the shooting of the foot image meets the preset shooting conditions includes:

[0029] Obtain the depth map of the foot image;

[0030] Determine the confidence region percentage of the depth map, and based on the confidence region percentage and a preset percentage value, determine whether the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

[0031] In an optional embodiment of the first aspect of this application, determining whether the shooting of the foot image conforms to the preset shooting distance and / or shooting content based on the confidence region ratio and a preset ratio value includes:

[0032] If the percentage of the confidence region is greater than or equal to the preset percentage value, it is determined that the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

[0033] In an optional embodiment of the first aspect of this application, the step of guiding the user to take a picture through shooting prompt information to obtain a foot target image that meets the preset shooting conditions includes:

[0034] If it is determined that the shooting of the foot image does not meet the preset shooting conditions, the shooting prompt message is sent to obtain a target foot image that meets the preset shooting conditions; wherein, the shooting prompt message is used to instruct the user to adjust the shooting angle, and / or, adjust the shooting distance, and / or, shoot predetermined content; the predetermined content includes the ground and a complete foot.

[0035] In an optional embodiment of the first aspect of this application, the preset shooting angle includes a view directly facing the instep or a view directly facing the heel.

[0036] In an optional embodiment of the first aspect of this application, obtaining two-dimensional and three-dimensional detection data of the foot based on the foot target image includes:

[0037] The color map of the foot target image is detected by a first deep learning model to obtain two-dimensional detection data including foot contour, foot detection box, and foot measurement auxiliary lines; the foot measurement auxiliary lines include heel measurement line and foot axis line.

[0038] The depth map of the foot target image is upsampled using a second deep learning model to obtain a resolution-enhanced depth map, and the three-dimensional detection data of the foot is obtained based on the resolution-enhanced depth map.

[0039] In an optional embodiment of the first aspect of this application, obtaining foot measurement data based on the two-dimensional and three-dimensional detection data of the foot includes:

[0040] The foot contour, heel measurement line, and foot axis line in the two-dimensional detection data are projected onto the ground of the camera coordinate system, and the projection points of the foot contour, the projection lines of the heel measurement line, and the projection lines of the foot axis line are obtained respectively.

[0041] The foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line;

[0042] The foot width data is determined based on the projection points of the foot contour and the projection lines of the foot axis.

[0043] Foot circumference data is determined based on the three-dimensional detection data, the projection points of the foot contour, and the projection lines of the foot axis.

[0044] In an optional embodiment of the first aspect of this application, determining the foot length data based on the projection point of the foot contour and the projection line of the heel measurement line includes:

[0045] The initial foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line;

[0046] Based on the pre-constructed functional relationship between the shooting angle and the measurement error of the forefoot, the initial foot length data is compensated for to determine the final foot length data.

[0047] A second aspect of this application provides a foot measurement data acquisition device, comprising:

[0048] The receiving module is used to receive foot images captured in real time by the camera of the terminal device;

[0049] The processing module is used to determine whether the shooting of the foot image meets the preset shooting conditions, which include at least one of the preset shooting angle, shooting distance, and shooting content;

[0050] The acquisition module is used to guide the user to take a picture by means of shooting prompt information, and to acquire a target image of the foot that meets the preset shooting conditions;

[0051] The processing module is further configured to acquire two-dimensional and three-dimensional detection data of the foot based on the foot target image; and acquire foot measurement data based on the two-dimensional and three-dimensional detection data of the foot.

[0052] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program; the computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any of the first aspects of this application.

[0053] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method as described in any of the first aspects of this application.

[0054] A fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects of this application.

[0055] This application provides a method, apparatus, device, and storage medium for acquiring foot measurement data. The method includes: receiving a real-time foot image captured by a user; performing image analysis on the foot image to determine whether the capture of the foot image meets preset shooting conditions, wherein the preset shooting conditions include at least one of a preset shooting angle, shooting distance, and shooting content; guiding the user to take a picture by sending a shooting prompt message; acquiring a foot target image that meets the preset shooting conditions from the real-time foot image; acquiring two-dimensional and three-dimensional detection data of the foot based on the foot target image; and finally obtaining foot measurement data based on the two-dimensional and three-dimensional detection data of the foot. This method, by constraining the user's shooting, only requires acquiring a single foot image that meets the preset shooting conditions, improving the accuracy of foot measurement, simplifying user operation, and enhancing the user experience. Attached Figure Description

[0056] Figure 1A A schematic diagram of the shooting angle facing the instep provided for an embodiment of this application;

[0057] Figure 1B A real photograph of the instep provided for an embodiment of this application;

[0058] Figure 2A A schematic diagram of the shooting angle directly facing the heel provided for an embodiment of this application;

[0059] Figure 2B A real-life photograph of the heel provided for an embodiment of this application;

[0060] Figure 3 A flowchart illustrating the method for acquiring foot measurement data provided in this application embodiment;

[0061] Figure 4 A schematic diagram of foot measurement data and foot measurement auxiliary lines provided in the embodiments of this application;

[0062] Figure 5 A schematic flowchart of the method for acquiring foot measurement data provided in this application embodiment is shown in Figure 2.

[0063] Figure 6 A flowchart illustrating the method for acquiring foot measurement data provided in this application embodiment. Figure 3 ;

[0064] Figure 7 This is a schematic diagram of the process for acquiring foot detection data provided in an embodiment of this application;

[0065] Figure 8 A schematic diagram illustrating foot length compensation provided in an embodiment of this application;

[0066] Figure 9 A schematic diagram of the structure of the foot measurement data acquisition device provided in the embodiments of this application is shown below;

[0067] Figure 10 Schematic diagram 2 of the structure of the foot measurement data acquisition device provided in the embodiments of this application;

[0068] Figure 11 This is a hardware structure diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated or described herein.

[0071] It should be understood that the terms “comprising” and “having” as used herein, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or apparatus.

[0072] In the description of the embodiments of this application, the term "correspondence" may indicate that there is a direct or indirect correspondence between two things, or that there is an association between two things, or that there is a relationship of instruction and being instructed, configuration and being configured, etc.

[0073] The following is a brief explanation of the technical terms involved in the embodiments of this application.

[0074] Depth camera: A type of sensor that captures depth maps, storing depth information for each pixel in the image, and can reconstruct a spatial 3D point cloud using camera intrinsic parameters.

[0075] Point cloud data: A collection of vectors in a three-dimensional coordinate system. These vectors are usually represented by X, Y, and Z coordinates and are generally used to represent the shape of the outer surface of an object. Furthermore, in addition to the spatial location information represented by (X, Y, Z), point cloud data can also represent the RGB color, grayscale value, depth, segmentation results, etc. of a point.

[0076] RGB-D camera: can output color map and depth map simultaneously, where D stands for depth and RGB stands for red, green and blue, representing color.

[0077] Depth map, also known as distance map, is an image that uses the distance (depth) from the image acquisition device to each point in the scene as pixel values. It directly reflects the geometry of the visible surfaces of objects. Depth map can be converted into point cloud data through coordinate transformation, and point cloud data with regularity and necessary information can also be converted into depth map data.

[0078] AR foot measurement technology mainly consists of two steps: foot modeling and foot measurement. First, a 3D model of the foot is created. Then, the corresponding foot dimensions are obtained by calculating the geometric relationship between measurement lines and measurement points. The following sections introduce relevant solutions based on different modeling methods.

[0079] (1) Reconstruction based on camera RGB image and reference object

[0080] Multi-view reconstruction based on a known-sized reference object, such as an A4 sheet of paper, can reconstruct the camera pose using the four corner points of the paper. Based on the size of the A4 paper, the true physical scale can be restored. Then, the foot contour is extracted using a segmentation algorithm, and a deformable model (3DMM) is used to fit the foot contour line, reconstructing the implicit parameters and 3D model of the foot. Finally, the foot length, width, and circumference are measured based on the 3D model. Alternatively, a magazine can be used as a reference object. Multi-view shooting results can be used for 3D reconstruction to restore the 3D point cloud of the foot. The true physical scale can be restored using the size of the magazine, and measurements can be performed based on the 3D point cloud of the foot. Theoretically, an accuracy of 2mm can be achieved.

[0081] The above methods require multi-angle shooting, involve a large amount of computation, and have high requirements for shooting. They also require additional reference objects of known size, such as A4 paper or magazines, making them relatively difficult for consumers to use. In addition, because multiple images are taken, motion blur, the flatness of the reference object, changes in lighting, and shooting angles can all introduce errors, resulting in uncontrollable measurement accuracy and a certain level of operational difficulty.

[0082] (2) Reconstruction based on specialized equipment

[0083] Currently, the industry has specialized equipment for foot measurement, such as a camera array constructed using five RGB-D cameras to directly model the feet for measurement. This requires high-precision camera calibration to unify the coordinate systems of multiple cameras, followed by point cloud fusion of the multi-camera results to reconstruct the foot's point cloud before measurement. There are also technical solutions that directly use laser or structured light scanning.

[0084] The above method is simple to measure and has high accuracy. For ordinary consumer-grade RGB-D camera arrays, it can achieve an accuracy of 2mm, and for high-precision structured light scanning, it can achieve an accuracy of 0.1mm. However, the dedicated equipment is expensive and is usually only available in shopping malls, which raises the barrier to entry for users.

[0085] With the widespread adoption of smart terminals containing LiDAR (such as mobile phones) and the continued growth of online shopping, the inventors considered using depth maps collected by LiDAR on smart terminals to improve the accuracy and stability of foot modeling, thereby providing users with an accurate and efficient AR foot measurement tool. Based on this need, the inventors designed a method for acquiring foot measurement data. First, by constraining the foot measurement scene, such as preset shooting angles and distances, the method analyzes the foot measurement scene based on the user-collected foot image, guiding the user to take a foot image that meets the shooting conditions. This foot image includes RGB color and depth information. Then, a multi-path deep learning model is used to detect and analyze the foot image, obtaining two-dimensional and three-dimensional detection data of the foot. Finally, based on the two-dimensional and three-dimensional detection data of the foot, foot measurement data including foot length, foot width, and foot circumference are obtained.

[0086] By constraining the foot measurement scenario, the accuracy of actual foot measurements is improved, simplifying user operation compared to taking multiple images. From a metrological perspective, the entire measurement process can be completed in 1 second on the terminal, improving data processing speed. To enhance the accuracy of the foot depth map, deep learning-based intelligent upsampling combines a high-resolution foot color map with a low-resolution foot depth map, thereby improving the resolution of the foot depth map and optimizing the accuracy of measurement points. Compared to AR size measurements that directly use foot depth maps, this further improves measurement accuracy.

[0087] The preset shooting angles provided in the embodiments of this application will be described below with reference to the accompanying drawings. In the embodiments of this application, the preset shooting angles include a view directly facing the instep and a view directly facing the heel.

[0088] Figure 1A This is a schematic diagram illustrating the shooting angle directly facing the instep, provided for an embodiment of this application. (See diagram below.) Figure 1A As shown, the user stands with their feet on the ground and can use props such as yoga blocks, boxes, or walls to align their heels with the props, trying to straighten their foot posture as much as possible, that is, to make the length of the foot as perpendicular as possible to the line of contact between the prop and the ground. The user holds their phone to take a frontal view of their foot, with the phone camera roughly directly above the instep. Figure 1B The image shown is a real-life photo taken with the camera pointing directly at the instep.

[0089] Figure 2A This is a schematic diagram illustrating a shooting angle directly facing the heel, provided as an embodiment of this application. Figure 2A As shown, after the user places their foot on the ground and aligns their foot posture, they place the phone camera directly above their heel and take a picture of their entire foot facing the heel by slightly adjusting the angle of the phone camera. Figure 2B The image shown is a real shot with the camera pointed directly at the heel.

[0090] It's important to note that besides adjusting the phone camera's shooting angle, the distance between the camera and the foot should be appropriate—neither too close nor too far. Too far a distance will result in a small footprint in the foot image, while too close a distance will affect depth information acquisition, both impacting measurement accuracy. Additionally, the captured content should meet requirements, such as including the user's complete foot and the ground beneath it, and free of other obstructions; in other words, the scene should be simple and the camera unobstructed.

[0091] In this embodiment, the user's handheld device is not limited to a mobile phone, but also includes other mobile devices with GRB-D shooting function, such as tablet computers, wearable devices, etc. This embodiment does not impose any restrictions on this.

[0092] The method for acquiring foot measurement data provided in this application will be described in detail below through specific embodiments. It should be noted that the technical solutions provided in this application may include some or all of the following content. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0093] Figure 3 A flowchart illustrating the method for acquiring foot measurement data provided in this application embodiment is shown below. Figure 3 As shown, the method for acquiring foot measurement data in this embodiment includes the following steps:

[0094] Step 301: Receive the foot image captured in real time by the camera of the terminal device.

[0095] In this embodiment, the terminal device camera includes a regular camera and a depth camera, wherein the regular camera captures a color map and the depth camera captures a depth map. The foot image includes RGB color information and depth information, that is, the foot image includes a color map and a depth map.

[0096] Step 302: Determine whether the shooting of the foot image meets the preset shooting conditions. The preset shooting conditions include at least one of the preset shooting angle, shooting distance, and shooting content.

[0097] In this embodiment, the preset shooting angle includes a shooting angle directly facing the instep ( Figure 1A , Figure 1B (as shown) and the shooting angle directly facing the heel ( Figure 2A , Figure 2B (As shown). Shooting distance refers to the distance between the camera on the user's handheld device and the foot. The shooting distance should be appropriate so that the user's entire foot occupies a preset percentage in the captured image, such as 60%. This preset percentage can be reasonably set based on test results; this embodiment does not impose any restrictions. Shooting content refers to the target included in the captured image. In a foot measurement scenario, the target includes the user's entire foot and the ground beneath it. The captured content should avoid including other interfering objects as much as possible.

[0098] In one optional embodiment of this example, foot pose information and ground position information are obtained based on the color map and depth map of the foot image, and the foot pose information and ground position information are used to determine whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance.

[0099] In one optional embodiment of this example, the depth map of the foot image is used to determine whether the shooting of the foot image meets the preset shooting distance and / or shooting content. It should be understood that shooting distances that are too close or too far will affect the quality of the depth map. Generally, within an appropriate shooting distance range, the quality of the depth map is better, and the measurement effect is better. It should also be understood that complex shooting content (i.e., many interfering objects in the shot) affects the reliability of the depth information. Analyzing the depth map can determine whether the shooting content meets the requirements, such as a simple shot.

[0100] Step 303: Guide the user to take a picture by providing shooting prompts, and obtain a target image of the feet that meets the preset shooting conditions.

[0101] In an optional embodiment of this example, if it is determined that the shooting of the foot image does not meet the preset shooting conditions, the terminal device sends a shooting prompt message. The shooting prompt message is used to instruct the user to adjust the shooting angle, and / or, adjust the shooting distance, and / or, shoot predetermined content. The predetermined content includes the ground and a complete foot.

[0102] Optionally, the shooting prompts can be voice prompts or text prompts on the shooting interface; this embodiment of the application does not impose any restrictions on this.

[0103] By sending shooting prompts, users are guided to take foot images according to preset shooting conditions, providing high-quality foot images for subsequent image detection and analysis, thereby improving the accuracy of foot measurement results.

[0104] Step 304: Obtain two-dimensional and three-dimensional detection data of the foot based on the foot target image.

[0105] Users capture real-time images of feet using a camera on a handheld device. Image analysis of these real-time images determines whether the capture meets preset shooting angles, distances, and content. If one of the real-time images is found to meet these preset requirements, it is designated as the target foot image. The analysis of the target foot image involves two steps:

[0106] One step involves analyzing the color map of the foot target image to obtain two-dimensional detection data. The two-dimensional detection data of the foot includes the foot outline, the foot detection box, and the foot measurement auxiliary lines, which include the heel measurement line and the foot axis.

[0107] One step involves analyzing the depth map of the foot target image to obtain 3D detection data of the foot. It should be noted that the 3D detection data of the foot is the 3D point cloud data of the foot.

[0108] Step 305: Obtain foot measurement data based on the two-dimensional and three-dimensional detection data of the foot.

[0109] In one optional embodiment of this example, the foot contour, heel measurement line, and foot axis line from the two-dimensional detection data are projected onto the ground in the camera coordinate system to obtain the projection points of the foot contour, the projection lines of the heel measurement line, and the projection lines of the foot axis line, respectively. Foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line; foot width data is determined based on the projection points of the foot contour and the projection lines of the foot axis line; and foot circumference data is determined based on the three-dimensional detection data, the projection points of the foot contour, and the projection lines of the foot axis line. In this embodiment, foot measurement data is obtained by projecting two-dimensional detection data onto three-dimensional space (camera coordinate system) to determine the measurement points and / or measurement lines corresponding to the foot measurement data.

[0110] Figure 4 This is a schematic diagram of foot measurement data and foot measurement auxiliary lines provided in an embodiment of this application. Figure 4 As shown, for foot length, the point furthest from the heel measurement line in the foot contour can be selected as the measurement point, and the foot length can be determined based on this measurement point and the heel measurement line. For foot width, the two endpoints of the front half of the foot contour that are the longest distances perpendicular to the foot axis can be selected, and these two endpoints can be used as measurement points to determine the foot width. For foot circumference, a cross-section perpendicular to the foot axis at a distance of 1 / 3 of the foot length from the front of the foot can be selected in the foot contour, and the spatial curve corresponding to this cross-section and the foot point cloud data can be obtained. The foot circumference can be determined based on the spatial curve. It should be noted that the above foot contour is a foot contour projected into three-dimensional space.

[0111] The method for acquiring foot measurement data illustrated in this application receives a real-time foot image taken by a user, performs image analysis on the foot image to determine whether the shooting of the foot image meets preset shooting conditions, wherein the preset shooting conditions include at least one of preset shooting angle, shooting distance, and shooting content. The user is guided to take the picture through shooting prompts, and a target foot image that meets the preset shooting conditions is obtained from the real-time foot image. Two-dimensional and three-dimensional detection data of the foot are obtained based on the target foot image, and finally, foot measurement data is obtained based on the two-dimensional and three-dimensional detection data of the foot. This method, by constraining the user's shooting, only requires acquiring a single foot image that meets the preset shooting conditions, improving the accuracy of foot measurement, simplifying user operation, and enhancing the user experience.

[0112] The following describes in detail the detection and analysis process of real-time user-captured images using two specific examples. The image detection and analysis process mainly involves the following aspects: foot pose detection, ground detection, shooting angle detection, and depth map quality analysis.

[0113] Figure 5A schematic flowchart of the method for acquiring foot measurement data provided in this application embodiment is shown in Figure 2. Figure 5 As shown, the method for acquiring foot measurement data in this embodiment includes the following steps:

[0114] Step 501: Receive the foot image captured in real time by the camera of the terminal device.

[0115] Step 502: Determine the foot pose information based on the color map of the foot image. The foot pose information includes the position information of the foot in the camera coordinate system and the pose information.

[0116] In an optional embodiment of this example, foot posture information can be obtained through the following steps:

[0117] Step 5021: Extract key point data of the foot from the color map using the foot key point detection model.

[0118] In this embodiment, the terminal device has a pre-installed lightweight foot keypoint detection model, which can be used to detect the position of the foot in an image. Due to the limited memory and computing power of the terminal device, a lightweight model structure should be selected. After model pruning and quantization compression, the model is pre-installed on the terminal device while meeting the preset detection accuracy requirements. Optional lightweight model structures include, for example, MobileNet v3 and ShuffleNet.

[0119] The key point data for the foot includes the location data of all key points of the foot, such as the big toe, the medial joint of the forefoot, the medial arch of the foot, the medial side of the heel, the back of the heel, the lateral side of the heel, and the lateral joint of the forefoot, as well as their coordinates in the color map.

[0120] Step 5022: Project the key point data of the foot in the color map onto the camera coordinate system to obtain the three-dimensional projection points of the foot.

[0121] In this embodiment, based on the projection relationship between the camera coordinate system and the image coordinate system, the key point data (two-dimensional data) of the foot in the color image is projected onto the three-dimensional space of the camera coordinate system to obtain the three-dimensional projection points of the foot, that is, the position coordinates of all key points of the foot in the camera coordinate system. The above projection relationship is related to the camera's intrinsic parameters (the camera's focal length f), and the specifics can be found in existing technologies, which will not be elaborated here.

[0122] Step 5023: Determine the foot pose information based on the three-dimensional projection points of the foot and the key points of the standard three-dimensional foot model.

[0123] In this embodiment, the position information of all key points of the foot in the camera coordinate system is determined based on the three-dimensional projection points of the foot, and the foot posture information is determined based on the position information of the key points of the standard three-dimensional foot model in the camera coordinate system.

[0124] It should be noted that, in order to guide the user's foot measurement posture, it is necessary to pre-configure the user's standard foot posture and establish a standard 3D model of the user's foot during measurement. By matching the key points of the standard foot 3D model, it is determined whether the user's current foot posture meets the measurement posture requirements.

[0125] Specifically, based on the key points of the user's current foot 3D projection point and the standard foot 3D model, the approximate posture of the user's current foot in the camera coordinate system is obtained, and the orientation information of the foot is extracted, such as leftward deviation (not aligned), rightward deviation (not aligned), or facing directly (aligned), thereby determining whether the user needs to adjust the foot posture.

[0126] Step 503: Determine the ground position information based on the depth map of the foot image. The ground position information includes the position information of the ground in the camera coordinate system.

[0127] In one optional embodiment of this example, point cloud data of the depth map is obtained based on the depth map and camera parameters; ground location information is determined based on the point cloud data. Specifically, the point cloud data of the depth map is obtained based on the depth map and camera memory (camera focal length f), and specific details can be found in existing technologies, which will not be elaborated here.

[0128] In one optional embodiment of this example, point cloud data of the depth map is obtained based on the depth map and camera parameters; gyroscope data from the terminal device is obtained, which is used to indicate the attitude information of the terminal device relative to the ground; and ground position information is determined based on the point cloud data and the gyroscope data. In this embodiment, the gyroscope data of the terminal device serves as auxiliary ground measurement data. By using point cloud plane fitting, the ground area, i.e., the position information of the ground in the camera coordinate system, can be quickly obtained, thereby determining the positional relationship between the foot and the ground (whether the foot is on the ground).

[0129] Step 504: Based on the foot pose information and ground position information, determine whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance.

[0130] The foot pose information includes the foot's position and posture in the camera coordinate system, while the ground position information includes the ground's position in the camera coordinate system. Specifically, firstly, based on the foot pose information and ground position information, the relative pose data of the terminal device's camera, the ground, and the foot are determined; then, based on the relative pose data, it is determined whether the foot image capture conforms to the preset shooting angle and / or shooting distance.

[0131] Step 505: If it is determined that the shooting angle and / or shooting distance of the foot image do not meet the preset shooting angle and / or shooting distance, a shooting prompt message is sent. The shooting prompt message is used to instruct the user to adjust the shooting angle and / or, adjust the shooting distance.

[0132] For example, if the distance between some key points of the user's foot and the ground is greater than 0, it indicates that the user's foot is not firmly planted on the ground, and the user can be prompted to firmly plant their foot. If the user's foot is pointing out of alignment, but the camera's pose relative to the ground meets the requirements, it indicates that the user's foot posture does not meet the requirements, and the user can be prompted to adjust their foot posture. If the user's foot is pointing out of alignment, and the camera's pose relative to the ground does not meet the requirements, it indicates that one of the reasons for the user's foot posture not meeting the requirements is the camera's pose relative to the ground not meeting the requirements, and the user can be prompted to adjust the camera's shooting angle, and then further analysis can be conducted to determine whether the user needs to further adjust their foot posture. If the positional relationship between the user's foot and the camera is less than a preset distance, it may result in the user's foot occupying a larger proportion of the image than a preset proportion, and the user can be prompted to adjust the shooting distance or the camera's focal length.

[0133] The method for acquiring foot measurement data illustrated in this application involves receiving a foot image captured in real-time by a terminal device's camera. This foot image includes a color map and a depth map. The following image analyses are performed on the color map and depth map: first, foot posture detection is performed on the color map to obtain foot posture information; second, ground detection is performed on the depth map to obtain ground position information. Combining the foot posture information and ground position information, it is ultimately determined whether the foot image capture meets the preset shooting angle and / or shooting distance, thereby guiding the user to capture a foot image that meets the requirements. This provides high-quality images for subsequent foot measurements, improving the accuracy of foot measurements.

[0134] Figure 6 A flowchart illustrating the method for acquiring foot measurement data provided in this application embodiment. Figure 3 .like Figure 6 As shown, the method for acquiring foot measurement data in this embodiment includes the following steps:

[0135] Step 601: Receive the foot image captured in real time by the camera of the terminal device.

[0136] Step 602: Obtain the depth map of the foot image and determine the confidence region percentage of the depth map.

[0137] In this embodiment, the confidence level of the depth map quality of the foot image is analyzed to determine the confidence level (credibility) of each pixel in the depth map. For example, the confidence level is divided into 0, 1, and 2, where 0 represents unreliable, 1 represents low confidence, and 2 represents high confidence. Based on the confidence level of each pixel in the depth map, the confidence region of the depth map is determined. For example, pixels with low and high confidence are both considered as pixels in the confidence region, thereby determining the proportion of the confidence region of the depth map, that is, the ratio of the confidence region of the depth map to the entire region.

[0138] Step 603: Based on the confidence region ratio and the preset ratio value, determine whether the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

[0139] In one possible scenario, if the confidence region percentage is greater than or equal to a preset percentage value, it is determined that the foot image was captured at the preset shooting distance and / or with the preset shooting content. In another possible scenario, if the confidence region percentage is less than a preset percentage value, it is determined that the foot image was captured at the preset shooting distance and / or with the preset shooting content.

[0140] Step 604: If it is determined that the shooting distance and / or shooting content of the foot image do not meet the preset shooting distance and / or shooting content, a shooting prompt message is sent. The shooting prompt message is used to instruct the user to adjust the shooting distance and / or shoot the predetermined content.

[0141] In this embodiment, the predetermined content includes the ground and a complete foot.

[0142] It should be understood that when the distance between the camera and the foot is less than a preset distance, the depth information captured by the camera in the foot image is insufficient, and the reliability of the depth map is low, and vice versa. Similarly, when there are obstructions between the camera and the foot, it will also lead to insufficient depth information captured by the camera, and the reliability of the depth map will be low, and vice versa. Therefore, reasons for low confidence in the depth map captured by the camera include shooting at too close a distance or the camera lens being obstructed. Thus, if it is determined that the confidence region of the depth map is less than a preset percentage, the user can be prompted to adjust the shooting distance or the content being photographed (avoiding obstructions).

[0143] The method for acquiring foot measurement data shown in this application embodiment receives foot images captured in real time by a terminal device camera, analyzes the confidence level of the depth map quality of the foot images, determines whether the user's shooting meets the preset shooting distance and / or shooting content, thereby guiding the user to shoot foot images that meet the requirements, providing high-quality images for subsequent foot measurements, and improving the accuracy of foot measurements.

[0144] Based on the above embodiments, it can be seen that when the foot image captured by the camera meets the requirements of shooting angle, shooting content (including a complete foot and the ground), shooting distance, and depth map quality, a foot target image is obtained. By performing further image analysis on the foot target image, the two-dimensional detection data and three-dimensional detection data of the foot can be determined, thereby determining the foot measurement data.

[0145] The following is in conjunction with the appendix Figure 7 The process of acquiring two-dimensional and three-dimensional detection data for the feet is explained.

[0146] Figure 7 This is a schematic diagram illustrating the process of acquiring foot detection data provided in an embodiment of this application. Figure 7 As shown, the method for acquiring foot detection data in this embodiment includes the following steps:

[0147] Step 701: Detect the color map of the foot target image using the first deep learning model to obtain two-dimensional detection data including foot contour, foot detection box, and foot measurement auxiliary lines.

[0148] In this embodiment, the foot measurement auxiliary lines include a heel measurement line and a foot axis line. The two-dimensional detection data of the foot contour includes the position coordinates of all detection points of the foot contour on the color map. The two-dimensional detection data of the foot detection frame includes the position coordinates of at least one endpoint of the foot detection frame (rectangle) on the color map, as well as the length and width of the detection frame. The two-dimensional detection data of the foot measurement auxiliary lines includes the position coordinates of at least two points of the heel measurement line on the color map, and the position coordinates of at least two points of the foot axis line on the color map.

[0149] Specifically, the color map of the foot target image is input into the first deep learning model. The first deep learning model includes three detection modules, which are used to detect the foot contour (foot segmentation), foot detection boxes, and foot measurement auxiliary lines, respectively. Optionally, the first deep learning model can be trained using a convolutional neural network (CNN).

[0150] For a foot image taken directly at the instep, the first deep learning model detects the intersection of an auxiliary object with the ground. The auxiliary object can be a straight line pattern on the ground, paper, a book, a box, etc. For a foot image taken directly at the heel, the first deep learning model detects a straight line perpendicular to the foot axis and tangent to the heel contour.

[0151] In this embodiment, the first deep learning model integrates segmentation and bounding box detection results to obtain accurate foot rectangle boundaries and foot contours. At the same time, the first deep learning model also verifies the perpendicularity of the heel measurement line to the foot axis to ensure that the heel measurement line is perpendicular to the foot axis, thereby improving the detection accuracy of the foot target image.

[0152] Step 702: Upsample the depth map of the foot target image using the second deep learning model to obtain a depth map with improved resolution.

[0153] Specifically, the color map of the foot target image is input into the second deep learning model. Based on the high-resolution foot color map and the low-resolution foot depth map, the second deep learning model predicts the high-resolution foot depth map, thereby improving the depth map accuracy of the two-foot scene. Finally, the high-resolution depth map is enlarged to the same size as the color map for subsequent foot measurements. Optionally, the second deep learning model can be trained using a convolutional neural network.

[0154] Step 703: Obtain 3D detection data of the feet based on the depth map with improved resolution.

[0155] Specifically, based on the enhanced depth map and camera parameters, the 3D point cloud data of the feet is obtained, that is, the position coordinates of the feet in the camera coordinate system.

[0156] The foot detection data acquisition method illustrated in this application uses a first deep learning model to detect the foot contour, foot detection box, and foot measurement auxiliary lines to acquire two-dimensional detection data. Simultaneously, a second deep learning model is used to improve the quality of the foot depth map, acquiring three-dimensional detection data. The two-dimensional and three-dimensional detection data acquired in this embodiment provide high-quality data for foot measurement, improving the accuracy of foot measurement data.

[0157] It should be noted that the measured foot length data may contain some errors, especially since the target image of the foot is taken from a perspective directly facing the heel. Length compensation is required; please refer to [link / reference needed] for details. Figure 8 .

[0158] Figure 8 This is a schematic diagram illustrating foot length compensation provided in an embodiment of this application. Figure 8 As shown, when the camera projection line is not perfectly aligned with the toes, it may not be tangent to the front of the foot, requiring length compensation.

[0159] Specifically, for foot target images taken directly at the instep or heel, after obtaining the projection points of the foot contour onto the camera coordinate system ground and the projection lines of the heel measurement line onto the camera coordinate system ground based on the above embodiments, the foot length data is determined according to the projection points of the foot contour and the projection lines of the heel measurement line, including:

[0160] The initial foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line. The final foot length data is then determined by length compensation based on a pre-constructed functional relationship between the shooting angle and the measurement error of the forefoot. The pre-constructed functional relationship between the shooting angle and the measurement error of the forefoot is obtained by modeling the relationship between the forefoot error and the viewing angle using the aforementioned standard 3D foot model and fitting it through a learning method.

[0161] Since users have been guided to shoot directly at the instep or heel, the length deviation in the width of the foot is small, so no length compensation is needed.

[0162] This embodiment illustrates a length compensation scheme in the foot length direction. By fitting the error function through model learning or by directly modeling the approximate relationship between the forefoot perspective and the error using mathematical formulas, foot length error compensation can be performed, which can further improve the accuracy of foot measurement data.

[0163] It should be noted that the execution subject of the above embodiments can be a terminal device (client) or a server (server), and this application does not impose any restrictions on this embodiment.

[0164] The foregoing described the method for acquiring foot measurement data provided in the embodiments of this application. The following describes the apparatus for acquiring foot measurement data provided in the embodiments of this application. The embodiments of this application can divide the apparatus for acquiring foot measurement data into functional modules based on the above method embodiments. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in the embodiments of this application is illustrative and only represents one logical functional division; other division methods may exist in actual implementation. The following description uses the division of functional modules according to each function as an example.

[0165] Figure 9 A schematic diagram of the structure of the foot measurement data acquisition device provided in the embodiments of this application is shown below. Figure 9 As shown, the foot measurement data acquisition device 900 of this embodiment includes a receiving module 901, a processing module 902, and an acquisition module 903.

[0166] The receiving module 901 is used to receive the foot image captured in real time by the camera of the terminal device.

[0167] The processing module 902 is used to determine whether the shooting of the foot image meets the preset shooting conditions, the preset shooting conditions including at least one of the preset shooting angle, shooting distance and shooting content;

[0168] The acquisition module 903 is used to guide the user to take a picture through shooting prompt information and acquire a foot target image that meets the preset shooting conditions;

[0169] The processing module 902 is further configured to acquire two-dimensional and three-dimensional detection data of the foot based on the foot target image; and acquire foot measurement data based on the two-dimensional and three-dimensional detection data of the foot.

[0170] In an optional embodiment of this example, the preset shooting conditions include a preset shooting angle and / or shooting distance, and the foot image includes a color image and a depth image; the processing module 902 is used for:

[0171] Determine foot posture information based on the color map;

[0172] Determine ground location information based on the depth map;

[0173] Based on the foot pose information and the ground position information, it is determined whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance; wherein, the foot pose information includes the position information and posture information of the foot in the camera coordinate system, and the ground position information includes the position information of the ground in the camera coordinate system.

[0174] In an optional embodiment of this example, the processing module 902 is configured to:

[0175] The key point data of the foot in the color map is extracted using the foot key point detection model;

[0176] The key point data is projected onto the camera coordinate system to obtain the three-dimensional projection points of the feet.

[0177] The foot pose information is determined based on the three-dimensional projection points of the foot and the key points of the standard three-dimensional foot model.

[0178] In an optional embodiment of this example, the processing module 902 is configured to:

[0179] Based on the depth map and camera parameters, obtain the point cloud data of the depth map;

[0180] The ground location information is determined based on the point cloud data.

[0181] Figure 10 A second schematic diagram of the structure of the foot measurement data acquisition device provided in this application embodiment. Figure 9 Based on the device shown, such as Figure 10 As shown, the foot measurement data acquisition device 900 of this embodiment also includes a transmission module 904.

[0182] The acquisition module 903 is used to acquire gyroscope data in the terminal device, and the gyroscope data is used to indicate the attitude information of the terminal device relative to the ground;

[0183] The processing module 902 is used to determine the ground location information based on the point cloud data and the gyroscope data.

[0184] In an optional embodiment of this example, the processing module 902 is configured to:

[0185] Based on the foot pose information and the ground position information, the relative pose data of the camera, the ground, and the foot of the terminal device are determined; based on the relative pose data, it is determined whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance.

[0186] In an optional embodiment of this example, the preset shooting conditions include a preset shooting distance and / or shooting content; the acquisition module 903 is used to acquire the depth map of the foot image;

[0187] The processing module 902 is used to determine the confidence region ratio of the depth map, and based on the confidence region ratio and a preset ratio value, determine whether the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

[0188] In an optional embodiment of this example, the processing module 902 is configured to:

[0189] If the percentage of the confidence region is greater than or equal to the preset percentage value, it is determined that the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

[0190] In an optional embodiment of this example, if the processing module 902 determines that the shooting of the foot image does not meet the preset shooting conditions, the sending module 904 is used to send shooting prompt information; the acquisition module 903 is used to acquire a foot target image that meets the preset shooting conditions.

[0191] The shooting prompt information is used to instruct the user to adjust the shooting angle, and / or, adjust the shooting distance, and / or, shoot predetermined content; the predetermined content includes the ground and a complete foot.

[0192] In one optional embodiment of this example, the preset shooting angle includes a view directly facing the instep or a view directly facing the heel.

[0193] In an optional embodiment of this example, the acquisition module 903 is used for:

[0194] The color map of the foot target image is detected by a first deep learning model to obtain two-dimensional detection data including foot contour, foot detection box, and foot measurement auxiliary lines; the foot measurement auxiliary lines include heel measurement line and foot axis line.

[0195] The depth map of the foot target image is upsampled using a second deep learning model to obtain a resolution-enhanced depth map, and the three-dimensional detection data of the foot is obtained based on the resolution-enhanced depth map.

[0196] In an optional embodiment of this example, the processing module 902 is configured to:

[0197] The foot contour, heel measurement line, and foot axis line in the two-dimensional detection data are projected onto the ground of the camera coordinate system, and the projection points of the foot contour, the projection lines of the heel measurement line, and the projection lines of the foot axis line are obtained respectively.

[0198] The foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line;

[0199] The foot width data is determined based on the projection points of the foot contour and the projection lines of the foot axis.

[0200] Foot circumference data is determined based on the three-dimensional detection data, the projection points of the foot contour, and the projection lines of the foot axis.

[0201] In an optional embodiment of this example, the processing module 902 is configured to:

[0202] The initial foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line;

[0203] Based on the pre-constructed functional relationship between the shooting angle and the measurement error of the forefoot, the initial foot length data is compensated for to determine the final foot length data.

[0204] The foot measurement data acquisition device provided in this embodiment can execute the technical solution of any of the aforementioned method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0205] Figure 11 This is a hardware structure diagram of the electronic device provided in an embodiment of this application. For example... Figure 11 As shown, the electronic device 1100 provided in this embodiment includes: a memory 1101, a processor 1102, and a computer program; wherein, the computer program is stored in the memory 1101 and is configured to be executed by the processor 1102 to implement the technical solution of any of the aforementioned method embodiments, and its implementation principle and technical effect are similar, and will not be described again here.

[0206] Optionally, the memory 1101 can be either standalone or integrated with the processor 1102. When the memory 1101 is a device independent of the processor 1102, the electronic device 1100 also includes a bus 1103 for connecting the memory 1101 and the processor 1102.

[0207] This application provides a computer-readable storage medium storing a computer program thereon, which is executed by a processor 1102 to implement the technical solutions of any of the foregoing method embodiments.

[0208] This application provides a computer program product, including a computer program, which, when executed by a processor, implements the technical solutions of any of the foregoing method embodiments.

[0209] This application provides a chip, including a processing module and a communication interface, wherein the processing module is capable of executing the technical solutions of any of the foregoing method embodiments.

[0210] Optionally, the chip further includes a storage module (e.g., a memory), which is used to store instructions, and the processing module is used to execute the instructions stored in the storage module. The execution of the instructions stored in the storage module causes the processing module to perform the technical solution of any of the foregoing method embodiments.

[0211] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0212] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0213] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0214] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0215] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for acquiring foot measurement data, characterized in that, Applied to a terminal device, the method includes: Receives real-time foot images captured by the camera of the terminal device; Determine whether the shooting of the foot image meets the preset shooting conditions, which include at least one of the preset shooting angle, shooting distance, and shooting content; The system guides the user to take a picture by providing shooting prompts, thereby obtaining a target image of the foot that meets the preset shooting conditions. Based on the foot target image, obtain two-dimensional and three-dimensional detection data of the foot; Based on the two-dimensional and three-dimensional detection data of the foot, foot measurement data is obtained; The process of obtaining two-dimensional and three-dimensional detection data of the foot based on the foot target image includes: The color map of the foot target image is detected by a first deep learning model to obtain two-dimensional detection data including foot contour, foot detection box, and foot measurement auxiliary lines; the foot measurement auxiliary lines include heel measurement lines and foot axis lines; wherein, the first deep learning model includes three detection modules, which are respectively used to detect the foot contour, foot detection box, and foot measurement auxiliary lines; The depth map of the foot target image is upsampled by the second deep learning model to obtain a resolution-enhanced depth map, and the three-dimensional detection data of the foot is obtained based on the resolution-enhanced depth map. The step of obtaining foot measurement data based on the two-dimensional and three-dimensional detection data of the foot includes: The foot contour, heel measurement line, and foot axis line from the two-dimensional detection data are projected onto the ground in the camera coordinate system. The projection points of the foot contour, the projection lines of the heel measurement line, and the projection lines of the foot axis line are obtained respectively. The foot length data is determined based on the projection points of the foot contour and the projection lines of the heel measurement line. The foot width data is determined based on the projection points of the foot contour and the projection lines of the foot axis line. The foot circumference data is determined based on the three-dimensional detection data, the projection points of the foot contour, and the projection lines of the foot axis line. The step of determining foot length data based on the projection points of the foot contour and the projection lines of the heel measurement line includes: determining initial foot length data based on the projection points of the foot contour and the projection lines of the heel measurement line; and performing length compensation on the initial foot length data based on a pre-constructed functional relationship between the shooting angle and the measurement error of the forefoot to determine the final foot length data. The pre-constructed functional relationship between the shooting angle and the measurement error of the forefoot is obtained by modeling the relationship between the forefoot error and the viewing angle using a standard three-dimensional foot model and fitting it through a learning method.

2. The method according to claim 1, characterized in that, The preset shooting conditions include preset shooting angles and / or shooting distances, and the foot image includes a color map and a depth map; determining whether the shooting of the foot image meets the preset shooting conditions includes: Determine foot posture information based on the color map; Determine ground location information based on the depth map; Based on the foot pose information and the ground position information, it is determined whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance; wherein, the foot pose information includes the position information and posture information of the foot in the camera coordinate system, and the ground position information includes the position information of the ground in the camera coordinate system.

3. The method according to claim 2, characterized in that, Determining foot pose information based on the color map includes: The key point data of the foot in the color map is extracted using the foot key point detection model; The key point data is projected onto the camera coordinate system to obtain the three-dimensional projection points of the feet. The foot pose information is determined based on the three-dimensional projection points of the foot and the key points of the standard three-dimensional foot model.

4. The method according to claim 2, characterized in that, Determining the ground location information based on the depth map includes: Based on the depth map and camera parameters, obtain the point cloud data of the depth map; The ground location information is determined based on the point cloud data.

5. The method according to claim 4, characterized in that, The method further includes: Acquire gyroscope data from the terminal device, the gyroscope data being used to indicate the attitude information of the terminal device relative to the ground; Determining the ground location information based on the point cloud data includes: Based on the point cloud data and the gyroscope data, the ground location information is determined.

6. The method according to any one of claims 2 to 5, characterized in that, The step of determining whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance based on the foot pose information and the ground position information includes: Based on the foot pose information and the ground position information, the relative pose data of the camera, the ground, and the foot of the terminal device are determined; based on the relative pose data, it is determined whether the shooting of the foot image conforms to the preset shooting angle and / or shooting distance.

7. The method according to claim 1, characterized in that, The preset shooting conditions include preset shooting distance and / or shooting content; Determining whether the shooting of the foot image meets the preset shooting conditions includes: Obtain the depth map of the foot image; Determine the confidence region percentage of the depth map, and based on the confidence region percentage and a preset percentage value, determine whether the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

8. The method according to claim 7, characterized in that, The step of determining whether the foot image capture conforms to the preset shooting distance and / or shooting content based on the confidence region ratio and the preset ratio value includes: If the percentage of the confidence region is greater than or equal to the preset percentage value, it is determined that the shooting of the foot image conforms to the preset shooting distance and / or shooting content.

9. The method according to claim 1, characterized in that, The step of guiding the user to take a picture through shooting prompts to obtain a target image of the feet that meets the preset shooting conditions includes: If it is determined that the shooting of the foot image does not meet the preset shooting conditions, the shooting prompt message is sent to obtain a target foot image that meets the preset shooting conditions; wherein, the shooting prompt message is used to instruct the user to adjust the shooting angle, and / or, adjust the shooting distance, and / or, shoot predetermined content; the predetermined content includes the ground and a complete foot.

10. The method according to claim 1, characterized in that, The preset shooting angles include a view directly facing the instep or a view directly facing the heel.

11. An electronic device, characterized in that, include: A memory, a processor, and a computer program; the computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1 to 10.

Images