A handheld 3D scanning sensing system, method and handheld 3D scanning device

Abstract

The present invention provides a handheld 3D scanning sensing system comprising: a pre-inspection unit, a scanning pattern matching unit, a scanning control unit, and a correction and reconstruction unit; the pre-inspection unit is used to verify user information and perform category identification on the object to be detected; the scanning pattern matching unit is used to match the corresponding scanning pattern; the scanning control unit is used to scan the user's foot shape according to the scanning pattern information, upload the scanned and corrected 3D data to a server, and receive a foot shape measurement report fed back by the server; the correction and reconstruction unit is used to perform data correction on the scanning data obtained during the scanning process and reconstruct an intuitive three-dimensional model surface. Accordingly, the present invention also discloses a handheld 3D scanning sensing method and a handheld 3D scanning device. The present invention reminds the user of the scanning posture by pre-inspecting the scanned object, and corrects and reconstructs the obtained three-dimensional point cloud data to obtain the final three-dimensional model, thereby improving data accuracy.

Description

Technical Field

[0001] The present invention relates to the field of footwear manufacturing technology, and in particular to a handheld 3D scanning sensing system, method, and handheld 3D scanning device. Background Art

[0002] 3D scanning technology is widely used in urban and architectural surveying, topographic mapping, industrial manufacturing, service industries, and the medical field. In particular, in the industrial manufacturing sector, it can help collect basic data in the early stages of design and perform dimensional scanning and inspection after the processing of large components.

[0003] In footwear manufacturing, 3D foot scanning can address the discrepancies between customer experiences when purchasing footwear in-store and online. Currently, data collection for custom footwear is mostly done through in-store scanning terminals, which capture the user's foot shape. However, these terminals are bulky and difficult to move. Furthermore, mobile devices used for collecting 3D foot shape data can sometimes be inaccurate due to differences in user operation.

[0004] In summary, providing a handheld 3D scanning sensor system, method and handheld 3D scanning device that can pre-inspect the scanned object, remind the user of the scanning posture, correct the data and improve the data accuracy is an issue that technicians in this field urgently need to solve. Summary of the Invention

[0005] In response to the above-mentioned problems and needs, this solution proposes a handheld 3D scanning sensing system, method and handheld 3D scanning device, which can solve the above-mentioned technical problems by adopting the following technical solutions.

[0006] To achieve the above-mentioned object, the present invention provides the following technical solutions: a handheld 3D scanning sensing system, comprising: a pre-detection unit, a scanning pattern matching unit, a scanning control unit, and a correction and reconstruction unit;

[0007] The pre-inspection unit is used to verify user information and perform category identification on the object to be detected;

[0008] The scanning pattern matching unit is electrically connected to the pre-detection unit, and is used to receive a scanning instruction from a user, match a corresponding scanning pattern, and send the corrected pattern information to the scanning control unit;

[0009] The scanning control unit is used to scan the user's foot shape according to the scanning mode information, upload the scanned and corrected 3D data to the server, and receive the foot shape measurement report fed back by the server;

[0010] The correction and reconstruction unit is used to perform data correction on the scan data acquired during the scanning process and reconstruct an intuitive three-dimensional model surface.

[0011] Furthermore, the pre-inspection unit includes an information verification module, an intelligent reminder module and an object detection module; the information verification module is electrically connected to the scanning pattern matching unit, and the information verification module includes a verification trigger module for detecting the user's verification trigger information and a verification module for verifying and identifying the user's identity information based on the trigger information. The verification module is connected to the verification trigger module, and the verification module receives a verification trigger signal from the verification trigger module. The user fingerprint image information and password information carried by the verification trigger signal are compared with the user information stored in the database of the verification module to complete the identity authentication. The verification module transmits the verification result to the scanning pattern matching unit to determine whether the user is a legal user; if the user is not a legal user, the scanning pattern matching unit controls the intelligent reminder module to prompt the illegal user; if the user is a legal user, the scanning pattern matching unit uploads the user fingerprint data and password information to the scanning control unit, and feeds back the selected mode signal to the object detection module.

[0012] Furthermore, the scanning mode matching unit includes a scanning mode selection module and a matching module;

[0013] The scanning mode selection module is used for the user to select the current scanning mode. The scanning mode selection module includes a command input module and a signal confirmation module. The command input module obtains the human body scanning command and the object scanning command input by the user. The signal confirmation module is electrically connected to the command input module. After the command signal input by the user is confirmed by the signal confirmation module, it is transmitted to the matching module.

[0014] The matching module is used to match the description information of the current mode according to the confirmation signal, and send the matching recognition signal containing the description information to the object detection module. The object detection module performs image processing and image recognition, and transmits the recognition feedback result to the matching module.

[0015] Furthermore, the object detection module includes a pattern information analysis module, an image acquisition module and an object recognition module;

[0016] The pattern information analysis module analyzes and determines the human body scanning pattern and the object scanning pattern according to the matching recognition signal, and sends the judgment result to the object recognition module;

[0017] The image acquisition module includes an image sensing module for acquiring object image information and an image processing module for processing the acquired object image and converting it into digital information. The image processing module is electrically connected to the image sensing module, and the image processing module receives the image signal sent by the image sensing module and sends the pre-processed object image information to the object recognition module.

[0018] The object recognition module uses a target recognition algorithm based on YOLOv3 to extract features of the target object, compares the extracted features with the object features stored in the database for detection and classification, and marks them as human or object, and determines whether the user selection mode matches. If so, the scanning control unit performs 3D scanning, otherwise it reminds the user to adjust the category of the object to be identified.

[0019] Furthermore, the scanning control unit includes a scanning data acquisition module, an information processing module, a voice interaction module and a display module;

[0020] When it is determined that the object to be scanned is a human body, the voice interaction module reminds the user to perform a three-dimensional scan of the foot, and the scanning data acquisition module obtains the distance information and foot depth image information of the foot to be scanned. The scanning data acquisition module compares the obtained distance information value with the preset distance information value. If the obtained distance information value is greater than the preset distance information value, an alarm message is sent to the display module for an alarm reminder, and the collected foot depth image information is sent to the information processing module;

[0021] The information processing module is used to reconstruct a three-dimensional model of the foot using the foot depth image information, and upload the complete three-dimensional data of the foot to the server via wireless transmission. The server calculates the scan data and feeds back a foot shape measurement report to the information processing module.

[0022] The display module is used for visually displaying the foot shape measurement report and the alarm information, and the display module is electrically connected to the information processing module.

[0023] Furthermore, the information processing module includes an image pre-processing module and a data conversion module;

[0024] The image preprocessing module is used to perform filtering and enhancement processing on the foot depth image information. The image preprocessing module first selects a frame of image from the acquired depth video sequence, and then selects the first m frames and the last m frames of image of the frame. Using the principle of temporal continuity, the module calculates the depth pixel value of the frame image, obtains the pixel points in the depth image corresponding to the pixel points that meet the color consistency constraint condition, uses the values ​​of these valid pixel points to estimate the values ​​of invalid pixel points, performs hole filling, and uses a median filter to perform filtering, repair a small number of invalid pixels, and smooth the image.

[0025] The data conversion module is used to obtain depth image feature points using the SURF algorithm, and to perform point cloud registration using the ICP algorithm to map the feature points into three-dimensional point cloud data points.

[0026] Furthermore, the scanning control unit also includes a light compensation module, which includes a light detection module for detecting ambient light, a control module, a light compensation lamp group and a constant current drive module. The light detection module sends the detected light information value to the control module, and the control module compares the light information value with the preset light level thresholds to determine the current ambient light level. The control signal is sent to the constant current drive module according to the level value, and the constant current drive module drives different numbers of light compensation lamp beads to perform light compensation.

[0027] Furthermore, the correction and reconstruction unit includes a data fusion module and a data correction output module; the data fusion module is used to calculate the TSDF value and weight of each point in the three-dimensional point cloud data and fuse them into a data cube; the data correction output module is used to scan the data cube, generate a frame of point cloud data to be used for alignment with the next frame, and after processing several frames of point cloud data, generate a visual three-dimensional model using the obtained data cube as the data source.

[0028] A handheld 3D scanning sensing method employs the above-mentioned handheld 3D scanning sensing system, and the specific steps are as follows:

[0029] Verify user information and identify the category of the object to be detected;

[0030] Receive a scanning instruction from the user, match the corresponding scanning mode, and send the corrected mode information to the scanning control unit;

[0031] Scan the user's foot shape according to the scanning mode information, upload the scanned and corrected 3D data to the server, and receive the foot shape measurement report fed back by the server;

[0032] The scan data acquired during the scanning process is corrected and an intuitive three-dimensional model surface is reconstructed.

[0033] A handheld 3D scanning device includes a data collector, a memory, and a processor. The data collector is used to collect verification data, image data, light data, and distance data. The data collector is electrically connected to the processor. The memory stores computer instructions that can be executed on the processor. When the processor executes the computer instructions, it executes the steps of the above-mentioned handheld 3D scanning sensing method.

[0034] It can be seen from the above technical solution that the beneficial effects of the present invention are: the present invention reminds the user of the scanning posture by pre-checking the scanned object, and corrects and reconstructs the obtained three-dimensional point cloud data to obtain the final three-dimensional model, thereby improving data accuracy.

[0035] In addition to the objects, features and advantages described above, the best embodiments for implementing the present invention will be described in more detail below with reference to the accompanying drawings so that the features and advantages of the present invention can be easily understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present invention or the description of the prior art. The drawings are only used to show some embodiments of the present invention, rather than to limit all embodiments of the present invention thereto.

[0037] Figure 1 This is a schematic diagram of the structure of a handheld 3D scanning sensor system in the present invention.

[0038] Figure 2 It is a schematic diagram of the composition structure of the pre-check unit in the present invention.

[0039] Figure 3 Schematic diagram of the composition structure of the scanning pattern matching unit in the present invention.

[0040] Figure 4 Schematic diagram of the structure of the scanning control unit in the present invention.

[0041] Figure 5 Schematic diagram of the specific steps of a handheld 3D scanning sensing method in the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the technical solution of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of specific embodiments of the present invention. The same figure marks in the drawings represent the same components. It should be noted that the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the current footwear customization process, many mobile 3D reconstruction software is used to reconstruct foot models based on captured foot images. However, due to the influence of user shooting techniques, data acquisition errors may occur. Therefore, the present invention provides a handheld 3D scanning sensor system for reducing data errors in the handheld scanning process. Figures 1 to 5As shown, the system includes: a pre-detection unit, a scanning pattern matching unit, a scanning control unit, and a correction and reconstruction unit.

[0044] Since many operators have different postures when performing 3D scanning using handheld devices, different object types lead to different reflected light, and distance and light issues cause the collected image data to be noisy and defective, a pre-inspection unit is designed to first classify and detect the objects to be scanned to distinguish whether the scanned objects are human body information.

[0045] Specifically, the pre-check unit is used to verify user information and identify the category of the object to be detected. The pre-check unit includes an information verification module, an intelligent reminder module, and an object detection module; the information verification module is electrically connected to the scanning pattern matching unit, and the information verification module includes a verification trigger module for detecting the user's verification trigger information and a verification module for verifying and identifying the user's identity information based on the trigger information. The verification module is connected to the verification trigger module, and the verification module receives a verification trigger signal from the verification trigger module. The user's fingerprint image information and password information carried in the verification trigger signal are compared with the user information stored in the database of the verification module to complete the identity authentication. The verification module transmits the verification result to the scanning pattern matching unit to determine whether the user is a legitimate user; if the user is not a legitimate user, the scanning pattern matching unit controls the intelligent reminder module to prompt the illegitimate user; if the user is a legitimate user, the scanning pattern matching unit uploads the user's fingerprint data and password information to the scanning control unit and feeds back the selected mode signal to the object detection module.

[0046] In this embodiment, a user verification module is provided to prevent unauthorized users from using the system, making it safer to use. The system can only be activated when verification is passed. The verification trigger module includes a capacitive switch, a fingerprint acquisition device, and a password button, which are used to collect verification information input by the user. The capacitive switch controls the on / off of the power supply circuit by outputting a changing electrical signal. When the system is powered on, the fingerprint acquisition device and the password button sequentially collect the verification information input by the user and transmit the processed information to the verification module. The intelligent reminder module includes a voice reminder module and a light reminder module. The voice reminder module uses a voice reminder device to remind the user according to the set voice message. The light reminder module uses three sets of blue LED running lights. When the user is determined to be an unauthorized user, the three sets of blue LED running lights are controlled to flash at a certain frequency.

[0047] More specifically, the object detection module includes a pattern information analysis module, an image acquisition module and an object recognition module; the pattern information analysis module analyzes and judges the human body scanning pattern and the object scanning pattern according to the matching recognition signal, and sends the judgment result to the object recognition module; the image acquisition module includes an image sensing module for collecting object image information and an image processing module for processing the collected object image and converting it into digital information. The image processing module is electrically connected to the image sensing module, the image processing module receives the image signal sent by the image sensing module, and sends the pre-processed object image information to the object recognition module; the object recognition module uses a target recognition algorithm based on YOLOv3 to extract features of the target object, compares the extracted features with the object features stored in the database for detection and classification, and marks it as a human body or an object, and judges whether the user-selected pattern matches. If so, the scanning control unit performs 3D scanning, otherwise the user is reminded to adjust the category of the object to be identified.

[0048] When performing target recognition, the original image is first scaled and adjusted using a scale pyramid structure. The scaled image is then divided into cells of equal size according to the scales of different feature maps using the DarkNet-53 feature extraction network. Feature fusion is then performed, and four values, including the coordinates of the upper left corner of the bounding box, the width, and the height of the target, are predicted for each bounding box in each cell. A clustering algorithm is used to obtain the size of the optimal anchor box. Finally, the probability of the object being contained in the anchor box is predicted, invalid prediction boxes are deleted, and binary cross entropy loss and logistic regression are used for category prediction to obtain the target recognition result.

[0049] In this embodiment, distance measurement and light detection are required in human scanning mode to remind the user to adjust the collection distance and automatically perform light compensation to make the collected data more accurate. In object scanning mode, only ordinary scanning methods are used to reduce operational difficulty and system power consumption.

[0050] The scanning pattern matching unit is electrically connected to the pre-inspection unit. The scanning pattern matching unit is used to receive the user's scanning instructions. The user's scanning instructions can be directly selected and input through buttons, matching the corresponding scanning mode, and sending the corrected mode information to the scanning control unit.

[0051] The scanning mode matching unit includes a scanning mode selection module and a matching module; the scanning mode selection module is used by the user to select the current scanning mode, and the scanning mode selection module includes a command input module and a signal confirmation module. The command input module obtains the human body scanning command and the object scanning command input by the user. The command input module can use a mode button, and the signal confirmation module detects and confirms the button information. The signal confirmation module is electrically connected to the command input module. The command signal input by the user is transmitted to the matching module after being confirmed by the signal confirmation module.

[0052] The matching module is used to match the description information of the current mode according to the confirmation signal, and send the matching recognition signal containing the description information to the object detection module. The object detection module performs image processing and image recognition, and transmits the recognition feedback result to the matching module. The description information includes a coding sequence or number.

[0053] The scanning control unit is configured to scan the user's foot shape based on the scanning mode information, upload the corrected 3D data to a server, and receive a foot shape measurement report from the server. In this embodiment, the server performs computational analysis on the acquired 3D data, uses standard values ​​for various foot measurements to determine the foot shape data, and generates a foot shape determination result. The cloud-based foot shape report is then converted into a QR code and sent to the user's mobile phone. The resulting foot shape data allows for the next step in the shoe type selection and customization process.

[0054] In this system, the scanning control unit includes a scanning data acquisition module, an information processing module, a voice interaction module and a display module; when it is determined that the object to be scanned is a human body, the voice interaction module reminds the user to perform a three-dimensional scan of the foot, the scanning data acquisition module obtains the distance information and foot depth image information of the foot to be scanned, the scanning data acquisition module compares the obtained distance information value with the preset distance information value, and if the obtained distance information value is greater than the preset distance information value, sends an alarm message to the display module for an alarm reminder, and sends the collected foot depth image information to the information processing module; the information processing module is used to use the foot depth image information to reconstruct a three-dimensional model of the foot, and upload the complete three-dimensional data of the foot to the server via wireless transmission. The server calculates the scanning data and feeds back a foot shape measurement report to the information processing module; the display module is used to visually display the foot shape measurement report and alarm information, and the display module is electrically connected to the information processing module.

[0055] In this embodiment, the scanning data acquisition module includes a 3D sensor, which transmits the acquired depth image information to the information processing module. The 3D sensor includes an infrared emitter and an infrared CMOS camera. The infrared CMOS camera receives light reflected by the object.

[0056] Specifically, the information processing module includes an image preprocessing module and a data conversion module; the image preprocessing module is used to filter and enhance the depth image information of the foot. The image preprocessing module first selects a frame image from the collected depth video sequence, and then selects the first m frames and the last m frames of the frame image. Using the principle of time continuity, the depth pixel value of the frame image is calculated to obtain the pixel points in the depth image corresponding to the pixel points that meet the color consistency constraint conditions. The values ​​of these valid pixel points are used to estimate the values ​​of invalid pixel points, perform hole filling, and use a median filter for filtering to repair a small part of invalid pixels and smooth the image; the data conversion module is used to obtain depth image feature points using the SURF algorithm, and use the ICP algorithm for point cloud registration to map the feature points into three-dimensional point cloud data points. The process of obtaining depth image feature points is as follows: constructing an integral image I(x,y), which refers to the sum of all pixel values ​​in the rectangular area formed from the upper left corner of the image to a certain pixel; using the Hessian matrix to construct a Gaussian pyramid scale space, and constructing the scale space by obtaining the Hessian determinant of each pixel point, and using a box filter and integral image operation instead of a second-order Gaussian filter to filter the image, thereby determining the image interest points and feature descriptions, and finally finding the corresponding points.

[0057] The scanning control unit also includes a light compensation module, which includes a light detection module for detecting ambient light, a control module, a light compensation lamp group and a constant current drive module. The light detection module sends the detected light information value to the control module. The control module compares the light information value with the preset light level thresholds, determines the current ambient light level, and sends a control signal to the constant current drive module based on the level value. The constant current drive module drives different numbers of light compensation lamp beads to perform light compensation.

[0058] The correction and reconstruction unit is used to perform data correction on the scanned data acquired during the scanning process and reconstruct a visual 3D model surface. The correction and reconstruction unit includes a data fusion module and a data correction output module; the data fusion module is used to calculate the TSDF value and weight of each point in the 3D point cloud data and fuse them into a data cube; the data correction output module is used to scan the data cube, generate a frame of point cloud data to be used for registration with the next frame, and after processing several frames of point cloud data, use the resulting data cube as the data source to generate a visual 3D model.

[0059] This application also discloses a handheld 3D scanning sensing method and a handheld 3D scanning device, which adopt the above-mentioned handheld 3D scanning sensing system. The specific steps are as follows:

[0060] S1: Verify user information and identify the category of the object to be detected;

[0061] S2: receiving a scanning instruction from the user, matching the corresponding scanning mode, and sending the corrected mode information to the scanning control unit;

[0062] S3: Scan the user's foot shape according to the scanning mode information, upload the scanned and corrected 3D data to the server, and receive the foot shape measurement report fed back by the server;

[0063] S4: Perform data correction on the scan data obtained during the scanning process and reconstruct an intuitive three-dimensional model surface.

[0064] The handheld 3D scanning device includes a data collector, a memory, and a processor. The data collector is used to collect verification data, image data, light data, and distance data. The data collector is electrically connected to the processor. The memory stores computer instructions that can be executed by the processor. When the processor executes the computer instructions, it performs the steps of the handheld 3D scanning sensing method described above. The present invention improves data accuracy by pre-checking the scanned object, reminding the user of the scanning posture, and correcting the data during model construction.

[0065] It should be noted that the embodiments described in the present invention are only preferred ways to implement the present invention, and any obvious modifications that belong to the overall concept of the present invention should fall within the scope of protection of the present invention.

Claims

1. A handheld 3D scanning sensor system, characterized in that: include: Pre-detection unit, scanning pattern matching unit, scanning control unit and correction and reconstruction unit; The pre-inspection unit is used to verify user information and perform category identification on the object to be detected; The scanning pattern matching unit is electrically connected to the pre-detection unit, and is used to receive a scanning instruction from a user, match a corresponding scanning pattern, and send the corrected pattern information to the scanning control unit; The scanning control unit is used to scan the user's foot shape according to the scanning mode information, upload the scanned and corrected 3D data to the server, and receive the foot shape measurement report fed back by the server; The correction and reconstruction unit is used to perform data correction on the scan data acquired during the scanning process and reconstruct an intuitive three-dimensional model surface; The pre-inspection unit includes an information verification module, an intelligent reminder module and an object detection module; the information verification module is electrically connected to the scanning pattern matching unit, and the information verification module includes a verification trigger module for detecting the user's verification trigger information and a verification module for verifying and identifying the user's identity information according to the trigger information. The verification module is connected to the verification trigger module, and the verification module receives a verification trigger signal from the verification trigger module. The user fingerprint image information and password information carried by the verification trigger signal are compared with the user information stored in the database of the verification module to complete the identity authentication. The verification module transmits the verification result to the scanning pattern matching unit to determine whether the user is a legitimate user; if the user is not a legitimate user, the scanning pattern matching unit controls the intelligent reminder module to prompt the illegal user; If the user is a legitimate user, the scanning pattern matching unit uploads the user's fingerprint data and password information to the scanning control unit and feeds back a selected pattern signal to the object detection module.

2. The handheld 3D scanning sensor system according to claim 1, wherein: The scanning mode matching unit includes a scanning mode selection module and a matching module; The scanning mode selection module is used for the user to select the current scanning mode. The scanning mode selection module includes a command input module and a signal confirmation module. The command input module obtains the human body scanning command and the object scanning command input by the user. The signal confirmation module is electrically connected to the command input module. After the command signal input by the user is confirmed by the signal confirmation module, it is transmitted to the matching module. The matching module is used to match the description information of the current mode according to the confirmation signal, and send the matching recognition signal containing the description information to the object detection module. The object detection module performs image processing and image recognition, and transmits the recognition feedback result to the matching module.

3. The handheld 3D scanning sensor system according to claim 2, wherein: The object detection module includes a pattern information analysis module, an image acquisition module and an object recognition module; The pattern information analysis module analyzes and determines the human body scanning pattern and the object scanning pattern according to the matching recognition signal, and sends the judgment result to the object recognition module; The image acquisition module includes an image sensing module for acquiring object image information and an image processing module for processing the acquired object image and converting it into digital information. The image processing module is electrically connected to the image sensing module, and the image processing module receives the image signal sent by the image sensing module and sends the pre-processed object image information to the object recognition module. The object recognition module uses a target recognition algorithm based on YOLOv3 to extract features of the target object, compares the extracted features with the object features stored in the database for detection and classification, and marks them as human or object, and determines whether the user selection mode matches. If so, the scanning control unit performs 3D scanning, otherwise it reminds the user to adjust the category of the object to be identified.

4. The handheld 3D scanning sensor system according to claim 3, wherein: The scanning control unit includes a scanning data acquisition module, an information processing module, a voice interaction module and a display module; When it is determined that the object to be scanned is a human body, the voice interaction module reminds the user to perform a three-dimensional scan of the foot, and the scanning data acquisition module obtains the distance information and foot depth image information of the foot to be scanned. The scanning data acquisition module compares the obtained distance information value with the preset distance information value. If the obtained distance information value is greater than the preset distance information value, an alarm message is sent to the display module for an alarm reminder, and the collected foot depth image information is sent to the information processing module; The information processing module is used to reconstruct a three-dimensional model of the foot using the foot depth image information, and upload the complete three-dimensional data of the foot to the server via wireless transmission. The server calculates the scan data and feeds back a foot shape measurement report to the information processing module. The display module is used for visually displaying the foot shape measurement report and the alarm information, and the display module is electrically connected to the information processing module.

5. The handheld 3D scanning sensor system according to claim 4, wherein: The information processing module includes an image preprocessing module and a data conversion module; The image preprocessing module is used to perform filtering and enhancement processing on the foot depth image information. The image preprocessing module first selects a frame of image from the acquired depth video sequence, and then selects the first m frames and the last m frames of image of the frame. Using the principle of temporal continuity, the module calculates the depth pixel value of the frame image, obtains the pixel points in the depth image corresponding to the pixel points that meet the color consistency constraint condition, uses the values ​​of these valid pixel points to estimate the values ​​of invalid pixel points, performs hole filling, and uses a median filter to perform filtering, repair a small number of invalid pixels, and smooth the image. The data conversion module is used to obtain depth image feature points using the SURF algorithm, and to perform point cloud registration using the ICP algorithm to map the feature points into three-dimensional point cloud data points.

6. The handheld 3D scanning sensor system according to claim 4, wherein: The scanning control unit also includes a light compensation module, which includes a light detection module for detecting ambient light, a control module, a light compensation lamp group and a constant current drive module. The light detection module sends the detected light information value to the control module. The control module compares the light information value with the preset light level thresholds, determines the current ambient light level, and sends a control signal to the constant current drive module based on the level value. The constant current drive module drives different numbers of light compensation lamp beads to perform light compensation.

7. The handheld 3D scanning sensor system according to claim 6, characterized in that The correction and reconstruction unit includes a data fusion module and a data correction output module; the data fusion module is used to calculate the TSDF value and weight of each point in the three-dimensional point cloud data and fuse them into a data cube; the data correction output module is used to scan the data cube, generate a frame of point cloud data to be used for registration with the next frame, and after processing several frames of point cloud data, use the obtained data cube as the data source to generate a visual three-dimensional model.

8. A handheld 3D scanning sensing method, using the handheld 3D scanning sensing system as described in claims 1-7, characterized in that , the specific steps are as follows: Verify user information and identify the category of the object to be detected; Receive a scanning instruction from the user, match the corresponding scanning mode, and send the corrected mode information to the scanning control unit; Scan the user's foot shape according to the scanning mode information, upload the scanned and corrected 3D data to the server, and receive the foot shape measurement report fed back by the server; The scan data acquired during the scanning process is corrected and an intuitive three-dimensional model surface is reconstructed.

9. A handheld 3D scanning device, comprising a data collector, a memory and a processor, characterized in that: The data collector is used to collect verification data, image data, light data and distance data. The data collector is electrically connected to the processor. The memory stores computer instructions that can be run on the processor. When the processor runs the computer instructions, it executes the steps of the handheld 3D scanning sensing method according to claim 8.

Images