Scanning data processing control, 3D scanning system, equipment and media

By recognizing hand gestures and mapping them to data control commands through image acquisition devices, the problem of two-handed operation in 3D scanning systems has been solved, enabling real-time interactive control without the need for external terminals and improving scanning efficiency and continuity.

CN123067701APending Publication Date: 2026-11-10SCANTECH (HANGZHOU) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610995334.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-03
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In existing 3D scanning systems, users need to use both hands to operate the scanning equipment and auxiliary terminal, which leads to hand fatigue, low scanning efficiency and discontinuous operation, especially in large objects or complex scenes.

Method used

By capturing images of the user's hand and recognizing gestures through an image acquisition device, a mapping relationship between gestures and data control commands is established. The 3D scan data acquired by the scanning device is then processed directly, enabling real-time interactive control without touching an external terminal.

Benefits of technology

It frees up users' hands, improves operational efficiency and response speed, enhances the continuity and real-time performance of the scanning process, reduces operational fatigue, and adapts to the scanning needs of large objects or complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123067701A_ABST
    Figure CN123067701A_ABST
Patent Text Reader

Abstract

This application discloses a scanning data processing control, a 3D scanning system, equipment, and medium, belonging to the field of 3D scanning technology, and applicable to scanning operation scenarios such as industrial 3D inspection, digital acquisition, and 3D reconstruction. The method includes: acquiring hand image data collected by an image acquisition device; performing gesture recognition on the hand image data to identify the user's current gesture action; determining the corresponding data control command from a preset mapping relationship based on the identified gesture action; and executing the data control command to perform corresponding data processing operations on the 3D scanning data acquired by the scanning device. This application solves the problems of fragmented operation, discontinuous interaction, and low efficiency in adjusting scanning data caused by relying on external input devices for interactive control in traditional 3D scanning processes, achieving direct, continuous, and real-time interactive control of 3D scanning data through gestures without touching an external terminal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of 3D scanning technology, and particularly relates to a scanning data processing control, a 3D scanning system, equipment and medium. Background Technology

[0002] With the widespread application of 3D scanning technology in fields such as industrial inspection, reverse engineering, cultural relic preservation, and medical modeling, the demand for real-time viewing, rotation, zooming, and mode switching of acquired data during scanning operations is increasing. Existing 3D scanning systems typically employ a collaborative working mode between the scanning equipment and an auxiliary terminal. This means that while scanning is underway, users need to monitor the quality and coverage of the acquired data in real time through the auxiliary terminal to adjust scanning strategies, supplement missing areas, or optimize data accuracy, ensuring the integrity and accuracy of the final scanned model.

[0003] In traditional technology, scanning operations and data viewing mainly involve a combination of scanning equipment and terminal equipment. Users need to operate the scanning equipment while simultaneously viewing, rotating, and zooming the acquired 3D data model in real time on the auxiliary terminal using a mouse or touch screen, and adjusting parameters such as the scanning mode.

[0004] However, all of the above-mentioned methods require users to work with both hands simultaneously: one hand to operate the scanning device and the other to continuously operate the auxiliary terminal. Maintaining this posture for extended periods can easily lead to hand fatigue, affecting the continuity and efficiency of the scanning operation, especially in scanning large objects or complex scenarios. Furthermore, when using a computer as an auxiliary terminal, the device cannot move synchronously with the user, requiring the user to frequently travel between the scanning site and the computer to view and process data, significantly reducing scanning efficiency. Even when using mobile devices such as tablets, users still need to continuously touch the screen to complete various operations, making the process cumbersome and preventing users from focusing on the scanning process itself, thus limiting the overall scanning experience and work efficiency. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a scanning data processing control, a 3D scanning system, device, and medium to solve the problems of fragmented operation, discontinuous interaction, and low efficiency of scanning data adjustment caused by relying on external input devices for interactive control in the existing 3D scanning process. It enables direct, continuous, and real-time interactive control of 3D scanning data through gestures without touching an external terminal.

[0006] In a first aspect, this application provides a scanning data processing control method applied to a three-dimensional scanning system, the three-dimensional scanning system including a scanning device and an image acquisition device, the method comprising: Acquire hand image data captured by the image acquisition device; The hand image data is used for gesture recognition to identify the gesture currently being performed by the user; Based on the recognized gestures, the corresponding data control commands are determined from the preset mapping relationship; The data control command is executed to perform corresponding data processing operations on the three-dimensional scan data acquired by the scanning device.

[0007] Secondly, this application provides a scanning data processing control device, the device comprising: A scanning data processing control device, characterized in that the device comprises: The image acquisition module is used to acquire hand image data acquired by the image acquisition device; The gesture recognition module is used to perform gesture recognition on the hand image data and identify the gesture action currently being performed by the user. The instruction parsing module is used to determine the corresponding data control instruction from the preset mapping relationship based on the recognized gesture action; The data processing module is used to execute the data control instructions to perform corresponding data processing operations on the three-dimensional scanning data acquired by the scanning device.

[0008] Thirdly, this application provides a three-dimensional scanning system, comprising: Scanning equipment used to acquire 3D scanning data; Image acquisition device used to acquire hand image data; A processing apparatus for the scanning data processing control method as described in the first aspect.

[0009] Fourthly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the scan data processing control method as described in the first aspect above.

[0010] Fifthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the scan data processing control method as described in the first aspect above.

[0011] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the scan data processing control method as described in the first aspect.

[0012] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the scan data processing control method as described in the first aspect above.

[0013] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: By capturing images of the user's hands and recognizing gestures through image acquisition devices, gestures are used as the input medium for human-computer interaction, replacing the traditional interaction method where users must use their free hand to operate a mouse or touch screen. Therefore, users do not need to divert their attention from the scanning site to the operation interface of the auxiliary terminal; they can issue commands simply through natural hand gestures, reducing the operational threshold and cognitive burden during the scanning process. Furthermore, by establishing a preset mapping relationship between gestures and data control commands, user-executed gestures are directly mapped to software-level data processing commands, significantly improving operational efficiency and response speed. Moreover, data control commands are applied directly to the 3D scan data already acquired by the scanning device, rather than to the hardware actions of the scanning device. Users can independently control the acquisition status and display of the acquired data without touching the auxiliary terminal, completely eliminating the constraints of traditional solutions that require two-handed operation and constant reliance on the auxiliary terminal. Furthermore, since the controlled object is a data processing operation rather than a device driver operation, gesture commands do not need to be converted through device drivers or hardware interfaces; command parsing and execution can be completed directly at the software level. This results in shorter command transmission paths, lower latency, and significantly improved real-time performance of scan data processing.

[0014] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0015] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram illustrating an application scenario of the scanning data processing and control method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the three-dimensional scanning system provided in the embodiments of this application; Figure 3 This is a flowchart illustrating the scanning data processing control method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the scanning data processing control flow based on gesture recognition provided in an embodiment of this application; Figure 5This is a schematic diagram of the structure of the scanning data processing control device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments that can be directly obtained based on the embodiments of this application are within the scope of protection of this application.

[0017] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0018] To address the problems inherent in traditional methods, this application provides a scanning data processing control method aimed at freeing users' hands and simplifying the data operation process during scanning. By acquiring hand images captured by a camera and performing gesture recognition, the method determines corresponding data control commands from a preset mapping relationship based on the recognized gesture actions. This enables the execution of corresponding data processing operations on the 3D scanning data already acquired by the scanning device, allowing a single gesture to complete all operations such as rotation, scaling, mode switching, and start / stop of 3D scanning data. This completely eliminates the reliance on hand-assisted operation terminals in traditional solutions, effectively reducing operator fatigue and improving work efficiency and overall scanning experience in large object scanning scenarios.

[0019] The scanning data processing control method, three-dimensional scanning system, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through embodiments and application scenarios.

[0020] The embodiments of this application are applicable to scanning operation scenarios such as industrial 3D inspection, reverse modeling, digital acquisition of cultural heritage, and medical 3D reconstruction. By sensing the user's hand movements through image acquisition equipment, real-time interactive control of 3D scanning data can be achieved.

[0021] Figure 1This is a schematic diagram illustrating an application scenario of the scanning data processing control method provided in this application embodiment. The scanning data processing control method provided in this application embodiment can be applied to, for example... Figure 1 The application scenario shown may include a scanning device 101, an image acquisition device 102, and a display terminal for displaying three-dimensional scan data. The display terminal can communicate with the 3D scanning system, for example, it can communicate with the scanning device 101 and the image acquisition device 102 via wired or wireless means and with the processing device.

[0022] The scanning device 101 is used to scan the target object and generate three-dimensional scan data. The image acquisition device 102 is used to acquire image data of the user's hand and to perceive and recognize the hand movements performed by the user. The display terminal is used to visualize the three-dimensional scan data generated by the scanning device 101.

[0023] In some embodiments, the image acquisition device 102 may be a monocular camera or an image sensor with visual acquisition capabilities installed in the 3D scanning system, or it may be an external camera that is independently installed and communicatively connected to the 3D scanning system. This application does not make specific limitations in this regard.

[0024] In some implementations, the display terminal can be an electronic device with graphics display capabilities, including but not limited to desktop computer monitors, laptop screens, industrial control terminal displays, tablet computers, smartphones, AR / VR headsets, and embedded display devices. Different types of display terminals can be used to achieve different resolutions, viewing angles, interaction methods, or immersive 3D display effects according to the application scenario, thereby adapting to different usage needs such as industrial inspection, mobile scanning, or immersive modeling.

[0025] In this embodiment, the gesture recognition process is completed based on the image acquired by the image acquisition device 102. It can also directly reuse the original camera equipment of the 3D scanning system without relying on other sensors or requiring the user to wear any wearable devices (such as data gloves, inertial sensors, etc.). This results in lower cost, simpler deployment, lower power consumption, and lower demand on computing resources in the 3D scanning system. Furthermore, compared to solutions that rely on wearable devices, users do not need to perform pre-scanning preparations such as wearing and calibrating the device, nor do they need to bear the extra weight and constraint brought by wearable devices. They can maintain a natural and comfortable operating state during the scanning operation, which lowers the threshold for device use and improves the scanning experience.

[0026] In one exemplary scenario, when a user is scanning a target object, they do not need to operate a computer or mobile terminal. They can trigger the image acquisition device 102 to acquire hand image data simply by performing hand gestures such as waving, clenching a fist, opening a palm, or making an OK gesture. Then, by performing gesture recognition on the hand image data, the user's gesture is identified, and data control commands are generated based on the gesture. This allows for direct data processing control of the 3D scan data generated by the scanning device 101, thus realizing data processing operations on the 3D scan data.

[0027] For example, data processing operations include, but are not limited to, one or more of the following: spatial pose adjustment, local region editing, viewpoint transformation, translation processing, rotation processing, scaling processing, data completion processing, and noise suppression processing of 3D scan data.

[0028] For example, spatial pose adjustment of 3D scan data can be achieved by rotating the point cloud model around the X or Y axis using gesture commands to observe structural details; local region editing of scan data can be achieved by selecting a target area with gestures and then removing or reconstructing that area to repair missing data; perspective transformation of scan data can be achieved by switching the current 3D model from a front view to a top view or any free viewpoint for multi-angle observation; translation of scan data can be achieved by dragging the model to change its position in the display space; scaling of scan data can be achieved by pinching or spreading two fingers to zoom in or out of the overall model; and noise suppression of scan data can be achieved by filtering out discrete or outlier points generated during the scanning process to improve the smoothness and integrity of the model.

[0029] The above provides an overall application environment for the embodiments of this application, illustrating the collaborative relationship between the scanning device, image acquisition device, and display terminal from the perspective of practical use scenarios. Based on this, the following section further details the above application environment from the perspective of functional module structure, in order to more clearly demonstrate the functional division and cooperation relationships between the various components.

[0030] Figure 2 This is a schematic diagram of the structure of the three-dimensional scanning system provided in an embodiment of this application. Figure 2 As shown in the figure, this application embodiment also provides a three-dimensional scanning system, including a scanning device 101, an image acquisition device 102, and a processing device 103.

[0031] The scanning device 101 and the image acquisition device 102 are described in the foregoing embodiments and will not be repeated here.

[0032] The processing device 103 is communicatively connected to the scanning device 101 and the image acquisition device 102. It is used to acquire hand image data acquired by the image acquisition device 102, process the acquired hand image data to generate data control commands, and then perform operations such as movement, rotation, scaling and post-processing on the three-dimensional scanning data generated by the scanning device 101 based on the data control commands.

[0033] With the aforementioned 3D scanning system, users do not need to put down the scanning equipment or touch any auxiliary terminals during the entire scanning process, completely freeing their hands. Especially in large object scanning scenarios, users can move freely along the scanning path, check the integrity and quality of the collected data at any time through gestures, and resume collection immediately after discovering missing areas. There is no need to frequently travel back and forth between the scanning site and the auxiliary terminal, significantly improving the continuity and overall efficiency of the scanning operation, and greatly reducing user fatigue.

[0034] The scanning data processing and control method provided in this application embodiment can be executed by a processing device. The processing device can be an electronic device or computing device with image processing capabilities and data interaction control capabilities, such as an embedded processing unit, an industrial computer, an edge computing device, a workstation server, a processing module in a mobile terminal, or a dedicated control chip and its combination structure integrated into a scanning system, used to perform hand image data processing, gesture recognition, and three-dimensional scanning data control operations.

[0035] The following description uses a processing device as an example to illustrate the scanning data processing and control method provided in the embodiments of this application.

[0036] Figure 3 This is a flowchart illustrating the scanning data processing control method provided in an embodiment of this application. Figure 3 As shown, the method includes steps 310 to 340.

[0037] Step 310: Acquire hand image data captured by the image acquisition device.

[0038] The image acquisition device continuously images the user's hand area at a preset frame rate, generating a continuous video image stream. In some embodiments, the processing device can extract image frames from the video image stream in chronological order to obtain hand image data containing the user's hand posture information.

[0039] In some implementations, the image acquisition device can limit the acquisition range based on a preset imaging area, so that only the hand area image data related to the scanning operation is acquired, thereby reducing background interference and improving the stability and processing efficiency of subsequent gesture recognition.

[0040] In some implementations, the image acquisition device can also be started synchronously during the scanning process of the scanning device, thereby achieving time-synchronous acquisition of hand image data and three-dimensional scan data, so that subsequent data processing control operations can be performed on the scan data based on gesture actions.

[0041] Step 320: Perform gesture recognition on the hand image data to identify the gesture currently being performed by the user.

[0042] In some implementations, the hand image data consists of consecutive video frames. The processing device first performs image preprocessing on each video frame to reduce the impact of lighting changes, background interference, and imaging noise on the recognition results, thereby making the hand region more separable in the image.

[0043] The processing device performs target detection processing on the preprocessed hand image data to locate the hand region, and crops the image based on the hand region to obtain a hand region image containing hand structure information.

[0044] Based on this, the processing device extracts key points from the hand region image to obtain a set of key points representing the hand posture. Then, the processing device performs gesture classification and recognition based on the key point set to identify the user's current gesture.

[0045] Typically, the hand consists of 21 key points, including the wrist and the joints of the five fingers. Each key point contains two-dimensional or three-dimensional spatial coordinate information. These key points can completely describe the current hand posture and provide basic geometric features for subsequent gesture recognition and motion analysis.

[0046] In some implementations, for continuous action gestures, the processing device can also analyze the key point set of multiple consecutive frames based on time series to improve the recognition stability and robustness of dynamic gestures such as waving, rotating, and scaling.

[0047] Step 330: Based on the recognized gesture, determine the corresponding data control command from the preset mapping relationship.

[0048] Among them, the preset mapping relationship refers to a set of association rules pre-established in the scanning data processing and control system, which is used to characterize the correspondence between different gesture actions performed by the user and the corresponding data control commands.

[0049] In some implementations, the preset mapping relationship includes at least a one-to-one correspondence or a one-to-many correspondence between gesture action identifiers and data control command identifiers, which is used to quickly determine the target data control command corresponding to the gesture action after the target gesture action is identified.

[0050] The gesture action identifiers in the preset mapping relationship are used to represent different types of gesture actions, such as clenching a fist, opening a palm, waving, or making an OK gesture; the data control command identifiers are used to represent the type of control operation performed on the interactive processing of the 3D scan data, including but not limited to one or more of the following: switching scan modes, starting data acquisition, pausing data acquisition, rotating the model, zooming the view, and ending data acquisition. This interactive process can be presented in real time on the display interface.

[0051] In some optional implementations, the preset mapping relationship can also be updated or expanded according to user operation configuration, device application scenario or historical interaction records, so as to achieve adaptation and adjustment between gesture actions and data control commands.

[0052] The "Switch Scan Mode" function corresponds to the operation of switching the processing mode of 3D scan data. It is used to switch between the display or processing logic of marker point related data and 3D reconstruction data, but does not change the acquisition parameters of the scanning device. For example, when the recognized gesture is a mode switching gesture, a control command to switch the scan mode is triggered, which is used to switch between the scanning marker point mode and the scanning data mode to adapt to the data processing needs of different scanning stages. "Start Data Acquisition" controls the generation status of 3D scan data, enabling the system to begin receiving and updating 3D scan data. "Pause Data Acquisition" pauses the update of 3D scan data, freezing the current display or update status of the 3D scan data. "Rotate Model" adjusts the spatial orientation of the 3D scan data, including at least left-right and / or up-down rotation, and can also be rotated at any angle depending on actual needs. "Zoom View" adjusts the display scale of the 3D scan data, including zoomed-in and / or zoomed-out views. "End Data Acquisition" terminates the processing of 3D scan data, ending the update of the current scan data and generating the final 3D scan result.

[0053] It should be noted that the scanning marker point mode refers to the working mode in which the scanning equipment focuses on marker point identification or calibration during the 3D scanning process. In this mode, the system prioritizes the acquisition and identification of feature marker points on the target object or in the environment to establish or calibrate scanning coordinate benchmarks, spatial reference relationships, or tracking constraints, thereby providing positional benchmarks and registration basis for the subsequent generation of 3D scanning data.

[0054] The scanning data mode refers to the working mode of the scanning device after the marker points have been established or calibrated, with the primary goal of acquiring the complete three-dimensional geometric information of the target object. In this mode, the system continuously scans and acquires data from the surface of the target object, generating complete scanning data containing point cloud data or 3D reconstruction data. This data is used to construct a 3D model of the target object and supports subsequent data display and interactive processing.

[0055] In some implementations, the scanning data mode is a general term for scanning data acquisition mode, which may include a variety of different scanning sub-modes to adapt to target objects with different shapes and structural features and scanning tasks with different accuracy requirements. The scanning sub-modes can be dynamically switched based on a preset switching relationship.

[0056] For example, the scanning data modes include, but are not limited to, infrared speckle scanning mode, single-line laser scanning mode, parallel laser scanning mode, and cross-line laser scanning mode. Specifically, the infrared speckle scanning mode is used to acquire the overall 3D shape of the target object without the need for attached markers; the single-line laser scanning mode is used to scan and acquire hole structures, small recessed areas, and local high-precision features to improve the accuracy of detail reconstruction; the parallel laser scanning mode is used to acquire the overall contour and medium-precision structural information of the target object to achieve a relatively balanced efficiency and accuracy; and the cross-line laser scanning mode is used for rapid global scanning of large-sized target objects to improve the efficiency of large-scene scanning coverage.

[0057] In some implementations, the infrared speckle scanning mode, single-line laser scanning mode, parallel laser scanning mode, and cross-line laser scanning mode can be switched according to at least one of the scanning task requirements, the geometric complexity of the target object, and the accuracy requirements. The processing device controls the scanning device to switch between different scanning data sub-modes based on the current scanning state or user input commands, so as to achieve adaptive adaptation to different scanning scenarios.

[0058] Therefore, by setting the above-mentioned preset mapping relationship, different gestures can be directly mapped to key control links in the scanning process, realizing unified control of scanning mode switching, data acquisition start and stop, and 3D model interactive operation. This allows users to complete the entire scanning process without relying on external input devices, significantly improving the continuity, interactive efficiency, and ease of operation of the scanning process, while enhancing the control flexibility and system adaptability of different scanning stages.

[0059] In some implementations, preset mapping relationships are stored in the storage unit of the processing device in the form of a mapping table or mapping index structure for quick retrieval and retrieval.

[0060] The processing device determines the corresponding data control command by matching and searching in a preset mapping relationship based on the recognized gesture action and using the gesture action as an index.

[0061] Different gestures correspond to different types of data control commands. For example, a fist gesture corresponds to the pause and resume control command for scanning data, an open palm gesture corresponds to the pause or freeze control command for real-time display of scanning data, a waving gesture corresponds to the view rotation control command for scanning data, two fingers opening or closing correspond to the zoom control command for scanning data, and an OK gesture corresponds to the control command for switching scanning modes or data processing modes.

[0062] In some optional implementations, the preset mapping relationship can also be dynamically adjusted according to user operating habits or historical usage data to optimize the matching relationship between gesture actions and data control commands, and improve interaction efficiency and recognition adaptability.

[0063] Step 340: Execute data control commands to perform corresponding data processing operations on the 3D scanning data acquired by the scanning device.

[0064] After determining the target data control command, the processing device applies the data control command to the three-dimensional scanning data generated by the scanning equipment, and updates the three-dimensional scanning data in real time through the data processing module to achieve dynamic interactive control of the three-dimensional scanning results.

[0065] In some implementations, when the data control command is a viewpoint transformation command, the processing device updates the display posture parameters of the 3D scan data to achieve rotation, translation, or scaling of the point cloud model or 3D reconstructed model; when the data control command is a scanning process control command, the processing device controls the update status of the scan data to achieve paused display or resumed update of the scan data; when the data control command is a post-processing command, the processing device performs corresponding data optimization processing on the 3D scan data, such as reconstructing local areas, filtering out discrete noise points, or completing missing areas.

[0066] In some alternative implementations, the 3D scanning data is visualized in a continuously updated manner on the display terminal. During the execution of data control instructions, the processing device only processes and updates the data layer without changing the acquisition parameters or scanning path of the scanning device, thereby achieving non-intrusive interactive control of the scanning results.

[0067] The above methods enable users to perform real-time control and editing operations on 3D scan data based on gesture recognition results, improving the interactive efficiency and data processing flexibility of the scanning process.

[0068] According to the scanning data processing control method provided in the embodiments of this application, the user's hand image is captured by the image acquisition device and the gesture is recognized. The gesture is used as the input medium for human-computer interaction, replacing the interaction method in the traditional solution where the user must use the free hand to operate the mouse or touch screen. Therefore, users do not need to divert their attention from the scanning site to the operation interface of the auxiliary terminal; they can issue commands simply through natural hand gestures, reducing the operational threshold and cognitive burden during the scanning process. Furthermore, by establishing a preset mapping relationship between gestures and data control commands, user-executed gestures are directly mapped to software-level data processing commands, significantly improving operational efficiency and response speed. Moreover, data control commands are applied directly to the 3D scan data already acquired by the scanning device, rather than to the hardware actions of the scanning device. Users can independently control the acquisition status and display of the acquired data without touching the auxiliary terminal, completely eliminating the constraints of traditional solutions that require two-handed operation and constant reliance on the auxiliary terminal. Furthermore, since the controlled object is a data processing operation rather than a device driver operation, gesture commands do not need to be converted through device drivers or hardware interfaces; command parsing and execution can be completed directly at the software level. This results in shorter command transmission paths, lower latency, and significantly improved real-time performance of scan data processing.

[0069] For example, in some specific application scenarios, when a user performs 3D scanning on a large object, the traditional method usually requires the user to scan a portion of the area, return to an auxiliary terminal to view the scan results, and then return to the target object to continue scanning based on the viewed results. This results in interruptions in the scanning path and reduced scanning efficiency. However, in the embodiments of this application, the user can directly adjust the display perspective and display scale of the 3D scan data on-site using natural gestures, and judge whether the model is complete based on the current visual feedback. If local missing parts, broken edges, incomplete holes, or insufficient details in recessed areas are found, the user can continue scanning the corresponding area by using gestures to restore data updates. Thus, the viewing, judgment, and supplementary scanning of scan data can be completed continuously in the same scanning process.

[0070] In other specific application scenarios, when the target object includes holes, small depressions, large outer contours, or complex curved surfaces, users can first switch to the scanning data mode via gestures and then switch between different scanning sub-modes. For example, in infrared speckle mode, the overall 3D shape can be acquired without attaching markers; in single-line laser mode, local details can be acquired for holes or small depressions; in parallel laser mode, the overall contour data of the target object can be acquired; and in cross-line laser mode, large objects can be quickly covered and scanned. The data control commands corresponding to the above mode switching can be linked with the display status of the 3D scanning data, allowing users to select the appropriate scanning method or data viewing method based on the feedback of the current 3D model at different scanning stages, thereby improving the scanning integrity and data usability of complex structural objects.

[0071] In some implementations, the steps of performing gesture recognition on hand image data include: performing hand detection on the hand image data to obtain a hand region image; inputting the hand region image into a hand keypoint network to obtain the keypoint coordinates of each joint of the hand; and performing gesture recognition based on the keypoint coordinates to identify the gesture action currently being performed by the user.

[0072] The processing device first performs hand detection processing on the acquired hand image data to locate the area where the user's hand is located from the original image, obtain the hand area image, and thus filter out the interference of background information on subsequent recognition.

[0073] Furthermore, the processing device inputs the hand region image into a pre-trained hand keypoint network, extracts features from the hand structure through the network, and outputs the keypoint coordinates that characterize the hand skeletal structure. The keypoint coordinates are used to describe the spatial positional relationship and posture change information of each joint of the hand.

[0074] Based on this, the processing device constructs a hand posture feature representation based on the key point coordinates and analyzes it in conjunction with the geometric relationship information between the key points, thereby generating posture feature data to characterize the current hand movement.

[0075] Finally, the processing device performs gesture recognition processing based on posture feature data, matching the current hand posture with preset gesture categories to identify the gesture action currently being performed by the user.

[0076] In the above embodiments, by adopting a hierarchical recognition process, gesture recognition is upgraded from pixel-level image processing to structured posture feature analysis, which effectively reduces the impact of background interference on the recognition results and improves the accuracy and robustness of gesture recognition. At the same time, the structural information of key points is used to enhance the expressive ability of dynamic gestures, thereby improving the recognition stability and adaptability in continuous action scenarios.

[0077] The gestures include static gestures and dynamic actions to adapt to different types of human-computer interaction during the scanning process.

[0078] As is easily understood, static gestures refer to the type of gesture formed by a user maintaining a relatively stable hand posture at a specific moment. They do not rely on motion changes across consecutive frames in the time dimension, but are recognized solely based on the spatial structural features of the hand represented by a single frame of hand image. Static gestures characterize hand posture through the distance relationships between key points of the hand and the angle relationships between joints, such as clenching a fist, opening the palm, or making an OK gesture. Their characteristic is that the coordinates of key points do not change significantly within a short time window.

[0079] Dynamic actions refer to gestures formed by continuous changes in a user's hand over time, relying on temporal change information composed of multiple consecutive frames of hand images for recognition. Dynamic actions are characterized by the positional changes, motion trajectories, and speed features of key hand points between consecutive frames, such as waving, swiping left or right, or zooming. Their characteristics are manifested by the continuous changing trend of key point coordinates in the time series, reflecting the dynamic behavior of the hand movement process.

[0080] Accordingly, the processor performs gesture recognition based on keypoint coordinates, including: calculating the distance features between each keypoint and the angle features at each joint based on the keypoint coordinates; for static gestures, identifying the gesture category based on the distance and angle features of the current single-frame image; for dynamic actions, using a sliding window mechanism to cache the keypoint coordinates of multiple consecutive frames to form a keypoint coordinate change sequence, and calculating the motion speed features of the keypoints between consecutive frames based on the keypoint coordinate change sequence; and identifying the action category based on the keypoint coordinate change sequence and motion speed features.

[0081] When recognizing gestures, the processing device does not rely solely on a single feature for judgment. Instead, it performs phased modeling of the spatial structural features and temporal change features of key hand points to distinguish between static gestures and dynamic actions.

[0082] After acquiring hand image data, the processing device first extracts features from the hand region using a pre-trained hand keypoint detection model, outputting keypoint coordinates including the wrist, palm, and finger joints, and then maps the keypoint coordinates to a normalized coordinate space to eliminate scale differences caused by different shooting distances and angles.

[0083] Key points refer to the set of discrete feature points used to characterize the structure and posture of the hand. Key points correspond to the joints or anatomical features of the hand, including but not limited to the wrist, palm, and the joints and fingertips of each finger.

[0084] In some implementations, key points are represented in two-dimensional or three-dimensional coordinates to describe the positional distribution of the hand in the image coordinate system or spatial coordinate system, thereby constructing a geometric model of the hand. The spatial distance and angular relationships between the key points can be used to characterize changes in hand posture and serve as basic feature data for gesture recognition and motion analysis.

[0085] For the static gesture recognition process, the processing device takes the coordinates of key points in a single frame as input, constructs a geometric relationship matrix between key points, including the Euclidean distance between each pair of key points and the angular distribution at each joint, and inputs the geometric relationship matrix into a preset classification model for discrimination, thereby outputting the corresponding static gesture category.

[0086] For the dynamic action recognition process, the processing device caches the coordinates of key points in multiple consecutive frames in a time series manner and constructs a key point trajectory sequence within a preset time window. Further, the processing device performs differential calculation on the trajectory sequence to obtain the displacement changes of key points between consecutive frames, and calculates motion speed features and direction features based on the displacement changes, thereby forming a dynamic action feature vector.

[0087] Subsequently, the processing device inputs the dynamic action feature vector into the sequence recognition model for classification and recognition, and outputs the corresponding dynamic action category to represent the continuous interactive operation performed by the user.

[0088] In the above embodiments, by having the processing device uniformly perform hierarchical recognition of static gestures and dynamic actions, and combining key point geometric features and temporal motion features for joint modeling, the system can simultaneously cover two types of scanning operation requirements: discrete control and continuous interaction. Static gestures are used to realize key control nodes such as mode switching and scanning start / stop, while dynamic actions are used to realize continuous interactive adjustment of 3D scanning data (such as rotation, scaling, and viewpoint changes), thereby improving the continuity, stability, and naturalness of scanning data interactive control, and reducing dependence on external interactive devices.

[0089] In some implementations, before performing feature extraction and gesture recognition, the processing device performs normalization preprocessing on the key point coordinates to improve the consistency and robustness of hand pose representation under different shooting conditions, thereby eliminating interference caused by changes in shooting position, scale, and hand orientation.

[0090] Accordingly, the normalization preprocessing steps may include: using the wrist key point as a reference, performing translation normalization on the key point coordinates to eliminate translation deviation; using the palm size as a reference, performing scale normalization on the key point coordinates to eliminate scale deviation; and using the palm direction as a reference, performing rotation normalization on the key point coordinates to eliminate rotation deviation.

[0091] The processing device first uses the wrist key point as a spatial reference benchmark and performs translation and normalization processing on the coordinates of all key points, so that the hand key points are translated as a whole to a local coordinate system with the wrist as the origin, thereby eliminating the influence of the position difference of the user's hand in the image on feature calculation.

[0092] The processing device uses the size of a palm as a scale benchmark to normalize the coordinates of key points, mapping the differences in hand size caused by cameras at different distances to a unified scale space, so as to ensure that distance features and angle features are comparable under different acquisition conditions.

[0093] The processing device uses the palm direction as the rotation reference and performs rotational normalization processing on the key point coordinates, so that the hand is uniformly mapped to the standard posture coordinate system under different orientations or rotation angles, thereby eliminating the posture deviation caused by hand rotation.

[0094] After completing the translation normalization, scale normalization, and rotation normalization processes described above, distance features and angle features are calculated based on the normalized key point coordinates, thereby further improving the stability and accuracy of subsequent gesture recognition.

[0095] Therefore, by introducing multi-dimensional normalization processing based on wrist, palm size, and palm orientation before feature calculation, the key points of the hand are mapped from the original image coordinate space to the standardized pose space, effectively eliminating the interference caused by changes in position, scale, and rotation, thereby significantly improving the stability and consistency of distance and angle features. Furthermore, this processing method enhances the robustness of gesture recognition under different shooting angles, different user hand shapes, and dynamic scanning environments, and improves the reliability and generalization ability of interactive control of scanning data.

[0096] To improve recognition stability, in some implementations, after completing static gesture recognition and dynamic action recognition, the processing device jointly judges the two types of recognition results to improve the accuracy of recognizing the user's true intent, thereby avoiding false triggering problems caused by a single recognition result. That is, when a static gesture category is identified based on the distance and angle features of the current single-frame image, and a dynamic action category is identified based on the key point coordinate change sequence of consecutive multiple frames, the dynamic action category is preferentially used as the final recognition result.

[0097] Specifically, when the processing device identifies a static gesture category based on the distance and angle features of the current single-frame image, and identifies a dynamic action category based on the sequence of key point coordinate changes in consecutive multi-frame images, the processing device will perform conflict determination and priority decision processing on the two recognition results.

[0098] During the priority decision-making process, the processing device prioritizes dynamic action categories as the final recognition result based on the characteristic that the continuous time series information corresponding to dynamic actions can better reflect the user's true interaction intentions, and ignores or delays the output of static gesture categories.

[0099] In some implementations, this priority strategy can be used to address the problem of momentary static posture misidentification that may occur when a user performs continuous operations such as waving, rotating, or scaling, thereby ensuring the continuity and stability of the interactive control of scan data.

[0100] The processing device also evaluates the confidence level of static gesture recognition results and dynamic action recognition results, and prioritizes outputting dynamic action results when the confidence level of dynamic action is higher than that of static gesture, so as to avoid interference with control commands due to momentary posture misjudgment.

[0101] Therefore, by introducing a conflict determination and priority selection mechanism between static gestures and dynamic actions, the system can prioritize retaining the dynamic action result, which has more temporal continuity and semantic integrity, when both types of recognition results exist simultaneously. This effectively avoids misjudgments caused by instantaneous hand postures and improves the reliability of gesture recognition results. At the same time, this mechanism enhances the system's stable control capability in continuous interaction scenarios, making the scanning data processing process more continuous, natural, and in line with the user's actual operation intentions.

[0102] In some implementations, when performing hand detection on hand image data, the processing device employs a hybrid processing mechanism that combines detection and tracking to reduce computational overhead and improve real-time performance while ensuring detection accuracy.

[0103] In some implementations, hand detection is performed on hand image data to obtain a hand region image. This can be achieved by inputting the hand image data into a hand detection network, which then processes the current hand image data and outputs the location of the hand region in the image, thereby obtaining a hand region image, which is then used for subsequent key point extraction and gesture recognition.

[0104] In the process of continuous frame processing, in order to reduce the computational burden caused by repeated detection, some implementations perform hand detection on the hand image data to obtain a hand region image. Alternatively, a tracking algorithm can be used to predict the hand region in the current frame image based on the hand region position of the previous frame image to obtain a hand region image.

[0105] The processing device can also use a tracking algorithm to predict and update the hand region in the current frame based on the position of the hand region in the previous frame image, thereby quickly obtaining the hand region image without performing a complete detection process.

[0106] To avoid hand area drift caused by the accumulation of tracking errors, the processing device re-executes the hand detection process to correct the results when preset conditions are met. The preset conditions include: performing detection correction once every preset number of frames, or triggering re-detection when the tracking confidence is lower than a preset threshold, thereby ensuring the stability and accuracy of hand area positioning.

[0107] By employing the above methods, a balance is achieved between detection accuracy and real-time performance in the hand detection process, providing a stable and reliable input foundation for subsequent gesture recognition and 3D scanning data interaction control.

[0108] In the above embodiments, by introducing a hybrid hand region acquisition mechanism that combines detection and tracking, and combining periodic correction and confidence-triggered correction strategies, the computational overhead of continuous frame detection can be significantly reduced while ensuring the accuracy of hand positioning. At the same time, by avoiding full frame-by-frame detection, the real-time response capability of gesture recognition processing is improved, and the stability and robustness in dynamic scanning scenarios are enhanced, thereby improving the continuity and efficiency of three-dimensional scanning data interaction control.

[0109] In some implementations, the processing device preprocesses the acquired hand image data before performing hand detection to improve the stability and accuracy of subsequent hand detection.

[0110] For example, the processing device performs size scaling on the hand image data, uniformly adjusting the input image to a preset resolution to adapt to the input requirements of the hand detection network.

[0111] The processing device can also perform color space conversion, such as converting RGB images to grayscale or normalized color spaces, to reduce the impact of lighting changes on the detection results.

[0112] The processing device can also normalize the image data to keep the pixel value distribution within a uniform range, thereby improving the stability of model inference.

[0113] After preprocessing, the processing device inputs the preprocessed image into the hand detection network to execute subsequent processes.

[0114] During the execution of data control commands, the processing device also provides synchronous visual feedback control of the display process of the 3D scan data through the display interface, so as to realize the integrated presentation of the scanning process and interactive operation.

[0115] The display interface is connected to the display terminal. As the interactive display layer of the display terminal, the display interface is used to carry the real-time rendering results, interactive status information and scanning process status information of the 3D scanning data, and receive the updated data output by the processing device to achieve dynamic refresh, so that the user can observe the change process of the 3D scanning data in real time on the display terminal.

[0116] In some implementations, when the processing device executes a data control command to switch scanning modes, the scanning mode indicator on the display interface switches between "scanning marker point mode" and "scanning data mode". For example, when scanning an engine component, the user triggers the mode switch by making a fist gesture, and the interface switches from "marker point mode" to "point cloud construction mode", simultaneously displaying the spatial distribution of the identified feature marker points on the surface of the component, or switching to displaying the generated engine point cloud model status.

[0117] In some implementations, when the processing device executes a data control command to start or pause data acquisition, the 3D data model on the display interface is gradually built or paused as scan data is written, and the 3D data model is updated in real time on the display interface. For example, when scanning the structure of a car dashboard, the user triggers the start of acquisition by opening their palm, and the 3D model of the dashboard gradually expands from a local outline to a complete point cloud structure; when the user executes the pause gesture again, the currently generated dashboard model remains unchanged and is used to observe the current scan quality.

[0118] In some implementations, when the processing device executes data control instructions to rotate the model, the 3D data model already constructed in the display interface rotates synchronously with the direction of the hand gesture. For example, when scanning a sculpture model, the user can use left and right hand gestures to rotate the sculpture model left and right around the vertical axis in the display interface, thereby observing its side structural details.

[0119] In some implementations, when the processing device executes a data control command to zoom in or out of the view, the view scale of the 3D data model in the display interface is synchronously zoomed in or out as two fingers are spread or contracted. For example, when scanning a building component, the user can zoom in on the model by spreading two fingers to observe crack details, and zoom out on the model by contracting two fingers to view the overall structural layout.

[0120] In some implementations, when the processing device executes a data control command to end data acquisition, the 3D data model on the display interface is confirmed as the final scan result and presented in the center of the display interface. For example, after completing the scan of an industrial part, the user ends the acquisition by making an OK gesture, and the system automatically centers and highlights the final complete point cloud model for subsequent export or measurement analysis.

[0121] In the above embodiments, by introducing a visual feedback mechanism synchronized with the display interface during the data control execution process, the generation process of 3D scanning data, the results of interactive operations, and the final model status can be presented in real time, thereby enhancing the user's ability to perceive the scanning progress and data changes. At the same time, by synchronously displaying the switching of scanning modes, changes in data acquisition status, and model interactive operations, the intuitiveness and consistency of system interaction are improved, allowing users to understand the current scanning status without additional explanation, thereby improving the controllability and operational efficiency of the 3D scanning process.

[0122] To more clearly illustrate the specific implementation method and engineering process of gesture recognition in actual scanning interaction, based on the above embodiments, the following further explains the specific implementation process of gesture recognition in 3D scanning data control and its collaborative relationship with the scanning process.

[0123] The gesture recognition-based scanning data processing control flow provided in this application embodiment is applied to a scanning data processing control system. This system only needs to be configured with an ordinary camera as an image acquisition device, without the need for additional complex auxiliary equipment. When operating the scanning device, the user does not need to be distracted by touching the computer or mobile device. They can complete the entire process of interactive control of the 3D scanning data through simple gestures such as waving, clenching a fist, opening the palm, OK gesture, opening or closing two fingers, and liking.

[0124] In some implementations, the image acquisition device is automatically activated upon software startup, and a preset one-to-one mapping relationship between gesture actions and scanning software functions is initialized. This mapping relationship includes at least the following: The OK gesture (index finger and thumb forming a ring) is used to switch scanning modes, switching between scan marker mode and scan data mode; A clenched fist (five fingers together) is used to start or resume a scan; Open your palm (fingers spread) to pause the scan; Waving your hand left and right is used to control the left and right rotation of the 3D model; Waving your hand up and down controls the vertical rotation of the 3D model. Spread your two fingers (index finger and thumb) to magnify the view; Bring your index finger and thumb together to zoom out; The thumbs-up gesture is used to complete the scanning process.

[0125] In practice, users can first switch to marker scanning mode using the OK gesture, and then start marker scanning by making a fist gesture. Once marker collection is complete, a thumbs-up gesture ends the phase and automatically switches to data scanning mode. After entering data scanning mode, another fist gesture initiates 3D data scanning.

[0126] During the scanning process, users can zoom in on the model to view local details by spreading two fingers, or zoom out on the overall structure by contracting two fingers. When it is necessary to check the scan quality, the scan can be paused by spreading the palm, and the model can be rotated by waving the palm left and right or up and down, combined with two-finger gestures to zoom, thereby checking the current 3D scan results from multiple angles. If missing areas or areas with poor scan quality are found, the scan can be resumed by clenching the fist for supplementary scanning. Finally, the entire scanning process is completed by giving a thumbs-up.

[0127] During system operation, the image acquisition device continuously acquires hand image data at a preset frame rate, forming a continuous video stream. Each frame is used as input data and fed into the processing unit. To ensure real-time performance and stability, the system first preprocesses the input images, including scaling, color space conversion, and data normalization, to adapt to the subsequent neural network input format and reduce the impact of lighting and imaging differences. Then, a hand detection network performs target detection on the images to obtain the hand bounding box coordinates. Taking advantage of the relatively small changes in hand position between consecutive frames, a target tracking mechanism is introduced. This mechanism predicts the hand region in the current frame based on the position of the previous frame and performs corrections using periodic detection, thereby reducing computational overhead while maintaining positioning stability. After obtaining the hand region, a region of interest (ROI) cropping method is used to extract local hand images to reduce background interference and improve subsequent recognition efficiency.

[0128] During the keypoint localization stage, the system analyzes the cropped hand region using a hand keypoint network to obtain coordinate information for 21 keypoints, including the wrist and finger joints. Each keypoint corresponds to a two-dimensional or three-dimensional spatial position, thereby constructing the hand skeleton structure. Based on this, to improve the consistency of feature representation under different shooting conditions, the system performs translational normalization based on the wrist keypoint, scale normalization based on the palm size, and rotational normalization based on the palm direction, mapping hand poses at different positions, distances, and orientations to a unified standard space.

[0129] In the feature construction stage, the system calculates distance and angle features based on keypoint coordinates to characterize finger opening and closing and joint flexion. It also further calculates the changes in keypoints across consecutive frames to form motion velocity features, which describe hand movement trends. Static gestures rely on single-frame features for recognition, while dynamic actions are cached using a sliding window or circular buffer mechanism to form a time-series structure that characterizes changes in hand movement trajectories. Based on this time series, displacement, velocity, and direction features are extracted to identify continuous actions such as waving, rotation, and scaling.

[0130] During the recognition phase, the system jointly judges static gestures and dynamic actions, prioritizing the recognition result of dynamic actions when both are present to avoid instantaneous posture misjudgment. Simultaneously, the recognition result is output to the result generation unit and converted into corresponding data control commands. These data control commands operate only on the 3D scan data itself, not on the scanning device's execution layer, and are used to implement the movement, rotation, scaling, and post-processing operations of the 3D model, including local area repair, noise suppression, or data completion.

[0131] During execution, the processing device applies data control commands to the 3D scan data and provides synchronous visual feedback on the display terminal. This allows the switching status between the scan marker mode and the scan data mode, the real-time construction or pause status of the scan data, and the rotation and scaling effects of the 3D model to be presented in real time. This forms a closed-loop linkage mechanism between the scanning process and interactive control, enabling users to complete the entire process of continuous acquisition, inspection, correction, and output of 3D scan data without relying on any external input devices.

[0132] Figure 4 This is a schematic diagram of the scanning data processing control flow based on gesture recognition provided in an embodiment of this application. Figure 4 As shown, the process consists of the image acquisition and preprocessing stage, the spatial perception and target localization stage, the temporal feature modeling and action recognition stage, and the result output stage in sequence.

[0133] The image acquisition stage first acquires image data of the user's hand through image acquisition, and then performs image preprocessing on the acquired image data. Image preprocessing includes size scaling, color space conversion, and data normalization to convert the original video frames into a standard input format suitable for subsequent model processing.

[0134] After preprocessing, the image proceeds to the spatial perception and target localization stage. In this stage, target detection is first performed on the image to determine the region where the hand is located. Then, based on the detection results, target tracking is performed to predict changes in hand position between consecutive frames, and corrections are made through re-detection if necessary. Next, region of interest extraction is performed on the hand region to obtain local images of the hand, thereby reducing background interference and improving computational efficiency. Further, geometric keypoint localization is performed based on the hand region. Keypoint network is used to extract the coordinates of keypoints at each joint of the hand, and this is combined with hand structural information to achieve geometric keypoint localization. Simultaneously, visual stabilization processing is performed on the hand position during consecutive frame processing to improve the consistency and stability of the detection results over time. The visual stabilization processing includes, but is not limited to, at least one of the following methods: target position prediction based on the hand region position of the previous frame, trajectory smoothing based on Kalman filtering, key point coordinate smoothing based on exponential moving average, cross-frame feature point association based on optical flow, and key point coordinate fusion based on confidence weighting. In particular, by applying smoothing constraints to the changes in hand position or key points between consecutive frames, the impact of jitter, occlusion, or instantaneous detection errors on the hand pose estimation results is reduced, thereby improving the stability and consistency of hand key points in the time dimension.

[0135] After completing spatial localization, the system proceeds to the temporal feature modeling and action recognition stage. In this stage, the system extracts temporal features from key points and standardizes their coordinates, including unified mapping related to translation scale and direction, to eliminate the influence of differences caused by varying shooting conditions. Based on this, a time series is constructed, organizing continuous frame key point data into a sequential structure. Dynamic behavior modeling is performed using this time series, and dynamic actions are recognized based on it. Simultaneously, single-frame structural information is combined to identify static postures, thus achieving dynamic action recognition and stable judgment of static gesture states. Dynamic actions reflect continuous changes in hand behavior, while static gestures represent instantaneous posture states.

[0136] After completing the recognition, the system enters the result output stage. The recognized gesture results are converted into corresponding data control commands and output the results to drive the interactive control of the 3D scan data. This enables the movement, rotation, scaling, and post-processing of the scan data. At the same time, continuous feedback on the scanning status is maintained during the output process. This makes the entire process, from image acquisition, image preprocessing, target detection, target tracking, region of interest extraction, geometric key point localization, visual stabilization, temporal features, standardization, time series construction, dynamic action recognition, gesture state stabilization, to the output results, form a complete closed-loop processing link, realizing real-time interactive control of gesture-based 3D scan data.

[0137] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0138] The scanning data processing control method provided in this application embodiment is applied to a three-dimensional scanning system including a scanning device, an image acquisition device, and a display terminal. It includes the following steps: While a user holds the scanning device to scan a target object and the display terminal displays the generated three-dimensional scanning data, hand image data acquired by the image acquisition device is obtained; gesture recognition is performed on the hand image data to identify the user's current gesture; based on the identified gesture, a corresponding data control command is determined from a preset mapping relationship; without interrupting the user's scanning operation on the target object, the data control command is executed to perform corresponding data processing operations on the three-dimensional scanning data displayed on the display terminal; wherein, the data processing operations include at least one of rotating, scaling, pausing updates, resuming updates, and switching scanning modes on the three-dimensional scanning data; and the visual feedback of the three-dimensional scanning data after data processing operations is used to assist in determining areas to be re-scanned or areas with abnormal scanning quality.

[0139] In some implementations, this application does not merely use gestures as a general input method to replace mouse, keyboard, or touch operations. Instead, it integrates gesture recognition results into the real-time processing of 3D scan data, enabling users to interactively control the acquired or generated 3D scan data while continuously performing scanning operations. In other words, data control commands act on the 3D scan data itself, rather than directly on the hardware acquisition actions or scanning path of the scanning device. For example, when a user is scanning a large workpiece, a complex curved surface part, or a target object with holes or recesses using a handheld scanning device, they can pause the update display of the current 3D scan data by opening their palm, rotate the constructed 3D model in the display interface by waving their palm left and right or up and down, and zoom in or out of the 3D model by spreading or contracting two fingers to view local details and determine if there are any missing or poorly scanned areas. After determining that there are areas to be scanned, the user can resume the update of the 3D scan data by clenching their fist, allowing the scanning device to continue acquiring 3D data of the corresponding area and supplementing the current 3D model with the newly acquired data. Therefore, the embodiments of this application form a closed-loop processing procedure: "the scanning device continuously acquires 3D data - the display terminal presents the 3D scan data in real time - the user interacts with and views the 3D scan data through gestures - the user judges the area to be scanned or the area with abnormal quality based on visual feedback - the user continues to acquire or scan." This closed-loop processing procedure allows the user to view the scan results in real time, judge the quality, and adjust the data without having to put down the scanning device or turn to a computer, tablet, or other auxiliary terminal for touch operation during the scanning process. This reduces the number of times the user switches between scanning operations and data viewing, and improves the continuity of the scanning process.

[0140] The gesture control in this embodiment is not only used to control the scanning device to start, stop, or adjust scanning parameters, but is mainly used to control the display, update, and processing status of data during the generation of 3D scanning data. In other words, this embodiment maps gesture input to data-layer interactive operations, enabling users to process and view 3D scanning data in real time without changing the hardware execution logic of the scanning device. This method decouples the acquisition process of the scanning device from the interactive control process of 3D scanning data, avoiding the impact on scanning continuity due to frequent triggering of device-layer control, while shortening the response path between gesture commands and data processing results, thus improving the real-time performance of scanning data interactive processing.

[0141] Through the above methods, the embodiments of this application not only solve the problem of operational fragmentation caused by the user needing to operate the scanning device and auxiliary terminal simultaneously during the scanning process, but also enable the user to complete real-time viewing, rotation, scaling, pausing, resumption, and rescanning judgment of 3D scanning data during continuous scanning. Therefore, it is particularly suitable for continuous scanning scenarios of large objects, complex curved surface objects, and objects with holes or recessed structures, improving scanning efficiency, data integrity, and user operation convenience.

[0142] It should be noted that in conventional gesture-controlled scanning solutions, gestures are typically understood as control inputs to the scanning device or the scanning process itself, such as starting a scan, stopping a scan, switching scan parameters, or triggering the scanning device to perform a certain acquisition action. The control object of this type of solution is the working state of the scanning device or acquisition commands. Its technical logic is to use gestures as a substitute for button, mouse, or touch input, and essentially still revolves around starting and stopping the scanning device, adjusting parameters, or acquisition actions.

[0143] In this embodiment, the gesture control object is not the hardware execution action of the scanning device, but rather the 3D scan data that the scanning device has already acquired or is generating. The former focuses on whether the device starts acquiring data, how it acquires data, and how the acquisition parameters are adjusted; the latter focuses on how the data already generated or being generated during the scanning process is viewed, updated, judged, and processed. The former solves the problem of device operation input method, while the latter solves the problem that data viewing, quality judgment, and rescanning decisions cannot be continuously connected with the scanning process.

[0144] Furthermore, in 3D scanning scenarios, users typically hold the scanning device in one hand and move it around the target object. If the other hand is used to operate a computer, tablet, or other auxiliary terminal, the scanning process will be interrupted. This is especially true when scanning large objects, complex curved surfaces, holes, or small recessed areas, where users often need to repeatedly check whether the current model is complete, whether local areas are missing, and whether the details meet the requirements. If gestures are only used to control the start and stop of the scanning device, it cannot solve the problem of users rotating, zooming, pausing to view, and judging the quality of the acquired 3D scan data in real time, nor can it form a closed-loop process for continuing to scan based on visual feedback.

[0145] This application's embodiments, based on the aforementioned scenario differences, map hand gestures into data control commands applied to 3D scan data. This allows users to view and interact with the 3D scan data on the display terminal in real time using natural gestures while the scanning device continuously acquires data, and to determine whether to continue acquisition or perform additional scanning based on the display feedback. This processing method does not simply replace the objects of existing gesture control commands, but rather embeds gesture recognition results into data processing to achieve coordination between the scanning device's acquisition process and the 3D scan data interaction process.

[0146] Therefore, by transferring gesture input from the device control layer to the 3D scan data processing layer, the user can complete the real-time viewing, interactive adjustment, and supplementary scan judgment of 3D scan data without interrupting the scanning action or touching the auxiliary terminal, thereby improving the continuity of the scanning process, data integrity, and interaction efficiency.

[0147] The scanning data processing control method provided in this application can be executed by a scanning data processing control device. This application uses the example of a scanning data processing control device executing the scanning data processing control method to illustrate the scanning data processing control device provided in this application.

[0148] Figure 5 This is a schematic diagram of the scanning data processing control device provided in an embodiment of this application. Figure 5 As shown, the scanning data processing control device 500 includes: an image acquisition module 501, a gesture recognition module 502, an instruction parsing module 503, and a data processing module 504.

[0149] Image acquisition module 501 is used to acquire hand image data acquired by image acquisition device.

[0150] The gesture recognition module 502 is used to perform gesture recognition on hand image data and identify the gesture action currently being performed by the user.

[0151] The instruction parsing module 503 is used to determine the corresponding data control instruction from the preset mapping relationship based on the recognized gesture.

[0152] The data processing module 504 is used to execute data control commands to perform corresponding data processing operations on the three-dimensional scanning data acquired by the scanning device.

[0153] In some implementations, the instruction parsing module 503 is also used to establish a correspondence between different gesture actions and data control instructions, wherein different gesture actions correspond to at least one of the following data control instructions: switching scanning mode, starting data acquisition, pausing data acquisition, rotating the model, zooming the view, and ending data acquisition; wherein switching scanning mode is used to switch between scanning marker point mode and scanning data mode, rotating the model includes left and right rotation and / or up and down rotation, and zooming the view includes zooming in and / or zooming out.

[0154] In some implementations, the gesture recognition module 502 is also used to perform hand detection on hand image data to obtain a hand region image; input the hand region image into a hand keypoint network to obtain the keypoint coordinates of each joint of the hand; and perform gesture recognition based on the keypoint coordinates to identify the gesture action currently performed by the user.

[0155] In some implementations, the gesture recognition module 502 is further configured to calculate the distance features between key points and the angle features at each joint based on the key point coordinates; for static gestures, the gesture category is identified based on the distance features and angle features of the current single-frame image; for dynamic actions, a sliding window mechanism is used to cache the key point coordinates of multiple consecutive frames to form a key point coordinate change sequence, and the motion speed features of the key points between consecutive frames are calculated based on the key point coordinate change sequence; the action category is identified based on the key point coordinate change sequence and the motion speed features.

[0156] In some implementations, the gesture recognition module 502 is also used to perform translation normalization processing on the key point coordinates based on the wrist key point to eliminate translation deviation; to perform scale normalization processing on the key point coordinates based on the palm size to eliminate scale deviation; and to perform rotation normalization processing on the key point coordinates based on the palm direction to eliminate rotation deviation.

[0157] In some implementations, the gesture recognition module 502 is also used to prioritize the dynamic action category as the final recognition result when both static gesture categories and dynamic action categories are recognized simultaneously.

[0158] In some embodiments, the image acquisition module 501 is also used to perform hand detection on hand image data, including at least one of the following methods: obtaining a hand region image based on a hand detection network; or predicting the hand region in the current frame image based on the hand region position of the previous frame image using a tracking algorithm; wherein, the detection process based on the hand detection network is re-executed every preset number of frames or when the tracking confidence is lower than a preset threshold, so as to correct the hand region in the current frame image.

[0159] In some embodiments, the image acquisition module 501 is further configured to perform the following processing before performing hand detection on the hand image data: preprocessing the hand image data, including size scaling, color space conversion, and data normalization; detecting the preprocessed image through a hand detection network to obtain the coordinates of the hand bounding box; and cropping the region of interest based on the hand bounding box coordinates to obtain the cropped hand image data.

[0160] In some embodiments, the above-mentioned device further includes a display module for synchronously presenting visual feedback of the 3D scan data on the display interface when executing data control commands; wherein: when executing a data control command to switch scanning modes, the scanning mode indicator on the display interface switches between scanning marker point mode and scanning data mode, and correspondingly displays the spatial distribution status of the scanned marker points or the status of the acquired 3D data model; when executing a data control command to start or pause data acquisition, the 3D data model is gradually built or paused as the scan data is written, and updated in real time on the display interface; when executing a data control command to rotate the model, the constructed 3D data model is synchronously rotated and displayed according to the gesture direction; when executing a data control command to zoom the view, the view scale of the 3D data model is synchronously enlarged or reduced according to the gesture action; when executing a data control command to end data acquisition, the 3D data model is confirmed as the final scan result and presented in the center of the display interface.

[0161] The scanning data processing control device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be any computing device with data processing and image sensing capabilities, such as a smartphone, tablet, laptop, desktop computer, industrial control computer, embedded device, edge computing terminal, server device, wearable device, or a combination of the aforementioned functional modules.

[0162] The scanning data processing control device provided in this application embodiment can realize the various processes implemented in the above-described scanning data processing control method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0163] Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, it implements the various processes of the above-described scanning data processing control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0164] This application provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described scanning data processing control method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0165] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable media, such as computer read-only memory (ROM), random-access memory (RAM), magnetic disks, or optical disks.

[0166] The computer-readable storage medium may include: read-only memory (ROM), random-access memory (RAM), magnetic disk or optical disk, etc.

[0167] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described scanning data processing control method.

[0168] This application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described scanning data processing control method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0169] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0170] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0171] From the above description of the embodiments, it is clear that the methods described in the embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0172] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the above-described embodiments. The above-described embodiments are merely illustrative and not restrictive. Under the guidance of this application, many other forms can be made without departing from the spirit and scope of the claims, and all of them are within the protection scope of this application.

[0173] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0174] Although embodiments of this application have been shown and described, it is understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A scanning data processing control method, characterized in that, Applied to a 3D scanning system, the 3D scanning system including a scanning device and an image acquisition device, the method includes: Acquire hand image data captured by the image acquisition device; The hand image data is used for gesture recognition to identify the gesture currently being performed by the user; Based on the recognized gestures, the corresponding data control commands are determined from the preset mapping relationship; The data control command is executed to perform corresponding data processing operations on the three-dimensional scan data acquired by the scanning device.

2. The scanning data processing control method according to claim 1, characterized in that, In the preset mapping relationship, different gesture actions correspond to at least one of the following data control commands: switch scanning mode, start data acquisition, pause data acquisition, rotate model, zoom view, end data acquisition; The switching scan mode is used to switch between scanning marker point mode and scanning data mode; the rotation model includes left and right rotation and / or up and down rotation; and the zoom view includes magnified view and / or zoomed-out view.

3. The scanning data processing control method according to claim 1, characterized in that, The step of performing gesture recognition on the hand image data includes: Hand detection is performed on the hand image data to obtain a hand region image; The hand region image is input into the hand keypoint network to obtain the keypoint coordinates of each joint of the hand; Gesture recognition is performed based on the coordinates of the key points to identify the gesture action currently being performed by the user.

4. The scanning data processing control method according to claim 3, characterized in that, The gestures include static gestures and dynamic gestures; the gesture recognition based on the key point coordinates includes: Based on the coordinates of the key points, calculate the distance features between each key point and the angle features at each joint. For static gestures, the gesture category is identified based on the distance and angle features of the current single-frame image; For dynamic actions, a sliding window mechanism is used to cache the coordinates of key points in multiple consecutive frames to form a sequence of key point coordinate changes, and the motion speed characteristics of key points between consecutive frames are calculated based on the sequence of key point coordinate changes. The action category is identified based on the sequence of key point coordinate changes and the motion speed characteristics.

5. The scanning data processing control method according to claim 4, characterized in that, Before calculating the distance features between key points and the angle features at each joint based on the key point coordinates, the method further includes: Using the wrist key points as a reference, the coordinates of the key points are translated and normalized to eliminate translation deviations; Using the size of a palm as a reference, the coordinates of key points are normalized to eliminate scale deviation. Using the palm direction as a reference, the coordinates of the key points are rotated and normalized to eliminate rotational deviations.

6. The scanning data processing control method according to claim 4, characterized in that, The method further includes: When a static gesture category is identified based on the distance and angle features of the current single-frame image, and a dynamic action category is identified based on the key point coordinate change sequence of multiple consecutive frames, the dynamic action category is preferentially used as the final identification result.

7. The scanning data processing control method according to claim 3, characterized in that, The process of performing hand detection on the hand image data to obtain a hand region image includes at least one of the following methods: The hand image data is input into the hand detection network to obtain a hand region image; or It uses a tracking algorithm to predict the hand region in the current frame image based on the position of the hand region in the previous frame image, and obtains the hand region image; Specifically, the detection process based on the hand detection network is re-executed every preset number of frames or when the tracking confidence is lower than a preset threshold, in order to correct the hand region in the current frame image.

8. The scanning data processing control method according to any one of claims 1 to 7, characterized in that, The method further includes: When the data control command is executed, a visual feedback of the 3D scan data is simultaneously presented on the display interface; wherein: When a data control command to switch scanning modes is executed, the scanning mode indicator in the display interface switches between scanning marker point mode and scanning data mode, and the spatial distribution status of the scanned marker points or the status of the acquired three-dimensional data model are displayed on the display interface accordingly. When a data control command to start or pause data acquisition is executed, the three-dimensional data model in the display interface is gradually built or paused as the scanned data is written, and the three-dimensional data model is updated in real time in the display interface. When the data control command for rotating the model is executed, the three-dimensional data model already constructed in the display interface rotates synchronously with the direction of the gesture. When the data control command to zoom the view is executed, the view scale of the three-dimensional data model in the display interface is synchronously enlarged or reduced as two fingers are spread or contracted. When the data control command to end data acquisition is executed, the three-dimensional data model in the display interface is confirmed as the final scan result and presented in the center of the display interface.

9. A scanning data processing control device, characterized in that, The device includes: The image acquisition module is used to acquire hand image data acquired by the image acquisition device; The gesture recognition module is used to perform gesture recognition on the hand image data and identify the gesture action currently being performed by the user. The instruction parsing module is used to determine the corresponding data control instruction from the preset mapping relationship based on the recognized gesture action; The data processing module is used to execute the data control instructions to perform corresponding data processing operations on the three-dimensional scanning data acquired by the scanning device.

10. A three-dimensional scanning system, characterized in that, include: Scanning equipment used to acquire 3D scanning data; Image acquisition device used to acquire hand image data; A processing device for the scanning data processing control method as described in any one of claims 1-8.