Scanning data display method, mobile device, three-dimensional scanning system and storage medium

Abstract

The invention relates to a scanning data display method, mobile equipment, a three-dimensional scanning system and a storage medium, and the method comprises the steps: transmitting a scanning command to a scanning execution end through the mobile equipment, so as to enable the scanning execution end to carry out the three-dimensional scanning of a scanning object; and the mobile device receives a scanning data stream transmitted by the scanning execution end in the process of executing the three-dimensional scanning, obtains display data according to the scanning data stream, and displays the display data. Based on the real-time communication interaction between the mobile device and the scanning execution end, the scanning data can be displayed through the mobile device, so that a remote display mechanism of the scanning data is provided for an operator, and the scanning convenience and the scanning efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of 3D scanning, and in particular to scanning data display methods, mobile devices, 3D scanning systems, and storage media. Background Technology

[0002] 3D scanning technology is a technique that uses optical, laser, structured light, or other sensing methods to quickly and accurately acquire three-dimensional geometric data of an object's surface. In current 3D scanning operations, operators need to control the scanning equipment to perform a 3D scan of the object and monitor the scanning progress through the scan data displayed on a computer screen to ensure that all scanned areas of the object are effectively acquired.

[0003] Due to spatial limitations in the scanning environment, operators sometimes need to move around the object being scanned. When the operator's viewing distance from the computer screen is far, it becomes difficult to quickly identify the scan data displayed on the screen. Therefore, to monitor the scanning progress in real time and confirm whether the object has been completely scanned, operators must frequently interrupt the scanning process to check the distant computer screen. This affects scanning continuity and increases the risk of misjudging missed areas due to visual fatigue, resulting in low scanning convenience and efficiency.

[0004] There is currently no effective solution to the problems of low scanning convenience and efficiency in related technologies. Summary of the Invention

[0005] This embodiment provides a scanning data display method, a mobile device, a 3D scanning system, and a storage medium to address the problems of low scanning convenience and efficiency in related technologies.

[0006] Firstly, this embodiment provides a scanning data display method for a mobile device, the mobile device being remotely connected to a scanning execution terminal, the method comprising:

[0007] Send a scan command to the scan execution terminal so that the scan execution terminal performs a three-dimensional scan on the scanned object;

[0008] The system receives the scanning data stream transmitted by the scanning execution end during the 3D scanning process, obtains display data based on the scanning data stream, and displays the display data.

[0009] In some embodiments, the scanning execution end includes an intermediate device and a scanning device, the intermediate device being communicatively connected to the scanning device; sending a scanning command to the scanning execution end to cause the scanning execution end to perform a three-dimensional scan of the object to be scanned includes:

[0010] A scan command is sent to the intermediate device, so that the intermediate device sends a scan command to the scanning device.

[0011] In some embodiments, the intermediate device includes a first processing module and a second processing module; the first processing module can call a first target graphics card, and the second processing module can call a second target graphics card;

[0012] The process of receiving the scan data stream transmitted by the scanning execution end during the execution of a 3D scan, obtaining display data based on the scan data stream, and displaying the display data includes:

[0013] The system receives a scan data stream obtained by encoding an image frame sequence by the second target graphics card, and decodes the scan data stream to obtain display data; wherein the image frame sequence is obtained by rendering point cloud data sent in real time by the first target graphics card from the scanning device.

[0014] In some embodiments, establishing a remote communication connection with the scanning execution terminal includes:

[0015] A search signal is initiated within a preset communication range, and a list of connectable devices is generated based on the search results; the list of connectable devices includes the communication information of the intermediate device;

[0016] In response to a user's selection of the intermediate device from the list of connectable devices, a remote communication connection is established with the intermediate device;

[0017] Receive the list of scanning devices sent by the intermediate device and determine the associated scanning devices.

[0018] In some embodiments, the scanning execution end is a scanning device;

[0019] Sending a scan command to the scan execution terminal to enable the scan execution terminal to perform a three-dimensional scan of the scanned object includes:

[0020] A scanning command is sent to the scanning device to enable the scanning device to perform a three-dimensional scan of the object to be scanned.

[0021] In some embodiments, the scan data stream is a point cloud data frame transmitted in real time by the scanning device; receiving the scan data stream transmitted by the scanning execution end during the execution of a 3D scan, obtaining display data based on the scan data stream, and displaying the display data includes:

[0022] The point cloud data frame is rendered to obtain display data, and the display data is then displayed.

[0023] In some embodiments, establishing a remote communication connection with the scanning execution terminal includes:

[0024] Initiate a search signal within the preset communication range;

[0025] The system receives response signals from each communication device within the communication range in response to the search signal, and generates a list of connectable devices; each of the communication devices includes the scanning device.

[0026] In response to the user's selection of the scanning device from the list of connectable devices, a remote communication connection is established with the scanning device.

[0027] In some embodiments, the method further includes:

[0028] In response to a user's display control command for the display data, the display data is controlled for display; the display control command includes at least one of the following: adjusting the display position of the display data, adjusting the display angle of the display data, and adjusting the display size of the display data.

[0029] Secondly, this embodiment provides a scanning data display method for a scanning execution terminal, which is remotely connected to a mobile device; the method includes:

[0030] Upon receiving a scanning command from a mobile device, a 3D scan is performed on the object to be scanned, resulting in a scan data stream.

[0031] The scanning data stream is transmitted to the mobile device in real time so that the mobile device can obtain display data based on the scanning data stream and display the display data.

[0032] Thirdly, this embodiment provides a mobile device that is remotely connected to the scanning execution terminal; the mobile device includes: a command sending module and a data receiving module; wherein:

[0033] The command sending module is used to send a scanning command to the scanning execution terminal so that the scanning execution terminal performs a three-dimensional scan on the object being scanned.

[0034] The data receiving module is used to receive the scanning data stream transmitted by the scanning execution end during the three-dimensional scanning process, obtain display data based on the scanning data stream, and display the display data.

[0035] Fourthly, this embodiment provides a three-dimensional scanning system, including: a mobile device and a scanning execution terminal; wherein:

[0036] The mobile device and the scanning execution terminal are connected via remote communication.

[0037] The mobile device is used to send scan commands to the scan execution terminal;

[0038] The scanning execution terminal is used to respond to the scanning command sent by the mobile device, perform a three-dimensional scan on the object to be scanned to obtain a scan data stream, and send the scan data stream to the mobile device;

[0039] The mobile device is also used to obtain display data based on the scan data stream and to display the display data.

[0040] In some embodiments, the 3D scanning system further includes several computing nodes; each computing node is pre-configured with different computing tasks; the scanning execution terminal and the mobile device are communicatively connected to at least a portion of the computing nodes.

[0041] Fifthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the scanning data display method described in the first aspect or the second aspect above.

[0042] Compared with related technologies, this embodiment provides a scanning data display method, a mobile device, a 3D scanning system, and a storage medium. The scanning data display method utilizes a mobile device to send a scanning command to a scanning execution terminal, enabling the terminal to perform a 3D scan of the object. The mobile device receives the scanning data stream transmitted by the execution terminal during the 3D scan, obtains display data based on the data stream, and displays the displayed data. This allows for real-time communication and interaction between the mobile device and the scanning execution terminal, providing operators with a remote display mechanism for scanning data, thus improving scanning convenience and efficiency.

[0043] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1 This is a hardware structure block diagram of the terminal for the scanning data display method according to an embodiment of this application;

[0046] Figure 2This is a flowchart of the scanning data display method according to an embodiment of this application;

[0047] Figure 3 These are timing diagrams showing scan data from some embodiments of this application;

[0048] Figure 4 This is another timing diagram of scan data display in some embodiments of this application;

[0049] Figure 5 This is a flowchart illustrating yet another scanning data display method according to an embodiment of this application;

[0050] Figure 6 This is a structural block diagram of a mobile device according to an embodiment of this application;

[0051] Figure 7 This is a schematic diagram of the structure of a three-dimensional scanning system according to an embodiment of this application. Detailed Implementation

[0052] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0053] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0054] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the scanning data display method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0055] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the scanning data display method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0056] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0057] This embodiment provides a scanning data display method for a mobile device, which is remotely connected to the scanning execution terminal; Figure 2 This is a flowchart of the scanning data display method in this embodiment, as shown below. Figure 2 As shown, the process includes the following steps:

[0058] Step S210: Send a scanning command to the scanning execution end so that the scanning execution end can perform a three-dimensional scan on the object to be scanned.

[0059] The mobile device can be an electronic device with data processing, data display, and communication capabilities, and is easy to carry or wear. For example, the mobile device can be a mobile phone, tablet, virtual reality (AR) glasses, etc. Specifically, based on the scanning data display method, a corresponding application can be implemented using a computer program, and this application can run within the mobile device to enable the mobile device to implement the scanning data display method. Understandably, the remote communication between the mobile device and the scanning execution end can specifically be remote communication between the application within the mobile device and the scanning execution end.

[0060] The scanning execution end can specifically be a device or a combination of devices capable of performing three-dimensional scanning of the object being scanned. For example, the scanning execution end can be a scanning device or a combination of a scanning device and other electronic devices. In some embodiments, the scanning execution end can include a scanning device and an intermediate device; the intermediate device can specifically be an electronic device capable of communicating with the scanning device and performing functions such as rendering and encoding the scanning data acquired by the scanning device.

[0061] The remote communication connection between the mobile device and the scanning execution terminal can be a direct remote communication connection between the mobile device and the scanning device, or a remote communication connection between the mobile device and an intermediate device (thus achieving an indirect remote communication connection with the scanning device). After establishing a remote communication connection between the mobile device and the scanning execution terminal, the mobile device can send a scanning command to the scanning execution terminal based on user instructions (e.g., the user clicks a button on the mobile device's display interface, or makes a trigger gesture on the mobile device), or it can be triggered according to a preset timed task, or when the external scanning environment meets certain conditions. That is, the timing and conditions for the mobile device to send a scanning command to the scanning execution terminal can be set according to the needs of the actual application scenario, and this embodiment does not impose specific limitations on this. In addition, the mobile device can establish a remote communication connection with one or more scanning execution terminals and send scanning commands to one or more scanning execution terminals. This remote communication connection includes, but is not limited to, transmission based on standard wireless protocols (Wi-Fi, Bluetooth, etc.), message queues, and real-time streaming media transmission.

[0062] Upon receiving a scan command, the scanning execution unit begins performing a 3D scan and acquires the scan data detected during the process. This scan data can specifically be point cloud data that characterizes the 3D features of the scanned object's surface. The scanning execution unit can either send each frame of point cloud data acquired during the scan directly to the mobile device as a scan data stream, or it can process the acquired raw point cloud data and then generate a scan data stream to send to the mobile device.

[0063] Step S220: Receive the scanning data stream transmitted by the scanning execution end during the 3D scanning process, obtain display data based on the scanning data stream, and display the display data.

[0064] The scanning execution end sends network packets or network data to the mobile device via a remote communication connection. The mobile device parses the received network packets or network data to receive the scan data stream. For example, when the scanning execution end includes an intermediate device and a scanning device, the intermediate device can fuse and render the point cloud data frames collected by the scanning device into an image frame sequence, and then encode the image frame sequence to obtain the scan data stream. The mobile device obtains the scan data stream by pulling the stream from the intermediate device. This enables the transmission of the scan data stream. Alternatively, when the scanning execution end is a scanning device, the scanning device can directly transmit the collected point cloud data frames to the mobile device via a direct remote communication connection with the mobile device.

[0065] After receiving a scan data stream, the mobile device can convert it into display data and then display it on the interface. Specifically, if the received data is a rendered and encoded scan data stream, it can decode the data and display it on the interface to show the user the 3D scan model corresponding to the scanned object. If the received data is a point cloud data frame, the mobile device performs fusion and rendering processes on the point cloud data frame to generate the corresponding 3D scan model, which is then presented on the interface.

[0066] Users can judge the scanning progress of the object on the display interface based on the completeness of the generated 3D scan model (e.g., whether there are large areas of missing data, holes, and whether the main viewpoints are complete), whether it contains the scanned details of interest, and the degree of reconstruction of the scanned object. Users can directly carry mobile devices and adjust the scanning process of the scanning execution end in real time based on the scanning progress reflected in the data displayed on the mobile device, either through the mobile device control or direct user control. For example, they can control the scanning viewpoint and scanning pace of the scanning execution end through communication interaction.

[0067] In related technologies, operators often need to control the scanning equipment to perform 3D scanning of the object and monitor the scanning progress through the scanning data displayed on the computer screen to ensure that all scanned areas of the object are effectively acquired. When the operator's viewing distance from the computer screen is far, the operator needs to move back and forth between the scanning work area and the area where the computer screen is located, resulting in low scanning continuity and low scanning convenience and efficiency.

[0068] In this embodiment, through steps S210 to S220, based on remote communication between the mobile device and the scanning execution end, the scanning results are displayed in real time on the mobile device. This breaks through the traditional limitations of space, hardware, and interaction, allowing operators to carry the mobile device with them and adjust the scanning process at any time based on the scanning progress displayed on the mobile device, achieving a convenient "walk-and-see" scanning method. Furthermore, based on the remote display of the scanning progress on the mobile device, multiple people can use different mobile devices to view the same scanning progress, thereby improving scanning efficiency in multi-person collaborative scanning scenarios. Specifically, in some embodiments, environmental data can be fed back to the scanning execution end based on sensors built into the mobile device (e.g., gyroscope, photometer) to dynamically adjust scanning parameters and optimize scanning effects (e.g., automatic supplemental lighting when insufficient light). In addition, a lightweight artificial intelligence (AI) model can be integrated into the mobile device to analyze scanning quality in real time and guide the scanning process.

[0069] Through steps S210 to S220, a scanning command is sent from the mobile device to the scanning execution terminal, enabling the scanning execution terminal to perform a 3D scan of the object. The mobile device receives the scanning data stream transmitted by the scanning execution terminal during the 3D scanning process, obtains display data based on the scanning data stream, and displays the display data. This allows for real-time communication and interaction between the mobile device and the scanning execution terminal, enabling the display of scan data via the mobile device, thus providing operators with a remote display mechanism for scan data and improving scanning convenience and efficiency.

[0070] In one embodiment, the scanning execution end includes an intermediate device and a scanning device, with the intermediate device communicatively connected to the scanning device; based on the above step S210, a scanning command is sent to the scanning execution end to enable the scanning execution end to perform a three-dimensional scan of the object to be scanned, which may specifically include:

[0071] Send a scan command to the intermediate device, so that the intermediate device sends a scan command to the scanning device.

[0072] The intermediate device can be a computer device capable of fusing, rendering, and encoding point cloud data, and possessing communication capabilities. The scanning device can be of different types of 3D scanning equipment. The intermediate device can support the connection of multiple types of 3D scanning devices, such as handheld laser scanners and tracking scanners, and supports hot-swappable connections. The intermediate device connects to both the scanning device and the mobile device. It can connect to the scanning device via wired or wireless connections.

[0073] In this process, the mobile device can send a signal carrying a scanning command to the intermediate device through a communication connection. The intermediate device receives the signal sent by the mobile device, parses out the scanning command from it, and then sends the scanning command to the scanning device so that the scanning device can start performing a three-dimensional scan of the object to be scanned.

[0074] This embodiment enables 3D scanning to be initiated based on communication between the mobile device and the intermediate device, thereby achieving control over the 3D scanning process. This allows users to remotely initiate 3D scanning as needed, further improving the convenience of 3D scanning process control.

[0075] In one embodiment, the intermediate device includes a first processing module and a second processing module; the first processing module can call a first target graphics card, and the second processing module can call a second target graphics card; based on the above step S220, it receives the scanning data stream transmitted by the scanning execution end during the execution of 3D scanning, obtains display data according to the scanning data stream, and displays the display data, which may specifically include:

[0076] The system receives a scan data stream obtained by encoding an image frame sequence using a second target graphics card, and decodes the scan data stream to obtain display data. The image frame sequence is obtained by rendering point cloud data sent in real time by a first target graphics card.

[0077] Different graphics cards can be configured within the intermediate device to handle different computing tasks. Specifically, the scanning object is scanned by a scanning device, and the point cloud data obtained from the real-time scanning is input into the first processing module of the intermediate device.

[0078] The first processing module of the intermediate device calls the first target graphics card to render the point cloud data of the target object. The first target graphics card can be an NVIDIA discrete graphics card, or other graphics cards with rendering capabilities, such as AMD graphics cards, can be used depending on the actual application requirements; no specific limitation is made here. In the first target graphics card, the received point cloud data can be preprocessed, including but not limited to denoising and point cloud calibration. The preprocessed point cloud data is then input into the rendering engine, and processed through the rendering pipeline, including lighting calculation and projection transformation, to generate an image frame sequence with a specific preset frame rate. The image frame sequence includes multiple RGB image frames. The rendering engine includes, but is not limited to, OpenGL, Direct3D, and Vulkan.

[0079] Next, the second processing module calls the second target graphics card to encode the image frame sequence. The second target graphics card is preferably an Intel integrated graphics card, but other graphics cards with video encoding capabilities, such as AMD graphics cards, can be used depending on the actual application requirements; no specific limitation is made here. In the second target graphics card, a hardware-accelerated encoder is used for core compression processing, such as H.265 hardware encoding or AV1 hardware encoding.

[0080] Specifically, in some embodiments, during the encoding process, the content complexity of image frames (such as motion intensity and texture details) and the state of the encoding buffer can be continuously analyzed. Based on the real-time analysis results, key settings such as quantization parameters are dynamically adjusted to achieve precise control of the output bitrate through dynamic bitrate control, balancing bandwidth and image quality requirements. It should be noted that before encoding or during the encoding loop, color space optimization can be performed on image frames to improve the compression efficiency of color information and enhance the subjective quality of the reconstructed image. Finally, the efficiently compressed data is encapsulated to generate a standardized scan data stream. After encoding is complete, the scan data stream can be pushed to the streaming module of an intermediate device, allowing mobile devices to request and pull the stream. For example, the mobile device sends a pull command to the intermediate device, and the intermediate device returns the corresponding scan data stream to the mobile device. The streaming module can achieve stable and reliable transmission of the scan data stream through dynamic load balancing and multi-protocol adaptation.

[0081] It should be further explained that multiple pairs of first and second processing modules can be set up according to actual needs. The parallel operation of multiple processing modules can be managed through a collaborative scheduling mechanism, thereby significantly improving system performance. This is suitable for scenarios with high concurrency and high real-time requirements.

[0082] In existing multimedia applications, graphics rendering and video encoding are typically integrated into the same process. However, when this method is applied to 3D scanning scenarios, it usually requires large-scale graphics rendering or inference tasks, leading to resource contention between rendering and encoding tasks, reduced overall performance of rendering and encoding, and difficulty in meeting the requirements for real-time processing and high-quality output.

[0083] Compared to existing technologies, this embodiment separates the rendering and encoding tasks of point cloud data, avoiding the mutual performance impact between the two tasks, thus making it more suitable for 3D scanning systems. Specifically, this embodiment enables rendering and video encoding to be performed in different processes, solving the problem of resource contention between rendering and encoding tasks, which leads to a decrease in overall rendering and encoding performance and makes it difficult to meet the requirements of real-time processing and high-quality output. By avoiding resource contention between rendering and encoding tasks, it avoids frame rate drops, screen stuttering, encoding failures, etc., thereby improving the overall performance of rendering and encoding to meet the requirements of real-time processing and high-quality output.

[0084] Furthermore, single-process architectures suffer from frequent context switching and complex resource scheduling when performing multi-task processing, making it difficult to achieve task isolation and stable control. This embodiment, however, can also improve system stability by using dual graphics card load isolation, thus facilitating subsequent functional expansion.

[0085] Specifically, compared to existing technologies where 3D scanning progress is typically displayed on a fixed computer, this embodiment introduces data stream pulling technology into the 3D scanning scenario, organically combining data stream transmission with 3D scanning technology, while maintaining communication between the mobile device and an intermediate remote device. First, point cloud fusion and rendering are used to process the original acquired point cloud data into an image frame sequence; then, the image frame sequence is encoded into a scanning data stream; finally, the mobile device uses a pulling method to read and process the scanning data stream from the intermediate device. This embodiment breaks down the barriers between data stream transmission and 3D scanning, thereby enabling real-time and accurate wireless transmission of scanning data.

[0086] In another embodiment, establishing a remote communication connection with the scanning execution end may specifically include:

[0087] Initiate a search signal within a preset communication range, generate a list of connectable devices based on the search results; the list of connectable devices includes the communication information of the intermediate device; in response to the user's selection of the intermediate device from the list of connectable devices, establish a remote communication connection with the intermediate device; receive the scanning device list sent by the intermediate device, and determine the associated scanning device.

[0088] In this system, mobile devices can initiate search signals within their communication range. Communication devices that receive the search signals within the range can send response signals to the mobile devices. Based on the received responses, the mobile devices generate a list of discoverable connectable devices and display it on the screen. Users can then select an intermediate device from the list of connectable devices on the screen to establish a remote communication connection with the intermediate device.

[0089] In other embodiments, provided that the mobile device and the intermediate device are on the same local area network, the mobile device can scan the Internet Protocol (IP) address of the local area network or directly enter the IP address of the intermediate device to search for the intermediate device and connect to it.

[0090] The intermediate device can also establish a connection with the scanning device beforehand. After the connection is established, the scanning device can report its own device information (e.g., device model, unique identifier, firmware version, scanning resolution, data format, etc.) to the intermediate device. Based on the device information of the scanning device, the intermediate device can configure the scanning device's drivers, communication protocols, scanning parameters, etc.

[0091] Furthermore, once a remote communication connection is established with the intermediate device, the intermediate device can synchronize the device information reported by the connected scanning devices to the mobile device. The mobile device can then display the associated scanning devices to the user on its interface. The user can initiate scanning commands to the intermediate device for one or more associated scanning devices. The intermediate device parses the scanning commands sent by the mobile device, determines the scanning device to initiate the scan, and sends the scanning commands to the corresponding scanning device.

[0092] In this embodiment, three-terminal communication interaction between the scanning device, the intermediate device, and the mobile device is realized. The intermediate device, as the medium for interaction between the scanning device and the mobile device, can ensure that the point cloud data collected by the scanning device is accurately and completely displayed on the mobile device. Thus, the remote display of the scanning progress on the mobile device is successfully realized, breaking through the spatial and hardware limitations of the current scanning data display and improving the convenience and efficiency of 3D scanning.

[0093] In another embodiment, the scanning execution end is a scanning device; based on the above step S210, a scanning command is sent to the scanning execution end so that the scanning execution end performs a three-dimensional scan of the scanning object, which may specifically include:

[0094] Send a scan command to the scanning device so that the scanning device can perform a 3D scan of the object to be scanned.

[0095] The mobile device can establish a wireless connection with the scanning device, such as Wi-Fi or Bluetooth. The mobile device and the scanning device are within each other's communication range. The mobile device directly sends a scanning command to the scanning device, which then begins to perform a 3D scan of the object based on this command.

[0096] Specifically, after establishing a communication link between the mobile device and the scanning device, the mobile device can send structured scanning commands to the scanning device, such as starting scanning and specific scanning parameter configurations. The scanning device parses the scanning commands sent by the mobile device, performs hardware initialization of the scanning device, and then begins to perform 3D scanning of the object to acquire point cloud data, which is then transmitted to the mobile device in real time or in batches.

[0097] In this embodiment, direct communication between the mobile device and the scanning device is realized, thereby enabling higher real-time communication between the mobile device and the scanning device, higher stability of data transmission, and higher human-computer interaction efficiency for users to operate and control the scanning device on the mobile device.

[0098] In one embodiment, the scanning data stream is a point cloud data frame transmitted in real time by the scanning device; based on the above step S220, the scanning execution end receives the scanning data stream transmitted during the 3D scanning process, obtains display data based on the scanning data stream, and displays the display data, which may specifically include:

[0099] The point cloud data frame is rendered to obtain display data, and the display data is then displayed.

[0100] In direct communication between a mobile device and a scanning device, the scanning device can directly transmit point cloud data frames to the mobile device, which then processes these frames of point cloud data through rendering. Specifically, the rendering process performed by the mobile device can be the conversion of the received point cloud data into a visualized image. This rendering process may include data preprocessing (denoising and downsampling of the point cloud data), visualization mapping of the point cloud, and post-processing (anti-aliasing, Gaussian blurring, etc.) to form a visible screen pixel image, thus obtaining display data. This display data can specifically be a 3D scanned model of the scanned object. Subsequently, this 3D scanned model is presented to the user on the mobile device's display interface.

[0101] In this embodiment, based on direct communication between the mobile device and the scanning device, the scanning device transmits point cloud data to the mobile device in real time. The mobile device renders and displays the received point cloud data, thereby achieving a data interaction method with higher real-time performance, stronger stability, and higher human-computer interaction efficiency. It also enables remote display of scanned data on the mobile device, thereby improving the convenience and efficiency of 3D scanning.

[0102] In one embodiment, establishing a remote communication connection with the scanning execution end may specifically include: initiating a search signal within a preset communication range; receiving response signals sent by each communication device within the communication range in response to the search signal, and generating a list of connectable devices; each communication device includes a scanning device; and establishing a remote communication connection with the scanning device in response to the user's selection operation of the scanning device from the list of connectable devices.

[0103] Specifically, the mobile device can initiate a search signal within its communication range. Each communication device within this range can then initiate a response signal based on this search signal. Based on the received response signals, the mobile device generates a list of connectable devices for the user on its display interface. The user selects the desired scanning device from this list, and the mobile device establishes a remote communication connection with the corresponding scanning device based on the user's selection. Therefore, this embodiment enables a direct communication connection between the scanning device and the mobile device, thus providing a foundation for the remote transmission of scanning data.

[0104] Furthermore, in one embodiment, the above-described scanning data display method may further include:

[0105] In response to user display control commands for display data, the system performs display control on the display data; the display control commands include at least one of the following: adjusting the display position of the display data, adjusting the display angle of the display data, and adjusting the display size of the display data.

[0106] The mobile device can receive display control commands from the user regarding the displayed data. These commands can take various forms, including but not limited to touch input, physical button presses, gesture recognition, and shaking or tapping the mobile device. The display position, angle, and size of the 3D scanned model on the display interface can be adjusted using these commands to optimize the model's appearance, improve user experience, and increase scanning efficiency.

[0107] The following explanation uses a user's touch operation on the mobile device's display screen as an example. When a user's long press operation for a preset period of time is detected, triggering the drag function of the 3D scanned model, and then the user presses and slides the 3D scanned model on the display screen, the display position of the 3D scanned model on the display screen can be dynamically adjusted in real time according to the distance and direction of the user's finger swiping on the display screen, so as to present the display effect of the 3D scanned model moving on the display screen with the user's finger.

[0108] If the user does not perform a long press operation and directly presses and holds the 3D scanned model on the display interface, the rotation function can be triggered. The display angle of the 3D scanned model on the display interface is dynamically adjusted according to the distance and direction of the user's finger sliding on the display interface. The range of change of the display angle can also be adapted to the distance of the user's finger sliding (for example, the greater the sliding distance, the greater the range of angle change), thus presenting the display effect of the 3D scanned model rotating with the user's finger.

[0109] Furthermore, if the user uses two fingers to swipe outwards in different directions on the display interface, the size of the 3D scanned model on the display interface can be increased; if the user uses two fingers to swipe between the two fingers, the size of the 3D scanned model on the display interface can be decreased. In this way, the size of the 3D scanned model can be increased or decreased according to the user's finger swipes.

[0110] It is understood that the above display control process is merely an example, and different forms of display control commands can be selected according to actual needs. This embodiment optimizes the display effect, improves the user experience, and increases 3D scanning efficiency by responding to the user's display control commands and adjusting the display position, display angle, and display size of the display data.

[0111] Figure 3 Timing diagrams of scan data displayed for some of the embodiments, such as Figure 3 As shown, the mobile device initiates a search signal within its communication range to search for the intermediate device; after completing the search, it establishes a connection with the intermediate device based on the search results. Once the scanning device is online (connected to the intermediate device), it can report its own device information to the intermediate device. The intermediate device synchronizes the scanning device's device information to the mobile device. The intermediate device can also configure scanning parameters, drivers, etc., based on the corresponding device information of the scanning device. The mobile device displays the device information of the associated scanning device to the user on its display interface. The mobile device can send a scanning command to the intermediate device passively or actively. After receiving the scanning command from the mobile device, the intermediate device sends a scanning command to the scanning device. The scanning device begins performing a 3D scan, acquiring point cloud data. The scanning device sends the point cloud data to the intermediate device. The intermediate device includes a first processing module and a second processing module. The first processing module calls a first target graphics card to fuse and render the point cloud data sent by the scanning device in real time to obtain an image frame sequence. The second processing module calls a second target graphics card to encode the image frame sequence to obtain a scan data stream. The mobile device sends a pull command to the intermediate device, and the intermediate device sends the scan data stream to the mobile device. The mobile device decodes and displays the received scan data stream.

[0112] Figure 4This is a timing diagram illustrating another type of scan data display in some embodiments. The mobile device initiates a search signal within its communication range to perform a device search. Based on the search signal initiated by the mobile device, the scanning device returns a response signal to complete the device response. Based on the response signal from the scanning device, the mobile device establishes a connection with the scanning device. Subsequently, the mobile device sends a scan command to the scanning device, either passively or actively triggered. The scanning device performs a 3D scan based on the scan command and sends point cloud data to the mobile device. The mobile device renders the received point cloud data and displays it on the display interface.

[0113] This embodiment also provides another scanning data display method, which is used on a scanning execution terminal that is remotely connected to a mobile device. Figure 5 This is a flowchart of another scanning data display method in this embodiment, such as... Figure 5 As shown, the scan data display method may include:

[0114] Step S501: Upon receiving a scanning command from a mobile device, perform a 3D scan on the object to be scanned to obtain a scan data stream;

[0115] Step S502: Transmit the scan data stream to the mobile device in real time so that the mobile device can obtain display data based on the scan data stream and display the display data.

[0116] Steps S501 to S502 described above enable real-time communication and interaction between the mobile device and the scanning execution terminal, allowing the mobile device to display the scanning data. This provides operators with a remote display mechanism for scanning data, improving scanning convenience and efficiency. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0117] This embodiment also provides a mobile device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0118] Figure 6 This is a structural block diagram of the mobile device 60 in this embodiment, which is remotely connected to the scanning execution end; as shown... Figure 6As shown, the mobile device 60 includes a command sending module 62 and a data receiving module 64; wherein: the command sending module 62 is used to send a scanning command to the scanning execution end so that the scanning execution end performs a three-dimensional scan on the scanning object; the data receiving module 64 is used to receive the scanning data stream transmitted by the scanning execution end during the three-dimensional scanning process, obtain display data according to the scanning data stream, and display the display data.

[0119] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0120] This embodiment also provides a three-dimensional scanning system. Figure 7 This is a schematic diagram of the structure of the three-dimensional scanning system 70 in this embodiment, as shown below. Figure 7 As shown, the 3D scanning system 70 includes: a mobile device 60 and a scanning execution terminal 72; wherein:

[0121] The mobile device 60 and the scanning execution terminal 72 are remotely connected; the mobile device 60 is used to send scanning commands to the scanning execution terminal 72; the scanning execution terminal 72 is used to respond to the scanning commands sent by the mobile device 60, perform a three-dimensional scan on the scanned object to obtain a scan data stream, and send the scan data stream to the mobile device 60; the mobile device 60 is also used to obtain display data based on the scan data stream and display the display data.

[0122] In one embodiment, the 3D scanning system 70 further includes several computing nodes; each computing node is pre-configured with different computing tasks; the scanning execution end and the mobile device are communicatively connected to at least a portion of the computing nodes.

[0123] In this 3D scanning system 70, multiple electronic devices with data processing and communication capabilities can be incorporated as different computing nodes. During the 3D scanning process, these nodes distribute different computational tasks, collaboratively completing the processing, transmission, and final display of the scanned data in a hierarchical manner. In this embodiment, hierarchical processing can involve dividing the computations involved in the 3D scanning process (such as the fusion and rendering of point cloud data) into tasks, assigning each task to a different computing node, with each node executing a single task. For example, when the amount of point cloud data is large, the tasks of receiving, preprocessing, fusing, rendering, encoding, and displaying the point cloud data can be processed hierarchically.

[0124] In this embodiment, by hierarchically processing the computational tasks in the 3D scanning process and configuring different computational tasks for different computing nodes, the problem of poor rendering effect and low data processing efficiency caused by hardware limitations of a single device can be avoided. In this way, the time and computational costs can be distributed among different computing nodes, thereby improving the overall performance of the 3D scanning process.

[0125] Furthermore, in conjunction with the scanning data display method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the scanning data display methods described in the above embodiments.

[0126] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0128] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0129] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method of displaying scan data, characterized by, The method for a mobile device, which is remotely connected with a scanning execution end, comprises: sending a scanning command to the scanning execution end to make the scanning execution end perform a three-dimensional scanning on a scanning object; receiving a scanning data stream transmitted by the scanning execution end in the process of performing the three-dimensional scanning, and obtaining display data according to the scanning data stream, and displaying the display data.

2. The method of scanning data display according to claim 1, wherein, The scanning execution end comprises an intermediate device and a scanning device, and the intermediate device is connected with the scanning device; the step of sending a scanning command to the scanning execution end to make the scanning execution end perform a three-dimensional scanning on a scanning object comprises: sending a scanning command to the intermediate device to make the intermediate device send a scanning command to the scanning device.

3. The method of scanning data display according to claim 2, wherein, The intermediate device comprises a first processing module and a second processing module; the first processing module can call a first target graphics card, and the second processing module can call a second target graphics card; The step of receiving a scanning data stream transmitted by the scanning execution end in the process of performing the three-dimensional scanning, and obtaining display data according to the scanning data stream, and displaying the display data comprises: receiving a scanning data stream obtained by encoding an image frame sequence by the second target graphics card, and decoding the scanning data stream to obtain display data; wherein the image frame sequence is obtained by rendering point cloud data transmitted by the first target graphics card to the scanning device in real time.

4. The method of scanning data display according to claim 2, wherein, The step of establishing a remote communication connection with the scanning execution end comprises: initiating a search signal within a preset communication range, and generating a connectable device list according to a search result; the connectable device list comprises communication information of the intermediate device; in response to a selection operation of the user on the intermediate device from the connectable device list, establishing a remote communication connection with the intermediate device; receiving a scanning device list sent by the intermediate device, and determining an associated scanning device.

5. The method of scanning data display according to claim 1, wherein, The scanning execution end is a scanning device; The step of sending a scanning command to the scanning execution end to make the scanning execution end perform a three-dimensional scanning on a scanning object comprises: sending a scanning command to the scanning device to make the scanning device perform a three-dimensional scanning on a scanning object.

6. The method of scanning data display according to claim 5, wherein, The scanning data stream is a point cloud data frame transmitted by the scanning device in real time; the step of receiving a scanning data stream transmitted by the scanning execution end in the process of performing the three-dimensional scanning, and obtaining display data according to the scanning data stream, and displaying the display data comprises: rendering the point cloud data frame to obtain display data, and displaying the display data.

7. The method of scanning data display according to claim 5, wherein, The step of establishing a remote communication connection with the scanning execution end comprises: initiating a search signal within a preset communication range; receiving a response signal sent by each communication device in the communication range in response to the search signal, and generating a connectable device list; each of the communication devices comprises the scanning device; in response to a selection operation of the user on the scanning device from the connectable device list, establishing a remote communication connection with the scanning device.

8. The scanning data display method according to any one of claims 1 to 7, characterized by, The method further comprises: In response to a display control instruction of the user for the display data, display control is performed on the display data; the display control instruction comprises at least one of the following: adjusting a display position of the display data, adjusting a display angle of the display data, and adjusting a display size of the display data.

9. A method of displaying scan data, characterized by For a scanning execution end, the scanning execution end is in remote communication connection with a mobile device; the method comprises: In a case where a scanning command sent by the mobile device is received, performing three-dimensional scanning on a scanning object to obtain a scanning data stream; Real-time transmission of the scanning data stream to the mobile device, so that the mobile device obtains display data based on the scanning data stream, and displays the display data.

10. A mobile device, comprising: The mobile device is in remote communication connection with a scanning execution end; the mobile device comprises a command sending module and a data receiving module; wherein: The command sending module is configured to send a scanning command to the scanning execution end, so that the scanning execution end performs three-dimensional scanning on a scanning object; The data receiving module is configured to receive a scanning data stream transmitted by the scanning execution end in a process of performing three-dimensional scanning, and obtain display data according to the scanning data stream, and display the display data.

11. A three-dimensional scanning system, characterized by Comprise: A mobile device and a scanning execution end; wherein: The mobile device and the scanning execution end are in remote communication connection; The mobile device is configured to send a scanning command to the scanning execution end; The scanning execution end is configured to perform three-dimensional scanning on a scanning object in response to the scanning command sent by the mobile device to obtain a scanning data stream, and send the scanning data stream to the mobile device; The mobile device is further configured to obtain display data according to the scanning data stream, and display the display data.

12. The three-dimensional scanning system of claim 11, wherein, The three-dimensional scanning system further comprises a plurality of computing nodes; each computing node is pre-configured with a different computing task; the scanning execution end and the mobile device are in communication connection with at least part of all the computing nodes.

13. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by a processor to realize the steps of the scanning data display method in any one of claims 1 to 8, or to realize the steps of the scanning data display method in claim 9.

Images