Rendering plug flow method and system based on point cloud data and storage medium

Abstract

The invention relates to a rendering plug flow method and system based on point cloud data and a storage medium, and the method comprises the steps: carrying out the rendering of the point cloud data obtained in real time in a first target video card, and obtaining a corresponding image frame sequence; transmitting the image frame sequence to a second target display card; and in the second target video card, encoding the image frame sequence to obtain a corresponding encoded video stream for the target terminal to obtain. According to the rendering and encoding method and device, the problems that the overall performance of rendering and encoding is greatly reduced and the requirements for real-time processing and high-quality output are difficult to meet due to the fact that the rendering and encoding tasks have serious resource competition are solved, resource competition between the rendering and encoding tasks is avoided, the overall performance of rendering and encoding is improved, and the rendering and encoding efficiency is improved. And the requirements of real-time processing and high-quality output are met.

Description

Technical Field

[0001] This application relates to the field of 3D scanning technology, and in particular to rendering and streaming methods, systems and storage media based on point cloud data. Background Technology

[0002] In existing multimedia applications, graphics rendering and video encoding are integrated into the same process. However, when this method is applied to 3D scanning scenarios, large-scale graphics rendering or inference tasks are usually required, leading to severe resource contention between rendering and encoding tasks. This significantly reduces the overall performance of rendering and encoding, making it difficult to meet the requirements of real-time processing and high-quality output.

[0003] There is currently no effective solution to the problem that severe resource contention exists between rendering and encoding tasks in related technologies, leading to a significant reduction in overall rendering and encoding performance and making it difficult to meet the requirements of real-time processing and high-quality output. Summary of the Invention

[0004] This embodiment provides a rendering and streaming method, system, and storage medium based on point cloud data to solve the problem in related technologies where severe resource competition between rendering and encoding tasks leads to a significant reduction in the overall performance of rendering and encoding, making it difficult to meet the requirements of real-time processing and high-quality output.

[0005] Firstly, this embodiment provides a rendering and streaming method based on point cloud data, applicable to a 3D scanning system; the 3D scanning system includes a processing end and a target terminal; the processing end includes a first processing end and a second processing end, the first processing end can call a first target graphics card, and the second processing end can call a second target graphics card; the method is applicable to the processing end side, and the method includes:

[0006] In the first target graphics card, the real-time acquired point cloud data is rendered to obtain the corresponding image frame sequence;

[0007] The image frame sequence is transmitted to the second target graphics card; in the second target graphics card, the image frame sequence is encoded to obtain the corresponding encoded video stream for the target terminal to acquire.

[0008] In some embodiments, transmitting the image frame sequence to the second target graphics card includes:

[0009] Write the image frame sequence output by the first target graphics card into a preset shared memory;

[0010] Each rendered image in the image frame sequence is read from the preset shared memory, and each rendered image is transmitted to the second target graphics card.

[0011] In some embodiments, reading each rendered image from the image frame sequence from the preset shared memory and transmitting each rendered image to the second target graphics card includes:

[0012] The second processing unit retrieves the rendered image from the preset shared memory in real time.

[0013] In some embodiments, the method further includes:

[0014] In the first target graphics card, the point cloud data acquired in real time is rendered to obtain the corresponding three-dimensional view;

[0015] The 3D view is transmitted to the front-end module of the 3D scanning system for real-time display.

[0016] In some embodiments, the method further includes:

[0017] Based on user operation commands input through the target terminal, the real-time display status of the 3D view in the front-end module is controlled.

[0018] In some embodiments, the method further includes:

[0019] When an encoding anomaly is detected in the second target graphics card, the system switches to software encoding mode to process the image frame sequence and obtains the encoded video stream.

[0020] In some embodiments, the data interaction between the first processing end and the second processing end adopts an event-driven asynchronous architecture for multi-process communication.

[0021] Secondly, this embodiment provides a three-dimensional scanning system, which includes a processing terminal and a target terminal; the processing terminal includes a first processing terminal and a second processing terminal; the first processing terminal can call a first target graphics card, and the second processing terminal can call a second target graphics card;

[0022] The processing terminal is used to execute the rendering and streaming method based on point cloud data described in the first aspect above.

[0023] The target terminal is used to request the processing terminal to obtain the encoded video stream generated by the processing terminal.

[0024] In some embodiments, the system further includes a three-dimensional scanning device;

[0025] The 3D scanning device is used to input the point cloud data acquired in real time into the first processing terminal.

[0026] Thirdly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the point cloud data-based rendering and streaming method described in the first aspect above.

[0027] Compared with related technologies, the rendering and streaming method, system, and storage medium based on point cloud data provided in this embodiment are applicable to 3D scanning systems. The 3D scanning system includes a processing end and a target terminal. The processing end includes a first processing end and a second processing end. The first processing end can call a first target graphics card, and the second processing end can call a second target graphics card. The method renders the real-time acquired point cloud data in the first target graphics card to obtain a corresponding image frame sequence; the image frame sequence is then transmitted to the second target graphics card; and the image frame sequence is encoded in the second target graphics card to obtain a corresponding encoded video stream for the target terminal to acquire. This solves the problem of severe resource contention between rendering and encoding tasks, which leads to a significant reduction in the overall performance of rendering and encoding, making it difficult to meet the requirements of real-time processing and high-quality output. It avoids resource contention between rendering and encoding tasks, improves the overall performance of rendering and encoding, and meets the requirements of real-time processing and high-quality output.

[0028] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 This is a hardware structure block diagram of a terminal device for a point cloud data-based rendering and streaming method provided in an embodiment of this application;

[0031] Figure 2 This is a flowchart of a rendering and streaming method based on point cloud data provided in an embodiment of this application;

[0032] Figure 3 This is a schematic diagram illustrating the interaction between the scanning main process and the encoding push stream process according to an embodiment of this application;

[0033] Figure 4 This is a flowchart illustrating a point cloud data-based rendering and streaming method according to an embodiment of this application.

[0034] Figure 5 This is a flowchart of a preferred embodiment of the rendering and streaming method based on point cloud data provided in this application;

[0035] Figure 6This is a structural block diagram of a three-dimensional scanning system provided in an embodiment of this application.

[0036] In the diagram: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 10, processing terminal; 20, target terminal; 30, 3D scanning device; 40, front-end module. Detailed Implementation

[0037] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0038] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0039] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the point cloud data-based rendering and streaming method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0040] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the point cloud data-based rendering and streaming method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0042] This embodiment provides a rendering and streaming method based on point cloud data, applicable to a 3D scanning system. The system includes a processing end and a target terminal. The processing end includes a first processing end and a second processing end. The first processing end can call a first target graphics card, and the second processing end can call a second target graphics card. Figure 2 This is a flowchart of the rendering and streaming method based on point cloud data in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0043] Step S210: In the first target graphics card, the real-time acquired point cloud data is rendered to obtain the corresponding image frame sequence.

[0044] Specifically, the target object is scanned using a 3D scanning device, and the point cloud data obtained from the real-time scan is input into the processing terminal. In this embodiment, multiple types of 3D scanning devices are supported for connection, including handheld laser scanners and tracking scanners, and hot-swappable device connections are also supported.

[0045] The first processing unit calls the first target graphics card to render the point cloud data of the target object. The first target graphics card is preferably an NVIDIA discrete graphics card, but other graphics cards with rendering capabilities, such as AMD graphics cards, can be used according to actual application requirements. No specific limitation is made here.

[0046] In the first target graphics card, the received point cloud data is preprocessed, including denoising and point cloud calibration. The preprocessed point cloud data is then input into the rendering engine, where it undergoes rendering pipeline processing such as lighting calculations and projection transformations to generate an image frame sequence with a specific preset frame rate. This image frame sequence includes multiple RGB image frames. The rendering engine includes, but is not limited to, OpenGL, Direct3D, and Vulkan.

[0047] Step S220: The image frame sequence is transmitted to the second target graphics card; in the second target graphics card, the image frame sequence is encoded to obtain the corresponding encoded video stream for the target terminal to acquire.

[0048] In this embodiment, the second processing terminal calls the second target graphics card to encode the image frame sequence. The second target graphics card is preferably an Intel integrated graphics card, but other graphics cards with video encoding capabilities, such as AMD graphics cards, can be used depending on the actual application requirements; no specific limitation is made here. In the second target graphics card, the encoding module uses a hardware-accelerated encoder for core compression processing, such as H.265 hardware encoding or AV1 hardware encoding.

[0049] Specifically, during the encoding process, the complexity of image frame content (such as motion intensity and texture detail) and the state of the encoding buffer are continuously analyzed. Based on real-time analysis results, key settings such as quantization parameters are dynamically adjusted to achieve precise control of the output bitrate through dynamic bitrate control, balancing bandwidth and image quality requirements. It should be noted that before encoding or within the encoding loop, color space optimization can be performed on image frames to improve the compression efficiency of color information and enhance the subjective quality of the reconstructed image. Finally, the efficiently compressed data is encapsulated to generate a standardized encoded video stream, which is then pushed to the streaming module for the target terminal's application (APP) to request. The target terminal can be a mobile phone, tablet, smart wearable device, or other mobile device. For example, a video stream request message is sent from the mobile APP to the second processing end. After parsing the request, the second processing end returns the corresponding video stream data to the mobile phone via the streaming module for decoding and playback.

[0050] The streaming module ensures stable reception and processing of the encoded video stream by using dynamic load balancing and multi-protocol adaptation. It also enables server maintenance, ensuring stable server operation and efficient resource scheduling. Furthermore, it manages the entire lifecycle of client connections through client session management, maintaining orderly connections between clients and servers. Adaptive packet loss compensation is employed to automatically activate forward error correction, data retransmission, or bitrate adjustment strategies when network packet loss or jitter occurs, compensating for data loss and optimizing transmission efficiency. Ultimately, this achieves stable and reliable transmission of the encoded video stream.

[0051] It should be further explained that multiple pairs of first and second processing ends can be set up according to actual needs. The parallel operation of multiple processing ends can be managed through a collaborative scheduling mechanism, thereby significantly improving system performance. This is suitable for scenarios with high concurrency and high real-time requirements.

[0052] In existing multimedia applications, graphics rendering and video encoding are integrated into the same process. However, when this method is applied to 3D scanning scenarios, large-scale graphics rendering or inference tasks are usually required, leading to severe resource contention between rendering and encoding tasks. This significantly reduces the overall performance of rendering and encoding, making it difficult to meet the requirements of real-time processing and high-quality output.

[0053] Compared to existing technologies, this application is applicable to 3D scanning systems. The 3D scanning system includes a processing end and a target terminal. The processing end includes a first processing end and a second processing end. The first processing end can call a first target graphics card, and the second processing end can call a second target graphics card. In the first target graphics card, real-time acquired point cloud data is rendered to obtain a corresponding image frame sequence. The image frame sequence is then transmitted to the second target graphics card. In the second target graphics card, the image frame sequence is encoded to obtain a corresponding encoded video stream for the target terminal to acquire. Based on this, by having the first processing end call the first target graphics card to perform the rendering task, and by having the second processing end call the second target graphics card to perform the video encoding task, rendering and video encoding can be achieved in different processes. This solves the problem of severe resource contention between rendering and encoding tasks, which leads to a significant decrease in overall rendering and encoding performance, making it difficult to meet the requirements of real-time processing and high-quality output. It avoids resource contention between rendering and encoding tasks, thereby preventing frame rate drops, screen stuttering, encoding failures, etc., and significantly improving the overall performance of rendering and encoding to meet the requirements of real-time processing and high-quality output.

[0054] Furthermore, single-process architectures suffer from frequent context switching and complex resource scheduling when performing multi-task processing, making it difficult to achieve task isolation and stable control. This application improves system stability by using dual graphics card load isolation, which facilitates subsequent functional expansion.

[0055] In some embodiments, the step S220 of transmitting the image frame sequence to the second target graphics card includes the following steps:

[0056] Step S221: Write the image frame sequence output by the first target graphics card into the preset shared memory;

[0057] Step S222: Read each rendered image from the preset shared memory in the image frame sequence and transfer each rendered image to the second target graphics card.

[0058] In this embodiment, a shared memory is preset between the first processing end and the second processing end, allowing the rendering and encoding processes to directly access the same memory area, thereby enabling the rendering and encoding processes to interact with each other through the preset shared memory.

[0059] Specifically, once the first target graphics card finishes rendering, the image frame sequence output by the first target graphics card is written to a preset shared memory, and each rendered image in the image frame sequence is read from the preset shared memory. Each rendered image is then transmitted to the second target graphics card to execute subsequent encoding processes. In other embodiments, precoding buffering technology can be further incorporated to effectively reduce end-to-end transmission latency, typically controlling the latency within 80ms, achieving ultra-low latency.

[0060] In this embodiment, the image frame sequence output by the first target graphics card is written into a preset shared memory, each rendered image in the image frame sequence is read from the preset shared memory, and each rendered image is transmitted to the second target graphics card, thereby reducing data transmission latency and achieving high-speed communication.

[0061] In some embodiments, step S222, which involves reading each rendered image from a preset shared memory sequence and transmitting each rendered image to a second target graphics card, includes the following steps:

[0062] The second processing unit retrieves the rendered image from the preset shared memory in real time.

[0063] Specifically, after the image frame sequence output by the first target graphics card is written into the preset shared memory, the second processing end retrieves the rendered image from the preset shared memory in real time and transmits it to the second target graphics card for encoding processing.

[0064] The pre-defined shared memory uses a multi-buffer design to avoid read / write blocking in a single buffer. It can also pre-allocate shared memory blocks of a fixed size to avoid memory fragmentation and help improve data access efficiency.

[0065] In this embodiment, the second processing end retrieves the rendered image from the preset shared memory in real time, so that the second target graphics card can perform the encoding process on the rendered image, thereby achieving low-latency transmission and ensuring real-time data processing.

[0066] In some embodiments, the above-described rendering and streaming method based on point cloud data further includes the following steps:

[0067] In the first target graphics card, the point cloud data acquired in real time is rendered to obtain the corresponding 3D view;

[0068] The 3D view is transmitted to the front-end module of the 3D scanning system for real-time display.

[0069] Specifically, in the first target graphics card, the received point cloud data is preprocessed, including denoising and point cloud calibration. The point cloud data is then voxelized and input into the rendering engine for rendering to generate an interactive 3D view. Specifically, based on the real-time lighting information of the current scanned scene, the voxelized point cloud data is rendered from multiple perspectives to obtain the 3D view.

[0070] Furthermore, the 3D view is transmitted to the front-end module for real-time display. The front-end module includes, but is not limited to, desktop and web interfaces. The front-end module displays the 3D view through a visual interface and provides user interaction functionality.

[0071] In this embodiment, the point cloud data acquired in real time is rendered in the first target graphics card to obtain the corresponding 3D view. The 3D view is then transmitted to the front-end module for real-time display to support interactive real-time viewing. At the same time, it can realize remote visual monitoring and conveniently view the 3D status of the target scanned object. It can be widely used in industrial inspection, smart cities and other fields.

[0072] In some embodiments, the above-described rendering and streaming method based on point cloud data further includes the following steps:

[0073] Based on user operation commands input through the target terminal, the real-time display status of the 3D view in the front-end module is controlled.

[0074] Specifically, a communication connection is established between the front-end module and the target terminal. This can be achieved through user operation commands input from the target terminal, such as mobile phones, tablets, or smart wearable devices. When a user operation command is captured, it is parsed in real time, and the real-time display status of the 3D view is dynamically adjusted based on the command parsing results. This includes, but is not limited to, rotation angle, scaling ratio, viewpoint switching, and hiding / showing specific components.

[0075] For example, when a user triggers a gesture zoom operation in the mobile app, the instruction is parsed and transmitted to the front-end rendering engine to adjust the display size of the 3D model in the view based on the instruction parameters; when a user triggers an orthographic view switching operation in the mobile app, the instruction is parsed and transmitted to the front-end rendering engine to adjust the display perspective of the 3D model in the view based on the instruction parameters, such as switching from the front view to the top view.

[0076] In this embodiment, based on user operation commands input through the target terminal, the real-time display status of the 3D view in the front-end module is controlled, thereby realizing real-time interaction between the user and the visualization interface. This supports multi-angle dynamic observation and parameter adjustment of the target scanned object, which helps to improve 3D scanning efficiency and ensure the output of high-quality 3D scanning results.

[0077] In some embodiments, the above-described rendering and streaming method based on point cloud data further includes the following steps:

[0078] When an encoding anomaly is detected in the second target graphics card, the system switches to software encoding mode to process the image frame sequence and obtain an encoded video stream.

[0079] Specifically, real-time monitoring of the second target graphics card for encoding anomalies can be achieved by monitoring various aspects, including graphics card status, encoding performance, and data interaction. Monitoring strategies can be set according to actual application requirements. For example, if the graphics card utilization rate of the second target graphics card exceeds a threshold, it indicates that the second target graphics card cannot perform encoding tasks normally; if the encoding latency consistently exceeds a preset threshold, it indicates that the second target graphics card is encoding abnormal; if timestamp jumps or repetitions are detected in the output bitstream, it indicates that the second target graphics card is encoding abnormal.

[0080] Furthermore, when an encoding anomaly is detected in the second target graphics card, the system switches to software encoding mode to process the image frame sequence in order to obtain the corresponding encoded video stream and improve the stability of the encoding task.

[0081] In this embodiment, when an encoding anomaly is detected in the second target graphics card, the system switches to software encoding mode to process the image frame sequence, thereby stably acquiring the encoded video stream, ensuring high-quality output of the bitstream, and ensuring the stable and continuous operation of the scanning process.

[0082] In some of these embodiments, the data interaction between the first processing end and the second processing end adopts an event-driven asynchronous architecture for multi-process communication.

[0083] In this embodiment, the first processing end and the second processing end achieve multi-process communication through an event-driven asynchronous architecture. Specifically, the first processing end calls the first target graphics card to perform image rendering, stores the generated image frame data in shared memory, and notifies the second processing end through an event mechanism, allowing it to continue processing the next frame without waiting for encoding to complete. The second processing end, by listening for event triggers, retrieves the data to be encoded from the shared memory and calls the second target graphics card to execute the encoding task, thereby achieving multi-process communication through event-driven mechanisms and asynchronous access to image frame data.

[0084] Understandably, by releasing the hardware's parallel computing power through the independent calls of dual graphics cards, the aforementioned event-driven mechanism can avoid mutual blocking between processing ends, form an efficient asynchronous pipeline between rendering and encoding stages, achieve efficient collaboration between rendering and encoding, significantly reduce data processing latency, and meet real-time requirements.

[0085] In this embodiment, the data interaction between the first processing end and the second processing end adopts an event-driven asynchronous architecture for multi-process communication, which can reduce data transmission latency, ensure the efficient operation of the system processing pipeline, and help improve the stability of the system.

[0086] The following is combined Figure 3 and Figure 4 This embodiment will be described and explained through specific examples.

[0087] The system uses a 3D scanning device to acquire raw point cloud data of the target object in real time, and then outputs the raw point cloud data to the device access module. During the main scanning process, the device access module calibrates the received raw point cloud data and sends the calibrated point cloud data to the rendering engine for processing. The rendering outputs an RGB frame sequence at a frame rate of 30fps, and the RGB frame sequence is then transmitted to the transmission module of the encoding and streaming process via the transmission module based on a preset shared memory.

[0088] During the encoding and streaming process, the received RGB frame sequence is transmitted to the encoding module, where H.265 hardware encoding is used to encode the RGB frame sequence, resulting in an H.265 encoded video stream. This encoded video stream is then pushed to the streaming module. In response to video stream acquisition requests from the mobile app, the streaming module distributes an adaptive bitstream to the mobile app. Furthermore, the mobile app can send control commands (such as rotation and scaling) to the front-end module to control the 3D view display status of the target object on the front-end visualization interface.

[0089] The present embodiment will now be described and illustrated through preferred embodiments.

[0090] Figure 5 This is a flowchart of the rendering and streaming method based on point cloud data according to a preferred embodiment, as follows: Figure 5 As shown, the rendering and streaming method based on point cloud data includes the following steps:

[0091] Step S510: In the first target graphics card, the real-time acquired point cloud data is rendered to obtain the corresponding image frame sequence.

[0092] Step S520: Write the image frame sequence output by the first target graphics card into a preset shared memory;

[0093] Step S530: Read each rendered image from the image frame sequence from the preset shared memory and transfer each rendered image to the second target graphics card;

[0094] In step S540, the image frame sequence is encoded in the second target graphics card to obtain the corresponding encoded video stream for the target terminal to acquire.

[0095] In this embodiment, the real-time acquired point cloud data is rendered in the first target graphics card to obtain the corresponding image frame sequence. The image frame sequence output by the first target graphics card is written to a preset shared memory, each rendered image in the image frame sequence is read from the preset shared memory, and each rendered image is transmitted to the second target graphics card.

[0096] Furthermore, in the second target graphics card, the image frame sequence is encoded to obtain the corresponding encoded video stream for the target terminal to obtain. This solves the problem that severe resource contention between rendering and encoding tasks leads to a significant reduction in the overall performance of rendering and encoding, making it difficult to meet the requirements of real-time processing and high-quality output. It avoids resource contention between rendering and encoding tasks, improves the overall performance of rendering and encoding, and meets the requirements of real-time processing and high-quality output.

[0097] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0098] This embodiment also provides a rendering and streaming system based on point cloud data. This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below can refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0099] Figure 6 This is a structural block diagram of the 3D scanning system of this embodiment, as shown below. Figure 6 As shown, the system includes a processing terminal 10 and a target terminal 20; the processing terminal 10 includes a first processing terminal and a second processing terminal, the first processing terminal can call a first target graphics card, and the second processing terminal can call a second target graphics card;

[0100] Processing terminal 10 is used to execute the steps in any of the above method embodiments;

[0101] The target terminal 20 is used to request the encoded video stream generated by the processing terminal 10 from the processing terminal 10.

[0102] The system provided in this embodiment renders the real-time acquired point cloud data in the first target graphics card to obtain the corresponding image frame sequence; transmits the image frame sequence to the second target graphics card; and encodes the image frame sequence in the second target graphics card to obtain the corresponding encoded video stream for the target terminal 20 to acquire. This solves the problem that severe resource contention between rendering and encoding tasks leads to a significant reduction in the overall performance of rendering and encoding, making it difficult to meet the requirements of real-time processing and high-quality output. It avoids resource contention between rendering and encoding tasks, improves the overall performance of rendering and encoding, and meets the requirements of real-time processing and high-quality output.

[0103] In some embodiments, the above-described three-dimensional scanning system further includes a three-dimensional scanning device 30;

[0104] The 3D scanning device 30 is used to input the real-time acquired point cloud data into the first processing terminal.

[0105] In some embodiments, the above-described 3D scanning system further includes a front-end module 40;

[0106] The first processing end is used to call the first target graphics card to render the point cloud data acquired in real time and obtain the corresponding 3D view;

[0107] Front-end module 40 is used to display the rendered 3D view in real time.

[0108] In some embodiments, the target terminal 20 is also used to control the real-time display status of the three-dimensional view in the front-end module 40 based on real-time input user operation commands.

[0109] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0110] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0111] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0112] S1, in the first target graphics card, the real-time acquired point cloud data is rendered to obtain the corresponding image frame sequence;

[0113] S2, transmit the image frame sequence to the second target graphics card; in the second target graphics card, encode the image frame sequence to obtain the corresponding encoded video stream for the target terminal to obtain.

[0114] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0115] Furthermore, in conjunction with the point cloud data-based rendering and streaming methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the point cloud data-based rendering and streaming methods described in the above embodiments.

[0116] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0117] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0118] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0119] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A rendering and streaming method based on point cloud data, characterized in that, Applicable to a 3D scanning system; the 3D scanning system includes a processing terminal and a target terminal; the processing terminal includes a first processing terminal and a second processing terminal, the first processing terminal can call a first target graphics card, and the second processing terminal can call a second target graphics card; The method is applicable to the processing end side, and the method includes: In the first target graphics card, the real-time acquired point cloud data is rendered to obtain the corresponding image frame sequence; The image frame sequence is transmitted to the second target graphics card; in the second target graphics card, the image frame sequence is encoded to obtain the corresponding encoded video stream for the target terminal to acquire.

2. The rendering and streaming method based on point cloud data according to claim 1, characterized in that, The step of transmitting the image frame sequence to the second target graphics card includes: Write the image frame sequence output by the first target graphics card into a preset shared memory; Each rendered image in the image frame sequence is read from the preset shared memory, and each rendered image is transmitted to the second target graphics card.

3. The rendering and streaming method based on point cloud data according to claim 2, characterized in that, The step of reading each rendered image from the image frame sequence from the preset shared memory and transmitting each rendered image to the second target graphics card includes: The second processing unit retrieves the rendered image from the preset shared memory in real time.

4. The rendering and streaming method based on point cloud data according to claim 1, characterized in that, The method further includes: In the first target graphics card, the point cloud data acquired in real time is rendered to obtain the corresponding three-dimensional view; The 3D view is transmitted to the front-end module of the 3D scanning system for real-time display.

5. The rendering and streaming method based on point cloud data according to claim 4, characterized in that, The method further includes: Based on user operation commands input through the target terminal, the real-time display status of the 3D view in the front-end module is controlled.

6. The rendering and streaming method based on point cloud data according to claim 1, characterized in that, The method further includes: When an encoding anomaly is detected in the second target graphics card, the system switches to software encoding mode to process the image frame sequence and obtains the encoded video stream.

7. The rendering and streaming method based on point cloud data according to any one of claims 1 to 6, characterized in that, The data interaction between the first processing end and the second processing end adopts an event-driven asynchronous architecture for multi-process communication.

8. A three-dimensional scanning system, characterized in that, The 3D scanning system includes a processing terminal and a target terminal; the processing terminal includes a first processing terminal and a second processing terminal; the first processing terminal can call a first target graphics card, and the second processing terminal can call a second target graphics card; The processing terminal is used to execute the steps of the rendering and streaming method based on point cloud data as described in any one of claims 1 to 7; The target terminal is used to request the processing terminal to obtain the encoded video stream generated by the processing terminal.

9. The three-dimensional scanning system according to claim 8, characterized in that, The system also includes a 3D scanning device; The 3D scanning device is used to input the point cloud data acquired in real time into the first processing terminal.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the rendering and streaming method based on point cloud data as described in any one of claims 1 to 7.

Images