A point cloud depth data transmission method and device, electronic equipment and storage medium

Abstract

The application provides a point cloud depth data transmission method and device, electronic equipment and a storage medium. The method comprises the following steps: configuring a running mode through a client, encapsulating the running mode into a corresponding control instruction, and sending the control instruction to a data acquisition device; receiving the control instruction through the data acquisition device, switching a corresponding data output format based on the control instruction to output depth data in the data output format; encapsulating the depth data into a unified frame structure through the data acquisition device, receiving the depth data frame through the client and converting the depth data frame, and transmitting the depth data frame to the client through Ethernet; receiving the depth data frame through the client, and reconstructing a three-dimensional point cloud coordinate based on the depth data frame.

Description

Technical Field

[0001] This application relates to the fields of 3D point cloud data processing and computer vision technology, and more specifically, to a point cloud depth data transmission method, apparatus, electronic device, and storage medium. Background Technology

[0002] 3D point cloud data, as a core data form for computer vision and 3D perception, is widely used in precision measurement systems such as industrial inspection, robot navigation, autonomous driving, and structured light / laser profilometers. In practical applications, data acquisition devices (such as laser profilometers, structured light cameras, and ToF depth cameras) acquire depth information of a scene through optical scanning and represent the geometric structure of an object's surface as a discrete set of points. This point cloud data needs to be transmitted to client software in real time for subsequent processing, display, or analysis. Therefore, data transmission efficiency and reconstruction accuracy directly determine the system's real-time performance, reliability, and the breadth of applicable scenarios.

[0003] Currently, existing 3D point cloud data transmission technologies mainly include three categories: (1) fixed-precision full-coordinate transmission scheme; (2) fixed low-width compression transmission scheme; and (3) point cloud depth data reconstruction technology.

[0004] However, fixed-precision full-coordinate transmission schemes consume a lot of bandwidth, putting a heavy burden on network transmission; fixed low-bit-width compression transmission schemes suffer from insufficient accuracy in high-speed mode, while high-precision mode exceeds bandwidth limits, making it difficult to achieve a dynamic balance between accuracy and speed; in point cloud depth data reconstruction technology, directly transmitting complete point cloud coordinates results in significant data redundancy and low transmission efficiency. Furthermore, the reliability and maintainability of depth data transmission are only average, and the data output format of the acquisition equipment is fixed, making it difficult to dynamically adjust according to needs. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method, apparatus, electronic device, and storage medium for point cloud depth data transmission. By transmitting only depth data and combining it with a dynamic bit-width switching mechanism, the 3D coordinate reconstruction of the point cloud is completed on the client side, reducing the bandwidth of point cloud data transmission. Furthermore, the client can select high-speed mode or high-precision mode according to application scenario requirements, achieving a dynamic balance between transmission accuracy and speed, reducing data redundancy, and improving transmission efficiency. In addition, a unified depth data transmission frame structure is provided, improving the reliability and maintainability of data transmission, and the client can actively and dynamically adjust the data output format of the acquisition device as needed.

[0006] In a first aspect, embodiments of this application provide a point cloud depth data transmission method, the method comprising: The operating mode is configured through the client, the operating mode is encapsulated into a corresponding control command, and the control command is sent to the data acquisition device; wherein, the operating mode includes high-speed mode and high-precision mode; the control command at least includes a mode identifier; The data acquisition device receives the control command and switches the corresponding data output format based on the control command to output depth data in that data output format; wherein, the data output format corresponding to the high-speed mode is the first data output format, and the data output format corresponding to the high-precision mode is the second data output format; The data acquisition device encapsulates depth data into a unified frame structure, the client receives the depth data frame and converts the depth data frame, and then transmits the depth data frame to the client via Ethernet. The client receives the depth data frame and reconstructs the 3D point cloud coordinates based on the depth data frame.

[0007] In one possible implementation, configuring the operating mode via the client includes: In response to user actions on the interactive interface provided by the client, select the corresponding operating mode; or... The client automatically determines and selects the corresponding operating mode based on real-time network bandwidth or system load.

[0008] In one possible implementation, switching the corresponding data output format based on the control command to output depth data in that data output format includes: The control commands are parsed using the data acquisition device, and the mode identifiers in the control commands are extracted. When the mode identifier is the first mode identifier, the original depth data is encoded in parallel using a hardware pipeline and output in the first data output format; When the mode identifier is the second mode identifier, the original depth data is parallel encoded in the second data output format and output.

[0009] In one possible implementation, the conversion of the depth data frame includes: The mode identifier in the depth data frame is parsed, and the current data output format is identified based on the mode identifier; The depth data frame is converted based on the data output format.

[0010] In one possible implementation, the conversion of the depth data frame based on the data output format includes: Convert the exponential offset of the depth data frame in the first data output format to the exponential offset corresponding to the second data output format; The implicit bits of the depth data frame in the first data output format are padded with a leading edge to obtain a normalized mantissa, and the first data output format of the depth data frame is converted into a second data output format.

[0011] In one possible implementation, the unified frame structure includes at least a data resolution width and a data resolution height; the reconstruction of 3D point cloud coordinates based on the depth data frame includes: A first sampling index is obtained based on the data resolution width of the depth data frame, a second sampling index is obtained based on the data resolution height, and spatial resolution parameters calibrated by the data acquisition device are obtained; wherein, the spatial resolution parameters include horizontal spatial resolution and depth spatial resolution; Based on the sampling index and the spatial resolution parameter rate, the horizontal coordinates and depth coordinates in the three-dimensional point cloud coordinates are calculated, and the depth value in the three-dimensional point cloud coordinates is obtained based on the depth data frame.

[0012] In one possible implementation, the unified frame structure includes a frame sequence number; the method further includes: Determine whether the frame numbers of the depth data frames are consecutive; If not, it is determined that the depth data frame has been dropped, and the current network status is detected; wherein, the network status includes at least interruption, timeout, and normal. The anomaly handling method is matched based on the network state, and the anomaly handling is performed based on the anomaly handling method.

[0013] Secondly, embodiments of this application also provide a point cloud depth data transmission device, the device comprising: The sending module is used to configure the operating mode through the client, encapsulate the operating mode into a corresponding control command, and send the control command to the data acquisition device; wherein, the operating mode includes a high-speed mode and a high-precision mode; and the control command includes at least a mode identifier; The switching module is used to receive the control command through the data acquisition device and switch the corresponding data output format based on the control command to output depth data in the data output format; wherein, the data output format corresponding to the high-speed mode is the first data output format, and the data output format corresponding to the high-precision mode is the second data output format; The conversion module is used to encapsulate depth data into a unified frame structure through the data acquisition device, receive depth data frames through the client and convert the depth data frames, and transmit the depth data frames to the client through Ethernet; The reconstruction module is used to receive the depth data frame through the client and reconstruct the three-dimensional point cloud coordinates based on the depth data frame.

[0014] In one possible implementation, the sending module is specifically used for: In response to user actions on the interactive interface provided by the client, select the corresponding operating mode; or... The client automatically determines and selects the corresponding operating mode based on real-time network bandwidth or system load.

[0015] In one possible implementation, the switching module is specifically used for: The control commands are parsed using the data acquisition device, and the mode identifiers in the control commands are extracted. When the mode identifier is the first mode identifier, the original depth data is encoded in parallel using a hardware pipeline and output in the first data output format; When the mode identifier is the second mode identifier, the original depth data is parallel encoded in the second data output format and output.

[0016] In one possible implementation, the conversion module is specifically used for: The mode identifier in the depth data frame is parsed, and the current data output format is identified based on the mode identifier; The depth data frame is converted based on the data output format.

[0017] In one possible implementation, the conversion module is specifically used for: Convert the exponential offset of the depth data frame in the first data output format to the exponential offset corresponding to the second data output format; The implicit bits of the depth data frame in the first data output format are padded with a leading edge to obtain a normalized mantissa, and the first data output format of the depth data frame is converted into a second data output format.

[0018] In one possible implementation, the unified frame structure includes at least a data resolution width and a data resolution height; the reconstruction module is specifically used for: A first sampling index is obtained based on the data resolution width of the depth data frame, a second sampling index is obtained based on the data resolution height, and spatial resolution parameters calibrated by the data acquisition device are obtained; wherein, the spatial resolution parameters include horizontal spatial resolution and depth spatial resolution; Based on the sampling index and the spatial resolution parameter rate, the horizontal coordinates and depth coordinates in the three-dimensional point cloud coordinates are calculated, and the depth value in the three-dimensional point cloud coordinates is obtained based on the depth data frame.

[0019] In one possible implementation, the unified frame structure includes a frame sequence number; the apparatus further includes: The judgment module is used to determine whether the frame sequence numbers of the depth data frames are consecutive; The detection module is used to determine if the depth data frame has been dropped if not, and to detect the current network status; wherein the network status includes at least interruption, timeout, and normal. The processing module is used to match the corresponding exception handling method based on the network status and perform exception handling based on the exception handling method.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the point cloud depth data transmission method as described in any of the first aspects.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the point cloud depth data transmission method described in any of the first aspects.

[0022] This application provides a point cloud depth data transmission method, apparatus, electronic device, and storage medium. The method involves configuring a running mode on a client side, encapsulating the running mode into corresponding control commands, and sending these commands to a data acquisition device. The data acquisition device receives the control commands and switches the corresponding data output format based on these commands to output depth data in that format. The data acquisition device encapsulates the depth data into a unified frame structure. The client receives and converts these depth data frames and transmits them via Ethernet. The client receives the depth data frames and reconstructs the 3D point cloud coordinates based on them. This application, by transmitting only depth data and combining it with a dynamic bit-width switching mechanism, completes the 3D point cloud coordinate reconstruction on the client side, reducing the bandwidth of point cloud data transmission. Furthermore, the client can select between high-speed and high-precision modes according to application requirements, achieving a dynamic balance between transmission accuracy and speed, reducing data redundancy, and improving transmission efficiency. In addition, the unified depth data transmission frame structure improves the reliability and maintainability of data transmission, and the client can actively and dynamically adjust the data output format of the acquisition device as needed.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a point cloud depth data transmission method provided according to an embodiment of this application; Figure 2 This is a schematic diagram of the overall process of point cloud depth data transmission; Figure 3 This is a schematic diagram of the point cloud depth data transmission device provided according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0027] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0029] Given that 3D point cloud data is a core data format for computer vision and 3D perception, it is widely used in precision measurement systems such as industrial inspection, robot navigation, autonomous driving, and structured light / laser profilometers. In practical applications, data acquisition devices (such as laser profilometers, structured light cameras, and ToF depth cameras) acquire depth information of a scene through optical scanning and represent the geometric structure of an object's surface as a discrete set of points. This point cloud data needs to be transmitted to client software in real time for subsequent processing, display, or analysis. Therefore, data transmission efficiency and reconstruction accuracy directly determine the system's real-time performance, reliability, and the breadth of applicable scenarios.

[0030] Currently, existing 3D point cloud data transmission technologies mainly include three categories: (1) fixed-precision full-coordinate transmission scheme; (2) fixed low-width compression transmission scheme; and (3) point cloud depth data reconstruction technology.

[0031] However, fixed-precision full-coordinate transmission schemes consume a lot of bandwidth, putting a heavy burden on network transmission; fixed low-bit-width compression transmission schemes suffer from insufficient accuracy in high-speed mode, while high-precision mode exceeds bandwidth limits, making it difficult to achieve a dynamic balance between accuracy and speed; in point cloud depth data reconstruction technology, directly transmitting complete point cloud coordinates results in significant data redundancy and low transmission efficiency. Furthermore, the reliability and maintainability of depth data transmission are only average, and the data output format of the acquisition equipment is fixed, making it difficult to dynamically adjust according to needs.

[0032] To address this issue, this application provides a point cloud depth data transmission method, apparatus, electronic device, and storage medium. By transmitting only depth data and combining it with a dynamic bit-width switching mechanism, the 3D coordinate reconstruction of the point cloud is completed on the client side, reducing the bandwidth of point cloud data transmission. Furthermore, the client can select between high-speed and high-precision modes according to application scenario requirements, achieving a dynamic balance between transmission accuracy and speed, reducing data redundancy, and improving transmission efficiency. In addition, a unified depth data transmission frame structure is provided, improving the reliability and maintainability of data transmission, and the client can actively and dynamically adjust the data output format of the acquisition device as needed.

[0033] Figure 1 This is a flowchart of a point cloud depth data transmission method provided according to an embodiment of this application. For example... Figure 1 As shown, the point cloud depth data transmission method of this application embodiment may specifically include: S101. Configure the operating mode through the client, encapsulate the operating mode into corresponding control commands, and send the control commands to the data acquisition device.

[0034] S102. Receive control commands through the data acquisition device, and switch the corresponding data output format based on the control commands to output depth data in that data output format.

[0035] S103. The depth data is encapsulated into a unified frame structure by the data acquisition device, the depth data frame is received by the client and converted, and the depth data frame is transmitted to the client via Ethernet.

[0036] S104. Receive depth data frames through the client and reconstruct the 3D point cloud coordinates based on the depth data frames.

[0037] In the aforementioned point cloud depth data transmission method, by transmitting only depth data and combining it with a dynamic bit-width switching mechanism, the 3D coordinate reconstruction of the point cloud is completed on the client side, reducing the bandwidth of point cloud data transmission. Furthermore, the client can select between high-speed and high-precision modes according to application scenario requirements, achieving a dynamic balance between transmission accuracy and speed, reducing data redundancy, and improving transmission efficiency. In addition, a unified depth data transmission frame structure is provided, improving the reliability and maintainability of data transmission, and the client can proactively and dynamically adjust the data output format of the acquisition device as needed.

[0038] The exemplary steps described above in the embodiments of this application are illustrated below with specific examples: S101 configures the operating mode through the client, encapsulates the operating mode into corresponding control commands, and sends the control commands to the data acquisition device (either through an independent control channel or embedded in the data stream).

[0039] In this embodiment, the operating modes include high-speed mode and high-precision mode; the control command at least includes a mode flag, which is 1 byte long. A mode flag of 0 indicates high-speed mode (M0), and a mode flag of 1 indicates high-precision mode (M1); the control command may also include a command identifier and verification information; the operating mode is set through the client, and the set operating mode is encapsulated into a control command and sent to the data acquisition device through the network for subsequent processing. For example, Figure 2 As shown, the client can be set to M=0 for high-speed mode or M=1 for high-precision mode.

[0040] The format of the control commands is shown in Table 1 below: Table 1

[0041] Optionally, control commands can be sent to the data acquisition device via a separate control channel or embedded in the data stream before being sent to the data acquisition device. Specifically, control commands can be sent via a separate control channel or embedded in the data stream for synchronous transmission, ensuring synchronization between the control commands and the data frames.

[0042] Optionally, when configuring the operating mode through the client, the corresponding operating mode is selected in response to the user's operation on the interactive interface provided by the client; or, the client can automatically determine and select the corresponding operating mode based on real-time network bandwidth or system load. For example, the client software provides an operating mode configuration interface, through which the user can select high-speed mode (M=0) or high-precision mode (M=1), or the system can automatically select based on network bandwidth or system load.

[0043] S102 receives control commands through the data acquisition device and switches the corresponding data output format based on the control commands to output depth data in that data output format.

[0044] In this embodiment, the data output format corresponding to the high-speed mode is the first data output format, and the data output format corresponding to the high-precision mode is the second data output format. This application uses half-precision floating-point format (float16) and single-precision floating-point format (float32) as the first and second data output formats, respectively, for illustrative purposes only and does not constitute a limitation. For example, when the mode identifier is 0, it indicates the high-speed mode, and the corresponding data output format is half-precision floating-point format; when the mode identifier is 1, it indicates the high-precision mode, and the corresponding data output format is single-precision floating-point format. The data acquisition device receives and switches the corresponding depth data output format based on control commands, and outputs the depth data in that data output format for subsequent processing. For example, as... Figure 2 As shown.

[0045] In one implementation, the control command is parsed by a data acquisition device to extract the mode identifier in the control command; when the mode identifier is a first mode identifier, the original depth data is parallel encoded and output in a first data output format using a hardware pipeline; when the mode identifier is a second mode identifier, the original depth data is parallel encoded and output in a second data output format.

[0046] Specifically, for example, after receiving control commands, the data acquisition device selects the corresponding data output format based on the mode identifier: when the mode identifier is the first mode identifier (value 0), it indicates M=0 (high-speed mode), and the depth data Z_out = Z_16 (float16, half-precision floating-point); when the mode identifier is the second mode identifier (value 1), it indicates M=1 (high-precision mode), and the depth data Z_out = Z_32 (float32, single-precision floating-point). In high-speed mode, a hardware pipeline approach can be used to perform parallel encoding processing of the depth data, reducing processing latency and ensuring real-time performance.

[0047] S103 encapsulates depth data into a unified frame structure through a data acquisition device, receives depth data frames through a client, converts the depth data frames, and transmits the depth data frames to the client via Ethernet.

[0048] In this embodiment, the unified frame structure includes at least data resolution width and data resolution height; Ethernet supports both TCP (Reliable Transmission) and UDP (Low-Latency Transmission) modes; the data acquisition device encapsulates depth data into a unified frame structure and sends the depth data frame to the client, which receives and converts the depth data frame, then transmits it back to the client via Ethernet for subsequent processing. For example, as... Figure 2 As shown.

[0049] Optionally, the application scenario is determined, and the corresponding Ethernet transmission mode is matched based on the application scenario.

[0050] It should be noted that depth data is encapsulated into a unified data frame structure, supporting extended fields and standardized parsing, as shown in Table 2 below:

[0051] It should be noted that when encapsulating depth data into a unified frame structure, the frame start identifier is set to a fixed value, and the current frame sequence number is written into the frame sequence number field; the data resolution width and data resolution height fields are filled with the width and height values ​​of the data resolution, respectively; the data region length field is filled with the number of bytes in the depth data field; and the data checksum field is filled with the data checksum calculated from the depth data field.

[0052] In some implementations, when converting depth data frames, the mode identifier in the depth data frame is parsed, and the current data output format is identified based on the mode identifier; the depth data frame is then converted based on the data output format.

[0053] Optionally, when converting the depth data frame based on the data output format, the exponential offset of the depth data frame in the first data output format is converted into the exponential offset corresponding to the second data output format; the implicit bits of the depth data frame in the first data output format are padded with leading bits to obtain the normalized mantissa, and the first data output format of the depth data frame is converted into the second data output format.

[0054] Specifically, for example, when the mode identifier is 0 (depth data is float16 data), precision recovery is required. The conversion from the first data output format float16 to the second data output format float32 not only involves bit width expansion but also exponent expansion and mantissa recovery. In exponent expansion, the float16 exponent offset is 15, and the float32 exponent offset is 127. Therefore, converting the float16 exponent offset from 15 to 127 increases the actual exponent offset by 112. In mantissa recovery, the implicit bits of float16 do not contain leading 1s; leading 1s need to be padded during conversion to obtain the normalized mantissa.

[0055] Therefore, by converting the qualified depth data frame (float16 data) to restore precision, the float32 value obtained from the float16 conversion is numerically close to the original float32 representation, thus ensuring that the converted depth data is numerically close to the original depth data and avoiding the precision loss caused by direct bit extension.

[0056] Optionally, timestamp information can be included in the unified data frame structure to achieve frame synchronization for data acquisition from multiple devices, and to dynamically adjust the acquisition frame rate based on network bandwidth conditions, further optimizing transmission efficiency. Additionally, compression identifiers can be extended into the unified data frame structure to support further data compression in high-bit-width mode. Device serial numbers can also be included in the unified data frame structure to support multi-device identification and management.

[0057] S104, the client receives the depth data frame and reconstructs the three-dimensional point cloud coordinates based on the depth data frame.

[0058] In this embodiment of the application, the client receives a depth data frame and reconstructs the 3D point cloud coordinates based on the depth data frame. For example, as... Figure 2 As shown.

[0059] Optionally, when reconstructing 3D point cloud coordinates based on depth data frames, a first sampling index is obtained based on the data resolution width of the depth data frame, a second sampling index is obtained based on the data resolution height, and spatial resolution parameters calibrated by the data acquisition device are acquired; based on the sampling index and spatial resolution parameter rate, the horizontal coordinates and depth coordinates in the 3D point cloud coordinates are calculated, and the depth value in the 3D point cloud coordinates is obtained based on the depth data frame.

[0060] It should be noted that the horizontal coordinates in the 3D point cloud coordinates are calculated based on the first sampling index and the horizontal spatial resolution; the depth coordinates in the 3D point cloud coordinates are calculated based on the second sampling index and the depth spatial resolution; and the depth values ​​in the 3D point cloud coordinates are obtained based on the depth data frame.

[0061] Specifically, based on the depth data, combined with the sampling index and spatial resolution parameter, the 3D point cloud is reconstructed as follows: X = u · sx (X is the horizontal coordinate, u is the sampling index, and sx is the horizontal spatial resolution), •Y = v · sy (Y is the depth coordinate, v is the sampling index, and sy is the depth spatial resolution), Z = Zout (Z is the depth value, and Zout is the output depth data), where u and v represent the sampling index (0 ≤ u < data resolution width Width, 0 ≤ v < data resolution height Height), sx and sy represent the spatial resolution parameter (determined by device calibration, unit: mm / point or μm / point), and Z_out represents the depth data (after float16 conversion or directly used from float32).

[0062] Continuing on, the horizontal coordinates, depth coordinates, and depth values ​​are combined into a three-dimensional point, and complete three-dimensional point cloud data is generated point by point.

[0063] Optionally, the spatial resolution parameters can be adjusted according to the equipment calibration parameters. Therefore, the spatial resolution parameters can be adjusted based on the equipment calibration parameters to support application scenarios with different field of view ranges and measurement accuracies.

[0064] The point cloud depth data transmission method provided in this application involves configuring a running mode on the client side, encapsulating the running mode into corresponding control commands, and sending the control commands to a data acquisition device. The data acquisition device receives the control commands and switches the corresponding data output format based on the commands to output depth data in that format. The data acquisition device encapsulates the depth data into a unified frame structure. The client receives and converts the depth data frames and transmits them to the client via Ethernet. The client receives the depth data frames and reconstructs the 3D point cloud coordinates based on them. This point cloud depth data transmission method, by transmitting only depth data and combining it with a dynamic bit-width switching mechanism, completes the 3D point cloud coordinate reconstruction on the client side, reducing the bandwidth of point cloud data transmission. Furthermore, the client can select a high-speed mode or a high-precision mode according to application scenario requirements, achieving a dynamic balance between transmission accuracy and speed, reducing data redundancy, and improving transmission efficiency. In addition, the unified depth data transmission frame structure improves the reliability and maintainability of data transmission, and the client can actively and dynamically adjust the data output format of the acquisition device as needed.

[0065] Furthermore, the unified frame structure includes frame sequence numbers; determining whether the frame sequence numbers of depth data frames are consecutive; if not, determining that depth data frames have been dropped, and detecting the current network status; matching the corresponding exception handling method based on the network status and performing exception handling based on the exception handling method. The network status includes at least interruption, timeout, and normal.

[0066] The point cloud depth data transmission method of this application is further described below with specific examples: In a 3D inspection scenario for industrial parts, a laser profilometer is used to perform 3D scanning inspection of parts on the production line. The field of view is 800×400 points / frame, with a frame rate of 30fps.

[0067] Scenario 1: High-speed detection mode (Mode Flag=0) When rapid filtering or real-time monitoring is required, select high-speed mode to transmit only float16 depth data. • Data volume: 800 × 400 × 2 bytes × 30fps ≈ 19.2 MB / s (compared to approximately 115 MB / s for XYZ full coordinate transmission, the bandwidth is reduced to about 1 / 6 of the original). • The client performs precision restoration, converting float16 to float32. • Real-time display or rapid initial screening after point cloud reconstruction. Scenario 2: High-precision detection mode (Mode Flag=1) When precise measurement or defect detection of parts is required, select the high-precision mode and transmit float32 depth data: • Data volume: 800 × 400 × 4 bytes × 30fps ≈ 38.4 MB / s • Clients can directly use float32 data without conversion. • Perform precise measurements or defect analysis after point cloud reconstruction. The results are shown in Table 2 below:

[0068] Therefore, this application only transmits depth data Z values ​​(approximately 2-4 bytes / point), significantly reducing data volume and transmission bandwidth compared to traditional XYZ full coordinate transmission (approximately 12 bytes / point). It provides a dynamic switching mechanism for operating modes: high-speed mode (M=0) uses float16 format for high transmission efficiency, suitable for real-time monitoring scenarios; high-precision mode (M=1) uses float32 format for high accuracy, suitable for precision detection scenarios. Users can flexibly choose according to their application scenario without replacing hardware, achieving a flexible balance between accuracy and speed. Utilizing the point cloud reconstruction capability of depth data eliminates redundant information in data transmission. The client completes point cloud reconstruction based on sampling index and resolution parameters, improving overall system efficiency and resource utilization. A unified data frame structure design enhances the reliability of data parsing and system maintainability. The client can dynamically adjust the data output format of the acquisition device through control commands, forming a complete two-way interaction mechanism, enabling bidirectional control between the client and the acquisition device. The system is highly flexible and adaptable to various application scenarios. (float16 to float32) During the conversion, exponential expansion and mantissa recovery are performed to avoid the accuracy loss caused by direct bit expansion and ensure the accuracy of point cloud reconstruction.

[0069] Figure 3 This is a schematic diagram of the structure of a point cloud depth data transmission device provided according to an embodiment of this application; as shown below. Figure 3 As shown, the point cloud depth data transmission device 300 of this application embodiment may specifically include: The sending module 301 is used to configure the operating mode through the client, encapsulate the operating mode into a corresponding control command, and send the control command to the data acquisition device; wherein, the operating mode includes high-speed mode and high-precision mode; the control command at least includes a mode identifier; The switching module 302 is used to receive control commands through the data acquisition device and switch the corresponding data output format based on the control commands to output depth data in the data output format; wherein, the data output format corresponding to the high-speed mode is the first data output format, and the data output format corresponding to the high-precision mode is the second data output format; The conversion module 303 is used to encapsulate depth data into a unified frame structure through the data acquisition device, receive depth data frames through the client and convert the depth data frames, and transmit the depth data frames to the client through Ethernet; The reconstruction module 304 is used to receive depth data frames through the client and reconstruct the coordinates of the three-dimensional point cloud based on the depth data frames.

[0070] In one possible implementation, the sending module is specifically used for: In response to user actions on the interactive interface provided by the client, select the corresponding operating mode; or... The client automatically determines and selects the corresponding operating mode based on real-time network bandwidth or system load.

[0071] In one possible implementation, the switching module is specifically used for: The control commands are parsed using data acquisition equipment to extract the mode identifiers from the control commands. When the mode identifier is the first mode identifier, the hardware pipeline is used to perform parallel encoding of the original depth data in the first data output format and output it. When the mode identifier is the second mode identifier, the original depth data is parallel encoded in the second data output format and output.

[0072] In one possible implementation, the conversion module is specifically used for: Parse the pattern identifier in the depth data frame and identify the current data output format based on the pattern identifier; The depth data frame is converted based on the data output format.

[0073] In one possible implementation, the conversion module is specifically used for: Convert the exponential offset of the depth data frame in the first data output format to the exponential offset corresponding to the second data output format; The implicit bits of the depth data frame in the first data output format are padded with a leading edge to obtain the normalized mantissa, and the first data output format of the depth data frame is converted into the second data output format.

[0074] In one possible implementation, the unified frame structure includes at least a data resolution width and a data resolution height; the reconstruction module is specifically used for: The first sampling index is obtained based on the data resolution width of the depth data frame, the second sampling index is obtained based on the data resolution height, and the spatial resolution parameters calibrated by the data acquisition device are obtained; wherein, the spatial resolution parameters include horizontal spatial resolution and depth spatial resolution; Based on the sampling index and spatial resolution parameter rate, the horizontal and depth coordinates in the 3D point cloud coordinates are calculated, and the depth value in the 3D point cloud coordinates is obtained based on the depth data frame.

[0075] In one possible implementation, the unified frame structure includes a frame sequence number; the apparatus further includes: The judgment module is used to determine whether the frame sequence numbers of depth data frames are consecutive; The detection module is used to determine if a depth data frame has been dropped, and to detect the current network status if not; the network status includes at least interruption, timeout, and normal. The processing module is used to match the corresponding exception handling method based on the network status and perform exception handling based on the exception handling method.

[0076] The point cloud depth data transmission device provided in this application configures the operating mode through a client, encapsulates the operating mode into corresponding control commands, and sends the control commands to a data acquisition device. The data acquisition device receives the control commands and switches the corresponding data output format based on the control commands to output depth data in that format. The data acquisition device encapsulates the depth data into a unified frame structure, receives the depth data frames through the client, converts the depth data frames, and transmits the depth data frames to the client via Ethernet. The client receives the depth data frames and reconstructs the 3D point cloud coordinates based on them. This point cloud depth data transmission device, by transmitting only depth data and combining it with a dynamic bit-width switching mechanism, completes the 3D coordinate reconstruction of the point cloud on the client side, reducing the bandwidth of point cloud data transmission. Furthermore, the client can select a high-speed mode or a high-precision mode according to application scenario requirements, achieving a dynamic balance between transmission accuracy and speed, reducing data redundancy, and improving transmission efficiency. In addition, the unified depth data transmission frame structure improves the reliability and maintainability of data transmission, and the client can actively and dynamically adjust the data output format of the acquisition device as needed.

[0077] like Figure 4 As shown in the embodiment of this application, an electronic device 400 includes a processor 401, a memory 402, and a bus. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device is running, the processor 401 communicates with the memory 402 via the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the point cloud depth data transmission method described above.

[0078] Specifically, the memory 402 and processor 401 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 401 runs the computer program stored in the memory 402, it can execute the point cloud depth data transmission method mentioned above.

[0079] Corresponding to the above-described point cloud depth data transmission method, this application embodiment also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the steps of the above-described point cloud depth data transmission method.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0081] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0083] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the deployment methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0084] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for transmitting point cloud depth data, characterized in that, The method includes: The operating mode is configured through the client, the operating mode is encapsulated into a corresponding control command, and the control command is sent to the data acquisition device; wherein, the operating mode includes high-speed mode and high-precision mode; the control command at least includes a mode identifier; The data acquisition device receives the control command and switches the corresponding data output format based on the control command to output depth data in that data output format; wherein, the data output format corresponding to the high-speed mode is the first data output format, and the data output format corresponding to the high-precision mode is the second data output format; The data acquisition device encapsulates depth data into a unified frame structure, the client receives the depth data frame and converts the depth data frame, and then transmits the depth data frame to the client via Ethernet. The client receives the depth data frame and reconstructs the 3D point cloud coordinates based on the depth data frame.

2. The method according to claim 1, characterized in that, The configuration of the operating mode via the client includes: In response to user actions on the interactive interface provided by the client, select the corresponding operating mode; or... The client automatically determines and selects the corresponding operating mode based on real-time network bandwidth or system load.

3. The method according to claim 1, characterized in that, The step of switching the corresponding data output format based on the control command to output depth data in that data output format includes: The control commands are parsed using the data acquisition device, and the mode identifiers in the control commands are extracted. When the mode identifier is the first mode identifier, the original depth data is encoded in parallel using a hardware pipeline and output in the first data output format; When the mode identifier is the second mode identifier, the original depth data is parallel encoded in the second data output format and output.

4. The method according to claim 1, characterized in that, The conversion of the depth data frame includes: The mode identifier in the depth data frame is parsed, and the current data output format is identified based on the mode identifier; The depth data frame is converted based on the data output format.

5. The method according to claim 4, characterized in that, The conversion of the depth data frame based on the data output format includes: Convert the exponential offset of the depth data frame in the first data output format to the exponential offset corresponding to the second data output format; The implicit bits of the depth data frame in the first data output format are padded with a leading edge to obtain a normalized mantissa, and the first data output format of the depth data frame is converted into a second data output format.

6. The method according to claim 1, characterized in that, The unified frame structure includes at least the data resolution width and the data resolution height. The reconstruction of 3D point cloud coordinates based on the depth data frame includes: A first sampling index is obtained based on the data resolution width of the depth data frame, a second sampling index is obtained based on the data resolution height, and spatial resolution parameters calibrated by the data acquisition device are obtained; wherein, the spatial resolution parameters include horizontal spatial resolution and depth spatial resolution; Based on the sampling index and the spatial resolution parameter rate, the horizontal coordinates and depth coordinates in the three-dimensional point cloud coordinates are calculated, and the depth value in the three-dimensional point cloud coordinates is obtained based on the depth data frame.

7. The method according to claim 1, characterized in that, The unified frame structure includes a frame sequence number; the method further includes: Determine whether the frame numbers of the depth data frames are consecutive; If not, it is determined that the depth data frame has been dropped, and the current network status is detected; wherein, the network status includes at least interruption, timeout, and normal. The anomaly handling method is matched based on the network state, and the anomaly handling is performed based on the anomaly handling method.

8. A point cloud depth data transmission device, characterized in that, The device includes: The sending module is used to configure the operating mode through the client, encapsulate the operating mode into a corresponding control command, and send the control command to the data acquisition device; wherein, the operating mode includes a high-speed mode and a high-precision mode; and the control command includes at least a mode identifier; The switching module is used to receive the control command through the data acquisition device and switch the corresponding data output format based on the control command to output depth data in the data output format; wherein, the data output format corresponding to the high-speed mode is the first data output format, and the data output format corresponding to the high-precision mode is the second data output format; The conversion module is used to encapsulate depth data into a unified frame structure through the data acquisition device, receive depth data frames through the client and convert the depth data frames, and transmit the depth data frames to the client through Ethernet; The reconstruction module is used to receive the depth data frame through the client and reconstruct the three-dimensional point cloud coordinates based on the depth data frame.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the point cloud depth data transmission method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the point cloud depth data transmission method as described in any one of claims 1 to 7.

Images