Intelligent laser obstacle clearing instrument remote monitoring system

Abstract

The present application relates to the technical field of image data transmission, in particular to a kind of intelligent laser obstacle clearing instrument remote monitoring system, laser radar depth and pixel data are collected and generate obstacle space positioning set, according to depth difference generation monitoring picture partition mark chart, set difference compression ratio and generate partition encoding transmission package, monitor transmission performance and correct generation encoding data package, fusion obstacle information synthesis remote monitoring picture data package.The present application, fusion laser radar, depth sensor and camera data, generate obstacle space positioning set under unified coordinate, realize the accurate acquisition of position and form, divide area by depth difference, detail area low compression ratio coding, ordinary area high compression ratio coding, reduce redundant transmission and improve efficiency, combined with link real-time adjustment to cope with delay packet loss and distortion, ensure image stability and quality, in remote monitoring fusion obstacle information and image, realize state accurate display, improve system responsiveness and reliability.

Description

Technical Field

[0001] This invention relates to the field of image data transmission technology, and in particular to a remote monitoring system for an intelligent laser obstacle clearing device. Background Technology

[0002] Image data transmission technology encompasses image acquisition, processing, transmission, and reproduction technologies. Its core content includes the research and application of image data encoding, compression, transmission, decoding, and display processes. With the development of information technology, image data transmission has been widely applied in various fields such as medicine, monitoring, communication, and entertainment. It primarily addresses the problem of efficient transmission of image data between different devices and reduces transmission bandwidth requirements through compression and encoding technologies. In recent years, with the increasing demand for remote monitoring technology, image data transmission technology has become a core supporting technology for applications such as intelligent monitoring systems, video conferencing, and remote medical care.

[0003] The intelligent laser obstacle clearing remote monitoring system refers to a system that uses laser technology to scan and clear obstacles and achieve remote monitoring. It primarily addresses the technical challenges of obstacle detection, clearing, and data transmission by enabling remote operation and monitoring of the laser obstacle clearing device through image data transmission technology. The system acquires obstacle information in the target area through laser scanning, uses an image sensor to collect images, and then transmits the collected image data wirelessly to a remote terminal to monitor the progress of obstacle clearing in real time. The implementation involves the coordinated operation of a laser scanning device, an image acquisition device, and a data transmission module.

[0004] Current technologies fail to perform refined image compression for different areas, potentially losing crucial details due to over-compression and affecting the accurate location of obstacles. Furthermore, image transmission does not consider real-time network environment changes, failing to dynamically adjust transmission quality based on latency, packet loss, and other factors, resulting in poor image transmission stability and distortion or delays. Existing technologies also do not effectively address image quality fluctuations; details and the accuracy of remote obstacle removal operations cannot be guaranteed in the monitoring footage, thus impacting the real-time performance and accuracy of obstacle detection and removal. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a remote monitoring system for an intelligent laser obstacle clearing device.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a remote monitoring system for an intelligent laser obstacle clearing device, the system comprising:

[0007] The environmental and target data acquisition module acquires the distance measurement value of the lidar scanning head, the depth matrix of the depth sensor, and the pixel matrix of the camera lens. Based on the same time marker, it matches the pixel coordinates with the depth and distance values ​​to calculate the three-dimensional position and connects the boundaries to generate an obstacle spatial positioning set.

[0008] The obstacle partitioning module calls the depth difference between the boundary pixels and surrounding pixels of the obstacle spatial positioning set, divides the detail preservation area and the compression area according to the boundary recognition threshold, merges adjacent similar markers into continuous areas, and generates a monitoring screen partitioning identification map.

[0009] The image compression and encoding module sets a low compression ratio for the detail-preserving area and a high compression ratio for the compressed area according to the monitoring screen partition identification map. It binds the compression ratio to color components and brightness area by area, compares the number of bytes in each area with the available bandwidth, shrinks the excess part proportionally, and generates partitioned encoded transmission packets.

[0010] The transmission link adjustment module monitors the delay, packet loss rate, and distortion of the partitioned coded transmission packet, compares them with the working judgment value, calculates the adjustment amount, and modifies the camera lens resolution and beam emission unit power, and synchronously rewrites the subsequent encoding to obtain the corrected coded data packet.

[0011] As a further aspect of the present invention, the obstacle spatial positioning set includes the measurement values ​​of the lidar scanning head, the depth matrix values ​​of the depth sensor array, and the pixel matrix values ​​of the camera acquisition lens; the monitoring screen partition identification map includes a detail-preserving area and a compression area; the partitioned encoded transmission packet includes a low compression ratio encoded area, low compression ratio encoded color components and brightness values, a high compression ratio encoded area, and binding data of the low compression ratio encoded area; and the corrected encoded data packet includes the adjusted resolution, adjusted beam emission power, delay value, packet loss rate value, and image distortion.

[0012] As a further aspect of the present invention, the environment and target data acquisition module includes:

[0013] The distance acquisition submodule acquires the distance measurement value of the lidar scanning head in the current scanning cycle, calls the measurement value and the cycle time mark to match point by point, arranges the distance measurement values ​​of each point in order based on the time mark, calculates the coordinate interval position of each point in the same cycle, generates a numerical matrix that corresponds to time and coordinate synchronously, and obtains the time coordinate matrix.

[0014] The depth stitching submodule compares the depth matrix values ​​of the depth sensor array in the same period point by point according to the time coordinate matrix, detects the row and column differences of the corresponding points in the three-dimensional spatial coordinates, calls the difference to correct the row and column positions in the depth matrix, establishes the corrected spatial depth matrix, and obtains a unified depth matrix.

[0015] The pixel fusion submodule calls the unified depth matrix and the pixel matrix value of the camera acquisition lens, and stitches the depth value and pixel value point by point in a unified spatial coordinate system. Based on the stitching result, it generates a three-dimensional coordinate set of obstacles and marks the boundary range of the points in the set to obtain the spatial positioning set of obstacles.

[0016] As a further aspect of the present invention, during the process of generating the obstacle spatial positioning set, the dynamic change rate of the three-dimensional coordinate points of each obstacle is further calculated, the motion state of the obstacle is determined based on the change rate, and the area of ​​the obstacle in motion is automatically promoted to the detail-preserving area in the monitoring screen partition identification map, so as to enhance the accuracy of remote monitoring of dynamic obstacles.

[0017] As a further aspect of the present invention, the image obstacle partitioning module includes:

[0018] The boundary detection submodule calls the boundary pixel coordinates of each obstacle in the obstacle spatial positioning set, obtains the row and column positions of the boundary coordinate points in the pixel matrix, detects the boundary extension range of adjacent pixels, establishes a continuous coordinate set of boundary pixels, and obtains the boundary coordinate set.

[0019] The depth discrimination submodule obtains the depth difference of surrounding pixels based on the boundary coordinate set, compares the difference with the obstacle boundary recognition threshold point by point, identifies points that are greater than the threshold, and generates the pixel coordinate distribution of the detail-preserving region to obtain the detail region distribution.

[0020] The region generation submodule calls the set of detailed region distribution and remaining pixel coordinates, marks the detailed regions as reserved regions, marks other regions as compressed regions, and generates a partitioned image matrix for encoding allocation in the monitoring screen, thus obtaining the monitoring screen partition identification map.

[0021] As a further aspect of the present invention, the image compression and encoding module includes:

[0022] The partition binding submodule obtains the detail retention area and compression area in the partition identifier map of the monitoring screen, calls the color component value and brightness value output by the camera acquisition lens, performs coordinate matching for each pixel in the corresponding partition, establishes the correspondence between the partition area and the color component value and brightness value, and obtains the partition pixel mapping table.

[0023] The compression setting submodule calls the pixel distribution of the detail-preserving region and the compression region according to the partitioned pixel mapping table, binds low compression ratio values ​​to the pixels in the detail-preserving region, binds high compression ratio values ​​to the pixels in the compression region, generates an encoding parameter matrix with partitioned compression ratio, and obtains the partitioned compression matrix.

[0024] The encoding generation submodule calls the compression ratio parameter in the partition compression matrix and the color component value and brightness value of the corresponding partition, performs pixel-by-pixel encoding value combination for the partition, forms a binary encapsulation structure that can be transmitted in the network, and generates a partition encoded transmission packet that can be used for data transmission.

[0025] As a further aspect of the present invention, during the image compression and encoding process, the transmission order of the partitioned encoding transmission packets is prioritized, with data packets corresponding to the detail-preserving region set to high priority and data packets corresponding to the compressed region set to low priority, so as to ensure the integrity of the image transmission in the detail-preserving region is prioritized when bandwidth is limited or the link is unstable.

[0026] As a further aspect of the present invention, the transmission link adjustment module includes:

[0027] The performance detection submodule measures the latency value during transmission based on the partitioned coded transmission packet, detects the data packet loss rate value in the transmission node, collects the image distortion value at the receiving end, and combines the three data items into a performance measurement matrix according to the time stamp order to obtain the link performance matrix.

[0028] The threshold determination submodule calls the latency value, packet loss rate value, and image distortion value in the link performance matrix, compares the three values ​​with the working performance threshold one by one, determines whether any value is greater than the threshold, generates a status flag table from the determination result, and obtains the threshold determination table.

[0029] The encoding correction submodule detects states that exceed the threshold according to the threshold judgment table, calls the resolution parameters of the camera acquisition lens and the transmission power parameters of the beam emission unit, synchronously adjusts the data in the partitioned encoding transmission packet, establishes the adapted encoding numerical structure, and obtains the corrected encoding data packet.

[0030] As a further aspect of the present invention, the system further includes:

[0031] The monitoring image synthesis module calls the corrected encoded data packet and the obstacle spatial positioning set to overlap in the same coordinate frame, draws and marks the obstacle boundary on the image, writes the time stamp and frame number into the data header, and establishes the remote monitoring image data packet;

[0032] The remote monitoring image data package includes image data, obstacle spatial location information, and fused data of image and obstacle information.

[0033] As a further aspect of the present invention, the monitoring image synthesis module includes:

[0034] The data fusion submodule calls the coded numerical structure in the corrected coded data packet and the three-dimensional coordinate points in the obstacle spatial positioning set, matches the coded values ​​and coordinate points according to the time stamp order, detects the spatial positional relationship between the color component values ​​and brightness values ​​in the coded values ​​and the coordinate points, generates the fused pixel coordinate matrix, and obtains the fused pixel matrix.

[0035] The coordinate overlay submodule calls the boundary coordinates of the obstacle spatial positioning set according to the fused pixel matrix, performs row-by-row and column-by-column matching of the pixel matrix of the image and the spatial coordinate points, performs overlay operation under the same coordinate frame, generates a data set containing the correspondence between spatial boundaries and pixel values, and obtains the coordinate overlay set;

[0036] The image generation submodule calls the pixel values ​​and spatial position relationships in the coordinate overlay set to synthesize the image matrix and obstacle boundary points point by point, and establishes an image data package that can be directly displayed on a remote terminal to obtain a remote monitoring image data package.

[0037] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0038] This invention integrates data from LiDAR scanning, depth sensors, and cameras, performing spatial matching and stitching within a unified coordinate system to accurately generate a spatial location set of obstacles, achieving high-precision acquisition of obstacle positions and shapes. Depth difference analysis is used to divide the image into regions; low-compression encoding is used for detail-preserving areas, while high-compression encoding is used for compressed areas, reducing unnecessary data transmission and improving transmission efficiency. Real-time adjustments to the transmission link dynamically address issues such as latency, packet loss, and image distortion, ensuring image quality and stability. In remote monitoring, the fused display of obstacle spatial location information and image data provides accurate obstacle clearance status, improving the responsiveness and reliability of the monitoring system. Attached Figure Description

[0039] Figure 1 This is a system flowchart of the present invention;

[0040] Figure 2 This is a flowchart illustrating the acquisition process of the environment and target data acquisition module of this invention.

[0041] Figure 3 This is a flowchart illustrating the acquisition process of the obstacle partitioning module in the image of the present invention.

[0042] Figure 4 This is a flowchart illustrating the acquisition process of the image compression and encoding module of the present invention.

[0043] Figure 5 This is a flowchart illustrating the acquisition process of the transmission link adjustment module of the present invention.

[0044] Figure 6 This is a flowchart illustrating the acquisition process of the monitoring image synthesis module of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0046] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0047] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0048] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0049] Please see Figure 1 This invention provides a technical solution: a remote monitoring system for an intelligent laser obstacle clearing device, the system comprising:

[0050] The environment and target data acquisition module acquires the distance measurement value of the lidar scanning head in the current scanning cycle, the depth matrix value of the depth sensor array in the same cycle, and the pixel matrix value of the camera acquisition lens. The three types of data are matched and stitched together in a unified spatial coordinate system according to the same time marker to generate an obstacle spatial positioning set for displaying the position and shape of obstacles.

[0051] The obstacle partitioning module calls the depth difference between the boundary pixel coordinates of each obstacle and the surrounding pixels in the obstacle spatial positioning set. It marks the areas with depth differences higher than the obstacle boundary recognition threshold as detail-preserving areas and other areas as compression areas, generating a monitoring screen partitioning map for encoding allocation in the monitoring screen.

[0052] The image compression and encoding module sets low compression ratio encoding for detail-preserving areas and high compression ratio encoding for compressed areas based on the zoning map of the monitoring screen. It binds the encoding settings with the color component values ​​and brightness values ​​output by the camera lens to generate zoning encoded transmission packets that can be used for data transmission.

[0053] The transmission link adjustment module measures the delay, packet loss rate, and image distortion during transmission based on the partitioned coded transmission packet, and compares them with the working performance threshold. When any value exceeds the threshold, it adjusts the resolution of the camera acquisition lens and the transmission power of the beam emission unit to generate a corrected coded data packet that can be directly sent after adaptation.

[0054] The monitoring image synthesis module calls the corrected encoded data packet and fuses it with the obstacle spatial positioning set. It then overlays and synthesizes the image and obstacle spatial position information within the same coordinate frame to generate a remote monitoring image data packet that can be directly displayed on a remote terminal.

[0055] The obstacle spatial positioning set includes the measurement values ​​of the lidar scanning head, the depth matrix values ​​of the depth sensor array, and the pixel matrix values ​​of the camera acquisition lens. The monitoring screen partition identification map includes the detail-preserving area and the compressed area. The partition encoding transmission packet includes the low compression ratio encoded area, the color components and brightness values ​​of the low compression ratio encoded area, the high compression ratio encoded area, and the binding data of the low compression ratio encoded area. The corrected encoding data packet includes the adjusted resolution, the adjusted beam emission power, the delay value, the packet loss rate value, and the image distortion. The remote monitoring screen data packet includes the image, the obstacle spatial location information, and the fusion data of the image and obstacle information.

[0056] Please see Figure 2 The environmental and target data acquisition module includes:

[0057] The distance acquisition submodule acquires the distance measurement value of the lidar scanning head in the current scanning cycle, calls the measurement value and the cycle time mark to match point by point, arranges the distance measurement values ​​of each point in order based on the time mark, calculates the coordinate interval position of each point in the same cycle, generates a numerical matrix that corresponds to time and coordinate synchronously, and obtains the time coordinate matrix.

[0058] The lidar scanning head is invoked during the current scanning cycle, which is set to 100 milliseconds. Distance measurements are acquired within this cycle. Specifically, the lidar scanning head rotates horizontally at a rate of 600 revolutions per minute, with its 16 internal laser emitters scanning synchronously. The horizontal angular resolution is [missing information]. Vertical angular resolution is One scan is completed within a 100-millisecond scan cycle. Scan, total collection The system marks each laser pulse's emission and reception process with a high-precision timestamp at a distance measurement point. For example, at the 25.055th millisecond of the cycle, the 7th laser emitter (with a vertical angle of...)... ) rotate horizontally to A distance measurement of 41.5 meters was obtained. The system then mapped this measurement to the corresponding time marker (25.055 milliseconds) point by point, forming a data tuple (25.055, 41.5, ...). , Subsequently, based on the time stamps from 0 milliseconds to 99.9xx milliseconds (the exact time depends on the last data collection point), all 28,800 data tuples are arranged into an ordered sequence. Next, the system calculates the coordinate interval position of each point within the same period, which uses the distance value from each data tuple. Horizontal angle and vertical angle This is performed through a transformation from spherical coordinates to Cartesian coordinates, and the calculation process is as follows: , , Taking the aforementioned data tuple as an example, let's substitute the values ​​and perform the calculation: rice, rice, The Z coordinate here is relative to the horizontal plane of the scanning head. This coordinate transformation is performed on all 28,800 points within the period to generate a numerical matrix containing five columns of data: [time marker, original distance value, X coordinate, Y coordinate, Z coordinate], thus obtaining the time coordinate matrix.

[0059] The depth stitching submodule compares the depth matrix values ​​of the depth sensor array in the same period point by point according to the time coordinate matrix, detects the row and column differences of the corresponding points in the three-dimensional spatial coordinates, calls the difference to correct the row and column positions in the depth matrix, establishes the corrected spatial depth matrix, and obtains a unified depth matrix.

[0060] Based on the time coordinate matrix, the depth matrix values ​​output by the depth sensor array installed 0.2 meters directly below and 0.1 meters directly in front of the lidar scanning head are compared point by point within the same 100-millisecond period. The depth sensor array has a resolution of 640x480 and a horizontal field of view. ,vertical The system detects the row and column differences of corresponding points in three-dimensional spatial coordinates. These differences originate from the physical spatial displacement between the two sensors. The specific correction process involves adjusting the coordinates of each point in the depth sensor's own coordinate system. Transforming to a unified coordinate system with the lidar as the origin, the transformation operation is performed through a translation vector. (Corresponding to displacement along the X, Y, and Z axes) This is achieved using corrected coordinates. The calculation is as follows ,Right now , , For example, if a depth sensor detects an obstacle point in its coordinate system with coordinates (29.18, 29.21, -3.42), then the aforementioned difference value is used to correct the row and column position of that point, and its coordinates in the unified coordinate system become: , , This correction process is applied to all pixels in the depth matrix, thereby establishing a corrected spatial depth matrix in which the spatial positions of all points are aligned with the LiDAR coordinate system, resulting in a unified depth matrix.

[0061] The pixel fusion submodule calls the unified depth matrix and the pixel matrix value of the camera acquisition lens, stitches the depth value and pixel value point by point in a unified spatial coordinate system, generates a three-dimensional coordinate set of obstacles based on the stitching result, and marks the boundary range of the points in the set to obtain the spatial positioning set of obstacles.

[0062] During the process of generating the spatial location set of obstacles, the dynamic change rate of the three-dimensional coordinate points of each obstacle is further calculated. Based on the change rate, the motion state of the obstacle is judged, and the area of ​​the obstacle in motion is automatically promoted to the detail preservation area in the partition identification map of the monitoring screen to enhance the accuracy of remote monitoring of dynamic obstacles.

[0063] The process involves calling a unified depth matrix and the pixel matrix values ​​of the camera (1920x1080 resolution), and then stitching the depth values ​​and pixel values ​​point-by-point in a unified spatial coordinate system. This process will assign each 3D coordinate point in the unified depth matrix a corresponding pixel value. The projection matrix of the camera is used to transform the image onto its two-dimensional image plane. The projection transformation is based on the pinhole camera model, and the transformation process is as follows: ,in Here, K represents the pixel coordinates, and K is the camera's intrinsic parameter matrix. The intrinsic parameter matrix K is pre-determined using the Zhang Zhengyou calibration method and set to [value missing]. The projection calculation first normalizes the three-dimensional coordinates: , Then, the pixel coordinates are calculated using the intrinsic parameter matrix: , Since the calculated pixel coordinates are outside the image area, indicating that this specific point is not within the camera's field of view, the system will process the next point, such as point [missing information]. The projection calculation result is pixel coordinates (1500, 860). The system then extracts the RGB value of this pixel position as (88, 95, 101) and compares this RGB value with the three-dimensional coordinates. Binding is performed to form fused data points. After traversing all points within the camera's field of view, a set of 3D coordinates of obstacles is generated based on the stitching result. The boundary range of points within the set is outlined. During the generation of the obstacle spatial positioning set, the system further calculates the dynamic change rate of each obstacle's 3D coordinate point, by using the current cycle... A certain obstacle point Compared with the previous cycle The corresponding point in Comparing the positions, their dynamic rate of change The calculation process is as follows: Substitute the values: meters per second, the system sets a threshold for motion state discrimination. The threshold is 0.15 m / s. This threshold is obtained by continuously collecting data for 1000 cycles in a static scene, calculating the apparent velocity of all stationary objects, and taking the 99.7 percentile of its statistical distribution (i.e., the mean plus three standard deviations). The calculated rate of change... meters per second higher than the threshold The system determines the motion state of the obstacle based on the rate of change (m / s), classifying it as a moving object. In the subsequent monitoring screen partitioning map, the area formed by the point and its surrounding pixels is automatically upgraded to a detail-preserving region. Specific data examples are shown in Table 1 below.

[0064] Table 1: Example Table of Spatial Location Set Data for Obstacles

[0065]

[0066] As shown in Table 1, this table lists some data points in the obstacle spatial positioning set, including their three-dimensional spatial coordinates, the pixel positions after projection, and the motion state determined by the dynamic change rate, thus obtaining the obstacle spatial positioning set.

[0067] Please see Figure 3 The obstacle partitioning module includes:

[0068] The boundary detection submodule calls the boundary pixel coordinates of each obstacle in the obstacle spatial localization set, obtains the row and column positions of the boundary coordinate points in the pixel matrix, detects the boundary extension range of adjacent pixels, establishes a continuous coordinate set of boundary pixels, and obtains the boundary coordinate set.

[0069] The system retrieves the boundary pixel coordinates of each obstacle from the obstacle spatial localization set. These boundary pixel coordinates were obtained in the previous step by performing an edge detection algorithm (such as the Canny operator) on the 2D projection map of the obstacle point cloud. The system then obtains the row and column positions of these boundary coordinates in a 1920x1080 pixel matrix. For example, if an initial boundary point with coordinates (1500, 850) is obtained, the system places it in a queue and detects its eight neighboring pixels, namely (1499, 849), (1499, 850), (1499, 851), (1500, 849), (1500, 851), (1501, 849), (1501, 850), (1501, 851). The system iterates through the neighboring points to determine if they also belong to the set of detected boundary points. If a neighboring point (1501, 850) is also identified as a boundary point, it adds it to the queue and marks the processed point (1500, 850) as visited. The system repeats this process, taking new points from the queue and detecting their neighbors to detect the boundary extension range of adjacent pixels until the queue is empty. This process establishes a set containing the coordinates of all pixels that constitute a continuous contour. For example, it eventually forms a continuous coordinate set containing a series of coordinates such as [(1500, 850), (1501, 850), (1502, 851), ..., (1498, 860)], thus obtaining the boundary coordinate set.

[0070] The depth discrimination submodule obtains the depth difference of surrounding pixels based on the boundary coordinate set, compares the difference with the obstacle boundary recognition threshold point by point, identifies points that are greater than the threshold, and generates the pixel coordinate distribution of the detail-preserving region to obtain the detail region distribution.

[0071] Based on the boundary coordinate set, the system obtains the depth difference of surrounding pixels. It retrieves a boundary point coordinate from the set, such as (1501, 850), queries its corresponding 3D coordinates, and obtains its depth value (i.e., distance from the sensor) as 8.2 meters. Next, the system calculates the normal direction of this boundary point on the image and searches for a background point 5 pixels outward along this normal direction. Assuming the found background point coordinates are (1505, 848), and its depth value is 45.6 meters, the system calculates the depth difference between these two points. Meters, the system calls an obstacle boundary recognition threshold. Point-by-point comparisons were performed, and the threshold was set according to the following experimental procedure: 100 scenes containing clear obstacles (such as vehicles, pedestrians, and trees) were selected. The obstacle outlines were manually marked, and the depth values ​​within a 5-pixel width region on both sides of the outline were extracted. The depth difference of all corresponding point pairs was calculated to form a dataset. Analysis of this dataset revealed that the depth difference at the boundaries of 98% of real obstacles was greater than 3.5 meters, while the depth fluctuations caused by sensor noise or changes in the surface material of objects were less than 1.0 meter. To ensure effective recognition and leave room for error, the threshold was set... The depth difference is set to 3.5 meters. Since the calculated difference of 37.4 meters is greater than the threshold of 3.5 meters, the system determines that the point (1501, 850) is a valid boundary point with a significant depth jump and records the coordinates of this point in a new list. The system repeats this comparison process for all points in the boundary coordinate set, and finally generates a pixel coordinate distribution that only contains valid boundary points verified by the depth difference, thus obtaining the detail region distribution.

[0072] The region generation submodule calls the set of detailed region distribution and remaining pixel coordinates, marks the detailed regions as reserved regions, marks other regions as compressed regions, generates a partition image matrix for encoding allocation in the monitoring screen, and obtains the monitoring screen partition identification map.

[0073] The system calls the set of coordinates of the detailed region distribution and all remaining pixels in the image. It creates a 1920x1080 two-dimensional matrix with the same resolution as the camera and initializes all its elements to 0. Then, the system traverses each pixel coordinate in the detailed region distribution, for example (1501, 850), and modifies the value at the corresponding position [1501, 850] in the two-dimensional matrix to 1. At the same time, in order to ensure the integrity of the details, a morphological dilation operation is performed on each point marked as 1, and the values ​​of all pixels in its surrounding 5x5 neighborhood are also set to 1. This operation connects all points identified as details into one or more complete regions. After traversing all points in the detailed region distribution and completing the dilation operation, the regions with a value of 1 in the two-dimensional matrix are the detail-preserving regions, while the regions with a value of 0 are the compression regions. Through this operation, the detailed regions are marked as preserved regions, and other regions are marked as compression regions. Finally, a partitioned image matrix for encoding allocation in the monitoring screen is generated, resulting in the monitoring screen partition identification map.

[0074] Please see Figure 4 The image compression and encoding module includes:

[0075] The partition binding submodule obtains the detail retention area and compression area in the partition identifier map of the monitoring screen, calls the color component value and brightness value output by the camera acquisition lens, performs coordinate matching for each pixel in the corresponding partition, establishes the correspondence between the partition area and the color component value and brightness value, and obtains the partition pixel mapping table.

[0076] The system acquires the detail-preserving region (regions with a value of 1) and the compressed region (regions with a value of 0) from the monitoring screen's partition map. It then retrieves the color component values ​​and luminance values ​​of the 1920x1080 resolution YUV420 format image output by the camera lens at the same time. The system scans the partition map pixel by pixel. When it scans the coordinates (1502, 851), it detects a value of 1, indicating it belongs to the detail-preserving region. The system then extracts the corresponding luminance value Y(1502, 851) and its corresponding chrominance values ​​U(751, 425) and V(751, 425) from the YUV image data (since it is YUV420 format, the resolution of the UV components is...). (Half of the Y component), assuming the extracted values ​​are Y=120, U=110, V=140, the system associates these values ​​with coordinates (1502, 851) and their region attributes (detail preservation). When the coordinates (10, 10) are scanned, their value is detected as 0, indicating that it belongs to the compressed region. The system extracts its corresponding YUV values, assuming they are Y=210, U=128, V=128, and associates them with coordinates (10, 10) and the region attributes (compression). After performing this coordinate matching operation on all 1920x1080 luminance pixels, a complete correspondence between the partitioned region and the color component values ​​and luminance values ​​is established, resulting in a partitioned pixel mapping table.

[0077] The compression setting submodule calls the pixel distribution of the detail-preserving region and the compression region according to the partition pixel mapping table, binds low compression ratio values ​​to the pixels in the detail-preserving region, binds high compression ratio values ​​to the pixels in the compression region, generates an encoding parameter matrix with partition compression ratio, and obtains the partition compression matrix.

[0078] The system retrieves the pixel distribution of the detail-preserving region and the compressed region based on the partitioned pixel mapping table. Pixels in the detail-preserving region are bound to low compression ratio values, and pixels in the compressed region are bound to high compression ratio values. The compression ratio is achieved by setting the quantization parameter (QP) of the H.265 encoder. The QP value ranges from 0 to 51; the smaller the value, the lower the compression ratio and the higher the quality. The system divides the QP value into three intervals: low compression ratio [20, 26], medium compression ratio [27, 35], and high compression ratio [36, 48]. Based on the partitioned pixel mapping table, the system binds a QP value of 22 to all pixels marked as detail-preserving regions and a QP value of 42 to all pixels marked as compressed regions. This binding process generates a 1920x1080 encoding parameter matrix with the same size as the original image. In this matrix, the element value corresponding to the detail-preserving region is 22, and the element value corresponding to the compressed region is 42, thus obtaining the partitioned compression matrix.

[0079] The encoding generation submodule calls the compression ratio parameter in the partition compression matrix and the color component value and brightness value of the corresponding partition, performs pixel-by-pixel encoding value combination for the partition, forms a binary encapsulation structure that can be transmitted in the network, and generates a partition encoded transmission packet that can be used for data transmission.

[0080] During the image compression and encoding process, the transmission order of the partitioned encoding transmission packets is prioritized, with data packets corresponding to the detail-preserving region set to high priority and data packets corresponding to the compressed region set to low priority, so as to ensure the integrity of the image transmission in the detail-preserving region is guaranteed first when bandwidth is limited or the link is unstable.

[0081] The video encoder processes images by calling the compression ratio parameter (QP value) in the partition compression matrix and the corresponding color component values ​​and luminance values ​​(YUV values) of the partition. It operates in units of 16x16 pixel coding units (CUs). For each CU, the system calculates the proportion of pixels belonging to the detail-preserving region. If the proportion exceeds 60%, the entire CU is encoded using a low QP value of 22; if the proportion is below 10%, the entire CU is encoded using a high QP value of 42; and if the proportion is between 10% and 60%, a medium QP value of 30 is used. This pixel-by-pixel encoding value combination for each partition forms a binary encapsulation structure that can be transmitted over the network. During image compression and encoding, the transmission order of the partition-encoded transmission packets is prioritized. When encapsulating video data packets encoded with QP values ​​22 and 30 into RTP (Real-time Transport Protocol) packets, the DSCP (Differentiated Services Code) is set in the Differentiated Services field of the IP header. The Point value is 0b101110 (indicating expedited forwarding EF), while the data packet generated using QP value 42 encoding is set to DSCP value 0b000000 (indicating best effort BE), generating a partitioned encoded transport packet that can be used for data transmission.

[0082] Please see Figure 5 The transmission link adjustment module includes:

[0083] The performance testing submodule measures the latency value during transmission based on the partitioned coded transmission packet, detects the data packet loss rate value in the transmission node, and collects the image distortion value at the receiving end. The three data items are combined into a performance measurement matrix according to the time stamp order to obtain the link performance matrix.

[0084] The system measures the latency during transmission based on partitioned encoding packets. This latency is calculated by taking half of the Round-Trip Time (RTT) from the timestamp information in the Sender's Report (SR) and Receiver's Report (RR) packets of RTCP (RTP Control Protocol). For example, if the measured RTT is 210 milliseconds, the one-way latency is 105 milliseconds. Simultaneously, the system continuously monitors the sequence numbers of RTP packets to detect the packet loss rate in the transmission nodes. If, within one second, the sender sends packets with the last sequence number 5500 and the first sequence number 4500, totaling 1001 packets, and the receiver receives 986 packets during this period, the packet loss rate is [missing information]. The system collects the image distortion value at the receiving end. This value is obtained by comparing the decoded detail-preserving region with the original (before encoding) detail-preserving region at the receiving end and calculating its peak signal-to-noise ratio (PSNR). For example, the calculated PSNR value is 29.5dB. The system combines these three data items in time stamp order to form a performance measurement matrix, as shown in Table 2 below.

[0085] Table 2: Real-time Link Performance Monitoring Table

[0086]

[0087] As shown in Table 2, this table records the key link performance indicators for three consecutive seconds, which are used for subsequent threshold determination to obtain the link performance matrix.

[0088] The threshold determination submodule calls the latency value, packet loss rate value, and image distortion value in the link performance matrix, compares the three values ​​with the working performance threshold one by one, determines whether any value is greater than the threshold, generates a status flag table from the determination results, and obtains the threshold determination table.

[0089] The latency, packet loss rate, and image distortion values ​​from the link performance matrix were retrieved, and each value was compared with a performance threshold. The performance threshold was set based on the following experiments and standards: Ten experienced operators were organized to conduct simulated remote fault clearing tasks, and their success rates under different network latencies were recorded. It was found that the error rate increased significantly after a latency exceeding 100 milliseconds; therefore, the latency threshold was set to [value missing]. ms; According to the ITU-T G.1010 standard, for interactive video applications, a packet loss rate exceeding 1% will begin to have a significant impact on user experience. Therefore, the packet loss rate threshold is set to ms. Subjective video quality assessment (MOS) was used to have observers rate video footage at different PSNR values. It was found that when the PSNR was below 30dB, most observers considered obstacle details to be blurry. Therefore, the image distortion threshold was set to... dB, the system takes the data with timestamp of 10:01:02 in Table 2 for comparison, and the specific determination process is shown in Table 3 below.

[0090] Table 3: Examples of Threshold Determination

[0091]

[0092] As shown in Table 3, the determination is... (Delay exceeds threshold) (Packet loss rate exceeds threshold) (Distortion exceeds threshold) Determine that any value (all three items here) exceeds the threshold, and generate a status flag table in the form of Boolean values: {Delay exceeds limit: True, Packet loss rate exceeds limit: True, Distortion exceeds limit: True}, to obtain the threshold determination table.

[0093] The encoding correction submodule detects states that exceed the threshold according to the threshold judgment table, calls the resolution parameters of the camera acquisition lens and the transmission power parameters of the beam emission unit, synchronously adjusts the data in the partitioned encoding transmission packet, establishes the adapted encoding numerical structure, and obtains the corrected encoding data packet.

[0094] If the system detects that latency, packet loss rate, and distortion all exceed the thresholds based on the threshold judgment table, it triggers a tiered adjustment strategy. First, it calls the resolution parameter of the camera acquisition lens and reduces it from 1920x1080 to 1280x720. This reduces the amount of original video data by about 56%. Then, the system re-evaluates the link performance. If the performance indicators still exceed the limits after 1 second, while maintaining the 1280x720 resolution, it calls the emission power parameter of the beam emission unit and reduces it from 100% to 85%. This reduces the amount of LiDAR point cloud data, thereby reducing the complexity of the obstacle spatial localization set. The data in the partitioned encoded transmission packet is then synchronously adjusted. That is, the video encoder is now processing a 1280x720 image and uses the new point cloud data to generate a partition identification map, establishing a lower total data volume encoding numerical structure to adapt to the current link conditions, resulting in a corrected encoded data packet.

[0095] The data fusion submodule calls the coded numerical structure in the corrected coded data packet and the three-dimensional coordinate points in the obstacle spatial positioning set, matches the coded values ​​with the coordinate points according to the time stamp order, detects the spatial positional relationship between the color component values ​​and brightness values ​​in the coded values ​​and the coordinate points, generates the fused pixel coordinate matrix, and obtains the fused pixel matrix.

[0096] The system calls upon the encoded numerical structure in the corrected encoded data packet (i.e., a 1280x720 resolution video frame) and the original, unsampled high-precision 3D coordinates in the obstacle spatial localization set. It maps the encoded values ​​of the video frame to the coordinates using the same time marker. The system detects the spatial relationship between the color component values ​​and luminance values ​​in the encoded values ​​and the coordinates. This process involves using a specific high-precision 3D coordinate point, for example... The image is then reprojected onto a 1280x720 resolution image plane, using an intrinsic parameter matrix matched to the new resolution. The system calculates the new pixel coordinates as (1000, 573), and then converts the original 3D coordinates... Data binding is performed with the YUV values ​​of the (1000,573) pixel in the corrected encoded data packet to generate a data structure in which each element contains a pixel value of a low-resolution image and a high-precision three-dimensional spatial coordinate, forming a fused pixel coordinate matrix, thus obtaining the fused pixel matrix.

[0097] The coordinate overlay submodule calls the boundary coordinates of the obstacle spatial positioning set according to the fused pixel matrix, performs row-by-row and column-by-column matching between the pixel matrix of the image and the spatial coordinate points, performs overlay operation under the same coordinate frame, and generates a data set containing the correspondence between spatial boundaries and pixel values, thus obtaining the coordinate overlay set.

[0098] Based on the fused pixel matrix, the system retrieves the original boundary coordinates of the obstacle spatial positioning set. The pixel matrix of the degraded 1280x720 image is then matched row-by-row and column-by-column with these high-precision spatial coordinate points. On the remote terminal's display interface (e.g., a 1920x1080 physical monitor), the system first upsamples the 1280x720 video image to fill the display. Then, an overlay operation is performed on this image. This operation projects each point in the original, high-precision obstacle boundary coordinate set onto the 1280x720 coordinate system, forming a series of new two-dimensional boundary point coordinates. For example, the original boundary point (1500, 850) corresponds to (1000, 567) at the new resolution. The system connects these new boundary points to form a vector contour, generating a data set containing upsampled pixel values ​​and precise vector contour data, resulting in a coordinate overlay set.

[0099] The image generation submodule calls the pixel values ​​and spatial position relationships in the coordinate overlay set to synthesize the image matrix and obstacle boundary points point by point, and establishes an image data package that can be directly displayed on the remote terminal to obtain the remote monitoring image data package.

[0100] By invoking the pixel values ​​and spatial relationships in the coordinate overlay set, the upsampled image matrix is ​​synthesized point-by-point with the vectorized obstacle boundary points. Specifically, the graphics rendering unit of the remote terminal first renders the 1280x720 video frame into the frame buffer. Then, it calls graphics drawing instructions (such as OpenGL or DirectX) to draw the precise boundary of the obstacle on the same content in the frame buffer, based on the vector contour data in the coordinate overlay set, using a red line with a width of 3 pixels and a color of (R:255, G:0, B:0). At the same time, for obstacles identified as moving, their inner contour area is covered by a semi-transparent yellow layer (R:255, G:255, B:0, Alpha:0.4). This process establishes an image data package that can be directly displayed on the remote terminal, with adaptively reduced video clarity but lossless enhancement of key target information through vector graphics, resulting in a remote monitoring screen data packet.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A remote monitoring system for an intelligent laser obstacle clearing device, characterized in that, The system comprises: An environment and target data acquisition module acquires distance measurement values of a laser radar scanning head, a depth matrix of a depth sensor, and a pixel matrix of a camera lens, matches positions and splices matrices under a unified spatial coordinate according to the same time marker, converts three-dimensional positions of pixel coordinates and depth values and distance values, and connects boundaries to generate an obstacle space positioning set; A picture obstacle partition module calls a depth difference between boundary pixels and surrounding pixels of the obstacle space positioning set, divides a detail reservation area and a compression area according to a boundary recognition threshold, merges adjacent same markers into continuous areas, and generates a monitoring picture partition identification map; An image compression and coding module sets a low compression ratio for the detail reservation area and a high compression ratio for the compression area according to the monitoring picture partition identification map, binds the compression ratios with color components and brightness in areas, compares byte amounts and available bandwidths of the areas, proportionally shrinks an excess part, and generates a partition coding transmission package; A transmission link adjustment module monitors delay, packet loss rate, and distortion degree based on the partition coding transmission package, compares the same with a working judgment value, calculates an adjustment amount, and changes camera lens resolution and beam emission unit power, synchronously rewrites subsequent coding, and obtains a corrected coding data package; A monitoring picture synthesis module calls the corrected coding data package and the obstacle space positioning set to overlap in the same coordinate frame, draws and labels obstacle boundaries on an image, writes time markers and frame numbers into a data header, and establishes a remote monitoring picture data package; The remote monitoring picture data package comprises a picture image, obstacle space position information, and fusion data of the picture and obstacle information.

2. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 1, characterized in that: The obstacle space positioning set comprises measurement values of a laser radar scanning head, depth matrix values of a depth sensor array, and pixel matrix values of a camera lens, the monitoring picture partition identification map comprises a detail reservation area and a compression area, the partition coding transmission package comprises low compression ratio coding areas, low compression ratio coded color components and brightness values, high compression ratio coding areas, and binding data of low compression ratio coding areas, and the corrected coding data package comprises adjusted resolution, adjusted beam emission power, delay values, packet loss rate values, and picture distortion degrees.

3. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 1, characterized in that, The environment and target data acquisition module comprises: A distance acquisition submodule acquires distance measurement values of a laser radar scanning head in a current scanning period, calls the measurement values and period time markers for point-by-point correspondence, arranges distance measurement values of each point based on time marker sequences, calculates coordinate interval position amounts of each point in the same period, generates a value matrix corresponding to time and coordinates synchronously, and obtains a time coordinate matrix; A depth splicing submodule compares depth matrix values of a depth sensor array in the same period point by point according to the time coordinate matrix, detects row and column difference amounts of corresponding points in three-dimensional spatial coordinates, calls the difference amounts to correct row and column positions in the depth matrix, establishes a corrected spatial depth matrix, and obtains a unified depth matrix; The pixel fusion sub-module calls the unified depth matrix and pixel matrix value of the camera lens, splices the depth value and the pixel value point by point in the unified space coordinate, generates a three-dimensional coordinate set of the obstacle based on the splicing result, and performs contour marking on the boundary range of the points in the set to obtain an obstacle space positioning set.

4. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 3, characterized in that, In the process of generating the obstacle space positioning set, the dynamic change rate of each obstacle three-dimensional coordinate point is further calculated, the motion state of the obstacle is distinguished based on the change rate, and the region of the obstacle in the motion state is automatically promoted to a detail reservation region in the partition identification map of the monitoring picture, so as to enhance the remote monitoring accuracy of the dynamic obstacle.

5. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 1, characterized in that, The picture obstacle partition module comprises: The boundary detection sub-module calls the boundary pixel coordinates of each obstacle in the obstacle space positioning set, obtains the row and column positions of the boundary coordinate points in the pixel matrix, detects the boundary extension range of the adjacent pixel points, establishes a continuous coordinate set of the boundary pixel points, and obtains a boundary coordinate set; The depth discrimination sub-module obtains the depth difference value of the surrounding pixels according to the boundary coordinate set, calls the difference value and the obstacle boundary recognition threshold for point-by-point comparison, discriminates the points greater than the threshold, and generates the pixel coordinate distribution of the detail reservation region to obtain the detail region distribution; The region generation sub-module calls the detail region distribution and the set of the remaining pixel coordinates, marks the detail region as a reservation region and marks other regions as compression regions, generates a partition image matrix used for encoding allocation in the monitoring picture, and obtains a monitoring picture partition identification map.

6. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 1, characterized in that, The image compression and encoding module comprises: The partition binding sub-module obtains the detail reservation region and the compression region in the monitoring picture partition identification map, calls the color component value and the brightness value output by the camera lens, performs coordinate matching on the pixels in the corresponding partition, establishes the corresponding relationship between the partition region and the color component value and the brightness value, and obtains a partition pixel mapping table; The compression setting sub-module calls the pixel distribution of the detail reservation region and the compression region according to the partition pixel mapping table, binds a low compression ratio value to the pixels of the detail reservation region, binds a high compression ratio value to the pixels of the compression region, generates an encoding parameter matrix with a partition compression ratio, and obtains a partition compression matrix; The encoding generation sub-module calls the compression ratio parameters in the partition compression matrix and the color component value and the brightness value of the corresponding partition, performs encoding value combination on the pixels in the partition, forms a binary encapsulation structure that can be transmitted in the network, and generates a partition encoding transmission package that can be used for data transmission.

7. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 6, characterized in that, In the image compression and encoding process, the transmission order of the partition encoding transmission package is prioritized, the data package corresponding to the detail reservation region is set as high priority, and the data package corresponding to the compression region is set as low priority, so as to ensure the picture transmission integrity of the detail reservation region when the bandwidth is limited or the link is unstable.

8. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 1, characterized in that, The transmission link adjustment module comprises: The performance detection sub-module detects the delay value in the transmission process based on the partitioned encoding transmission packet, detects the data packet loss rate value in the transmission node, collects the picture distortion value of the receiving end, combines the three data values in the order of time markers into a performance detection matrix, and obtains a link performance matrix; The threshold determination sub-module compares the delay value, the packet loss rate value and the picture distortion value in the link performance matrix with the working performance threshold value one by one, determines whether any value is greater than the threshold value, generates a state marker table according to the determination result, and obtains a threshold determination table; The encoding correction sub-module detects the state of exceeding the threshold value according to the threshold determination table, calls the resolution parameter of the camera lens and the transmission power parameter of the beam emission unit, synchronously adjusts the data in the partitioned encoding transmission packet, establishes an adapted encoding value structure, and obtains a corrected encoding data packet.

9. The intelligent laser obstacle clearing instrument remote monitoring system according to claim 1, characterized in that, The monitoring picture synthesis module comprises: The data fusion sub-module calls the encoding value structure in the corrected encoding data packet and the three-dimensional coordinate points in the obstacle space positioning set, corresponds the encoding value and the coordinate points in the order of time markers, detects the spatial position relationship between the color component value and the brightness value in the encoding value and the coordinate points, generates a fused pixel coordinate matrix, and obtains a fused pixel matrix; The coordinate superposition sub-module calls the boundary coordinates of the obstacle space positioning set according to the fused pixel matrix, performs row-by-column matching on the pixel matrix of the picture image and the spatial coordinate points, performs superposition operation in the same coordinate frame, generates a data set containing the spatial boundary and the pixel value correspondence relationship, and obtains a coordinate superposition set; The picture generation sub-module calls the pixel value and the spatial position relationship in the coordinate superposition set, performs point-by-point synthesis on the image matrix and the obstacle boundary points, establishes an image data package that can be directly displayed on the remote terminal, and obtains a remote monitoring picture data packet.

Images